WO2013173861A1 - Object position determination - Google Patents

Object position determination Download PDF

Info

Publication number
WO2013173861A1
WO2013173861A1 PCT/AU2013/000411 AU2013000411W WO2013173861A1 WO 2013173861 A1 WO2013173861 A1 WO 2013173861A1 AU 2013000411 W AU2013000411 W AU 2013000411W WO 2013173861 A1 WO2013173861 A1 WO 2013173861A1
Authority
WO
WIPO (PCT)
Prior art keywords
points
point data
sensing device
area
cluster
Prior art date
Application number
PCT/AU2013/000411
Other languages
French (fr)
Inventor
Mark Darrell BARRY
Kevin John GOSSCHALK
Steven Colin MARTIN
Timothy Charles GURNETT
Original Assignee
Queensland University Of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Queensland University Of Technology filed Critical Queensland University Of Technology
Publication of WO2013173861A1 publication Critical patent/WO2013173861A1/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/042—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means
    • G06F3/0425—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by opto-electronic means using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected

Definitions

  • the present invention relates to a method and apparatus for determining the position of an object relative to an area, and in one example, for determining the position of an object relative to an image displayed in the area.
  • US2008191864 describes an interactive training system capable of generating continuous feedback for physical therapy and training applications based on capturing and analyzing the movement of a user on an interactive surface.
  • the training system captures sophisticated input such as the entire areas in contact with the interactive surface, center of gravity, pressure distribution, velocity, acceleration, direction, orientation etc.
  • the training system also captures and/or calculates and/or estimates the position of a body part while in the air, not touching the interactive surface, and also while sensor input is unavailable.
  • the training system can also provide alerts for predefined events such as a fall or the beginning of a fall.
  • US2011292036 describes a method for processing data including receiving a depth map of a scene containing a body of a humanoid subject.
  • the depth map includes a matrix of pixels, each pixel corresponding to a respective location in the scene and having a respective pixel depth value indicative of a distance from a reference plane to the respective location.
  • the depth map is processed in a digital processor to extract a skeleton of at least a part of the body, the skeleton including multiple joints having respective coordinates.
  • An application program interface (API) indicates at least the coordinates of the joints.
  • API application program interface
  • the present invention seeks to provide a method for determining the position of at least one object relative to an area, the method including, in an electronic processing device:
  • sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object
  • the method includes, in the electronic processing device:
  • the area coordinate space includes a display device space defined relative to a display device and an image space defined relative to an image displayed in the area by the display device, and wherein the method includes, for at least some of the points:
  • the method includes, in the electronic processing device:
  • the method includes, in the electronic processing device:
  • the sensing device includes at least two sensors, and wherein the method includes, for at least some of the points:
  • the method includes, in the electronic processing device: a) at least for points within the region, transforming the first and second sensing device point data into combined sensing device point data indicative of the position of points in a combined sensing device depth space defined relative to both the first and second sensors; and,
  • the sensing device includes at least two sensors, and wherein the method includes, in the electronic processing device:
  • first and second sensing device depth point data into first and second sensing device point data indicative of a position of points in first and second sensing device spaces
  • the method includes, in the electronic processing device determining an object contact point representing a point of contact between an object and the area, using the cluster position.
  • the method includes: a) comparing the contact point to previously determined contact points; and, b) classifying the contact point in accordance with the results of the comparison.
  • the method includes:
  • the method is used in an interactive display system, the display system including a display device for displaying at least one image on the area, and wherein the method includes, in the electronic processing device, determining interaction between a displayed at least one image and an object in accordance with at least one of:
  • the present invention seeks to provide apparatus for determining the position of at least one object relative to an area, the apparatus including an electronic processing device that:
  • sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object
  • d) for points within the region performs clustering of points using the sensing device point data to determine at least one cluster of points representing an object; and, e) for each cluster of points, uses area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the ⁇ position of an object.
  • an interactive display system including:
  • a display device for displaying at least one image on an area
  • sensing device for determining sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object;
  • ii) transforms the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area;
  • iii) determines from the area space point data if the points are positioned within a region defined relative to the area
  • iv for points within the region, performs clustering of points using the sensing device point data to determine at least one cluster of points representing an object
  • v) for each cluster of points uses area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the position of an object;
  • vi) causes the display device to display at least one image in accordance with the determined object position.
  • Figure 1A is a schematic diagram of an example of apparatus for determining the position of an object relative to an area
  • Figure IB is a schematic side view of the arrangement of the display device, sensing device and area of Figure 1A;
  • Figure 1C is a schematic plan view of the arrangement of the display device, sensing device and area of Figure 1A;
  • Figure 2 is a flow chart of an example of a method for determining the position of an object
  • Figure 3 A is a schematic plan view of an example of the capture area of a single sensing device
  • Figure 3B is a schematic plan view of an example of the capture area of two sensing devices
  • Figure 3C is a schematic plan view of an example of first and second sensing device depth spaces
  • Figure 3D is a schematic plan view of an example of a combined sensing device depth space
  • Figure 4 is an example of a process for calibrating apparatus for determining the position of an object.
  • Figures 5 A to 5C are a flow chart of a second example of a process for determining the position of an object.
  • the apparatus 100 includes a processing system 110 coupled to an optional display device 120 and to a sensing device 121.
  • the sensing device 121 is adapted to sense the presence of one or more objects provided in the vicinity of the area 130, with the processing system 110 interpreting signals from the sensing device 121 to determine the position of the one or more objects.
  • the sensing device 121 can be any form of sensing device that is capable of capturing positional information regarding the position of an object, such as a person. Accordingly, the sensing device can include an imaging device, such as a camera or the like, and in one particular example, the sensing device is a system such as the Microsoft KinectTM or ASUS Xtion ProTM system.
  • the processing system 110 is adapted to receive data regarding the position of sensed objects from the sensing device 121, and then interpret this to allow the position of one or more objects relative to the area 130 to be determined. Accordingly, the processing system
  • 1 10 can include any suitable form of electronic processing system that is capable of receiving and interpreting signals from the sensing device 121.
  • the processing system 110 includes a processor 1 1 1, a memory 112, an input/output (I/O) device 113, such as a keyboard and display, and an external interface 114 coupled together via a bus 115.
  • the external interface is used for coupling the processing system 110 to peripheral devices, such as the display device 120 and sensing device 121, as well as to devices, such as communications networks, databases, other storage devices, or the like.
  • peripheral devices such as the display device 120 and sensing device 121
  • devices such as communications networks, databases, other storage devices, or the like.
  • the processor 11 1 executes instructions in the form of applications software stored in the memory 112 to allow signals from the sensing device 121 to be interpreted and optionally used, for example to control the display device 120, as will be described in more detail below. Accordingly, for the purposes of the following description, it will be appreciated that actions performed by the processing system 110 are typically performed by the processor 1 1 1 under control of instructions stored in the memory 112, and this will not therefore be described in further detail below.
  • the computer system 110 may be formed from any suitably programmed processing system, such as a suitably programmed PC, Internet terminal, lap-top, hand-held PC, tablet PC, slate PC, iPadTM, mobile phone, smart phone, PDA (Personal Data Assistant), or other communications device. Accordingly, the processor
  • 1 11 can be any form of electronic processing device such as a microprocessor, microchip processor, logic gate configuration, firmware optionally associated with implementing logic such as an FPGA (Field Programmable Gate Array), or any other electronic device, system or arrangement capable of interacting with the sensing device 121 and optionally the display device 120.
  • a microprocessor microchip processor
  • logic gate configuration firmware optionally associated with implementing logic
  • FPGA Field Programmable Gate Array
  • the apparatus is utilised to provide an interactive display system.
  • the display device 120 can be in the form of a projector, or the like, which is used to project images on to the area 130.
  • the processing system 1 10 determines the position of objects relative to the area 130, allowing the position of the objects to be interpreted as control inputs for controlling the images displayed in the area.
  • this arrangement allows the processing system 110 to determine user interaction with images displayed within the area 130, based on a position of parts of the user, such as their hands and feet.
  • the area 130 is a mat which is provided on a floor or other similar surface. This allows users, such as children, to interact with images generated by the processing system 110 using their hands and feet. Thus, this arrangement can be utilised to allow games to be implemented by the processing system 110, with , a visual representation of the game being displayed in the area 130, allowing users to interact with the game, as will be appreciated by persons skilled in the art.
  • the processing system 110 may implement one or more software modules having respective functions.
  • the processor 111 may execute an object positioning module for determining the position of objects within the area. Information from this module, in the form of an object or contact position, can then be provided to a game module, allowing a game to be executed. It will be appreciated that this arrangement allows a single module to be used for determining the position of an object, with this being able to provide inputs to a wide range of different game modules, thereby allowing a number of different games to be easily implemented on a common platform.
  • the display device 120 and sensing device 121 are typically positioned offset to, and at a level above that of the area 130, as shown in Figures IB and 1C.
  • the display device 120, sensing device 121 and processing system 110 are provided in a common housing 140, which in use is positioned substantially adjacent to the area 130, which is typically defined by a mat or similar.
  • the sensing device 121 is oriented to image along a Z axis and determine the position of an object in an X-Y plane as shown, and as will be described in more detail below.
  • the z-axis is positioned at an angle of approximately 40° relative to a surface of the area, although it will be appreciated that a wide range of arrangements can be used, depending on the preferred implementation.
  • the sensing device 121 is adapted to determine the position of an object in the X-Y plane, and along the Z axis, positional information obtained directly from the sensing device 121 cannot easily be used to determine whether an object is within or in contact with the area. Accordingly, in order to achieve this, the processing system implements a particular procedure for determining the position of an object relative to the area 130, as will now be described with reference to Figure 2.
  • sensing device point data indicative of position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, are determined.
  • This information is typically generated by the sensing device 121, although this may also involve having the processing system 110 perform some preliminary processing of data generated by the sensing device 121, as will be described in more detail below.
  • the processing system 110 transforms the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area 130.
  • this will typically involve converting the data through one or more geometric transformations to provide the point data in a format that defines the position of points relative to the area 130, as opposed to relative to the sensing device 121.
  • it is determined if points are within a defined region using the area space point data.
  • the defined region typically corresponds to the perimeter of the area 130 and may also include a predetermined height above the area, such as 10-40 cm, depending on the preferred implementation.
  • the processing system 110 performs clustering of points in the sensing device coordinate space. This is performed for points determined to be within the region, so that points outside the region can be excluded from the clustering process, thereby reducing the amount of data that requires interpretation, and hence computational requirements. Clustering is performed to identify clusters of points which correspond to objects, and this may be achieved using any suitable clustering algorithm, as will be appreciated by persons skilled in the art.
  • the area space point data is then used to determine a cluster position in the area coordinate space at step 240, with the cluster position being indicative of the position of an object.
  • clusters corresponding to objects have been determined, their position can be determined in the area coordinate space, allowing this information to be used by the processing system 1 10, for example to control interaction between users and the processing system 110.
  • this allows an image can be displayed in the area 130 by the display device 120, with the position of a user's foot or hand being used to identify interaction with a part of the image.
  • This allows the processing system 110 to interpret the interaction and update the displayed image accordingly.
  • this can be used to allow games to be played, for example by displaying a game environment on the area 130, and allowing user interaction with the environment, based on the location of the user's hands and feet.
  • a single sensing device 121 is used.
  • a capture area 300 for which a single sensing device 121 may capture images of an object may not extend over the entirety of the area 130, depending on the particular relative positioning of the sensing device 121 and area 130.
  • first and second sensing device depth spaces Two separate sensing device coordinate spaces, referred to generally as first and second sensing device depth spaces, which can be combined through a suitable transform into a common sensing device depth space, shown generally in Figure 3D, allowing this to take into account objects in the overlap region 303.
  • the sensing devices 121, 122 include Microsoft KinectTM systems utilising the in-built Infra-Red (IR) camera to perform object detection.
  • IR Infra-Red
  • coordinate spaces are typically defined as set out below.
  • the sensing device coordinate space includes: • First and second sensing device depth spaces (dXi,dYi,Depthi dX2,dY ,Depth 2 ): This is the information provided by the Microsoft Source Development Kit (SDK).
  • SDK Microsoft Source Development Kit
  • the dXj and dY x values relate to the position of points in the coordinates of a depth frame, and Depthj is a measure of the distance of the detected points from the respective sensing device.
  • the first and second sensing device depth spaces are shown in Figure 3C.
  • First and second sensing device space coordinates are cartesian coordinates with units of metres.
  • the Z axis points in the direction of the sensing devices 121, 122, the Y axis is above the sensing device, and X is to the left of the respective sensing device when facing the Z-axis, as described in the SDK documentation.
  • the Z axis of each sensing device 121 , 122 is at an angle of approximately 40° relative to a surface of the area 130.
  • Combined sensing device depth space (cdX,cdY): This a 2D space combining the two first and second sensing device depth spaces. Each depth space is rotated and translated so that objects in the region 303 share the same cdX and cdY coordinates.
  • the combined sensing device depth space is shown in Figure 3C.
  • Display device space ( ⁇ , ⁇ , ⁇ ) Display device space coordinates are cartesian coordinates with units of metres. First and second sensing device spaces are transformed such that the pX-pZ plane is parallel to the area 130 onto which images are projected.
  • Image space (X,Y,Height): The Image space coordinates are used for subsequent control of applications implemented by the processing system 1 10.
  • Display device space coordinates, pX, pZ are converted into image space coordinates matching the location of pixels in images displayed by the display device 120.
  • the Height value, pY, is a distance above the area 130.
  • the calibration process may be performed in any appropriate manner. An example of this will now be described in more detail with reference to Figure 4.
  • the relative orientation in depth space of the first and second sensing devices 121, 122 is determined. This can be achieved in any one of a number of ways depending on the preferred implementation. In one example, this can be based on the physical configuration of the sensing devices 121, 122 within the housing 140. Alternatively, this can be performed by imaging a single, or more preferably two, objects, which are positioned in the area of overlap between the two sensing devices, allowing at least a relative orientation of the two sensing devices to be determined.
  • the relative orientations are used by the processing system 110 to create a combined depth space transform, which is a geometrical transform allowing point data to be converted between the separate first and second sensing device spaces and the combined sensing device space.
  • the relative orientation of the display device 120 and sensing devices 121, 122, relative to the area 130 are determined.
  • this information is used by the processing device 110 to define a sensing device to display device space transformation, as well as a display device space to image space transformation, at step 420.
  • the sensing devices are Microsoft KinectTM sensing devices
  • sensing of objects is performed utilising an infrared sensor.
  • the MicrosoftTM inect centre also includes an additional separate visual camera which can be used to image the area 130, with concordance between captured infrared and visible images being performed utilising software applications being provided as part of the Microsoft KinectTM SDK.
  • the position of the display device 120 relative to the area 130 can be achieved using any suitable technique, such as displaying a calibration image on the area 130, and ensuring this is correctly displayed within the area 130.
  • the processing system 1 10 receives signals from the sensing devices 121, 122 indicative of point data in first and second sensing device depth spaces, with these being converted to the first and second sensing device spaces at step 505. It will be appreciated that this can be performed by the processing system 1 10 executing the Kinect SDK and this process will not therefore be described in any further detail.
  • the processing system 110 transforms the first and second sensing device point data in the first and second sensing device spaces into display device point data in display device space.
  • the processing system 1 10 selects a next point in the display device point data, and compares this to a defined region at step 520, which is usually defined in terms of the perimeter of the area 130, and a predetermined height above the area 130. [0074] This is performed to determine if the point is within the region at step 525. If so, at step 530, the processing system 110 converts the position of the point into image space point data, and combined sensing device point data in the image space and combined sensing device spaces, respectively. Otherwise, the point is discounted from further analysis.
  • step 540 to perform clustering to identify clusters of points in the combined sensing device depth space.
  • Clustering can be performed using any suitable technique.
  • the number of cluster groups is unknown and will typically change from one instance to the next.
  • the number of points in a cluster may vary between clusters, and whilst there is a lower bound in the form of a minimum number of points required to define a cluster, there is no upper bound on the number of points in a cluster.
  • many clustering techniques are able to work higher dimensional spaces, the computational cost of calculating the 'distance function' between points will increase dramatically.
  • the clustering process is performed many times a second on modest 'consumer' computer hardware, and hence it is desirable to take the computational complexity of the clustering process into account to avoid the clustering, process being unduly computational expensive.
  • the three dimensional distribution of points within a cluster can be used to help identify the orientation of the object such as a foot or a hand, as well as where the hand or foot can be deemed to be located.
  • the cluster location is identified to correspond to the ball of the foot, which can be identified from the cluster height and direction.
  • the cluster height and direction can be identified using any known method, for example, Principal Component Analysis (PCA), or the like.
  • the separation is compared to a first threshold and if the separation is less than the first threshold, the contact point is defined as an existing contact point at step 580. If above the first threshold, the separation is compared to a second threshold at step 585, and if less than the second threshold but above the first threshold, the contact point is defined as a moved existing contact point at step 590. Otherwise, the contact point is defined as a new contact point at step 595.
  • step 600 it is determined if all clusters are complete and if not, the process returns to step 545 allowing the next cluster to be examined. Otherwise, at step 600 contact point information is transferred to a display application, typically executed by the processing system 1 10, which controls operation of the display device 120, thereby allowing for example, the display application to interpret the contact point as input data.
  • a display application typically executed by the processing system 1 10, which controls operation of the display device 120, thereby allowing for example, the display application to interpret the contact point as input data.
  • the above described apparatus and method allow the position of objects within an area to be determined.
  • this further allows the position of objects relative to displayed images to be determined, thereby allowing the method and apparatus to be used in the control of image display, and hence to allow interaction with displayed images.

Abstract

A method for determining the position of at least one object relative to an area, the method including, determining, from a sensing device, sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, transforming the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space, determining from the area space point data if the points are positioned within a region defined relative to the area, for points within the region, performing clustering of points using the sensing device point data to determine at least one cluster of points representing an object, and, for each cluster of points, using area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the position of an object.

Description

OBJECT POSITION DETERMINATION Background of the Invention
[0001] The present invention relates to a method and apparatus for determining the position of an object relative to an area, and in one example, for determining the position of an object relative to an image displayed in the area.
Description of the Prior Art
[0002] The reference in this specification to any prior publication (or information derived from it), or to any matter which is known, is not, and should not be taken as an acknowledgment or admission or any form of suggestion that the prior publication (or information derived from it) or known matter forms part of the common general knowledge in the field of endeavour to which this specification relates.
[0003] It is known to provide interactive display systems using an interactive surface. For example, US2008191864 describes an interactive training system capable of generating continuous feedback for physical therapy and training applications based on capturing and analyzing the movement of a user on an interactive surface. The training system captures sophisticated input such as the entire areas in contact with the interactive surface, center of gravity, pressure distribution, velocity, acceleration, direction, orientation etc. The training system also captures and/or calculates and/or estimates the position of a body part while in the air, not touching the interactive surface, and also while sensor input is unavailable. The training system can also provide alerts for predefined events such as a fall or the beginning of a fall.
[0004] However, such arrangements rely on the use of an array of sensors embedded within the interactive surface, rendering the surface expensive to manufacture. Furthermore, as sensitive components are incorporated into the surface on which the user interacts, this can often result in sensor damage, with the embedded sensors being complex to replace. This therefore limits the usefulness of the arrangement. [0005] Camera based motion sensing systems are also known. For example, US2011292036 describes a method for processing data including receiving a depth map of a scene containing a body of a humanoid subject. The depth map includes a matrix of pixels, each pixel corresponding to a respective location in the scene and having a respective pixel depth value indicative of a distance from a reference plane to the respective location. The depth map is processed in a digital processor to extract a skeleton of at least a part of the body, the skeleton including multiple joints having respective coordinates. An application program interface (API) indicates at least the coordinates of the joints.
[0006] However, such arrangements are typically only capable of capturing motion of a subject, and not determining the absolute position of the subject relative to an external reference, such as a interactive display.
Summary of the Present Invention
[0007] In a first broad form the present invention seeks to provide a method for determining the position of at least one object relative to an area, the method including, in an electronic processing device:
a) determining, from a sensing device, sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object;
b) transforming the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area;
c) determining from the area space point data if the points are positioned within a region defined relative to the area;
d) for points within the region, performing clustering of points using the sensing device point data to determine at least one cluster of points representing an object; and,
e) for each cluster of points, using area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the position of an object. [0008] Typically the method includes, in the electronic processing device:
a) for each cluster, using the area space point data to determine a cluster height and cluster direction relative to the area; and,
b) determining the cluster position using the cluster height and cluster direction.
[0009] Typically the area coordinate space includes a display device space defined relative to a display device and an image space defined relative to an image displayed in the area by the display device, and wherein the method includes, for at least some of the points:
a) determining display device point data indicative of a position of the plurality of points in display device space; and,
b) determining image space point data representing the position of the points in the image space.
[0010] Typically the method includes, in the electronic processing device:
a) transforming the sensing device point data into display device point data; and, b) determining from the display device point data if the points are within the region.
[0011] Typically the method includes, in the electronic processing device:
a) for points within the region, transforming the display device point data into image space point data; and,
b) for each cluster of points, using image space point data to determine the cluster position.
[0012] Typically the sensing device includes at least two sensors, and wherein the method includes, for at least some of the points:
a) determining, from the first and second sensors, respective first and second sensing device point data indicative of the position of points in first and second sensing device spaces defined relative to the first and second sensors of the sensing device, respectively; and,
b) transforming the first and second sensing device point data into area space point data to determine if the points are positioned within the region.
[0013] Typically the method includes, in the electronic processing device: a) at least for points within the region, transforming the first and second sensing device point data into combined sensing device point data indicative of the position of points in a combined sensing device depth space defined relative to both the first and second sensors; and,
b) performing clustering of points using the combined sensing device point data.
[0014] Typically the sensing device includes at least two sensors, and wherein the method includes, in the electronic processing device:
a) determining first and second sensing device depth point data indicative of a position of points in first and second sensing device depth spaces;
b) converting first and second sensing device depth point data into first and second sensing device point data indicative of a position of points in first and second sensing device spaces;
c) converting first and second sensing device point data into display device point data indicative of a position of points in display device space;
d) determining from the display device point data if the points are positioned within a region defined relative to the area;
e) for points within the region:
i) converting first and second sensing device depth point data into combined sensing device depth point data indicative of a position of points in a combined sensing device space;
ii) converting display device point data into image space point data indicative of a position of points in an image space;
iii) performing clustering of points using the combined sensing device depth point data; and, 1
iv) for each cluster of points, determining a cluster position using the image space point data.
[0015] Typically the method includes, in the electronic processing device determining an object contact point representing a point of contact between an object and the area, using the cluster position.
[0016] Typically the method includes: a) comparing the contact point to previously determined contact points; and, b) classifying the contact point in accordance with the results of the comparison.
[0017] Typically the method includes:
a) determining a separation between the contact point and a previously determined contact point;
b) comparing the separation to a threshold; and,
c) classifying the contact point as at least one of:
i) a previous contact point;
ii) a moved existing contact point; and,
iii) a new contact point.
[0018] Typically the method is used in an interactive display system, the display system including a display device for displaying at least one image on the area, and wherein the method includes, in the electronic processing device, determining interaction between a displayed at least one image and an object in accordance with at least one of:
a) an object position; and,
b) an object contact point.
[0019] In a second broad form the present invention seeks to provide apparatus for determining the position of at least one object relative to an area, the apparatus including an electronic processing device that:
a) determines, from a sensing device, sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object;
b) transforms the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area;
c) determines from the area space point data if the points are positioned within a region defined relative to the area;
d) for points within the region, performs clustering of points using the sensing device point data to determine at least one cluster of points representing an object; and, e) for each cluster of points, uses area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the ■ position of an object.
[0020] In a third broad form the present invention seeks to provide an interactive display system including:
a) a display device for displaying at least one image on an area;
b) a sensing device for determining sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object; and,
c) an electronic processing device that:
i) receives sensing device point data from the sensing device;
ii) transforms the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area;
iii) determines from the area space point data if the points are positioned within a region defined relative to the area;
iv) for points within the region, performs clustering of points using the sensing device point data to determine at least one cluster of points representing an object;
v) for each cluster of points, uses area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the position of an object; and,
vi) causes the display device to display at least one image in accordance with the determined object position.
Brief Description of the Drawings
[0021] An example of the present invention will now be described with reference to the accompanying drawings, in which: -
[0022] Figure 1A is a schematic diagram of an example of apparatus for determining the position of an object relative to an area; [0023] Figure IB is a schematic side view of the arrangement of the display device, sensing device and area of Figure 1A;
[0024] Figure 1C is a schematic plan view of the arrangement of the display device, sensing device and area of Figure 1A;
[0025] Figure 2 is a flow chart of an example of a method for determining the position of an object;
[0026] Figure 3 A is a schematic plan view of an example of the capture area of a single sensing device;
[0027] Figure 3B is a schematic plan view of an example of the capture area of two sensing devices;
[0028] Figure 3C is a schematic plan view of an example of first and second sensing device depth spaces;
[0029] Figure 3D is a schematic plan view of an example of a combined sensing device depth space;
[0030] Figure 4 is an example of a process for calibrating apparatus for determining the position of an object; and,
[0031) Figures 5 A to 5C are a flow chart of a second example of a process for determining the position of an object.
Detailed Description of the Preferred Embodiments
[0032] An example of apparatus for determining the position of an object will now be described with reference to Figures 1 A to 1C.
[0033] In this example, the apparatus 100 includes a processing system 110 coupled to an optional display device 120 and to a sensing device 121. In use, the sensing device 121 is adapted to sense the presence of one or more objects provided in the vicinity of the area 130, with the processing system 110 interpreting signals from the sensing device 121 to determine the position of the one or more objects.
[0034] The sensing device 121 can be any form of sensing device that is capable of capturing positional information regarding the position of an object, such as a person. Accordingly, the sensing device can include an imaging device, such as a camera or the like, and in one particular example, the sensing device is a system such as the Microsoft Kinect™ or ASUS Xtion Pro™ system.
[0035] The processing system 110 is adapted to receive data regarding the position of sensed objects from the sensing device 121, and then interpret this to allow the position of one or more objects relative to the area 130 to be determined. Accordingly, the processing system
1 10 can include any suitable form of electronic processing system that is capable of receiving and interpreting signals from the sensing device 121.
[0036] In one example, the processing system 110 includes a processor 1 1 1, a memory 112, an input/output (I/O) device 113, such as a keyboard and display, and an external interface 114 coupled together via a bus 115. The external interface is used for coupling the processing system 110 to peripheral devices, such as the display device 120 and sensing device 121, as well as to devices, such as communications networks, databases, other storage devices, or the like. Although a single external interface is shown, this is for the purpose of example only, and in practice multiple interfaces using various methods (e.g. Ethernet, serial, USB, wireless, mobile networks or the like) may be provided. It will also be appreciated that additional hardware components, may be incorporated into the computer system 110, depending on the particular implementation.
[0037] In use, the processor 11 1 executes instructions in the form of applications software stored in the memory 112 to allow signals from the sensing device 121 to be interpreted and optionally used, for example to control the display device 120, as will be described in more detail below. Accordingly, for the purposes of the following description, it will be appreciated that actions performed by the processing system 110 are typically performed by the processor 1 1 1 under control of instructions stored in the memory 112, and this will not therefore be described in further detail below.
[0038] Accordingly, it will be appreciated that the computer system 110 may be formed from any suitably programmed processing system, such as a suitably programmed PC, Internet terminal, lap-top, hand-held PC, tablet PC, slate PC, iPad™, mobile phone, smart phone, PDA (Personal Data Assistant), or other communications device. Accordingly, the processor
1 11 can be any form of electronic processing device such as a microprocessor, microchip processor, logic gate configuration, firmware optionally associated with implementing logic such as an FPGA (Field Programmable Gate Array), or any other electronic device, system or arrangement capable of interacting with the sensing device 121 and optionally the display device 120.
[0039] In one example, the apparatus is utilised to provide an interactive display system. In this scenario, the display device 120 can be in the form of a projector, or the like, which is used to project images on to the area 130. The processing system 1 10 determines the position of objects relative to the area 130, allowing the position of the objects to be interpreted as control inputs for controlling the images displayed in the area. Thus, this arrangement allows the processing system 110 to determine user interaction with images displayed within the area 130, based on a position of parts of the user, such as their hands and feet.
[0040] Whilst this can have a number of uses, in one particular example, the area 130 is a mat which is provided on a floor or other similar surface. This allows users, such as children, to interact with images generated by the processing system 110 using their hands and feet. Thus, this arrangement can be utilised to allow games to be implemented by the processing system 110, with , a visual representation of the game being displayed in the area 130, allowing users to interact with the game, as will be appreciated by persons skilled in the art.
[0041] In the above described example, the processing system 110 may implement one or more software modules having respective functions. For example, the processor 111 may execute an object positioning module for determining the position of objects within the area. Information from this module, in the form of an object or contact position, can then be provided to a game module, allowing a game to be executed. It will be appreciated that this arrangement allows a single module to be used for determining the position of an object, with this being able to provide inputs to a wide range of different game modules, thereby allowing a number of different games to be easily implemented on a common platform.
[0042] It will also be appreciated however, that the determination of an object position within an area 130 can be utilised for other applications and this will not therefore be described in further detail. [0043] In any event, in order to allow display of images, and sensing of objects in the area 130, the display device 120 and sensing device 121 are typically positioned offset to, and at a level above that of the area 130, as shown in Figures IB and 1C.
[0044] In this example, the display device 120, sensing device 121 and processing system 110 are provided in a common housing 140, which in use is positioned substantially adjacent to the area 130, which is typically defined by a mat or similar. In this example, the sensing device 121 is oriented to image along a Z axis and determine the position of an object in an X-Y plane as shown, and as will be described in more detail below.
[0045] In one particular example, to allow the use of the common housing 140, and to allow the housing to be positioned near the area 130, the z-axis is positioned at an angle of approximately 40° relative to a surface of the area, although it will be appreciated that a wide range of arrangements can be used, depending on the preferred implementation. In any event, as the sensing device 121 is adapted to determine the position of an object in the X-Y plane, and along the Z axis, positional information obtained directly from the sensing device 121 cannot easily be used to determine whether an object is within or in contact with the area. Accordingly, in order to achieve this, the processing system implements a particular procedure for determining the position of an object relative to the area 130, as will now be described with reference to Figure 2.
[0046] In this example, at step 200, sensing, device point data indicative of position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, are determined. This information is typically generated by the sensing device 121, although this may also involve having the processing system 110 perform some preliminary processing of data generated by the sensing device 121, as will be described in more detail below.
[0047] At step 210, the processing system 110 transforms the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area 130. Thus, this will typically involve converting the data through one or more geometric transformations to provide the point data in a format that defines the position of points relative to the area 130, as opposed to relative to the sensing device 121. [0048] At step 220, it is determined if points are within a defined region using the area space point data. The defined region typically corresponds to the perimeter of the area 130 and may also include a predetermined height above the area, such as 10-40 cm, depending on the preferred implementation.
[0049] At step 230, the processing system 110 performs clustering of points in the sensing device coordinate space. This is performed for points determined to be within the region, so that points outside the region can be excluded from the clustering process, thereby reducing the amount of data that requires interpretation, and hence computational requirements. Clustering is performed to identify clusters of points which correspond to objects, and this may be achieved using any suitable clustering algorithm, as will be appreciated by persons skilled in the art.
[0050] Once clusters have been determined, the area space point data is then used to determine a cluster position in the area coordinate space at step 240, with the cluster position being indicative of the position of an object.
[0051] Accordingly, the above-described process operates to determine the position of objects in the area, including up to a predetermined height above the area. This is achieved by examining whether points are within a region using an area coordinate space, in which the position of points are defined relative to an area. Clustering of points within the region is then performed on point data in a sensing device coordinate space, defined relative to the sensing devices. However, this is not essential, and clustering of points could also be performed in area coordinate space. Similarly, clustering is typically performed in two dimensions for computational efficiency, however clustering could also be performed in higher dimensions, for example using all three dimensions of point data in sensing device coordinate space.
[0052] Once clusters corresponding to objects have been determined, their position can be determined in the area coordinate space, allowing this information to be used by the processing system 1 10, for example to control interaction between users and the processing system 110. [0053] In one particular example, this allows an image can be displayed in the area 130 by the display device 120, with the position of a user's foot or hand being used to identify interaction with a part of the image. This allows the processing system 110 to interpret the interaction and update the displayed image accordingly. In one particular example, this can be used to allow games to be played, for example by displaying a game environment on the area 130, and allowing user interaction with the environment, based on the location of the user's hands and feet.
[0054] In the above-described example, a single sensing device 121 is used. However, as shown in Figure 3 A, a capture area 300 for which a single sensing device 121 may capture images of an object, may not extend over the entirety of the area 130, depending on the particular relative positioning of the sensing device 121 and area 130.
[0055] Accordingly, in an alternative example, the apparatus 100 includes two sensing devices 121, 122, which are positioned relative to each other to allow sensing to be performed over the entire area 130, as shown by capture areas 301, 302 in Figure 3B. In this example, an overlap region 303 occurs, with objects in this region being detected by both sensing devices 121, 122. s
[0056] The use of two sensing devices 121, 122 also results in two separate sensing device coordinate spaces, referred to generally as first and second sensing device depth spaces, which can be combined through a suitable transform into a common sensing device depth space, shown generally in Figure 3D, allowing this to take into account objects in the overlap region 303.
[0057] A specific example of the operation of apparatus 100 including two sensing devices 121, 122, will now be described in more detail with reference to Figures 4 and 5 A to 5C. For the purpose of this example, the sensing devices 121, 122 include Microsoft Kinect™ systems utilising the in-built Infra-Red (IR) camera to perform object detection. For the purpose of this example, coordinate spaces are typically defined as set out below.
[0058] In particular, the sensing device coordinate space includes: • First and second sensing device depth spaces (dXi,dYi,Depthi dX2,dY ,Depth2): This is the information provided by the Microsoft Source Development Kit (SDK). The dXj and dYx values relate to the position of points in the coordinates of a depth frame, and Depthj is a measure of the distance of the detected points from the respective sensing device. The first and second sensing device depth spaces are shown in Figure 3C.
• First and second sensing device spaces (kX/,kYi,kZ , kX kY kZ?) First and second sensing device space coordinates are cartesian coordinates with units of metres. The Z axis points in the direction of the sensing devices 121, 122, the Y axis is above the sensing device, and X is to the left of the respective sensing device when facing the Z-axis, as described in the SDK documentation. As the sensing devices are directed towards the area 130, the Z axis of each sensing device 121 , 122 is at an angle of approximately 40° relative to a surface of the area 130.
• Combined sensing device depth space (cdX,cdY): This a 2D space combining the two first and second sensing device depth spaces. Each depth space is rotated and translated so that objects in the region 303 share the same cdX and cdY coordinates. The combined sensing device depth space is shown in Figure 3C.
The area coordinate space includes:
• Display device space (ρΧ,ρΥ,ρΖ): Display device space coordinates are cartesian coordinates with units of metres. First and second sensing device spaces are transformed such that the pX-pZ plane is parallel to the area 130 onto which images are projected.
• Image space (X,Y,Height): The Image space coordinates are used for subsequent control of applications implemented by the processing system 1 10. Display device space coordinates, pX, pZ are converted into image space coordinates matching the location of pixels in images displayed by the display device 120. The Height value, pY, is a distance above the area 130. [0060] In order to determine appropriate transformations between the different spaces, it is typical to undergo a calibration process in which the relative orientations of the display device 120, sensing devices 121, 122 and the area 130 are determined.
[0061] The calibration process may be performed in any appropriate manner. An example of this will now be described in more detail with reference to Figure 4.
[0062] In this example at step 400, the relative orientation in depth space of the first and second sensing devices 121, 122 is determined. This can be achieved in any one of a number of ways depending on the preferred implementation. In one example, this can be based on the physical configuration of the sensing devices 121, 122 within the housing 140. Alternatively, this can be performed by imaging a single, or more preferably two, objects, which are positioned in the area of overlap between the two sensing devices, allowing at least a relative orientation of the two sensing devices to be determined.
[0063] Alternatively, calibration can be performed utilising a manual process in which images captured by the sensing devices 121, 122 are displayed to an operator with these then being manipulated by the user to define common points and the relative position relative to the area.
[0064] At step 405, the relative orientations are used by the processing system 110 to create a combined depth space transform, which is a geometrical transform allowing point data to be converted between the separate first and second sensing device spaces and the combined sensing device space.
[0065] At step 410, the relative orientation of the display device 120 and sensing devices 121, 122, relative to the area 130, are determined. At step 415, this information is used by the processing device 110 to define a sensing device to display device space transformation, as well as a display device space to image space transformation, at step 420.
[0066] In one particular example, in which the sensing devices are Microsoft Kinect™ sensing devices, sensing of objects is performed Utilising an infrared sensor. In this example, it is not possible to detect the area 130 using the infrared sensor unless a temperature differential between the different parts of the area is induced, for example, through the use of heated or cooled objects. However, the Microsoft™ inect centre also includes an additional separate visual camera which can be used to image the area 130, with concordance between captured infrared and visible images being performed utilising software applications being provided as part of the Microsoft Kinect™ SDK.
[0067] The position of the display device 120 relative to the area 130 can be achieved using any suitable technique, such as displaying a calibration image on the area 130, and ensuring this is correctly displayed within the area 130.
[0068] It will be appreciated by persons skilled in the art that any suitable technique for determining the transformations may be used and this will not therefore be described in further detail.
[0069] An example process for determining the position of an object will now be described in more detail with reference to Figures 5A to 5C.
[0070] For the purpose of this example, it will be assumed that transformation of point data between the different spaces is performed using geometrical transformations derived during the calibration process, and this will not therefore be described in any further detail.
[0071] In this example at step 500, the processing system 1 10 receives signals from the sensing devices 121, 122 indicative of point data in first and second sensing device depth spaces, with these being converted to the first and second sensing device spaces at step 505. It will be appreciated that this can be performed by the processing system 1 10 executing the Kinect SDK and this process will not therefore be described in any further detail.
[0072] At step 510, the processing system 110 transforms the first and second sensing device point data in the first and second sensing device spaces into display device point data in display device space.
[0073] At step 515, the processing system 1 10 selects a next point in the display device point data, and compares this to a defined region at step 520, which is usually defined in terms of the perimeter of the area 130, and a predetermined height above the area 130. [0074] This is performed to determine if the point is within the region at step 525. If so, at step 530, the processing system 110 converts the position of the point into image space point data, and combined sensing device point data in the image space and combined sensing device spaces, respectively. Otherwise, the point is discounted from further analysis.
[0075] At step 535, the processing system 110 determines if all points for both the first and second system point data have been considered, and if not the process returns to step 510 to select a next point.
[0076] Otherwise, once all points are complete, the process moves on to step 540 to perform clustering to identify clusters of points in the combined sensing device depth space.
[0077J Clustering can be performed using any suitable technique. In this regard, in determining the clustering technique to be used, it should be considered that the number of cluster groups is unknown and will typically change from one instance to the next. The number of points in a cluster may vary between clusters, and whilst there is a lower bound in the form of a minimum number of points required to define a cluster, there is no upper bound on the number of points in a cluster. Furthermore, whilst many clustering techniques are able to work higher dimensional spaces, the computational cost of calculating the 'distance function' between points will increase dramatically.
[0078] In one example, the clustering process is performed many times a second on modest 'consumer' computer hardware, and hence it is desirable to take the computational complexity of the clustering process into account to avoid the clustering, process being unduly computational expensive.
[0079] In one example, clustering is performed using a density-based spatial clustering algorithm, such as DBSCAN, as described for example in "A density-based algorithm for discovering clusters in large spatial databases with noise" by Martin Ester, Hans-Peter Kriegel, Jorg Sander, Xiaowei Xu (1996). In this example, performing the clustering in combined sensing device depth space has the advantage that cdX and cdY axe integer coordinates, and therefore it is computationally more efficient than clustering in display device space or image space. [0080] By performing clustering in combined sensing device depth space, different points which have been identified by different sensing devices 121, 122 for the same object will fall within the same cluster and this avoids the need to try and identify and eliminate duplicate points that have been captured by the two sensing devices 121, 122, for a single object.
[0081] At step 545, the processing system 110 selects a next cluster before going on to calculate a cluster height and direction at step 550, using the image space point data. At step 555, the processing system 1 10 then determines a cluster location in image space representing a contact point, based on the cluster height and direction.
[0082] In particular, the three dimensional distribution of points within a cluster can be used to help identify the orientation of the object such as a foot or a hand, as well as where the hand or foot can be deemed to be located. For example, for a foot the cluster location is identified to correspond to the ball of the foot, which can be identified from the cluster height and direction. Furthermore, in the event that an object cluster height in image space is equal to 0, then the object is deemed to be in contact with the area 130. It will be appreciated that the cluster height and direction can be identified using any known method, for example, Principal Component Analysis (PCA), or the like.
[0083] At this point, the object and an associated contact point have been located and accordingly, this information can then be utilised by the processing system 110, for example for use in controlling a game.
[0084] An example of this will now be described. In particular, at step 560, the processing system 110 compares the current contact point to previously stored contact points corresponding to the contact point of objects identified during a previous iteration of the algorithm and hence at a previous time point. At step 565 the processing system 110 determines the separation from a nearest previously identified cluster before comparing the separation to thresholds in order to classify the contact point at step 570.
[0085] This can be done in a number of manners but in one example, at step 575 the separation is compared to a first threshold and if the separation is less than the first threshold, the contact point is defined as an existing contact point at step 580. If above the first threshold, the separation is compared to a second threshold at step 585, and if less than the second threshold but above the first threshold, the contact point is defined as a moved existing contact point at step 590. Otherwise, the contact point is defined as a new contact point at step 595.
[0086] At step 600 it is determined if all clusters are complete and if not, the process returns to step 545 allowing the next cluster to be examined. Otherwise, at step 600 contact point information is transferred to a display application, typically executed by the processing system 1 10, which controls operation of the display device 120, thereby allowing for example, the display application to interpret the contact point as input data.
[0087] Accordingly, the above described apparatus and method allow the position of objects within an area to be determined. In one particular example, this further allows the position of objects relative to displayed images to be determined, thereby allowing the method and apparatus to be used in the control of image display, and hence to allow interaction with displayed images.
[0088] Throughout this specification and claims which follow, unless the context requires otherwise, the word "comprise", and variations such as "comprises" or "comprising", will be understood to imply the inclusion of a stated integer or group of integers or steps but not the exclusion of any other integer or group of integers.
[0089] Persons skilled in the art will appreciate that numerous variations and modifications will become apparent. All such variations and modifications which become apparent to persons skilled in the art, should be considered to fall within the spirit and scope that the invention broadly appearing before described.

Claims

THE CLAIMS DEFINING THE INVENTION ARE AS FOLLOWS :
1) A method for determining the position of at least one object relative to an area, the method including, in an electronic processing device:
a) determining, from a sensing device, sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object; b) transforming the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area;
c) determining from the area space point data if the points are positioned within a region defined relative to the area;
d) for points within the region, performing clustering of points using the sensing device point data to determine at least one cluster of points representing an object; and, e) for each cluster of points, using area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the position of an object.
2) A method according to claim 1, wherein the method includes, in the electronic processing device:
a) for each cluster, using the area space point data to determine a cluster height and cluster direction relative to the area; and,
b) determining the cluster position using the cluster height and cluster direction.
3) A method according to claim 1 or claim 2, wherein the area coordinate space includes a display device space defined relative to a display device and an image space defined relative to an image displayed in the area by the display device, and wherein the method includes, for at least some of the points:
a) determining display device point data indicative of a position of the plurality of points in display device space; and,
b) determining image space point data representing the position of the points in the image space.
4) A method according to claim 3, wherein the method includes, in the electronic processing device: a) transforming the sensing device point data into display device point data; and, b) determining from the display device point data if the points are within the region.
5) A method according to claim 4, wherein the method includes, in the electronic processing device:
a) for points within the region, transforming the display device point data into image space point data; and,
b) for each cluster of points, using image space point data to determine the cluster position.
6) A method according to any one of the claims 1 to 5, wherein the sensing device includes at least two sensors, and wherein the method includes, for at least some of the points: a) determining, from the first and second sensors, respective first and second sensing device point data indicative of the position of points in first and second sensing device spaces defined relative to the first and second sensors of the sensing device, respectively; and,
b) transforming the first and second sensing device point data into area space point data to determine if the points are positioned within the region.
7) A method according to claim 6, wherein the method includes, in the electronic processing device:
a) at least for points within the region, transforming the first and second sensing device point data into combined sensing device point data indicative of the position of points in a combined sensing device depth space defined relative to both the first and second sensors; and,
b) performing clustering of points using the combined sensing device point data.
8) A method according to any one of the claims 1 to 7, wherein the sensing device includes at least two sensors, and wherein the method includes, in the electronic processing device: a) determining first and second sensing device depth point data indicative of a position of points in first and second sensing device depth spaces;
b) converting first and second sensing device depth point data into first and second sensing device point data indicative of a position of points in first and second sensing device spaces; c) converting first and second sensing device point data into display device point data indicative of a position of points in display device space;
d) determining from the display device point data if the points are positioned within a region defined relative to the area;
e) for points within the region:
i) converting first and second sensing device depth point data into combined sensing device depth point data indicative of a position of points in a combined sensing device space;
ii) converting display device point data into image space point data indicative of a position of points in an image space;
iii) performing clustering of points using the combined sensing device depth point data; and,
iv) for each cluster of points, determining a cluster position using the image space point data.
9) A method according to any one of the claims 1 to 8, wherein the method includes, in the electronic processing device determining an object contact point representing a point of contact between an object and the area, using the cluster position.
10) A method according to claim 9, wherein the method includes:
a) comparing the contact point to previously determined contact points; and,
b) classifying the contact point in accordance with the results of the comparison.
11) A method according to claim 10, wherein the method includes:
a) determining a separation between the contact point and a previously determined contact point;
b) comparing the separation to a threshold; and,
c) classifying the contact point as at least one of:
i) a previous contact point;
ii) a moved existing contact point; and,
iii) a new contact point.
12) A method according to any one of the claims 1 to 12, wherein the method is used in an interactive display system, the display system including a display device for displaying at least one image on the area, and wherein the method includes, in the electronic processing device, determining interaction between a displayed at least one image and an object in accordance with at least one of:
a) an object position; and,
b) an object contact point.
13) Apparatus for determining the position of at least one object relative to an area, the apparatus including an electronic processing device that:
a) determines, from a sensing device, sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object; b) transforms the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area;
c) determines from the area space point data if the points are positioned within a region defined relative to the area;
d) for points within the region, performs clustering of points using the sensing device point data to determine at least one cluster of points representing an object; and, e) for each cluster of points, uses area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the position of an object.
14) An interactive display system including:
a) a display device for displaying at least one image on an area;
b) a sensing device for determining sensing device point data indicative of a position of a plurality of points in a sensing device coordinate space defined relative to the sensing device, the points corresponding to surface points of the at least one object; and,
c) an electronic processing device that:
i) receives sensing device point data from the sensing device;
ii) transforms the sensing device point data into area space point data indicative of a position of the plurality of points in an area coordinate space defined relative to the area; iii) determines from the area space point data if the points are positioned within a region defined relative to the area;
iv) for points within the region, performs clustering of points using the sensing device point data to determine at least one cluster of points representing an object;
v) Tor each cluster of points, uses area space point data to determine a cluster position in area coordinate space, the cluster position being indicative of the position of an object; and,
vi) causes the display device to display at least one image in accordance with the determined object position.
r
PCT/AU2013/000411 2012-05-22 2013-04-19 Object position determination WO2013173861A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261650032P 2012-05-22 2012-05-22
US61/650,032 2012-05-22

Publications (1)

Publication Number Publication Date
WO2013173861A1 true WO2013173861A1 (en) 2013-11-28

Family

ID=49622928

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2013/000411 WO2013173861A1 (en) 2012-05-22 2013-04-19 Object position determination

Country Status (1)

Country Link
WO (1) WO2013173861A1 (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040045339A1 (en) * 2002-09-05 2004-03-11 Sanjay Nichani Stereo door sensor
US20100199228A1 (en) * 2009-01-30 2010-08-05 Microsoft Corporation Gesture Keyboarding

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040045339A1 (en) * 2002-09-05 2004-03-11 Sanjay Nichani Stereo door sensor
US20100199228A1 (en) * 2009-01-30 2010-08-05 Microsoft Corporation Gesture Keyboarding

Similar Documents

Publication Publication Date Title
US10762386B2 (en) Method of determining a similarity transformation between first and second coordinates of 3D features
US10518414B1 (en) Navigation method, navigation system, movement control system and mobile robot
US20200097091A1 (en) Method and Apparatus of Interactive Display Based on Gesture Recognition
Sanna et al. A Kinect-based natural interface for quadrotor control
KR101184170B1 (en) Volume recognition method and system
US20140139429A1 (en) System and method for computer vision based hand gesture identification
EP3455686A1 (en) Systems and methods for initializing a robot to autonomously travel a trained route
Frank et al. Toward mobile mixed-reality interaction with multi-robot systems
Caputo et al. 3D hand gesture recognition based on sensor fusion of commodity hardware
US9008442B2 (en) Information processing apparatus, information processing method, and computer program
JP6054831B2 (en) Image processing apparatus, image processing method, and image processing program
CN102708355A (en) Information processing device, authoring method, and program
CN104081307A (en) Image processing apparatus, image processing method, and program
Tara et al. Hand segmentation from depth image using anthropometric approach in natural interface development
JP7162079B2 (en) A recording medium for recording a method, system and computer program for remotely controlling a display device via head gestures
CN106125994B (en) Coordinate matching method and the control method and terminal for using the coordinate matching method
CN105892633A (en) Gesture identification method and virtual reality display output device
CN112657176A (en) Binocular projection man-machine interaction method combined with portrait behavior information
Yin et al. Estimation of the fundamental matrix from uncalibrated stereo hand images for 3D hand gesture recognition
KR102063408B1 (en) Method and apparatus for interaction with virtual objects
JP2009216480A (en) Three-dimensional position and attitude measuring method and system
Kapgate et al. Human following robot using kinect in embedded platform
Heindl et al. Metric pose estimation for human-machine interaction using monocular vision
WO2013173861A1 (en) Object position determination
Boyali et al. 3D and 6 DOF user input platform for computer vision applications and virtual reality

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 13793726

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 13793726

Country of ref document: EP

Kind code of ref document: A1