Classifications

• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06F—ELECTRIC DIGITAL DATA PROCESSING
  • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
  • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
  • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
  • G06F3/0304—Detection arrangements using opto-electronic means
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
  • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
• • G—PHYSICS
  • G06—COMPUTING; CALCULATING OR COUNTING
  • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
  • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
  • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
  • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition

Definitions

• the present invention generally relates to user interfaces and more particularly to methods and systems of 3D Human-Machine-Interface.
• the job would generate a printout, containing final results or (all too often) an abort notice with an attached error log. Successful runs might also write a result on magnetic tape or generate some data cards to be used in later computation.
• the turnaround time for a single job often spanned entire days. If one were very lucky, it might be hours; real-time response was unheard of. But there were worse fates than the card queue; some computers actually required an even more tedious and error-prone process of toggling in programs in binary code using console switches. The very earliest machines actually had to be partly rewired to incorporated program logic into themselves, using devices known as plugboards.
• Command-line interfaces were closely associated with the rise of timesharing computers.
• the concept of timesharing dates back to the 1950s; the most influential early experiment was the MULTICS operating system after 1965; and by far the most influential of present-day command-line interfaces is that of Unix itself, which dates from 1969 and has exerted a shaping influence on most of what came after it.
• VDTs video-display terminals
• the PDP-1 console display had been descended from the radar display tubes of World War II, twenty years earlier, reflecting the fact that some key pioneers of minicomputing at MIT's Lincoln Labs were former radar technicians. Across the continent in that same year of 1962, another former radar technician was beginning to blaze a different trail at Stanford Research Institute. His name was Doug Engelbart. He had been inspired by both his personal experiences with these very early graphical displays and by Vannevar Bush's seminal essay As We May Think, which had presented in 1945 a vision of what we would today call hypertext.
• VR can confuse the human proprioceptive system; VR motion at even moderate speeds can induce dizziness and nausea as the brain tries to reconcile the visual simulation of motion with the inner ear's report of the body's real-world motions.
• Jef Raskin's THE project (The Humane Environment) is exploring the zoom world model of GUIs, described in that spatializes them without going 3D.
• THI the screen becomes a window on a 2-D virtual world where data and programs are organized by spatial locality.
• Objects in the world can be presented at several levels of detail depending on one's height above the reference plane, and the most besic selection operation is to zoom in and land on them.
• 3D human machine interface may include (1) an image acquisition assembly, (2) an initializing module, (3) an image segmentation module, (4) a segmented data processing module, (5) a scoring module, (6) a projection module, (7) a fitting module,(8) a scoring and error detection module, (9) a recovery module, (10) a three dimensional correlation module, (11) a three dimensional skeleton prediction module, and (12) an output module.
• 3D HMI 3D human machine interface
• 3D HMI may include (1) an image acquisition assembly, (2) an initializing module, (3) an image segmentation module, (4) a segmented data processing module, (5) a scoring module, (6) a projection module, (7) a fitting module,(8) a scoring and error detection module, (9) a recovery module, (10) a three dimensional correlation module, (11) a three dimensional skeleton prediction module, and (12) an output module.
• the image acquisition assembly may be adapted to acquire a set of images, wherein substantially each image is associated with a different point in time.
• the images may be of a single user or multiple users.
• the initialization module may be adapted to detect and define the user's (1) colors, (2) organ's parameters, surrounding, and other parameters which are associated with the user.
• the user may be any person and/or animal and/or moving object which enters the frame.
• the image segmentation module may be adapted to extract segmented data from the image.
• the segmented data may also comprise: • Color
• the segmented data processing module may be adapted to process the segmented data.
• the segmented data may be processed in the following way:
• Color - using known color parameters to detect elements and/or light changes for example, use skin color to detect palms and face.
• Edge detection detect the edges of the image.
• the segmented data processing module may be adapted to detect deviation in the distance of an organ from the image acquisition assembly, in accordance with the deviation of the organs relative size.
• the scoring module may be adapted to (1) examine the processed segmented data, (2) estimate the quality of the processed segmented data, and according to the quality (3) decide which portions of the segmented data are reliable enough to be used by the HMI system.
• the three dimensional skeleton prediction module may be adapted to predict the position of the three dimensional skeleton which will have the best match or correlation with the processed image.
• the three dimensional prediction module may use constraints which derive from the type of skeleton used, for example, if the skeleton is of a human figure, the head of the skeleton can't rotate 360 degrees.
• the three dimensional prediction module may also use a set of dynamic and motion process to predict the position of the three dimensional skeleton.
• the projection module may be adapted to project the skeleton onto the image.
• the projection may be applied in the two-dimensional plane.
• the fitting module may be adapted to fit segmented data to the projected skeleton.
• the fitting module may be adapted to associate portions of the segmented data with portions of the projected skeleton.
• the scoring and error detection module may be adapted (1) to examine the processed skeleton after it was associated with segmented data, (2) to evaluate the fitting quality of said skeleton and (3) determine whether an error has occurred during the skeleton prediction process or the association of segmented data.
• the recovery module may be adapted to recover from a detected error.
• the recovery may be a process of multiple processing layers, re segmenting the image, using the 3D skeleton motion history to re predict correct position, re projecting and re fitting the 3D skeleton.
• the recovery module may also decide to skip a frame if the image information is corrupt.
• the three dimensional correlation module may be adapted to update the position of the three dimensional skeleton in accordance with the position of the fitted skeleton.
• said updating process associates the 3D skeleton on the fitted skeleton, fits between the 3D skeleton and the fitted skeleton, and updates the 3D skeleton to the correct position.
• FIG. 1 there is shown a block diagram depicting a system in accordance with some embodiments of the present invention.
• FIG. 2 there is shown a flow-chart depicting the steps of an HMI system in accordance with some embodiments of the present invention.
• FIG. 3 there is shown a block diagram depicting a system in accordance with some embodiments of the present invention.
• FIG. 4 there is shown a flow-chart depicting the steps of an HMI system in accordance with some embodiments of the present invention.
• FIG. 5 there is shown a block diagram depicting a system in accordance with some embodiments of the present invention.
• FIG. 6 there is shown a flow-chart depicting the steps of an HMI system in accordance with some embodiments of the present invention.
• Embodiments of the present invention may include apparatuses for performing the operations herein. This apparatus may be specially constructed for the desired purposes, or it may comprise a general purpose computer selectively activated or reconfigured by a computer program stored in the computer.
• Such a computer program may be stored in a computer readable storage medium, such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs) electrically programmable read-only memories (EPROMs), electrically erasable and programmable read only memories (EEPROMs), magnetic or optical cards, or any other type of media suitable for storing electronic instructions, and capable of being coupled to a computer system bus.
• a computer readable storage medium such as, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, magnetic-optical disks, read-only memories (ROMs), random access memories (RAMs) electrically programmable read-only memories (EPROMs), electrically erasable and programmable read only memories (EEPROMs), magnetic or optical cards, or any other type of media suitable for storing electronic instructions, and capable of being coupled to a computer system bus.
• 3D human machine interface may include (1) an image acquisition assembly, (2) an initializing module, (3) an image segmentation module, (4) a segmented data processing module, (5) a scoring module, (6) a projection module, (7) a fitting module,(8) a scoring and error detection module, (9) a recovery module, (10) a three dimensional correlation module, (11) a three dimensional skeleton prediction module, and (12) an output module.
• 3D HMI 3D human machine interface
• 3D HMI may include (1) an image acquisition assembly, (2) an initializing module, (3) an image segmentation module, (4) a segmented data processing module, (5) a scoring module, (6) a projection module, (7) a fitting module,(8) a scoring and error detection module, (9) a recovery module, (10) a three dimensional correlation module, (11) a three dimensional skeleton prediction module, and (12) an output module.
• the image acquisition assembly may be adapted to acquire a set of images, wherein substantially each image is associated with a different point in time.
• the images may be of a single user or multiple users.
• the initialization module may be adapted to detect and define the user's (1) colors, (2) organ's parameters, surrounding, and other parameters which are associated with the user and decide on the best way for image segmentation in the next steps (thresholds, score for every image segmentation etc.)
• the image segmentation module may be adapted to extract segmented data from the image.
• the segmented data may also comprise:
• the segmented data processing module may be adapted to process the segmented data.
• the segmented data may be processed in the following way:
• Color - using known color parameters to detect elements and/or light changes for example, use skin color to detect palms and face.
• Edge detection detect the edges of the image.
• the scoring module may be adapted to (1) examine the processed segmented data, (2) estimate the quality of the processed segmented data, and according to the quality (3) decide which portions of the segmented data are reliable enough to be used by the HMI system.
• the three dimensional skeleton prediction module may be adapted to predict the position of the three dimensional skeleton which will have the best match or correlation with the processed image.
• the three dimensional prediction module may use constraints which derive from the type of skeleton used, for example, if the skeleton is of a human figure, the head of the skeleton can't rotate 360 degrees.
• the three dimensional prediction module may use a set of dynamic and motion process to predict the position of the three dimensional skeleton.
• the projection module may be adapted to project the skeleton onto the image.
• the projection may be applied in the two-dimensional plane.
• the fitting module may be adapted to fit segmented data to the projected skeleton.
• the fitting module may be adapted to associate portions of the segmented data with portions of the projected skeleton.
• the scoring and error detection module may be adapted (1) to examine the processed skeleton after it was associated with segmented data, (2) to evaluate the fitting quality of said skeleton and (3) determine whether an error has occurred during the skeleton prediction process or the association of segmented data.
• the recovery module may be adapted to recover from a detected error.
• the recovery may be a process of multiple processing layers, re segmenting the image, using the 3D skeleton motion history to re predict correct position, re projecting and re fitting the 3D skeleton. The recovery module may also decide to skip a frame if the image information is corrupt.
• the three dimensional correlation module may be adapted to update the position of the three dimensional skeleton in accordance with the position of the fitted skeleton.
• said updating process associates the 3D skeleton on the fitted skeleton, fits between the 3D skeleton and the fitted skeleton, and updates the 3D skeleton the correct position.
• FIG. 1 there is shown an exemplary HMI system in accordance with some embodiments of the present invention, which system may be best described in conjunction with Figure 2, there is shown a flow chart depicting the steps of such an HMI system.
• Figure 1 shows a 3D human machine interface (“3D HMI"), which 3D HMI may include (1) an image acquisition assembly 1000, (2) an initializing module 1100, (3) an image segmentation module 1200, (4) a segmented data processing module 1300, (5) a scoring module 1400, (6) a projection module 1450, (7) a fitting module 1500,(8) a scoring and error detection module 1550, (9) a recovery module 1600, (10) a three dimensional correlation module 1700, (11) a three dimensional skeleton prediction module 1800, and (12) an output module 1900.
• 3D HMI 3D human machine interface
• the image acquisition assembly may be adapted to acquire a set of images, as seen in step 2000, wherein substantially each image is associated with a different point in time.
• the images may be of a single user or multiple users.
• the image acquisition assembly may comprise of a digital camera, a web camera, a film camera, a video camera, a web camera, a digital video camera, an analogue
• the system may enter an initialization phase, step 2100, which is performed by the initialization module 1100.
• which initialization module may be adapted to detect and define the user's (1) colors,
• organ's parameters (2) organ's parameters, (3) surrounding, and other parameters which are associated with the user.
• the system may be adapted to extract segmentation data, as shown in step 2200, which segmented data may comprise:
• the image segmentation module 1200 may be adapted to extract the segmented data from the image.
• the system may be adapted to process the segmented data, as shown in step 2300.
• the segmented data may be processed in the following way:
• Color - using known color parameters to detect elements and/or light changes for example, use skin color to detect palms and face.
• Edge detection detect the edges in the image.
• the segmented data processing module, 1300 may be adapted to process the segmented data.
• the system may be adapted to evaluate the quality of the segmented data, as shown in step 2400, the evaluation is performed by (1) examining the processed segmented data, (2) estimating the quality of the processed segmented data, and according to the estimated quality (3) decide which portions of the segmented data are reliable enough to be used by the HMI system.
• the scoring module 1400 may be adapted evaluate the quality of the segmented information.
• the system may be adapted to predict the position of the three dimensional skeleton, as shown in step 2800, which position will have the best match or correlation with the processed image.
• the prediction may be more accurate with the use of constraints which derive from the type of skeleton used, for example, if the skeleton is of a human figure, the head of the skeleton can't rotate 360 degrees without a motion of the shoulders.
• the prediction sequence may also use a set of dynamic and motion process and so on.
• the three dimensional skeleton prediction module 1800 may be adapted to predict the position of the three dimensional skeleton.
• the system may be further adapted to project the skeleton onto the image, as shown in step
• the projection may be applied in the two-dimensional plane.
• the projection module, 2450 may be adapted to project the skeleton onto the image.
• the system may be further adapted to fit the segmented data with the projected skeleton, as shown in step 2500.
• the fitting process may comprise the association of portions of the segmented data with portions of the projected skeleton.
• fitting the segmented data may comprise associating portions of the extracted segmented data with current skeleton parameters, which current skeleton parameters may support the associated portions of extracted segmented data.
• the outcome of this process is a "fitted skeleton”.
• the fitting module, 2500 may be adapted to associate the segmented data with the projected skeleton.
• the system may be further adapted to give score to the fitted skeleton and detect errors, as shown in step 2550.
• giving score and detecting errors may comprise of (1) examining the fitted skeleton, (2) evaluating the fitting quality of said skeleton and (3) determining whether an error has occurred during the skeleton prediction process or the association of segmented data.
• the scoring and error detection module 1550 may be adapted to give score and detect errors.
• the system may enter a recovery phase, as shown in step 2600.
• the recover process may be a process of multiple processing layers.
• the recovery phase may comprise re-segmenting the image, re-predicting the 3D skeleton position, re-projecting and re-fitting the skeleton using extended effort.
• the recovery module may also decide to skip a frame or more if the image information is corrupt.
• the system may be adapted to detect that the object its tracking is not in the frame.
• the system may be adapted to skip one or more frames until the object is back in the frame.
• the recovery phase may direct the system back to the initialization step.
• the recovery module 2600 may be adapted to perform the recovery process.
• the system may be adapted to update the position of the three dimensional skeleton in accordance with the position of the fitted skeleton, as shown in step 2700.
• the updating process may comprise (1) projecting the 3D skeleton on the fitted skeleton, (2) associating the 3D skeleton with the fitted skeleton, and (3) updating the position of the 3D skeleton.
• the three dimensional correlation module, 1700 may be adapted to update the position of the three dimensional skeleton.
• the three- dimensional correlation module 1700 and the skeleton prediction module 1800 may use some or all of the algorithms and processes which were disclosed in PCT application serial number.PCT/IL2005/000813, filed on 31 July 2005 under the same assignee as the present application.
• FIG. 3 there is shown an exemplary HMI system in accordance with some embodiments of the present invention, which system may be best described in conjunction with Figure 4, there is shown a flow chart depicting the steps of such an HMI system.
• Figure 3 shows a 3D human machine interface (“3D HMI"), which 3D HMI may include (1) a Zlens image acquisition assembly 3000, (2) an initializing module 3100, (3) an image segmentation module 3200, (4) a segmented data processing module 3300, (5) a fitting module 3500. (6) a scoring module 3550, (7)a three dimensional correlation module 3700, and (8) an output module 3900.
• the Zlens acquisition assembly may be adapted to acquire a set of images, as seen in step 4000 wherein substantially each image is associated with a different point in time.
• the images may be of a single user or multiple users.
• the Zlens acquisition assembly may be mounted on another image acquisition assembly, i.e. element 1000 of Fig.1.
• the Zlens acquisition assembly (3000) may be best described in conjunction with PCT/IL2006/001254 filed on October 31 2006 under the same assignee as the present application and with US Patent application 60/731 ,274 US filed on October 31 2005 under the same assignee as the present application.
• system is further adapted to enter an initialization phase, as shown in step 4100,
• the initialization module 3100 which is performed by the initialization module 3100. Which initialization module may be adapted to detect and define the user's (1) colors, (2) organ's parameters, surroundings, (3) and other parameters which are associated with the user.
• the system may be adapted to extract segmentation data, as shown in step 4200, which segmented data may comprise:
• the image segmentation module 3200 may be adapted to extract the segmented
• the system may be adapted to process the segmented data, as shown in step 4300.
• Color - using known color parameters to detect elements and/or light changes for example, use skin color to detect palms and face.
• Edge detection detect the contours of every organ.
• the segmented data processing module, 3300 may be adapted to process the segmented data.
• the system may be further adapted to fit portions of the extracted segmented data with the acquired image, as shown in step 4500.
• the fitting process may comprise associating portions of the extracted segmented data with dedicated areas of the acquired image.
• the dedicated areas may be stored in the system or may be determined during the initialization phase.
• the dedicated areas may be specific organs of the user (hands, head, feet) or any other element which may be acquired during step 3000.
• the fitting process may comprise testing whether the extracted segmented data defines parameters which are relevant to the dedicated areas.
• the fitting module, 3500 may be adapted to the associate portions of the segmented data with the acquired image.
• the outcome of this process is a "fitted image”.
• the system may be further adapted to evaluate the quality of the fitted segmented data, as shown in step 4550.
• evaluating the quality of the fitted segmented data may comprise of (1) examining the processed segmented data, (2) estimating the quality of the processed segmented data, and according to the estimated quality (3) decide which portions of the segmented data are reliable enough to be used by the HMI system, (4) examine the fitted image, (5) evaluate the fitting quality of said image and (6) determining whether an error has occurred during the association of segmented data.
• the scoring module, 3550 may be adapted to evaluate the quality of the fitted segmented data.
• the system may comprise an error detection mechanism and a recovery mechanism as was described hereinabove.
• the system may be adapted to update the position of a three dimensional body in accordance with the fitted image and the extrapolation of a depth map using the Zlens image acquisition assembly , as shown in step 4700.
• the updating process may comprise associating the extracted depth map with the extracted segmented data, and (3) updating the position of the three-dimensional body of the output model.
• the three dimensional correlation module, 3700 may be adapted to update the position of the three dimensional body.
• the functionality of the three-dimensional correlation module 3700 and the Zlens image acquisition assembly 3000 and particularly the extrapolation of depth from an image acquired using the Zlens apparatus may best be described in conjunction with PCTVI L2006/001254 filed on October 31 2006 under the same assignee as the present application and with US Patent application 60/731,274
• FIG. 5 there is shown an exemplary HMI system in accordance with some embodiments of the present invention, which system may be best described in conjunction with Figure 6, there is shown a flow chart depicting the steps of such an HMI system.
• Figure 5 shows a 3D human machine interface (“3D HMI"), which 3D HMI may include (1) a Zlens acquisition assembly 5000, (2) an initializing module 5100, (3) an image segmentation module 5200, (4) a segmented data processing module 5300, (5) a scoring module 5400, (6) a projection module 5450, (7) a fitting module 5500,(8) a scoring and error detection module 5550, (9) a recovery module 5600, (10) a three dimensional correlation module 5700, (11) a three dimensional skeleton prediction module 5800, (12) an output module 5900 and an optional (13) depth extraction module 5050.
• 3D HMI 3D human machine interface
• the Zlens acquisition assembly may be adapted to acquire a set of images, as seen in step 6000, wherein substantially each image is associated with a different point in time.
• the images may be of a single user or multiple users.
• the Zlens acquisition assembly may be mounted on another image acquisition assembly, i.e. element 1000 of Fig.1.
• the Zlens acquisition assembly (5000) may be best described in conjunction with PCT/IL2006/001254 filed on October 31 2006 under the same assignee as the present application and with US Patent application 60/731 ,274 US filed on October 31 2005 under the same assignee as the present application.
• the system may enter an initialization phase, step 6100, which is performed by the initialization module 5100.
• which initialization module may be adapted to detect and define the user's (1) colors, (2) organ's parameters, (3) surroundings, and other parameters which are associated with the user.
• the system may be adapted to extract segmentation data, as shown in step 6200, which segmented data may comprise:
• the image segmentation module 5200 may be adapted to extract the segmented data from the image.
• the system may be adapted to process the segmented data, as shown in step 6300.
• the segmented data may be processed in the following way:
• Color - using known color parameters to detect elements and/or light changes for example, use skin color to detect palms and face.
• Edge detection detect the contours of every organ.
• the segmented data processing module, 5300 may be adapted to process the
• the system may be adapted to evaluate the quality of the segmented data, as shown in step 6400, the evaluation is performed by (1) examining the processed segmented data, (2) estimating the quality of the processed segmented data, and according to the estimated quality (3) decide which portions of the segmented data are reliable enough to be used by the HMI system.
• the scoring module, 5400 may be adapted evaluate the quality of the segmented information.
• the system may be adapted to predict the position of the three dimensional skeleton, as shown in step 6800, which position will have the best match or correlation with the processed image.
• the prediction may be more accurate with the use of constraints which derive from the type of skeleton used, for example, if the skeleton is of a human figure, the head of the skeleton can't rotate 360 degrees without a motion of the shoulders.
• the prediction sequence may also use a set of dynamic and motion process.
• the three dimensional skeleton prediction module 5800 may be adapted to predict the position of the three dimensional skeleton.
• the system may be further adapted to extract depth using the Zlens acquisition assembly, as shown in step 6050, the extraction of depth using a Zlens acquisition assembly is described in (1) PCT/IL2006/001254 filed on October 31 2006 under the same assignee as the present application and with (2)US Patent application 60/731,274 US filed on October 31 2005 under the same assignee as the present application.
• the depth extraction module, 5050 may be adapted to extract depth from the acquired image.
• the system may be further adapted to project the skeleton onto the image, as shown in step 6450.
• the projection may be applied in the two-dimensional plane.
• the projection of the skeleton may be applied in the three-dimensional plane if module 5050 is used.
• the projection may be onto a three-dimensional image and/or a three dimensional cloud of points.
• the projection module, 6450 may be adapted to project the skeleton onto the image.
• the system may be further adapted to fit the segmented data with the projected skeleton, as shown in step 6500.
• the fitting process may comprise the association of portions of the segmented data with portions of the projected skeleton.
• fitting the segmented data may comprise associating portions of the extracted segmented data with current skeleton parameters, which current skeleton parameters may support the associated portions of extracted segmented data.
• the outcome of this process is a "fitted skeleton”.
• the fitting module, 5500 may be adapted to associate the segmented data with the projected skeleton.
• the system may be further adapted to give score to the fitted skeleton and detect errors, as shown in step 6550.
• giving score and detecting errors may comprise of (1) examining the fitted skeleton, (2) evaluating the fitting quality of said skeleton and (3) determining whether an error has occurred during the skeleton prediction process or the association of segmented data.
• the scoring and error detection module, 5550 may be adapted to give score and detect errors
• the system may enter a recovery phase, as shown in step 6600.
• the recovery process may be a process of multiple processing layers.
• the recovery phase may comprise re-segmenting the image, re-predicting the 3D skeleton position, re-projecting and re-fitting the skeleton using extended effort.
• the recovery module may also decide to skip a frame or more if the image information is corrupt.
• the system may be adapted to detect that the object its tracking is not in the frame.
• the system may be adapted to skip one or more frames until the object is back in the frame.
• the recovery phase may direct the system back to the initialization step.
• the recovery module 5600 may be adapted to perform the recovery process.
• the system may be adapted to update the position of the three dimensional skeleton in accordance with the position of the fitted skeleton, as shown in step 6700.
• the updating process may comprise (1) projecting the 3D skeleton on the fitted skeleton, (2) associating the three dimensional skeleton with the fitted skeleton, (3) extract depth using the Zlens assembly, (4) associating the three- dimensional skeleton with depth parameters and (5) updating the position of the 3D skeleton.
• the three dimensional correlation module, 5700 may be adapted to update the position of the three dimensional skeleton.
• the three-dimensional correlation module 5700 and the skeleton prediction module 5800 may use some or all of the algorithms and processes which were disclosed in PCT application serial number.PCT/IL2005/000813, filed on 31 July 2005 under the same assignee as the present application.
• the functionality of (1) the three-dimensional correlation module 5700 (2) the Zlens image acquisition assembly 5000 the (3) depth extraction module 5050 and (4) particularly the extrapolation of depth from an image acquired using the Zlens apparatus may best be described in conjunction with PCT/IL2006/001254 filed on October 31 2006 under the same assignee as the present application and with US Patent application 60/731 ,274 US filed on October 31 2005 under the same assignee as the present application.
• the systems described hereinabove may be adapted to receive from an exterior source depth images and/or three-dimensional images. According to yet further embodiments of the present invention, if a depth images and/or three- dimensional images is received the system is adapted to extract its parameters in the relevant modules.
• the processes and displays presented herein are not inherently related to any particular computer or other apparatus. Various general purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct a more specialized apparatus to perform the desired method. The desired structure for a variety of these systems will appear from the description below. In addition, embodiments of the present invention are not described with reference to any particular programming language. It will be appreciated that a variety of programming languages may be used to implement the teachings of the inventions as described herein. One of ordinary skill in the art should understand that the described invention may be used for all kinds of wireless or wire-line system.

Landscapes

• Engineering & Computer Science (AREA)
• Theoretical Computer Science (AREA)
• Physics & Mathematics (AREA)
• General Engineering & Computer Science (AREA)
• General Physics & Mathematics (AREA)
• Human Computer Interaction (AREA)
• Multimedia (AREA)
• Software Systems (AREA)
• Geometry (AREA)
• Computer Graphics (AREA)
• Processing Or Creating Images (AREA)
• Image Processing (AREA)
• User Interface Of Digital Computer (AREA)

Priority Applications (8)

Applications Claiming Priority (1)

Related Child Applications (4)

Publications (2)

Family

ID=39864447

Family Applications (1)

Country Status (6)

Cited By (20)

Family Cites Families (12)

• 2007
  • 2007-04-15 KR KR1020097023103A patent/KR101379074B1/ko not_active IP Right Cessation
  • 2007-04-15 EP EP07736215A patent/EP2147393A4/en not_active Withdrawn
  • 2007-04-15 CA CA2684020A patent/CA2684020C/en not_active Expired - Fee Related
  • 2007-04-15 WO PCT/IL2007/000475 patent/WO2008126069A2/en active Application Filing
  • 2007-04-15 JP JP2010502633A patent/JP5147933B2/ja not_active Expired - Fee Related
• 2009
  • 2009-10-14 IL IL201514A patent/IL201514A/he not_active IP Right Cessation

Non-Patent Citations (1)

Also Published As

Similar Documents

Legal Events