CA2928401A1 - A computer-implemented method for making rapid periodic movements visible to the human eye - Google Patents
A computer-implemented method for making rapid periodic movements visible to the human eye Download PDFInfo
- Publication number
- CA2928401A1 CA2928401A1 CA2928401A CA2928401A CA2928401A1 CA 2928401 A1 CA2928401 A1 CA 2928401A1 CA 2928401 A CA2928401 A CA 2928401A CA 2928401 A CA2928401 A CA 2928401A CA 2928401 A1 CA2928401 A1 CA 2928401A1
- Authority
- CA
- Canada
- Prior art keywords
- video
- subject
- photographs
- computer
- still
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Classifications
-
- G—PHYSICS
- G03—PHOTOGRAPHY; CINEMATOGRAPHY; ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ELECTROGRAPHY; HOLOGRAPHY
- G03B—APPARATUS OR ARRANGEMENTS FOR TAKING PHOTOGRAPHS OR FOR PROJECTING OR VIEWING THEM; APPARATUS OR ARRANGEMENTS EMPLOYING ANALOGOUS TECHNIQUES USING WAVES OTHER THAN OPTICAL WAVES; ACCESSORIES THEREFOR
- G03B39/00—High-speed photography
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Studio Devices (AREA)
Abstract
A computer-implemented method for making rapid periodic movements of a subject of examination visible to the human eye, comprising the steps of: sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, determining whether still or slow motion photographs or video of said subject is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, determining exposure and brightness settings for capture of the still or slow motion photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
Description
A COMPUTER-IMPLEMENTED METHOD FOR MAKING RAPID PERIODIC
MOVEMENTS VISIBLE TO THE HUMAN EYE
FIELD OF THE INVENTION
The present invention relates to the visualization of rapid periodic movements. More particularly, the present invention relates to a method to visualize rapid periodic movements, such as oscillations or rotations, by means of controlling the capture frame rate of such movements.
BACKGROUND OF THE INVENTION
Super high-speed cameras (devices generally capable of image exposures in excess of at least 250 FPS (frames per second)) are often used to record rapid periodic movements, such as the oscillations of vocal cords (which generally vibrate anywhere between 80 and 400 Hz (cycles per second)) or the rotations of a turbo prop airplane rotor (which commonly rotates between 2500 and 9000 RPM (revolutions per minute)), as photographic images.
The recorded images can then be played back individually or in slow-motion for study/analysis purposes.
As an example, one could analyze a person's vocals chords that are oscillating at 300Hz by employing the use of a high-speed camera that captures images at 3600 FPS. The camera would capture 12 frames of images for every vocal cycle, which, if played back at a video speed of 36 FPS would display 3 vocal cycles in slow motion every second for analysis purposes. This, however, is still a relatively fast movement for observation purposes, but slowing playback to 1 Hz would require a high speed capture rate of 10800 FPS. The problem with the use of such high-speed cameras is that they are very expensive and collect large quantities of data. In addition, such high-speed cameras require very bright lights for the needed exposure for the film/images, which has been known to cause blurring of images, and even destruction of the subject of examination, because of the heat of the lighting.
An alternative approach, which overcomes some of the problems associated with the use of high-speed cameras noted above, is to capture a single frame from each periodic movement (e.g. vocal cycle), or every nth periodic movement, through the use of a synchronized strobe light in order to observe the oscillation or rotational movement as a still image or in a slow motion manner for analysis purposes. In particular, by matching the strobe light frequency to the periodic frequency (e.g. oscillation or rotational periodicity), one is capable of capturing a freeze frame view of the subject (by capturing the same position in the cycle over time). By slightly offsetting synchronization, one is capable of capturing movement in forward or backward slow motion.
SUMMARY OF THE INVENTION
The computer-implemented method of the present invention is capable of making rapid periodic movements visible to the human eye. By precisely matching the capture frequency of the subject being examined to the periodic frequency of movement (e.g.
oscillation or rotational periodicity), one is capable of capturing freeze frame views of the subject (by capturing the same position in each or nth cycle over time), whereas by slightly offsetting synchronization, one is capable of capturing movement in an apparent forward or backward slow motion manner.
In one aspect, the present invention provides a computer-implemented method for making rapid periodic movements of a
MOVEMENTS VISIBLE TO THE HUMAN EYE
FIELD OF THE INVENTION
The present invention relates to the visualization of rapid periodic movements. More particularly, the present invention relates to a method to visualize rapid periodic movements, such as oscillations or rotations, by means of controlling the capture frame rate of such movements.
BACKGROUND OF THE INVENTION
Super high-speed cameras (devices generally capable of image exposures in excess of at least 250 FPS (frames per second)) are often used to record rapid periodic movements, such as the oscillations of vocal cords (which generally vibrate anywhere between 80 and 400 Hz (cycles per second)) or the rotations of a turbo prop airplane rotor (which commonly rotates between 2500 and 9000 RPM (revolutions per minute)), as photographic images.
The recorded images can then be played back individually or in slow-motion for study/analysis purposes.
As an example, one could analyze a person's vocals chords that are oscillating at 300Hz by employing the use of a high-speed camera that captures images at 3600 FPS. The camera would capture 12 frames of images for every vocal cycle, which, if played back at a video speed of 36 FPS would display 3 vocal cycles in slow motion every second for analysis purposes. This, however, is still a relatively fast movement for observation purposes, but slowing playback to 1 Hz would require a high speed capture rate of 10800 FPS. The problem with the use of such high-speed cameras is that they are very expensive and collect large quantities of data. In addition, such high-speed cameras require very bright lights for the needed exposure for the film/images, which has been known to cause blurring of images, and even destruction of the subject of examination, because of the heat of the lighting.
An alternative approach, which overcomes some of the problems associated with the use of high-speed cameras noted above, is to capture a single frame from each periodic movement (e.g. vocal cycle), or every nth periodic movement, through the use of a synchronized strobe light in order to observe the oscillation or rotational movement as a still image or in a slow motion manner for analysis purposes. In particular, by matching the strobe light frequency to the periodic frequency (e.g. oscillation or rotational periodicity), one is capable of capturing a freeze frame view of the subject (by capturing the same position in the cycle over time). By slightly offsetting synchronization, one is capable of capturing movement in forward or backward slow motion.
SUMMARY OF THE INVENTION
The computer-implemented method of the present invention is capable of making rapid periodic movements visible to the human eye. By precisely matching the capture frequency of the subject being examined to the periodic frequency of movement (e.g.
oscillation or rotational periodicity), one is capable of capturing freeze frame views of the subject (by capturing the same position in each or nth cycle over time), whereas by slightly offsetting synchronization, one is capable of capturing movement in an apparent forward or backward slow motion manner.
In one aspect, the present invention provides a computer-implemented method for making rapid periodic movements of a
2 subject of examination visible to the human eye, comprising the steps of: sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, determining whether still or slow motion photographs or video of said subject is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, determining exposure and brightness settings for capture of the still or slow motion photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting. The still capture rate may be based on a natural number divisor of the fundamental frequency. The computer-implemented method may further comprise the step of tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject. The computer-implemented method may also further comprise the step of playing back any recorded capture of the subject of examination. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
In another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In a further aspect, the present invention provides a computer readable memory having recorded thereon statements and
In another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In a further aspect, the present invention provides a computer readable memory having recorded thereon statements and
3 instructions for execution by a computer for making rapid periodic movements of a subject of examination visible to the human eye, said statements and instructions comprising: code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for determining whether still or slow motion photographs or video of said subject is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, code means for determining exposure and brightness settings for capture of the still or slow motion photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting. The computer readable memory may further comprise code means for tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject. The computer readable memory may also further comprise code means for playing back any recorded capture of the subject of examination. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
In yet a further aspect, the present invention provides a computer-implemented method for making rapid periodic movements of a subject of examination visible to the human eye in a slower motion manner, comprising the steps of: sampling the subject of examination and determining a fundamental frequency of said
In yet a further aspect, the present invention provides a computer-implemented method for making rapid periodic movements of a subject of examination visible to the human eye in a slower motion manner, comprising the steps of: sampling the subject of examination and determining a fundamental frequency of said
4 subject, determining a still capture rate based on the fundamental frequency of said subject, determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, determining exposure and brightness settings for capture of slow motion photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting. The still capture rate may be based on a natural number divisor of the fundamental frequency. The computer-implemented method may further comprise the step of tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject. The computer-implemented method may also further comprise the step of playing back any recorded capture of the subject of examination. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
In yet another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet a further aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of a subject of examination visible to the human eye in a slower motion manner, said statements and
In yet another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet a further aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of a subject of examination visible to the human eye in a slower motion manner, said statements and
5 instructions comprising: code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, code means for determining exposure and brightness settings for capture of slow motion photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and . brightness setting.
The computer readable memory may further comprise code means for tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject. The computer readable memory may also further comprise code means for playing back any recorded capture of the subject of examination. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
In another aspect, the present invention provides a computer-implemented method for making rapid periodic movements of a subject of examination visible to the human eye in a still freeze-frame manner, comprising the steps of: sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, setting a sampling frequency to the still capture rate, determining exposure and
The computer readable memory may further comprise code means for tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject. The computer readable memory may also further comprise code means for playing back any recorded capture of the subject of examination. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
In another aspect, the present invention provides a computer-implemented method for making rapid periodic movements of a subject of examination visible to the human eye in a still freeze-frame manner, comprising the steps of: sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, setting a sampling frequency to the still capture rate, determining exposure and
6 brightness settings for capture of still freeze-frame photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting. The still capture rate may be based on a natural number divisor of the fundamental frequency.
The computer-implemented method may further comprise the step of tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject. The computer-implemented method may also further comprise the step of playing back any recorded capture of the subject of examination. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
In yet another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet a further aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of a subject of examination visible to the human eye in a still freeze-frame manner, said statements and instructions comprising: code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for setting a sampling frequency to the still capture rate, code
The computer-implemented method may further comprise the step of tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject. The computer-implemented method may also further comprise the step of playing back any recorded capture of the subject of examination. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
In yet another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet a further aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of a subject of examination visible to the human eye in a still freeze-frame manner, said statements and instructions comprising: code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for setting a sampling frequency to the still capture rate, code
7 means for determining exposure and brightness settings for capture of still freeze-frame photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting. The computer readable memory may further comprise code means for tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject. The computer readable memory may also further comprise code means for playing back any recorded capture of the subject of examination. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
In another aspect, the present invention provides a computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye on a portable electronic device attached to an endoscope, comprising the steps of: sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, determining whether still or slow motion photographs or video of said vocal chords is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, determining exposure and brightness settings for capture of the still or slow motion photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
The still capture rate may be based on a natural number divisor
In another aspect, the present invention provides a computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye on a portable electronic device attached to an endoscope, comprising the steps of: sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, determining whether still or slow motion photographs or video of said vocal chords is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, determining exposure and brightness settings for capture of the still or slow motion photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
The still capture rate may be based on a natural number divisor
8 of the fundamental frequency. The computer-implemented method may further comprise the step of tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer-implemented method may also further comprise the step of playing back any recorded capture of the vocal chords. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In yet another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet a further aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye on a portable electronic device attached to an endoscope, said statements and instructions comprising: code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for determining whether still or slow motion photographs or video of said vocal chords is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, code means for determining exposure and brightness settings for capture of the still or slow motion
In yet another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet a further aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye on a portable electronic device attached to an endoscope, said statements and instructions comprising: code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for determining whether still or slow motion photographs or video of said vocal chords is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, code means for determining exposure and brightness settings for capture of the still or slow motion
9 photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
The computer readable memory may further comprise code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer readable memory may also further comprise code means for playing back any recorded capture of the vocal chords. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In another aspect, the present invention provides a computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye in a slower motion manner on a portable electronic device attached to an endoscope, comprising the steps of: sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, determining exposure and brightness settings for capture of slow motion photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting. The still capture rate may be based on a natural number divisor of the fundamental frequency. The computer-implemented method may further comprise the step of tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer-implemented method may also further comprise the step of playing back any recorded capture of the vocal chords. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In a further aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet another aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye in a slower motion manner on a portable electronic device attached to an endoscope, said statements and instructions comprising: code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, code means for determining exposure and brightness settings for capture of slow motion photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting. The computer readable memory may further comprise code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer readable memory may also further comprise code means for playing back any recorded capture of the vocal chords. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In another aspect, the present invention provides a computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye in a still freeze-frame manner on a portable electronic device attached to an endoscope, comprising the steps of: sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, setting a sampling frequency to the still capture rate, determining exposure and brightness settings for capture of still freeze-frame photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting. The still capture rate may be based on a natural number divisor of the fundamental frequency. The computer-implemented method may further comprise the step of tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer-implemented method may also further comprise the step of playing back any recorded capture of the vocal chords. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In yet another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet a further aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye in a still freeze-frame manner on a portable electronic device attached to an endoscope, said statements and instructions comprising: code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for setting a sampling frequency to the still capture rate, code means for determining exposure and brightness settings for capture of still freeze-frame photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting. The computer readable memory may further comprise code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer readable memory may also further comprise code means for playing back any recorded capture of the vocal chords. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 is a Master Flow Diagram showing the general steps involved in the method of the present invention.
Figure 2 is a diagram showing vocal frequency with 141 capture rate.
Figure 3 is a series of screen captures of a rotor running at 3500 RPM using a freeze frame capture rate. The rotor appears frozen in time. Subsequent frames show a shift of the observation point through user control.
Figure 4 is a series of screen captures showing slow motion movement of the vocal cords when singing the letter "e" at a frequency of approximately 200 Hz.
Figure 5 is a detailed logic tree of the steps involved in the method of the present invention.
Figure 6 shows pseudocode for calculating the fundamental frequency.
Figure 7 shows the core method to determine the maximum frequency. The algorithm tracks small variations in frequency and is capable of suppressing frequency spikes.
DETAILED DESCRIPTION OF THE INVENTION
The following description is presented to enable a person skilled in the art to make and use the invention, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the scope of the invention.
Thus, the present invention is not intended to be limited to the embodiments disclosed, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
As shown in Figure 1, the computer-implemented method of the present invention involves the following general steps (which steps may not always be necessary or in the following definitive order): 1) sampling the subject of examination (1) to determine periodicity of rapid movement (i.e. fundamental frequency, e.g.
Hz, RPM, or the like); 2) determining freeze-frame (still) capture rate (2) for desired visualization; 3) determining appropriate exposure and brightness setting (3) for the determined fundamental frequency; 4) determining whether freeze-frame or slow motion photographs and/or video (4) is to be captured and performing any necessary calculations accordingly;
and 5) capturing photographs and/or video of the subject of examination at the determined capture rate, exposure setting, and brightness level. The method of the present invention may involve the additional steps of 6) tracking the fundamental frequency for any changes over time and adjusting same while viewing and/or recording photographs and/or video of the subject of examination; and 7) playing back any recorded capture of the subject of examination.
Sampling the subject of examination (step 1) to determine periodicity of rapid movement (e.g. Hz, RPM, or the like) can be undertaken manually or more preferably automatically using an appropriate sensor. In this respect, the sensor may include any type of sensor that can adequately determine the periodicity of rapid movement or oscillation frequency. In the case of the visualization of vocal cords, for instance, the built-in microphone or a specialized Bluetooth microphone that can be located close to the sound source, can be used to collect sound samples of the vocal cord oscillations and send this data to an electronic device, such as a connected laptop computer or mobile electronic device such as an iPad, iPhone, or the like. The electronic device will apply a frequency tracking algorithm to determine the fundamental frequency of the vocal cord (see Figure 6). For rotational movements, an electromagnetic or optical sensor may be used to establish revolutions per second.
Air pressure sensors, mechanical switches, ultrasound sensors, etc. are just a few of the other numerous sensor technologies that can be used to determine and convey oscillation frequency of the observed vibration or rotational movement of the subject of examination, as appropriate.
Once periodicity/frequency of rapid movement has been determined, one can then determine the freeze-frame (still) capture rate (step 2) to implement the desired visualization. As to desired visualization (see step 4), by matching the capture rate appropriately to the periodic frequency of movement of the subject being examined (e.g. oscillation or rotational periodicity), one is capable of capturing freeze frame views of the subject (by capturing the same position in the cycle over time). In this respect, the capture rate may be tuned to a precise sub sampling frequency that matches a natural number divisor. For example, a vocal vibration of 240 Hz could be sampled every 4th oscillation using a capture frequency of 60 FPS
as shown in Figure 2. In this way, since the capture rate is precisely set to a natural number fraction of the observed frequency, the image will appear still. The following equation(s) therefore provide the capture rate needed to obtain freeze-frame/still viewing:
SCR = RPS / N
or SCR = F / N
Where, SCR is Still Capture Rate RPS is Rotations per second F is Frequency in Hz (Oscillations per second) N is a natural number, N = {1, 2, 3, .... XI
Most mobile computing devices are able to adjust the capture rate in a limited range up to a maximum of 120 or 240 FPS, but it is therefore possible to match the capture rate to the rate of oscillation or a natural number divisor thereof to give the appearance of freeze frame viewing.
It is important to note that the observed point in time can also be shifted to other points in the periodic movement through a phase shift of the sampling point. The phase shift can be specified in steps of 1 up to the sampling frequency. Phase shifting the sampling point allows for static analysis by moving the observation point in time. For example, an airplane rotor can be examined at different points along its movement path or a physician can inspect the larynx at different stages of the sound formation (see, for instance, Figures 3 and 4).
If, however, on the other hand, one wishes to see movement in an apparent slow-motion manner (see step 4), then the capture rate can be slightly offset/adjusted from the freeze-frame synchronization to provide the desired visualization in either an apparent forward or backward slow-motion manner. In this respect, to show observable movement, we need to introduce a small capture rate offset (CRO) that will shift the sampling point through the waveform at a speed that is observable by the human eye. If, for example, viewing playback is set at 30 FPS, we need 30 sampled frames per wavelength to observe one cycle per second. So, if a vocal cord vibration of 120 Hz is sampled at 60 Hz, it will capture 2 times the playback frame rate of 30 FPS, and the playback would show a full wavelength every 2 seconds. If the sampling rate is changed to 61 Hz, then the capture rate offset (CRO) is 1 Hz, and we would see slow-motion movement. The sampling frequency (SF; CRO multiplied by N) determines the number of discrete sampling points along the periodic movement. The higher the sampling frequency, the more discrete points are available. In the example provided above, the CRO would be 1. The number of frames (NF) is the oscillation speed or frequency divided by SF. The following equation(s) are therefore germane:
SF = N * CRO
Where, SF is Sampling frequency CRO is Capture rate offset N is a natural number divisor of the fundamental frequency for the sub-sampling rate, N = {1, 2, 3,_ X}
NF = F / SF
Where NF is Number of frames F is Frequency of Oscillation or RPS
SF is Sampling frequency As a person skilled in the art would appreciate, because N
multiplies the CRO, higher sampling frequencies are less sensitive to drift and allow for more precise control.
The capture rate offset (CRO) also determines the speed of the observable frequency. By controlling the CRO, the observable periodic movement can be made to appear faster or slower to the point of freezing the movement. Positive and negative offsets will produce observable movements in opposite directions. For example, a rotor would turn counter clockwise for positive CROs and clockwise for negative CROs. The ability to precisely control the speed and direction of the observable movement through fine control of the CRO allows for dynamic analysis in observable frequencies.
The frequency of the slow motion playback is dependent on the playback rate. In our example provided above, playback is set to 30 FPS. The observer would see a full wavelength motion with a = frequency of 0.5 Hz, or in other words, a full oscillation would take 2 seconds to display. The following equation is therefore germane:
PF = PR / NF
Where, PF is Playback frequency NF is Number of frames PR is Playback rate The speed of the visual movement depends on the offset from the freeze frame capture rate, the playback rate and the size of the divisor, which will have an amplification effect. A high playback rate would require more frames per second whereas a slower playback rate would require less. For example, in our above example of 60 frames for one sampled waveform, a playback rate (PR) of 30 Hz would require 30 frames per second. The playback frequency (PF) would be 0.5 Hz or 2 seconds per completed waveform. If PF was 60 Hz, PF would be 1 Hz or 1 second per waveform. The higher the number of frames per waveform, the slower the PF is. For small capture rate offsets (CRO), we achieve a relatively high number of frames per sampled waveform.
A visual approach may be used to adjust the capture rate manually via a user control. In this respect, once a user adjusts the capture rate to a point where it nears a natural number divisor of the oscillation frequency, the oscillation will move slower and slower until it finally appears frozen at the point where the capture rate matches a natural number divisor of the oscillation frequency. Manual adjustment can be difficult, however, as even relatively small offsets can display fast moving oscillations that the human eye cannot observe. If the frequency of movement is known, it may also be possible to manually set the capture rate or a natural number divisor of the oscillation frequency at a rate within the supported range of the electronic device in order to achieve the desired visualization.
Sensor input may also be used to establish the correct capture rate automatically. For vocal cord vibration at a specific sound such as when pronouncing the letter "e", for instance, a microphone can be used to establish the fundamental frequency, which is the main frequency at which the vocal cord vibrates for that sound. This fundamental frequency may then be automatically divided using a natural number, as discussed above, to set the capture rate in a range that is supported by the electronic device.
For the computer-implemented method of the present invention, it is also important to determine the appropriate exposure and brightness setting (step 3) for the determined capture rate of the subject of examination. Fast moving objects require a very short exposure time to avoid blurring. The maximum exposure time required is determined by the speed of the observed object moving through the frame. The faster the movement of the object, the shorter the exposure time required to avoid blurring. In this respect, a minimum factor of 10 times the observed periodic movement is required to avoid blurring. For movement at 200 to 400 Hz, this means an exposure time of 1/2000 to 1/4000 is required. The following equation is germane:
EXP = 1 / (P * F) Where, EXP is the Exposure time in seconds P is the Multiplication factor (minimum 10) F is the Frequency of periodic movement in Hz The Field of View (FOV) of the camera also influences the speed of the object within the frame. If a periodically moving object fills a frame because the object is closer to the camera or a telephoto lens is used, then the movement of the object appears faster than if the object is further away and only fills a portion of the screen.
A very short exposure time limits the amount of light that will arrive at the photo sensor. One way to compensate for the lack of light is to use a stronger light source. If this approach is not practical, then the sensitivity of the photo sensor has to be increased. The sensitivity setting of photo sensors are usually expressed in terms of ISO, and the higher the ISO the more sensitive the sensor and the less light is needed to fully expose the frame. The higher the sensitivity setting of the sensor, the higher the digital noise will be. At very high ISO
settings, visible grain can appear and image details may be lost.
The aperture or opening of the camera lens is fixed for most mobile computer cameras. However, if a variable aperture is available, the larger the opening, the more light will be available at the sensor. The aperture controls the depth of field, with a larger aperture producing a shallow depth of field and a small aperture providing a large depth of field. The shallower the depth of field, the smaller the range of the image that will be sharp.
Therefore, to produce a well-exposed, sharp image, the right exposure, ISO and aperture setting has to be used. The presented invention may include a method to optimize the exposure and ISO
settings by leveraging the sensor input to determine the frequency of the periodic movement. For low light situations, as is the case for endoscopic procedures, it is useful to use the maximum or very high ISO setting and an exposure of 1200 to 1/1500 seconds. Very bright scenes will use an exposure of 1/(10 * Fundamental Frequency) or the highest available capture rate (whichever is lower). The ISO setting can then automatically follow the brightness of the scene which can be determined automatically through the photo sensor or can be established manually through user input. Once the required exposure is determined through the sensor input, the maximum aperture and ISO are set accordingly. Cameras with a variable aperture may also automatically set the aperture to the maximum required according to the desired depth of field.
If the available camera controls and the scene lighting are optimized, digital methods are also available to increase the brightness of the image. The present invention may utilize automatic histogram equalization to increase the brightness of the image. A good description of histogram-based equalization can be found at:
https://en.wikipedia.org/wiki/Histogram equalization In addition, if the periodic movement of the subject being examined varies in frequency over time, then tracking changes to the frequency and making adjustments to the capture rate in real time become very important (and thus the need for an additional step of tracking the frequency of capture and adjusting same (step 6) while viewing and/or recording photographs and/or video of the subject of examination). The presented invention may utilize a frequency-tracking algorithm to calculate and set the capture frequency during video capture.
There are also other issues that at times may need to be considered. Digital sensor data is typically read line by line in a scanning fashion. In the result, very fast moving objects can produce a "rolling shutter" effect if the subject moves during the tithe it takes from the start of the scan to the end of the scan. What this means is that object line-by-line scans may not line up and instead may be shifted along the scan axis.
For example, if straight lines are recorded while the camera is moved rapidly from side to side, the lines will appear to bend.
This effect can appear when the movement axis coincides with the scan axis. If a rolling shutter effect appears in the image, it may be possible to eliminate this effect by turning the phone so that the scan axis is perpendicular to the movement of the object. Motion of the camera lens relative to the subject may also produce choppy video. For this situation, optical or digital image stabilization techniques can be used.
Given the foregoing, we now provide an example of how to make and work the present invention in the context of visualizing vocal chord movements. A person skilled in the art would readily understand that the method disclosed hereinafter can be easily adjusted or modified to apply to the visualization of other rapid periodic movements, as desired.
Example The present example describes in detail the method used to visualize and capture vocal chord movements. In this respect, in this example the images and sound are captured by an iPhone attached to an adapter (such as that disclosed in International Patent Publication No. WO 2014/07667) that couples the iPhone to an endoscope, such as model Olympus ENF-P4.
As described above, and as shown in the more detailed logic tree displayed at Figure 5, the first step (1) is to record and sample a digitized voice by using the internal microphone of the iPhone (sound is sampled continuously and samples are stored in a sound sample array), and then determine the fundamental frequency of the voice samples (step 2). This is the most critical and complex step of the method of the present invention. The pseudo code shown at Figure 6 illustrates the method used to calculate the fundamental frequency. In particular, the time domain samples are transformed to the frequency domain and the most prominent frequency (max_ frequency) is determined for each sample. This max frequency is compared to a number of maximum frequencies that have been collected for previous samples. This step involves determining the new fundamental frequency by comparing the new max frequency to the previously collected max frequencies and determining if there is a shift (see Figure 7). If the frequency jump is above a MAX DELTA (10Hz by default; but a person skilled in the art could modify this in accordance with his/her needs), it rejects the jump and keeps the old fundamental frequency but appends the new max frequency so the algorithm adapts over time if the shift is persistent. This method adapts to small drifts, as present in human voices singing a single tone. At the same time, it suppresses spikes due to harmonics or noise.
As shown in Figure 5, the next step (3) is to calculate the freeze frame (still) capture rate from the max frequency established at (2). Firstly, we determine an appropriate natural number divisor, as previously described, to assist in establishing a freeze frame (still) capture rate. In this respect, for this example with vocal chords, we use the following equation:
N = (int) (max frequency / 30 ) Where, N is the natural number divisor to establish a still capture rate frequency around 30 Hz (30 is used for illustration because it has been found to work well with the iPhone) This is then used to determine the still capture rate in accordance with the following equation:
SCR = max frequency / N
Where, SCR is the precise floating point number representing the capture frequency required for a freeze frame capture. In other words, if the oscillation frequency is stable, the captured video will appear frozen in time Once we have the still or freeze frame capture rate (SCR), the user needs to decide if he/she desires motion or still frame capture (4). If freeze-frame viewing is desired, SCR is used as the sampling frequency (5). The user then determines if he/she wants to change the point of observation along the oscillation (6). The user, in a preferred embodiment, is able to determine the shift point (7) using a graphical slider control that maps the slider range from 0 to 360 to correspond to the phase shift of the displayed oscillation. The slider range corresponds to one full oscillation of the subject under observation. As the user changes the observation point through manipulation of the slider value from 0 to 360, the freeze frame appears at different points along the oscillation which allows for a deeper analysis of the movement. In the case of vocal cords, the medical professional is able to examine how the vocal cords appear at different points in the oscillation so anomalies can be detected on how vocal cords move when the sound is formed.
For example, if we have 60 samples per waveform, we have one sample per 6 degrees of shift (360 / 60). To shift 90 degrees, we shift the start of the waveform by 15 samples. 0 degrees corresponds to the start of the oscillation which is equivalent to a sample offset of 0. 360 corresponds to the end of the oscillation and is equivalent to 60 samples at which point the next oscillation starts again.
Phase shift and still capture rate settings are transferred to the camera controller (10), and the camera takes photos at the calculated frequency. The camera controller comprises all camera settings that are applied in photo and video mode. In addition, the camera controller reacts to user input for taking a picture and starting and stopping video recording. Settings that are changed in the camera controller are applied when the camera is active. For example, when the user enters preview mode for a picture, the brightness and exposure settings are applied so that the user can see what the picture will look like. The same settings are applied when the user takes the picture. In addition, in video mode, the capture rate is applied during preview and actual recording.
If, on the other hand, the user decides that he/she wants slow motion viewing at (4), then the speed of the motion should preferably be obtained from a graphical user interface control for "Slow motion speed setting" as depicted at (8) of Figure 5.
The value for speed is then translated to a capture rate offset in Hz, as previously described. For example, a relatively slow motion movement would translate into a sampling rate offset from the sampling frequency of approximately 0.5 Hz. The slow motion speed setting is directly mapped to fixed capture rate offset (CRO). For this example, the GUI value for slow motion is translated to a CRO of 0.5 Hz.
In our example, the capture rate offset is added to the still capture rate (SCR) to produce the slight offset needed for slow motion visualization.
SR (sampling rate) = SCR + CRO
The resulting capture rate settings are then transferred to the camera control module (10).
For the examination of vocal cords, it is important to note that the available light is relatively low. For this reason, it is preferable to automatically set the sensitivity to the highest setting (approx. 924 for the iPhone 6). The exposure needs to be relatively fast as the vocal cord vibration would otherwise appear blurry. It has been found that setting exposure to a minimum value of 1/1200 to 1/1500 s is appropriate. This setting may be done automatically at any time in vocal cord or other low light examinations (9). In brighter environments, sensor input from the camera can be used to automatically control ISO and exposure. Alternatively, ISO and exposure settings can be controlled by the user in manual mode. For our example, setting ISO directly to the maximum value of 924 and setting the exposure time to 1/1200 second works well with the devices used in this method for visualizing vocal chords. The exposure and ISO settings are transferred to the camera control module (10) for recording photos / video.
The user should also preferably be able to control when to start and stop recording through a user control button on the iPhone app screen.
In accordance with the method described above, Figure 3 shows a series of screen captures of a rotor running at 3500 RPM using a freeze frame capture rate. The rotor appears frozen in time.
Subsequent frames show a shift of the observation point through user control. This is an example of using no motion in accordance with the logic tree implementation described above.
Figure 4 shows a series of screen captures showing slow motion movement of a person's vocal cords when singing the letter "e"
(which is at a frequency of approximately 200 Hz). This is an ' example of the slow movement visualization in accordance with the logic tree implementation described above.
Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. The invention achieves multiple objectives and because the invention can be used in different applications for different purposes, not every embodiment falling within the scope of the attached claims will achieve every objective.
Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, processes, machines, manufactures, means, methods or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present invention.
Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, means, methods, or steps.
The computer readable memory may further comprise code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer readable memory may also further comprise code means for playing back any recorded capture of the vocal chords. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In another aspect, the present invention provides a computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye in a slower motion manner on a portable electronic device attached to an endoscope, comprising the steps of: sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, determining exposure and brightness settings for capture of slow motion photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting. The still capture rate may be based on a natural number divisor of the fundamental frequency. The computer-implemented method may further comprise the step of tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer-implemented method may also further comprise the step of playing back any recorded capture of the vocal chords. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In a further aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet another aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye in a slower motion manner on a portable electronic device attached to an endoscope, said statements and instructions comprising: code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, code means for determining exposure and brightness settings for capture of slow motion photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting. The computer readable memory may further comprise code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer readable memory may also further comprise code means for playing back any recorded capture of the vocal chords. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In another aspect, the present invention provides a computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye in a still freeze-frame manner on a portable electronic device attached to an endoscope, comprising the steps of: sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, setting a sampling frequency to the still capture rate, determining exposure and brightness settings for capture of still freeze-frame photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting. The still capture rate may be based on a natural number divisor of the fundamental frequency. The computer-implemented method may further comprise the step of tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer-implemented method may also further comprise the step of playing back any recorded capture of the vocal chords. In addition, the computer-implemented method may further comprise the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
In yet another aspect, the present invention provides a computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform any of the foregoing method steps.
In yet a further aspect, the present invention provides a computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye in a still freeze-frame manner on a portable electronic device attached to an endoscope, said statements and instructions comprising: code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for setting a sampling frequency to the still capture rate, code means for determining exposure and brightness settings for capture of still freeze-frame photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting. The computer readable memory may further comprise code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords. The computer readable memory may also further comprise code means for playing back any recorded capture of the vocal chords. In addition, the computer readable memory may further comprise code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
BRIEF DESCRIPTION OF THE DRAWINGS
Figure 1 is a Master Flow Diagram showing the general steps involved in the method of the present invention.
Figure 2 is a diagram showing vocal frequency with 141 capture rate.
Figure 3 is a series of screen captures of a rotor running at 3500 RPM using a freeze frame capture rate. The rotor appears frozen in time. Subsequent frames show a shift of the observation point through user control.
Figure 4 is a series of screen captures showing slow motion movement of the vocal cords when singing the letter "e" at a frequency of approximately 200 Hz.
Figure 5 is a detailed logic tree of the steps involved in the method of the present invention.
Figure 6 shows pseudocode for calculating the fundamental frequency.
Figure 7 shows the core method to determine the maximum frequency. The algorithm tracks small variations in frequency and is capable of suppressing frequency spikes.
DETAILED DESCRIPTION OF THE INVENTION
The following description is presented to enable a person skilled in the art to make and use the invention, and is provided in the context of a particular application and its requirements. Various modifications to the disclosed embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the scope of the invention.
Thus, the present invention is not intended to be limited to the embodiments disclosed, but is to be accorded the widest scope consistent with the principles and features disclosed herein.
As shown in Figure 1, the computer-implemented method of the present invention involves the following general steps (which steps may not always be necessary or in the following definitive order): 1) sampling the subject of examination (1) to determine periodicity of rapid movement (i.e. fundamental frequency, e.g.
Hz, RPM, or the like); 2) determining freeze-frame (still) capture rate (2) for desired visualization; 3) determining appropriate exposure and brightness setting (3) for the determined fundamental frequency; 4) determining whether freeze-frame or slow motion photographs and/or video (4) is to be captured and performing any necessary calculations accordingly;
and 5) capturing photographs and/or video of the subject of examination at the determined capture rate, exposure setting, and brightness level. The method of the present invention may involve the additional steps of 6) tracking the fundamental frequency for any changes over time and adjusting same while viewing and/or recording photographs and/or video of the subject of examination; and 7) playing back any recorded capture of the subject of examination.
Sampling the subject of examination (step 1) to determine periodicity of rapid movement (e.g. Hz, RPM, or the like) can be undertaken manually or more preferably automatically using an appropriate sensor. In this respect, the sensor may include any type of sensor that can adequately determine the periodicity of rapid movement or oscillation frequency. In the case of the visualization of vocal cords, for instance, the built-in microphone or a specialized Bluetooth microphone that can be located close to the sound source, can be used to collect sound samples of the vocal cord oscillations and send this data to an electronic device, such as a connected laptop computer or mobile electronic device such as an iPad, iPhone, or the like. The electronic device will apply a frequency tracking algorithm to determine the fundamental frequency of the vocal cord (see Figure 6). For rotational movements, an electromagnetic or optical sensor may be used to establish revolutions per second.
Air pressure sensors, mechanical switches, ultrasound sensors, etc. are just a few of the other numerous sensor technologies that can be used to determine and convey oscillation frequency of the observed vibration or rotational movement of the subject of examination, as appropriate.
Once periodicity/frequency of rapid movement has been determined, one can then determine the freeze-frame (still) capture rate (step 2) to implement the desired visualization. As to desired visualization (see step 4), by matching the capture rate appropriately to the periodic frequency of movement of the subject being examined (e.g. oscillation or rotational periodicity), one is capable of capturing freeze frame views of the subject (by capturing the same position in the cycle over time). In this respect, the capture rate may be tuned to a precise sub sampling frequency that matches a natural number divisor. For example, a vocal vibration of 240 Hz could be sampled every 4th oscillation using a capture frequency of 60 FPS
as shown in Figure 2. In this way, since the capture rate is precisely set to a natural number fraction of the observed frequency, the image will appear still. The following equation(s) therefore provide the capture rate needed to obtain freeze-frame/still viewing:
SCR = RPS / N
or SCR = F / N
Where, SCR is Still Capture Rate RPS is Rotations per second F is Frequency in Hz (Oscillations per second) N is a natural number, N = {1, 2, 3, .... XI
Most mobile computing devices are able to adjust the capture rate in a limited range up to a maximum of 120 or 240 FPS, but it is therefore possible to match the capture rate to the rate of oscillation or a natural number divisor thereof to give the appearance of freeze frame viewing.
It is important to note that the observed point in time can also be shifted to other points in the periodic movement through a phase shift of the sampling point. The phase shift can be specified in steps of 1 up to the sampling frequency. Phase shifting the sampling point allows for static analysis by moving the observation point in time. For example, an airplane rotor can be examined at different points along its movement path or a physician can inspect the larynx at different stages of the sound formation (see, for instance, Figures 3 and 4).
If, however, on the other hand, one wishes to see movement in an apparent slow-motion manner (see step 4), then the capture rate can be slightly offset/adjusted from the freeze-frame synchronization to provide the desired visualization in either an apparent forward or backward slow-motion manner. In this respect, to show observable movement, we need to introduce a small capture rate offset (CRO) that will shift the sampling point through the waveform at a speed that is observable by the human eye. If, for example, viewing playback is set at 30 FPS, we need 30 sampled frames per wavelength to observe one cycle per second. So, if a vocal cord vibration of 120 Hz is sampled at 60 Hz, it will capture 2 times the playback frame rate of 30 FPS, and the playback would show a full wavelength every 2 seconds. If the sampling rate is changed to 61 Hz, then the capture rate offset (CRO) is 1 Hz, and we would see slow-motion movement. The sampling frequency (SF; CRO multiplied by N) determines the number of discrete sampling points along the periodic movement. The higher the sampling frequency, the more discrete points are available. In the example provided above, the CRO would be 1. The number of frames (NF) is the oscillation speed or frequency divided by SF. The following equation(s) are therefore germane:
SF = N * CRO
Where, SF is Sampling frequency CRO is Capture rate offset N is a natural number divisor of the fundamental frequency for the sub-sampling rate, N = {1, 2, 3,_ X}
NF = F / SF
Where NF is Number of frames F is Frequency of Oscillation or RPS
SF is Sampling frequency As a person skilled in the art would appreciate, because N
multiplies the CRO, higher sampling frequencies are less sensitive to drift and allow for more precise control.
The capture rate offset (CRO) also determines the speed of the observable frequency. By controlling the CRO, the observable periodic movement can be made to appear faster or slower to the point of freezing the movement. Positive and negative offsets will produce observable movements in opposite directions. For example, a rotor would turn counter clockwise for positive CROs and clockwise for negative CROs. The ability to precisely control the speed and direction of the observable movement through fine control of the CRO allows for dynamic analysis in observable frequencies.
The frequency of the slow motion playback is dependent on the playback rate. In our example provided above, playback is set to 30 FPS. The observer would see a full wavelength motion with a = frequency of 0.5 Hz, or in other words, a full oscillation would take 2 seconds to display. The following equation is therefore germane:
PF = PR / NF
Where, PF is Playback frequency NF is Number of frames PR is Playback rate The speed of the visual movement depends on the offset from the freeze frame capture rate, the playback rate and the size of the divisor, which will have an amplification effect. A high playback rate would require more frames per second whereas a slower playback rate would require less. For example, in our above example of 60 frames for one sampled waveform, a playback rate (PR) of 30 Hz would require 30 frames per second. The playback frequency (PF) would be 0.5 Hz or 2 seconds per completed waveform. If PF was 60 Hz, PF would be 1 Hz or 1 second per waveform. The higher the number of frames per waveform, the slower the PF is. For small capture rate offsets (CRO), we achieve a relatively high number of frames per sampled waveform.
A visual approach may be used to adjust the capture rate manually via a user control. In this respect, once a user adjusts the capture rate to a point where it nears a natural number divisor of the oscillation frequency, the oscillation will move slower and slower until it finally appears frozen at the point where the capture rate matches a natural number divisor of the oscillation frequency. Manual adjustment can be difficult, however, as even relatively small offsets can display fast moving oscillations that the human eye cannot observe. If the frequency of movement is known, it may also be possible to manually set the capture rate or a natural number divisor of the oscillation frequency at a rate within the supported range of the electronic device in order to achieve the desired visualization.
Sensor input may also be used to establish the correct capture rate automatically. For vocal cord vibration at a specific sound such as when pronouncing the letter "e", for instance, a microphone can be used to establish the fundamental frequency, which is the main frequency at which the vocal cord vibrates for that sound. This fundamental frequency may then be automatically divided using a natural number, as discussed above, to set the capture rate in a range that is supported by the electronic device.
For the computer-implemented method of the present invention, it is also important to determine the appropriate exposure and brightness setting (step 3) for the determined capture rate of the subject of examination. Fast moving objects require a very short exposure time to avoid blurring. The maximum exposure time required is determined by the speed of the observed object moving through the frame. The faster the movement of the object, the shorter the exposure time required to avoid blurring. In this respect, a minimum factor of 10 times the observed periodic movement is required to avoid blurring. For movement at 200 to 400 Hz, this means an exposure time of 1/2000 to 1/4000 is required. The following equation is germane:
EXP = 1 / (P * F) Where, EXP is the Exposure time in seconds P is the Multiplication factor (minimum 10) F is the Frequency of periodic movement in Hz The Field of View (FOV) of the camera also influences the speed of the object within the frame. If a periodically moving object fills a frame because the object is closer to the camera or a telephoto lens is used, then the movement of the object appears faster than if the object is further away and only fills a portion of the screen.
A very short exposure time limits the amount of light that will arrive at the photo sensor. One way to compensate for the lack of light is to use a stronger light source. If this approach is not practical, then the sensitivity of the photo sensor has to be increased. The sensitivity setting of photo sensors are usually expressed in terms of ISO, and the higher the ISO the more sensitive the sensor and the less light is needed to fully expose the frame. The higher the sensitivity setting of the sensor, the higher the digital noise will be. At very high ISO
settings, visible grain can appear and image details may be lost.
The aperture or opening of the camera lens is fixed for most mobile computer cameras. However, if a variable aperture is available, the larger the opening, the more light will be available at the sensor. The aperture controls the depth of field, with a larger aperture producing a shallow depth of field and a small aperture providing a large depth of field. The shallower the depth of field, the smaller the range of the image that will be sharp.
Therefore, to produce a well-exposed, sharp image, the right exposure, ISO and aperture setting has to be used. The presented invention may include a method to optimize the exposure and ISO
settings by leveraging the sensor input to determine the frequency of the periodic movement. For low light situations, as is the case for endoscopic procedures, it is useful to use the maximum or very high ISO setting and an exposure of 1200 to 1/1500 seconds. Very bright scenes will use an exposure of 1/(10 * Fundamental Frequency) or the highest available capture rate (whichever is lower). The ISO setting can then automatically follow the brightness of the scene which can be determined automatically through the photo sensor or can be established manually through user input. Once the required exposure is determined through the sensor input, the maximum aperture and ISO are set accordingly. Cameras with a variable aperture may also automatically set the aperture to the maximum required according to the desired depth of field.
If the available camera controls and the scene lighting are optimized, digital methods are also available to increase the brightness of the image. The present invention may utilize automatic histogram equalization to increase the brightness of the image. A good description of histogram-based equalization can be found at:
https://en.wikipedia.org/wiki/Histogram equalization In addition, if the periodic movement of the subject being examined varies in frequency over time, then tracking changes to the frequency and making adjustments to the capture rate in real time become very important (and thus the need for an additional step of tracking the frequency of capture and adjusting same (step 6) while viewing and/or recording photographs and/or video of the subject of examination). The presented invention may utilize a frequency-tracking algorithm to calculate and set the capture frequency during video capture.
There are also other issues that at times may need to be considered. Digital sensor data is typically read line by line in a scanning fashion. In the result, very fast moving objects can produce a "rolling shutter" effect if the subject moves during the tithe it takes from the start of the scan to the end of the scan. What this means is that object line-by-line scans may not line up and instead may be shifted along the scan axis.
For example, if straight lines are recorded while the camera is moved rapidly from side to side, the lines will appear to bend.
This effect can appear when the movement axis coincides with the scan axis. If a rolling shutter effect appears in the image, it may be possible to eliminate this effect by turning the phone so that the scan axis is perpendicular to the movement of the object. Motion of the camera lens relative to the subject may also produce choppy video. For this situation, optical or digital image stabilization techniques can be used.
Given the foregoing, we now provide an example of how to make and work the present invention in the context of visualizing vocal chord movements. A person skilled in the art would readily understand that the method disclosed hereinafter can be easily adjusted or modified to apply to the visualization of other rapid periodic movements, as desired.
Example The present example describes in detail the method used to visualize and capture vocal chord movements. In this respect, in this example the images and sound are captured by an iPhone attached to an adapter (such as that disclosed in International Patent Publication No. WO 2014/07667) that couples the iPhone to an endoscope, such as model Olympus ENF-P4.
As described above, and as shown in the more detailed logic tree displayed at Figure 5, the first step (1) is to record and sample a digitized voice by using the internal microphone of the iPhone (sound is sampled continuously and samples are stored in a sound sample array), and then determine the fundamental frequency of the voice samples (step 2). This is the most critical and complex step of the method of the present invention. The pseudo code shown at Figure 6 illustrates the method used to calculate the fundamental frequency. In particular, the time domain samples are transformed to the frequency domain and the most prominent frequency (max_ frequency) is determined for each sample. This max frequency is compared to a number of maximum frequencies that have been collected for previous samples. This step involves determining the new fundamental frequency by comparing the new max frequency to the previously collected max frequencies and determining if there is a shift (see Figure 7). If the frequency jump is above a MAX DELTA (10Hz by default; but a person skilled in the art could modify this in accordance with his/her needs), it rejects the jump and keeps the old fundamental frequency but appends the new max frequency so the algorithm adapts over time if the shift is persistent. This method adapts to small drifts, as present in human voices singing a single tone. At the same time, it suppresses spikes due to harmonics or noise.
As shown in Figure 5, the next step (3) is to calculate the freeze frame (still) capture rate from the max frequency established at (2). Firstly, we determine an appropriate natural number divisor, as previously described, to assist in establishing a freeze frame (still) capture rate. In this respect, for this example with vocal chords, we use the following equation:
N = (int) (max frequency / 30 ) Where, N is the natural number divisor to establish a still capture rate frequency around 30 Hz (30 is used for illustration because it has been found to work well with the iPhone) This is then used to determine the still capture rate in accordance with the following equation:
SCR = max frequency / N
Where, SCR is the precise floating point number representing the capture frequency required for a freeze frame capture. In other words, if the oscillation frequency is stable, the captured video will appear frozen in time Once we have the still or freeze frame capture rate (SCR), the user needs to decide if he/she desires motion or still frame capture (4). If freeze-frame viewing is desired, SCR is used as the sampling frequency (5). The user then determines if he/she wants to change the point of observation along the oscillation (6). The user, in a preferred embodiment, is able to determine the shift point (7) using a graphical slider control that maps the slider range from 0 to 360 to correspond to the phase shift of the displayed oscillation. The slider range corresponds to one full oscillation of the subject under observation. As the user changes the observation point through manipulation of the slider value from 0 to 360, the freeze frame appears at different points along the oscillation which allows for a deeper analysis of the movement. In the case of vocal cords, the medical professional is able to examine how the vocal cords appear at different points in the oscillation so anomalies can be detected on how vocal cords move when the sound is formed.
For example, if we have 60 samples per waveform, we have one sample per 6 degrees of shift (360 / 60). To shift 90 degrees, we shift the start of the waveform by 15 samples. 0 degrees corresponds to the start of the oscillation which is equivalent to a sample offset of 0. 360 corresponds to the end of the oscillation and is equivalent to 60 samples at which point the next oscillation starts again.
Phase shift and still capture rate settings are transferred to the camera controller (10), and the camera takes photos at the calculated frequency. The camera controller comprises all camera settings that are applied in photo and video mode. In addition, the camera controller reacts to user input for taking a picture and starting and stopping video recording. Settings that are changed in the camera controller are applied when the camera is active. For example, when the user enters preview mode for a picture, the brightness and exposure settings are applied so that the user can see what the picture will look like. The same settings are applied when the user takes the picture. In addition, in video mode, the capture rate is applied during preview and actual recording.
If, on the other hand, the user decides that he/she wants slow motion viewing at (4), then the speed of the motion should preferably be obtained from a graphical user interface control for "Slow motion speed setting" as depicted at (8) of Figure 5.
The value for speed is then translated to a capture rate offset in Hz, as previously described. For example, a relatively slow motion movement would translate into a sampling rate offset from the sampling frequency of approximately 0.5 Hz. The slow motion speed setting is directly mapped to fixed capture rate offset (CRO). For this example, the GUI value for slow motion is translated to a CRO of 0.5 Hz.
In our example, the capture rate offset is added to the still capture rate (SCR) to produce the slight offset needed for slow motion visualization.
SR (sampling rate) = SCR + CRO
The resulting capture rate settings are then transferred to the camera control module (10).
For the examination of vocal cords, it is important to note that the available light is relatively low. For this reason, it is preferable to automatically set the sensitivity to the highest setting (approx. 924 for the iPhone 6). The exposure needs to be relatively fast as the vocal cord vibration would otherwise appear blurry. It has been found that setting exposure to a minimum value of 1/1200 to 1/1500 s is appropriate. This setting may be done automatically at any time in vocal cord or other low light examinations (9). In brighter environments, sensor input from the camera can be used to automatically control ISO and exposure. Alternatively, ISO and exposure settings can be controlled by the user in manual mode. For our example, setting ISO directly to the maximum value of 924 and setting the exposure time to 1/1200 second works well with the devices used in this method for visualizing vocal chords. The exposure and ISO settings are transferred to the camera control module (10) for recording photos / video.
The user should also preferably be able to control when to start and stop recording through a user control button on the iPhone app screen.
In accordance with the method described above, Figure 3 shows a series of screen captures of a rotor running at 3500 RPM using a freeze frame capture rate. The rotor appears frozen in time.
Subsequent frames show a shift of the observation point through user control. This is an example of using no motion in accordance with the logic tree implementation described above.
Figure 4 shows a series of screen captures showing slow motion movement of a person's vocal cords when singing the letter "e"
(which is at a frequency of approximately 200 Hz). This is an ' example of the slow movement visualization in accordance with the logic tree implementation described above.
Although the present invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made herein without departing from the spirit and scope of the invention as defined by the appended claims. The invention achieves multiple objectives and because the invention can be used in different applications for different purposes, not every embodiment falling within the scope of the attached claims will achieve every objective.
Moreover, the scope of the present application is not intended to be limited to the particular embodiments of the process, machine, manufacture, means, methods and steps described in the specification. As one of ordinary skill in the art will readily appreciate from the disclosure of the present invention, processes, machines, manufactures, means, methods or steps, presently existing or later to be developed that perform substantially the same function or achieve substantially the same result as the corresponding embodiments described herein may be utilized according to the present invention.
Accordingly, the appended claims are intended to include within their scope such processes, machines, manufacture, means, methods, or steps.
Claims (60)
1. A computer-implemented method for making rapid periodic movements of a subject of examination visible to the human eye, comprising the steps of:
sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, determining whether still or slow motion photographs or video of said subject is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, determining exposure and brightness settings for capture of the still or slow motion photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, determining whether still or slow motion photographs or video of said subject is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, determining exposure and brightness settings for capture of the still or slow motion photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
2. The computer-implemented method of claim 1 further comprising the step of tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject.
3. The computer-implemented method of one of claims 1 or 2 further comprising the step of playing back any recorded capture of the subject of examination.
4. The computer-implemented method of any one of claims 1 to 3 further comprising the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
5. The computer-implemented method of any one of claims 1 to 4 wherein the still capture rate is based on a natural number divisor of the fundamental frequency.
6. A computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of a subject of examination visible to the human eye, said statements and instructions comprising:
code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for determining whether still or slow motion photographs or video of said subject is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, code means for determining exposure and brightness settings for capture of the still or slow motion photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for determining whether still or slow motion photographs or video of said subject is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, code means for determining exposure and brightness settings for capture of the still or slow motion photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
7. The computer readable memory of claim 6 further comprising code means for tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject.
8. The computer readable memory of any one of claims 6 or 7 further comprising code means for playing back any recorded capture of the subject of examination.
9. The computer readable memory of any one of claims 6 to 8 further comprising code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
10.A computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform the method steps of any one of claims 1 to 5.
11. A computer-implemented method for making rapid periodic movements of a subject of examination visible to the human eye in a slower motion manner, comprising the steps of:
sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, determining exposure and brightness settings for capture of slow motion photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, determining exposure and brightness settings for capture of slow motion photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
12. The computer-implemented method of claim 11 further comprising the step of tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject.
13. The computer-implemented method of one of claims 11 or 12 further comprising the step of_playing back any recorded capture of the subject of examination.
14. The computer-implemented method of any one of claims 11 to 13 further comprising the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
15. The computer-implemented method of any one of claims 11 to 14 wherein the still capture rate is based on a natural number divisor of the fundamental frequency.
16. A computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of a subject of examination visible to the human eye in a slower motion manner, said statements and instructions comprising:
code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, code means for determining exposure and brightness settings for capture of slow motion photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, code means for determining exposure and brightness settings for capture of slow motion photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
17. The computer readable memory of claim 16 further comprising code means for tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject.
18. The computer readable memory of one of claims 16 or 17 further comprising code means for playing back any recorded capture of the subject of examination.
19. The computer readable memory of any one of claims 16 to 18 further comprising code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
20. A computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform the method steps of any one of claims 11 to 15.
21. A computer-implemented method for making rapid periodic movements of a subject of examination visible to the human eye in a still freeze-frame manner, comprising the steps of:
sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, setting a sampling frequency to the still capture rate, determining exposure and brightness settings for capture of still freeze-frame photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
sampling the subject of examination and determining a fundamental frequency of said subject, determining a still capture rate based on the fundamental frequency of said subject, setting a sampling frequency to the still capture rate, determining exposure and brightness settings for capture of still freeze-frame photographs or video of the subject, and capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
22. The computer-implemented method of claim 21 further comprising the step of tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject.
23. The computer-implemented method of one of claims 21 or 22 further comprising the step of playing back any recorded capture of the subject of examination.
24. The computer-implemented method of any one of claims 21 to 23 further comprising the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
25. The computer-implemented method of any one of claims 21 to 24 wherein the still capture rate is based on a natural number divisor of the fundamental frequency.
26. A computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of a subject of examination visible to the human eye in a still freeze-frame manner, said statements and instructions comprising:
code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for setting a sampling frequency to the still capture rate, code means for determining exposure and brightness settings for capture of still freeze-frame photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
code means for sampling the subject of examination and determining a fundamental frequency of said subject, code means for determining a still capture rate based on the fundamental frequency of said subject, code means for setting a sampling frequency to the still capture rate, code means for determining exposure and brightness settings for capture of still freeze-frame photographs or video of the subject, and code means for capturing photographs or video of the subject at the sampling frequency, exposure setting, and brightness setting.
27. The computer readable memory of claim 26 further comprising code means for tracking the fundamental frequency of the subject of examination over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the subject.
28. The computer readable memory of one of claims 26 or 27 further comprising code means for playing back any recorded capture of the subject of examination.
29. The computer readable memory of any one of claims 26 to 28 further comprising code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the subject of examination.
30. A computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform the method steps of any one of claims 21 to 25.
31. A computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye on a portable electronic device attached to an endoscope, comprising the steps of:
sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, determining whether still or slow motion photographs or video of said vocal chords is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, determining exposure and brightness settings for capture of the still or slow motion photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, determining whether still or slow motion photographs or video of said vocal chords is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, determining exposure and brightness settings for capture of the still or slow motion photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
32. The computer-implemented method of claim 31 further comprising the step of tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords.
33. The computer-implemented method of one of claims 31 or 32 further comprising the step of playing back any recorded capture of the vocal chords.
34. The computer-implemented method of any one of claims 31 to 33 further comprising the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
35. The computer-implemented method of any one of claims 31 to 34 wherein the still capture rate is based on a natural number divisor of the fundamental frequency.
36. A computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye on a portable electronic device attached to an endoscope, said statements and instructions comprising:
code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for determining whether still or slow motion photographs or video of said vocal chords is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, code means for determining exposure and brightness settings for capture of the still or slow motion photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for determining whether still or slow motion photographs or video of said vocal chords is to be captured and determining a sampling frequency needed for capturing said still or slow motion photographs or video, code means for determining exposure and brightness settings for capture of the still or slow motion photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
37. The computer readable memory of claim 36 further comprising code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords.
38. The computer readable memory of any one of claims 36 or 37 further comprising code means for playing back any recorded capture of the vocal chords.
39. The computer readable memory of any one of claims 36 to 38 further comprising code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
40. A computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform the method steps of any one of claims 31 to 35.
41. A computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye in a slower motion manner on a portable electronic device attached to an endoscope, comprising the steps of:
sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, determining exposure and brightness settings for capture of slow motion photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, determining exposure and brightness settings for capture of slow motion photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
42. The computer-implemented method of claim 41 further comprising the step of tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords.
43. The computer-implemented method of one of claims 41 or 42 further comprising the step of playing back any recorded capture of the vocal chords.
44. The computer-implemented method of any one of claims 41 to 43 further comprising the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
45. The computer-implemented method of any one of claims 41 to 44 wherein the still capture rate is based on a natural number divisor of the fundamental frequency.
46. A computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye in a slower motion manner on a portable electronic device attached to an endoscope, said statements and instructions comprising:
code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, code means for determining exposure and brightness settings for capture of slow motion photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for determining a capture rate offset from the still capture rate and setting a sampling frequency to a sum of the still capture rate and capture rate offset, code means for determining exposure and brightness settings for capture of slow motion photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
47. The computer readable memory of claim 46 further comprising code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate, capture rate offset and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords.
48. The computer readable memory of one of claims 46 or 47 further comprising code means for playing back any recorded capture of the vocal chords.
49. The computer readable memory of any one of claims 46 to 48 further comprising code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
50. A computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform the method steps of any one of claims 41 to 45.
51. A computer-implemented method for making rapid periodic movements of vocal chords visible to the human eye in a still freeze-frame manner on a portable electronic device attached to an endoscope, comprising the steps of:
sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, setting a sampling frequency to the still capture rate, determining exposure and brightness settings for capture of still freeze-frame photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
sampling the vocal chords and determining a fundamental frequency of said vocal chords, determining a still capture rate based on the fundamental frequency of said vocal chords, setting a sampling frequency to the still capture rate, determining exposure and brightness settings for capture of still freeze-frame photographs or video of the vocal chords, and capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
52. The computer-implemented method of claim 51 further comprising the step of tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords.
53. The computer-implemented method of one of claims 51 or 52 further comprising the step of playing back any recorded capture of the vocal chords.
54. The computer-implemented method of any one of claims 51 to 53 further comprising the step of phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
55. The computer-implemented method of any one of claims 51 to 54 wherein the still capture rate is based on a natural number divisor of the fundamental frequency.
56. A computer readable memory having recorded thereon statements and instructions for execution by a computer for making rapid periodic movements of vocal chords visible to the human eye in a still freeze-frame manner on a portable electronic device attached to an endoscope, said statements and instructions comprising:
code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for setting a sampling frequency to the still capture rate, code means for determining exposure and brightness settings for capture of still freeze-frame photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
code means for sampling the vocal chords and determining a fundamental frequency of said vocal chords, code means for determining a still capture rate based on the fundamental frequency of said vocal chords, code means for setting a sampling frequency to the still capture rate, code means for determining exposure and brightness settings for capture of still freeze-frame photographs or video of the vocal chords, and code means for capturing photographs or video of the vocal chords at the sampling frequency, exposure setting, and brightness setting.
57. The computer readable memory of claim 56 further comprising code means for tracking the fundamental frequency of the vocal chords over time and adjusting the still capture rate and sampling frequency to match any changes in the fundamental frequency for capturing photographs or video of the vocal chords.
58. The computer readable memory of one of claims 56 or 57 further comprising code means for playing back any recorded capture of the vocal chords.
59. The computer readable memory of any one of claims 56 to 58 further comprising code means for phase shifting the captured photographs or video at the sampling frequency to capture other photographs or video of the rapid periodic movement of the vocal chords.
60. A computer program product comprising a computer readable memory storing computer executable instructions thereon that when executed by a computer perform the method steps of any one of claims 51 to 55.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA2928401A CA2928401A1 (en) | 2016-04-28 | 2016-04-28 | A computer-implemented method for making rapid periodic movements visible to the human eye |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CA2928401A CA2928401A1 (en) | 2016-04-28 | 2016-04-28 | A computer-implemented method for making rapid periodic movements visible to the human eye |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CA2928401A1 true CA2928401A1 (en) | 2017-10-28 |
Family
ID=60156668
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CA2928401A Abandoned CA2928401A1 (en) | 2016-04-28 | 2016-04-28 | A computer-implemented method for making rapid periodic movements visible to the human eye |
Country Status (1)
| Country | Link |
|---|---|
| CA (1) | CA2928401A1 (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112422804A (en) * | 2019-08-20 | 2021-02-26 | 华为技术有限公司 | Video special effect generation method and terminal |
-
2016
- 2016-04-28 CA CA2928401A patent/CA2928401A1/en not_active Abandoned
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112422804A (en) * | 2019-08-20 | 2021-02-26 | 华为技术有限公司 | Video special effect generation method and terminal |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102739955B (en) | Image capturing apparatus for enabling generation of data of panoramic image with wide dynamic range | |
| JP5589548B2 (en) | Imaging apparatus, image processing method, and program storage medium | |
| JP2018513640A (en) | Automatic panning shot generation | |
| CN104717422B (en) | Show equipment and display methods | |
| KR102640227B1 (en) | System and method for creating background blur in camera panning or motion | |
| JP2017520135A (en) | Method and apparatus for acquiring images with motion blur | |
| US11190703B2 (en) | Image-capturing apparatus, program, and electronic device that controls image sensor based on moving velocity | |
| US20120197079A1 (en) | Control device, endoscope apparatus, aperture control method, and information storage medium | |
| US11470253B2 (en) | Display device and program | |
| JP2017208073A (en) | Composing and realizing viewer's interaction with digital media | |
| JP2016530827A (en) | System, device and method for tracking objects on a display | |
| US20190239725A1 (en) | Endoscope apparatus, control method of endoscope apparatus, and recording medium | |
| JP2005354166A (en) | Image processing apparatus, image processing system, image processing method, and storage medium | |
| CA2928401A1 (en) | A computer-implemented method for making rapid periodic movements visible to the human eye | |
| WO2019026923A1 (en) | Image processing device, image processing method, image processing program, and image capture device | |
| US20230308758A1 (en) | Systems and methods for assessing stabilization of videos | |
| US20190052791A1 (en) | Image processing apparatus and image processing method | |
| JP5608803B1 (en) | Imaging device, imaging method, and imaging control program | |
| Mullen et al. | How to shoot and edit high-quality surgical videos for hand and upper extremity surgery | |
| JP5377073B2 (en) | Imaging apparatus, blur repair method, and program | |
| JP2016099598A (en) | Imaging apparatus, method for controlling the same, and control program | |
| JP2010258765A (en) | Imaging apparatus | |
| EP3059951A1 (en) | Method and apparatus for generating lens-related metadata | |
| US20160127708A1 (en) | Method for recording and processing at least one video sequence comprising at least one video track and a sound track | |
| JP2015149611A (en) | Photographing apparatus, photographing support method, and photographing support program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| FZDE | Discontinued |
Effective date: 20220301 |