US20190305759A1 - Navigation device with track smoothing - Google Patents
Navigation device with track smoothing Download PDFInfo
- Publication number
- US20190305759A1 US20190305759A1 US16/446,699 US201916446699A US2019305759A1 US 20190305759 A1 US20190305759 A1 US 20190305759A1 US 201916446699 A US201916446699 A US 201916446699A US 2019305759 A1 US2019305759 A1 US 2019305759A1
- Authority
- US
- United States
- Prior art keywords
- remainder
- quotient
- navigation device
- buffer
- digital filter
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Classifications
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03H—IMPEDANCE NETWORKS, e.g. RESONANT CIRCUITS; RESONATORS
- H03H17/00—Networks using digital techniques
- H03H17/02—Frequency selective networks
- H03H17/0248—Filters characterised by a particular frequency response or filtering method
-
- H—ELECTRICITY
- H03—ELECTRONIC CIRCUITRY
- H03H—IMPEDANCE NETWORKS, e.g. RESONANT CIRCUITS; RESONATORS
- H03H17/00—Networks using digital techniques
- H03H17/02—Frequency selective networks
- H03H17/0248—Filters characterised by a particular frequency response or filtering method
- H03H17/026—Averaging filters
Definitions
- This disclosure generally relates to a navigation device and, more particularly, to a navigation device capable of performing track smoothing and a track smoothing method thereof.
- an optical mouse is one of the peripheral devices for interacting with the computer system having a display screen.
- the optical mouse generally includes a light source, an image sensor and a processing unit.
- the light source emits light to a working surface.
- the image sensor receives reflected light from the working surface and successively outputs image frames.
- the processing unit calculates a displacement of the optical mouse with respect to the working surface according to the image frames and sends the displacement to a computer system at a report rate for corresponding control.
- the image frames captured by the image sensor may contain noise which can reflect in the displacement calculated by the processing unit.
- the displacement will not be obviously affected by a position offset caused by the noise when the displacement is relatively larger than the position offset, when the displacement is very small and substantially identical to or just a little bit larger than the position offset, the track can be disturbed to have a zigzag path.
- the unsmooth track becomes obvious and can be easily noticed by the operator.
- the present disclosure provides a navigation device capable of performing track smoothing and a track smoothing method thereof.
- the present disclosure provides a navigation device and a track smoothing method thereof that adopt an N-tap filter to perform the averaging operation of input data.
- the present disclosure further provides a navigation device and a track smoothing method thereof that may adjust a smoothing strength according to the resolution setting.
- the present disclosure provides a navigation device configured to detect motion and output smoothed displacement with respect to a working surface upon which the navigation device is operated.
- the navigation device includes an image sensor and a processor.
- the image sensor is configured to sense reflection light beams from the working surface and output input data.
- the processor has a digital filter which is coupled to the image sensor to receive the input data.
- the digital filter includes a delay buffer, an accumulation buffer, a divider, a remainder buffer and an adder.
- the delay buffer has multiple taps configured to sequentially receive a data series of the input data to be stored in the multiple taps from the image sensor, and output a tap sum, which is a sum of each data of the data series stored in the multiple taps.
- the accumulation buffer is configured to store an accumulation value.
- the divider is coupled to the accumulation buffer to receive the accumulation value, and configured to calculate a quotient and a remainder by dividing the accumulation value by an average number.
- the remainder buffer is coupled to the divider to receive the remainder, wherein the divider sets a calculated non-zero remainder to 0 when the quotient is 0, while the remainder is stored in the remainder buffer only when the quotient is larger than 0.
- the adder is coupled to the delay buffer, the remainder buffer and the accumulation buffer, and configured to add the tap sum and the remainder stored in the remainder buffer to generate the accumulation value when the quotient is larger than 0.
- the digital filter is configured to output the quotient as the smoothed displacement to control a cursor or a pointer on a display.
- the present disclosure further provides a track smoothing method of a navigation device.
- the navigation device detects motion and outputs smoothed displacement with respect to a working surface upon which the navigation device is operated.
- the navigation device includes an image sensor for sensing reflection light beams from the working surface to output input data and a processor which has a digital filter with an average number and coupled to the image sensor to receive the input data.
- the track smoothing method includes the steps of: receiving a data series of the input data from the image sensor to be sequentially stored in multiple taps of the digital filter; calculating a tap sum of each data of the data series of the input data stored in every tap in the digital filter; updating an accumulation value with the tap sum; calculating a quotient and a remainder by dividing the accumulation value by the average number; setting a calculated non-zero remainder to 0 when the quotient is equal to 0, while storing the remainder in a remainder buffer only when the quotient is larger than 0; and outputting the quotient as the smoothed displacement to control a cursor or a pointer on a display.
- the present disclosure further provides a navigation device configured to detect motion and output smoothed displacement with respect to a working surface upon which the navigation device is operated.
- the navigation device includes an image sensor and a processor.
- the image sensor is configured to receive reflection light beams from the working surface and output input data.
- the processor has a digital filter which is coupled to the image sensor to receive a data series of the input data.
- the digital filter includes a divider and an adder.
- the divider is configured to calculate a quotient and a remainder by dividing an accumulation value by an average number, wherein the divider sets a calculated non-zero remainder to 0 when the quotient is 0, while the remainder is stored in a remainder buffer only when the quotient is larger than 0.
- the adder is configured to add a next tap sum to the accumulation value stored in an accumulation buffer to update the accumulation value when the quotient is equal to 0, and add the next tap sum and the remainder stored in the remainder buffer to generate a new accumulation value when the quotient is larger than 0.
- the next tap sum is a sum of each data of the data series of the input data stored in every of multiple taps in the digital filter received from the image sensor, and the data series of the input data are sequentially stored in the multiple taps.
- the digital filter is configured to output the quotient as the smoothed displacement to control a cursor or a pointer on a display.
- the digital filter is, for example, an N-tap filter, wherein N is a delay number and an average number.
- the tap sum is a sum of data of every tap in the digital filter.
- the digital filter is adjustable to an 8-tap filter, a 16-tap filter, a 32-tap filter, a 64-tap filter and so on according to a required average strength.
- FIG. 1 is a schematic block diagram of a navigation device according to one embodiment of the present disclosure.
- FIG. 2 is an operational schematic diagram of a navigation device according to one embodiment of the present disclosure.
- FIGS. 3-4 are flow charts of the track smoothing method of a navigation device according to some embodiments of the present disclosure.
- FIG. 1 it is a schematic block diagram of a navigation device according to one embodiment of the present disclosure.
- the navigation device 1 of the present disclosure is, for example, operated on a working surface S, and configured to detect the motion with respect to the working surface S and output smoothed displacement. It should be mentioned that although in FIG. 1 the navigation device 1 is shown to be operated on the working surface S, the present disclosure is not limited thereto.
- the navigation device 1 is also adaptable to handheld navigation devices such as a remote device, a pointing device or the like without particular limitations.
- the navigation device 1 includes a light source 20 , an image sensor 30 and a digital filter 10 , wherein the digital filter 10 is, for example, included in a processor.
- the light source 20 , the image sensor 30 and the digital filter 10 are all included in an image sensor package such as an image sensing chip, and the image sensor package is configured to output a smoothed track.
- the light source 20 and the image sensor 30 are included in an image sensor package but the digital filter 10 is in an external processor, e.g. a central processing unit (CPU), a microcontroller (MCU) or the like outside of the image sensor package.
- CPU central processing unit
- MCU microcontroller
- the light source 20 is, for example, a coherent light source, a partially coherent light source or a non-coherent light source such as a light emitting diode or a laser diode, but not limited thereto.
- the light source 20 is configured to emit light of an identifiable spectrum to illuminate the working surface S.
- the image sensor 30 includes, for example, a CCD image sensing element, a CMOS image sensing element or other optoelectronic elements capable of detecting light signals and output electrical signals.
- the image sensor 30 is configured to receive reflected light from the working surface S to generate digital image frames (e.g. including an analog-to-digital converter therein for digitizing), and calculate displacement according to the digital image frames, e.g. calculating according to the correlation between image frames.
- the displacement data outputted by the image sensor 30 is not smoothed.
- the digital filter 10 receives the displacement data outputted from the image sensor 30 and performs the smoothing process so as to output smoothed displacement data.
- FIG. 1 shows that the image sensor 30 and the light source 20 are arranged symmetrical to a normal line of the working surface S for receiving a main reflection light beam Lr from the working surface S, also referred as a bright field arrangement, but the present disclosure is not limited thereto.
- the image sensor 30 and the light source 20 may form a dark field arrangement, i.e. the image sensor 30 and the light source 20 not symmetrical to the normal line of the working surface S, and in this case the image sensor 30 is configured to receive scattered light from the working surface S.
- FIG. 2 is an operational schematic diagram of a navigation device according to one embodiment of the present disclosure.
- the digital filter 10 includes a delay buffer 101 , an adder 103 , an accumulation buffer 105 , a calculation unit 107 and a remainder buffer 109 , wherein every element in the digital filter 10 may be implemented by software and/or firmware without particular limitation.
- the digital filter 10 is configured to receive input data from the image sensor 30 and generate output data after the smoothing process. For example, FIG.
- FIG. 2 shows a data series of the input data [1 ⁇ 1 0 1 1 1 3 2 1 2 0 0 0] which are sequentially inputted into the digital filter 10 at times t 0 to t 12 , wherein each data of the data series is, for example, displacement data and the negative sign indicates an opposite direction.
- a movement of the navigation device 1 on the working surface S may be two-dimensional displacement
- FIG. 2 shows only one-dimensional displacement data (e.g. X-axis), and the operation of another dimension (e.g. Y-axis) is identical to that of FIG. 2 and thus details thereof are not repeated herein.
- the processor including the digital filter 10 calculates a sum of vector thereof using vector arithmetic and outputs the movement corresponding to the sum of vector according to the resolution setting.
- the delay buffer 101 has a delay number and is configured to sequentially receive input data and output a tap sum. In one embodiment, when a number of input data is larger than the delay number, the most first input data is firstly removed from the delay buffer 101 so as to form a first-in first-out operation.
- the digital filter 10 is configured to calculate a quotient and a remainder by dividing an accumulation value by an average number. In one embodiment, when the quotient is smaller than 1 (e.g. equal to 0), the digital filter 10 adds next tap sum to the accumulation value so as to update the accumulation value; whereas, when the quotient is larger than 0, the digital filter 10 adds the next tap sum and the remainder to be served as a new accumulation value.
- the digital filter 10 divides the updated accumulation value or the new accumulation value by the average number.
- the smoothing of the input data is performed by repeatedly calculating in this way.
- the digital filter 10 adds the next tap sum and the remainder to be served as a new accumulation value (illustrated with examples below).
- the delay number is equal to the average number.
- the tap sum is a sum of data of every tap (e.g. Tap1 to Tap8) in the delay buffer 101 .
- the tap sum is a sum of two data in Tap1 and Tap2 as well as data in Tap3 to Tap8.
- the tap sum at times t 2 to t 12 is obtainable in a similar way.
- the accumulation buffer 105 is configured to store an accumulation value, which is updated at every clock cycle (e.g. t 0 to t 12 ).
- the calculation unit 107 is implemented, for example, by a divider which is configured to calculate a quotient and a remainder by dividing the accumulation value by the average number N, wherein the digital filter 10 outputs the quotient as the output data (e.g. smoothed displacement data), and the remainder may be stored in the remainder buffer 109 .
- the adder 103 is configured to add the tap sum and the remainder so as to generate the accumulation value.
- the accumulation value is obtainable according to at least two ways as illustrated below with examples.
- a method of updating the accumulation value is determined according to the quotient. For example, when the quotient is smaller than 1 (e.g. equal to 0), the calculation unit 107 sets the remainder to 0, and a next tap sum is directly added to the accumulation value already stored in the accumulation buffer 105 so as to update the accumulation value; whereas, when the quotient is larger than 0, the adder 103 adds the next tap sum and the remainder to be served as a new accumulation value, which is then stored in the accumulation buffer 105 .
- the tap sum is calculated as “1”. It is assumed that an initial accumulation value is “0”, and the accumulation value is equal to “1” which is calculated by adding the tap sum “1” to the initial accumulation value “0”.
- the calculation unit 107 divides the accumulation value “1” by the average number N so as to obtain a quotient equal to “0” and a remainder equal to “1”. Meanwhile, as the quotient is equal to “0”, the calculation unit 107 either sets the remainder to “0” or does not store the remainder to the remainder buffer 109 . For example, the digital filter 10 resets the remainder stored in the remainder buffer 109 to “0” when the quotient is smaller than 1. Meanwhile, the digital filter 10 outputs zero displacement.
- the tap sum is calculated as “0”.
- the accumulation value is still equal to “1” which is calculated by adding a next (i.e. time t 1 ) tap sum “0” to the accumulation value “1” already stored in the accumulation buffer 105 .
- the calculation unit 107 performs the division operation so as to obtain another quotient equal to “0” and another remainder equal to “1”. Similarly, as the quotient is equal to “0”, the calculation unit 107 sets the remainder to “0” and the digital filter 10 outputs zero displacement.
- the accumulation value may be calculated by adding the next tap sum, the remainder and the accumulation value, and the calculation result is not changed.
- the calculation unit 107 may not set the remainder to 0 as long as the remainder is ignored in calculating the accumulation value, e.g. disable the adder 103 or not reading the accumulation buffer 109 .
- the calculation unit 107 obtains a quotient equal to “1” (larger than 0) and a remainder equal to “5” at time t 6 , the accumulation value is equal to “13” which is calculated by adding the tap sum “8” and a previous (i.e. at time t 6 ) remainder “5”.
- the calculation unit 107 performs the division operation so as to obtain another quotient equal to “1” and another remainder equal to “5”. More specifically, as the quotient is not equal to “0” at time t 6 , the remainder buffer 109 is not reset.
- the digital filter 10 receives one input data at each clock time t 1 to t 12 and obtains a quotient and a remainder after the averaging operation (i.e. dividing the accumulation value by the average value).
- the remainder is stored in the remainder buffer 109 for updating.
- the quotient is smaller than 1 (e.g. equal to 0)
- the remainder may be stored in the remainder buffer 109 or the storage of the remainder buffer 109 may be reset to 0. More specifically, in this embodiment, when the quotient is smaller than 1, the digital filter 10 outputs zero displacement and resets the storage in the remainder buffer 109 to zero or ignores the remainder (i.e. the remainder stored or not stored in the remained buffer 109 ); whereas, when the quotient is larger than 0, the digital filter 10 outputs nonzero smoothed displacement and the obtained remainder is stored in the remainder buffer 109 .
- the accumulation value is always a sum of the tap sum and the remainder. That is, the calculation unit 107 updates the remainder stored in the remainder buffer 109 at each clock cycle (e.g. t 0 to t 12 ).
- the tap sum is calculated as “1”. It is assumed that an initial remainder is “0”, and the accumulation value is equal to “1” which is calculated by adding the tap sum “1” and the initial remainder “0”.
- the calculation unit 107 divides the accumulation value “1” by the average number N so as to obtain a quotient equal to “0” and a remainder equal to “1”. Meanwhile, the digital filter 10 outputs zero displacement and the remainder “1” is stored in the remainder buffer 109 .
- the tap sum is calculated as “0”.
- the accumulation value is still equal to “1” which is calculated by adding a next (i.e. time t 1 ) tap sum “0” and the remainder “1” already stored in the remainder buffer 109 .
- the calculation unit 107 also performs the division operation so as to obtain another quotient equal to “0” and another remainder equal to “1”. Accordingly, the digital filter 10 outputs zero displacement and the remainder “1” is used to update the remainder in the reminder buffer 109 .
- the digital filter 10 outputs the quotient Q obtained by the calculation unit 107 to be served as the smoothed displacement data.
- a delay time exists between a time (e.g. t 0 ) that the navigation device 1 starts to move and a time (e.g. t 6 ) that the digital filter 10 starts to output nonzero displacement, e.g. 6 clock cycles being delayed in FIG. 2 , wherein an actual delay time is determined according to the input data and not limited to that shown in FIG. 2 .
- an actual delay time is determined according to the input data and not limited to that shown in FIG. 2 .
- the digital filter 10 directly sets the output data to 0 when a predetermined number of successive input data is 0, wherein the predetermined number is preferably smaller than the average number N, but not limited thereto.
- said delay time may not influence the operation such that the digital filter 10 may continuously output the output data (i.e. quotient Q) obtained by the calculation unit 107 without setting the output data to 0.
- the average number N is positively correlated with a resolution of the navigation device 1 .
- a resolution of the navigation device 1 When the resolution is higher, the influence on the navigation device 1 from the noise or the working surface S becomes more obvious, and thus a higher smoothing strength (i.e. higher N) is selected so as to increase the track smooth.
- the average number N may be adjusted to 8, 16, 32 or 64 corresponding to different resolutions.
- the average number N is correspondingly adjusted.
- said delay time is shorter and the digital filter 10 may be arranged to continuously output the output data obtained by the calculation unit 107 without setting the output data to 0.
- the track smoothing method includes the steps of: calculating a tap sum of every tap in a digital filter (Step S 31 ); updating an accumulation value with the tap sum (Step S 32 ); calculating a quotient and a remainder by dividing the accumulation value by an average number (Step S 33 ); and outputting the quotient to be served as output data (Step S 34 ).
- Step S 31 As shown in FIG. 2 , the delay buffer 10 is configured to output a tap sum of every tap therein (e.g. Tap1 to Tap8), and a tap number is determined according to the required smoothing strength.
- a tap sum of every tap therein e.g. Tap1 to Tap8
- Step S 32 an accumulation value is obtainable according to at least two ways.
- a quotient Q is smaller than 1 (e.g. equal to 0)
- the tap sum is directly added to the accumulation value stored in the accumulation buffer 105 (Step S 321 shown in FIG. 4 ) for updating, and a remainder R may be set to 0 or ignored; whereas, when the quotient Q is larger than 0, the adder 103 adds the remainder R to the tap sum (Step S 322 shown in FIG. 4 ) to be served as a new accumulation value which is then stored in the accumulation buffer 105 .
- the accumulation value is always a sum of the tap sum and the remainder R.
- Step S 33 Next, the calculation unit 107 divides the accumulation value by the average number N so as to obtain the quotient Q and the remainder R.
- Step S 34 The navigation device 10 outputs the quotient Q to be served as output data (i.e. smoothed displacement data).
- the remainder R may be stored in the remainder buffer 109 , ignored or set to 0 according to different applications.
- the track smoothing method of this embodiment may be referred to FIGS. 1 to 2 and corresponding descriptions, and thus details thereof are not repeated herein.
- the track smoothing method of this embodiment may also include the step: setting the output data to 0 when a predetermined number of successive input data is 0.
- values in the above embodiment such as the average number, delay number, resolution, input data, output data, tap sum, accumulation value, remainder and quotient are only intended to illustrate but not to limit the present disclosure.
- values shown in FIG. 2 are all integers, the present disclosure is not limited thereto, and the values in FIG. 2 may also be decimals.
- the present disclosure further provides a navigation device ( FIG. 1 ) and a track smoothing method thereof ( FIGS. 3-4 ) that may smooth the track of a plurality of input data and the smoothing strength is adjustable according to the device resolution thereby increasing the adaptability thereof.
Landscapes
- Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Computer Hardware Design (AREA)
- Mathematical Physics (AREA)
- Image Processing (AREA)
Abstract
Description
- This application is a continuation application of U.S. Ser. No. 14/605,007, filed on Jan. 26, 2015, the full disclosure of which is incorporated herein by reference.
- This disclosure generally relates to a navigation device and, more particularly, to a navigation device capable of performing track smoothing and a track smoothing method thereof.
- Conventionally, an optical mouse is one of the peripheral devices for interacting with the computer system having a display screen. The optical mouse generally includes a light source, an image sensor and a processing unit. The light source emits light to a working surface. The image sensor receives reflected light from the working surface and successively outputs image frames. The processing unit calculates a displacement of the optical mouse with respect to the working surface according to the image frames and sends the displacement to a computer system at a report rate for corresponding control.
- However, as the image frames captured by the image sensor may contain noise which can reflect in the displacement calculated by the processing unit. Although the displacement will not be obviously affected by a position offset caused by the noise when the displacement is relatively larger than the position offset, when the displacement is very small and substantially identical to or just a little bit larger than the position offset, the track can be disturbed to have a zigzag path. Particularly in the high resolution setting, the unsmooth track becomes obvious and can be easily noticed by the operator.
- Accordingly, the present disclosure provides a navigation device capable of performing track smoothing and a track smoothing method thereof.
- The present disclosure provides a navigation device and a track smoothing method thereof that adopt an N-tap filter to perform the averaging operation of input data.
- The present disclosure further provides a navigation device and a track smoothing method thereof that may adjust a smoothing strength according to the resolution setting.
- The present disclosure provides a navigation device configured to detect motion and output smoothed displacement with respect to a working surface upon which the navigation device is operated. The navigation device includes an image sensor and a processor. The image sensor is configured to sense reflection light beams from the working surface and output input data. The processor has a digital filter which is coupled to the image sensor to receive the input data. The digital filter includes a delay buffer, an accumulation buffer, a divider, a remainder buffer and an adder. The delay buffer has multiple taps configured to sequentially receive a data series of the input data to be stored in the multiple taps from the image sensor, and output a tap sum, which is a sum of each data of the data series stored in the multiple taps. The accumulation buffer is configured to store an accumulation value. The divider is coupled to the accumulation buffer to receive the accumulation value, and configured to calculate a quotient and a remainder by dividing the accumulation value by an average number. The remainder buffer is coupled to the divider to receive the remainder, wherein the divider sets a calculated non-zero remainder to 0 when the quotient is 0, while the remainder is stored in the remainder buffer only when the quotient is larger than 0. The adder is coupled to the delay buffer, the remainder buffer and the accumulation buffer, and configured to add the tap sum and the remainder stored in the remainder buffer to generate the accumulation value when the quotient is larger than 0. The digital filter is configured to output the quotient as the smoothed displacement to control a cursor or a pointer on a display.
- The present disclosure further provides a track smoothing method of a navigation device. The navigation device detects motion and outputs smoothed displacement with respect to a working surface upon which the navigation device is operated. The navigation device includes an image sensor for sensing reflection light beams from the working surface to output input data and a processor which has a digital filter with an average number and coupled to the image sensor to receive the input data. The track smoothing method includes the steps of: receiving a data series of the input data from the image sensor to be sequentially stored in multiple taps of the digital filter; calculating a tap sum of each data of the data series of the input data stored in every tap in the digital filter; updating an accumulation value with the tap sum; calculating a quotient and a remainder by dividing the accumulation value by the average number; setting a calculated non-zero remainder to 0 when the quotient is equal to 0, while storing the remainder in a remainder buffer only when the quotient is larger than 0; and outputting the quotient as the smoothed displacement to control a cursor or a pointer on a display.
- The present disclosure further provides a navigation device configured to detect motion and output smoothed displacement with respect to a working surface upon which the navigation device is operated. The navigation device includes an image sensor and a processor. The image sensor is configured to receive reflection light beams from the working surface and output input data. The processor has a digital filter which is coupled to the image sensor to receive a data series of the input data. The digital filter includes a divider and an adder. The divider is configured to calculate a quotient and a remainder by dividing an accumulation value by an average number, wherein the divider sets a calculated non-zero remainder to 0 when the quotient is 0, while the remainder is stored in a remainder buffer only when the quotient is larger than 0. The adder is configured to add a next tap sum to the accumulation value stored in an accumulation buffer to update the accumulation value when the quotient is equal to 0, and add the next tap sum and the remainder stored in the remainder buffer to generate a new accumulation value when the quotient is larger than 0. The next tap sum is a sum of each data of the data series of the input data stored in every of multiple taps in the digital filter received from the image sensor, and the data series of the input data are sequentially stored in the multiple taps. The digital filter is configured to output the quotient as the smoothed displacement to control a cursor or a pointer on a display.
- In the navigation device and the track smoothing method according to the present disclosure, the digital filter is, for example, an N-tap filter, wherein N is a delay number and an average number. In one embodiment, the tap sum is a sum of data of every tap in the digital filter. In one embodiment, the digital filter is adjustable to an 8-tap filter, a 16-tap filter, a 32-tap filter, a 64-tap filter and so on according to a required average strength.
- Other objects, advantages, and novel features of the present disclosure will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings.
-
FIG. 1 is a schematic block diagram of a navigation device according to one embodiment of the present disclosure. -
FIG. 2 is an operational schematic diagram of a navigation device according to one embodiment of the present disclosure. -
FIGS. 3-4 are flow charts of the track smoothing method of a navigation device according to some embodiments of the present disclosure. - It should be noted that, wherever possible, the same reference numbers will be used throughout the drawings to refer to the same or like parts.
- Referring to
FIG. 1 , it is a schematic block diagram of a navigation device according to one embodiment of the present disclosure. Thenavigation device 1 of the present disclosure is, for example, operated on a working surface S, and configured to detect the motion with respect to the working surface S and output smoothed displacement. It should be mentioned that although inFIG. 1 thenavigation device 1 is shown to be operated on the working surface S, the present disclosure is not limited thereto. Thenavigation device 1 is also adaptable to handheld navigation devices such as a remote device, a pointing device or the like without particular limitations. - The
navigation device 1 includes alight source 20, animage sensor 30 and adigital filter 10, wherein thedigital filter 10 is, for example, included in a processor. In some embodiments, thelight source 20, theimage sensor 30 and thedigital filter 10 are all included in an image sensor package such as an image sensing chip, and the image sensor package is configured to output a smoothed track. In other embodiments, thelight source 20 and theimage sensor 30 are included in an image sensor package but thedigital filter 10 is in an external processor, e.g. a central processing unit (CPU), a microcontroller (MCU) or the like outside of the image sensor package. It should be mentioned that when thenavigation device 1 is adapted to a handheld navigation device, thelight source 20 may not be implemented. - The
light source 20 is, for example, a coherent light source, a partially coherent light source or a non-coherent light source such as a light emitting diode or a laser diode, but not limited thereto. Thelight source 20 is configured to emit light of an identifiable spectrum to illuminate the working surface S. - The
image sensor 30 includes, for example, a CCD image sensing element, a CMOS image sensing element or other optoelectronic elements capable of detecting light signals and output electrical signals. Theimage sensor 30 is configured to receive reflected light from the working surface S to generate digital image frames (e.g. including an analog-to-digital converter therein for digitizing), and calculate displacement according to the digital image frames, e.g. calculating according to the correlation between image frames. The displacement data outputted by theimage sensor 30 is not smoothed. In the present disclosure, thedigital filter 10 receives the displacement data outputted from theimage sensor 30 and performs the smoothing process so as to output smoothed displacement data. - It should be mentioned that although
FIG. 1 shows that theimage sensor 30 and thelight source 20 are arranged symmetrical to a normal line of the working surface S for receiving a main reflection light beam Lr from the working surface S, also referred as a bright field arrangement, but the present disclosure is not limited thereto. In other embodiments, theimage sensor 30 and thelight source 20 may form a dark field arrangement, i.e. theimage sensor 30 and thelight source 20 not symmetrical to the normal line of the working surface S, and in this case theimage sensor 30 is configured to receive scattered light from the working surface S. - Referring to
FIGS. 1 and 2 together,FIG. 2 is an operational schematic diagram of a navigation device according to one embodiment of the present disclosure. Thedigital filter 10 includes adelay buffer 101, anadder 103, anaccumulation buffer 105, acalculation unit 107 and aremainder buffer 109, wherein every element in thedigital filter 10 may be implemented by software and/or firmware without particular limitation. Thedigital filter 10 is configured to receive input data from theimage sensor 30 and generate output data after the smoothing process. For example,FIG. 2 shows a data series of the input data [1 −1 0 1 1 1 3 2 1 2 0 0 0] which are sequentially inputted into thedigital filter 10 at times t0 to t12, wherein each data of the data series is, for example, displacement data and the negative sign indicates an opposite direction. It should be mentioned that a movement of thenavigation device 1 on the working surface S may be two-dimensional displacement, andFIG. 2 shows only one-dimensional displacement data (e.g. X-axis), and the operation of another dimension (e.g. Y-axis) is identical to that ofFIG. 2 and thus details thereof are not repeated herein. More specifically, after each dimension of the two-dimensional displacement is respectively obtained by the digital filter 10 (e.g. according toFIGS. 1 and 2 ), the processor including thedigital filter 10 then calculates a sum of vector thereof using vector arithmetic and outputs the movement corresponding to the sum of vector according to the resolution setting. - The
delay buffer 101 has a delay number and is configured to sequentially receive input data and output a tap sum. In one embodiment, when a number of input data is larger than the delay number, the most first input data is firstly removed from thedelay buffer 101 so as to form a first-in first-out operation. Thedigital filter 10 is configured to calculate a quotient and a remainder by dividing an accumulation value by an average number. In one embodiment, when the quotient is smaller than 1 (e.g. equal to 0), thedigital filter 10 adds next tap sum to the accumulation value so as to update the accumulation value; whereas, when the quotient is larger than 0, thedigital filter 10 adds the next tap sum and the remainder to be served as a new accumulation value. Next, thedigital filter 10 divides the updated accumulation value or the new accumulation value by the average number. The smoothing of the input data is performed by repeatedly calculating in this way. In other embodiments, no matter whether the quotient is larger than or equal to 0, thedigital filter 10 adds the next tap sum and the remainder to be served as a new accumulation value (illustrated with examples below). Preferably the delay number is equal to the average number. - For example, the
digital filter 10 is an N-tap filter (e.g. N=8 shown herein, but not limited thereto), i.e. the average number and the delay number are both equal to N. - In one embodiment, the tap sum is a sum of data of every tap (e.g. Tap1 to Tap8) in the
delay buffer 101. Referring toFIG. 2 , for example at time t0, the tap sum is a sum of single data in Tap1 as well as data in Tap2 to Tap8 (e.g. a symbol “X” shown herein due to no data being received, and X=0 actually). For example at time t1, the tap sum is a sum of two data in Tap1 and Tap2 as well as data in Tap3 to Tap8. Similarly, the tap sum at times t2 to t12 is obtainable in a similar way. - The
accumulation buffer 105 is configured to store an accumulation value, which is updated at every clock cycle (e.g. t0 to t12). Thecalculation unit 107 is implemented, for example, by a divider which is configured to calculate a quotient and a remainder by dividing the accumulation value by the average number N, wherein thedigital filter 10 outputs the quotient as the output data (e.g. smoothed displacement data), and the remainder may be stored in theremainder buffer 109. Theadder 103 is configured to add the tap sum and the remainder so as to generate the accumulation value. - In the present disclosure, the accumulation value is obtainable according to at least two ways as illustrated below with examples.
- In one embodiment, a method of updating the accumulation value is determined according to the quotient. For example, when the quotient is smaller than 1 (e.g. equal to 0), the
calculation unit 107 sets the remainder to 0, and a next tap sum is directly added to the accumulation value already stored in theaccumulation buffer 105 so as to update the accumulation value; whereas, when the quotient is larger than 0, theadder 103 adds the next tap sum and the remainder to be served as a new accumulation value, which is then stored in theaccumulation buffer 105. - For example referring to
FIG. 2 , at time t0, as the data of Tap1 is “1” and the data of Tap2 to Tap8 is “0”, the tap sum is calculated as “1”. It is assumed that an initial accumulation value is “0”, and the accumulation value is equal to “1” which is calculated by adding the tap sum “1” to the initial accumulation value “0”. Next, thecalculation unit 107 divides the accumulation value “1” by the average number N so as to obtain a quotient equal to “0” and a remainder equal to “1”. Meanwhile, as the quotient is equal to “0”, thecalculation unit 107 either sets the remainder to “0” or does not store the remainder to theremainder buffer 109. For example, thedigital filter 10 resets the remainder stored in theremainder buffer 109 to “0” when the quotient is smaller than 1. Meanwhile, thedigital filter 10 outputs zero displacement. - At time t1, as the data of Tap2 is “1”, the data of Tap1 is “−1” and the data of Tap3 to Tap8 is “0”, the tap sum is calculated as “0”. The accumulation value is still equal to “1” which is calculated by adding a next (i.e. time t1) tap sum “0” to the accumulation value “1” already stored in the
accumulation buffer 105. Next, thecalculation unit 107 performs the division operation so as to obtain another quotient equal to “0” and another remainder equal to “1”. Similarly, as the quotient is equal to “0”, thecalculation unit 107 sets the remainder to “0” and thedigital filter 10 outputs zero displacement. - In some embodiments, as the remainder is set to 0, the accumulation value may be calculated by adding the next tap sum, the remainder and the accumulation value, and the calculation result is not changed. In other embodiments, the
calculation unit 107 may not set the remainder to 0 as long as the remainder is ignored in calculating the accumulation value, e.g. disable theadder 103 or not reading theaccumulation buffer 109. - Similarly, the calculation of the accumulation value at times t2 to t6 is obtainable in a similar way.
- At time t7, as the data from Tap8 to Tap1 is [1 −1 0 1 1 1 3 2], the tap sum is calculated as “8” (=1+(−1)+0+1+1+1+3+2). Meanwhile as the
calculation unit 107 obtains a quotient equal to “1” (larger than 0) and a remainder equal to “5” at time t6, the accumulation value is equal to “13” which is calculated by adding the tap sum “8” and a previous (i.e. at time t6) remainder “5”. Next, thecalculation unit 107 performs the division operation so as to obtain another quotient equal to “1” and another remainder equal to “5”. More specifically, as the quotient is not equal to “0” at time t6, theremainder buffer 109 is not reset. - Similarly, the calculation of the accumulation value at times t8 to t12 is obtainable in a similar way.
- In this embodiment, the
digital filter 10 receives one input data at each clock time t1 to t12 and obtains a quotient and a remainder after the averaging operation (i.e. dividing the accumulation value by the average value). When the quotient is larger than 0, the remainder is stored in theremainder buffer 109 for updating. When the quotient is smaller than 1 (e.g. equal to 0), the remainder may be stored in theremainder buffer 109 or the storage of theremainder buffer 109 may be reset to 0. More specifically, in this embodiment, when the quotient is smaller than 1, thedigital filter 10 outputs zero displacement and resets the storage in theremainder buffer 109 to zero or ignores the remainder (i.e. the remainder stored or not stored in the remained buffer 109); whereas, when the quotient is larger than 0, thedigital filter 10 outputs nonzero smoothed displacement and the obtained remainder is stored in theremainder buffer 109. - In another embodiment, the accumulation value is always a sum of the tap sum and the remainder. That is, the
calculation unit 107 updates the remainder stored in theremainder buffer 109 at each clock cycle (e.g. t0 to t12). - Referring to
FIG. 2 , for example at time t2, as the data of Tap1 is “1” and the data of other taps Tap2 to Tap8 is “0”, the tap sum is calculated as “1”. It is assumed that an initial remainder is “0”, and the accumulation value is equal to “1” which is calculated by adding the tap sum “1” and the initial remainder “0”. Next, thecalculation unit 107 divides the accumulation value “1” by the average number N so as to obtain a quotient equal to “0” and a remainder equal to “1”. Meanwhile, thedigital filter 10 outputs zero displacement and the remainder “1” is stored in theremainder buffer 109. - At time t1, as the data of Tap2 is “1”, the data of Tap1 is “−1” and the data of other taps Tap3 to Tap8 is “0”, the tap sum is calculated as “0”. The accumulation value is still equal to “1” which is calculated by adding a next (i.e. time t1) tap sum “0” and the remainder “1” already stored in the
remainder buffer 109. Next, thecalculation unit 107 also performs the division operation so as to obtain another quotient equal to “0” and another remainder equal to “1”. Accordingly, thedigital filter 10 outputs zero displacement and the remainder “1” is used to update the remainder in thereminder buffer 109. - Similarly, the calculation of the accumulation value at times t2 to t12 is obtainable in a similar way.
- In the present disclosure, at each time t0 to t12 the
digital filter 10 outputs the quotient Q obtained by thecalculation unit 107 to be served as the smoothed displacement data. In addition, it is known fromFIG. 2 that a delay time exists between a time (e.g. t0) that thenavigation device 1 starts to move and a time (e.g. t6) that thedigital filter 10 starts to output nonzero displacement, e.g. 6 clock cycles being delayed inFIG. 2 , wherein an actual delay time is determined according to the input data and not limited to that shown inFIG. 2 . Accordingly, in order to allow thedigital filter 10 not to continuously output nonzero displacement when thenavigation device 1 stops moving with respect to the working surface S, e.g. the quotient Q not zero at times t0 to t12 inFIG. 2 when successive input data becomes zero, in some embodiments thedigital filter 10 directly sets the output data to 0 when a predetermined number of successive input data is 0, wherein the predetermined number is preferably smaller than the average number N, but not limited thereto. In some embodiments, when thenavigation device 1 operates at a high frame rate and the average number N is not high, said delay time may not influence the operation such that thedigital filter 10 may continuously output the output data (i.e. quotient Q) obtained by thecalculation unit 107 without setting the output data to 0. - In some embodiments, the average number N is positively correlated with a resolution of the
navigation device 1. When the resolution is higher, the influence on thenavigation device 1 from the noise or the working surface S becomes more obvious, and thus a higher smoothing strength (i.e. higher N) is selected so as to increase the track smooth. For example as shown in Table 1 below, in the present disclosure the average number N may be adjusted to 8, 16, 32 or 64 corresponding to different resolutions. When the resolution of thenavigation device 1 is changed, the average number N is correspondingly adjusted. In some embodiments, when the average number N is low (e.g. N=8 or 16), said delay time is shorter and thedigital filter 10 may be arranged to continuously output the output data obtained by thecalculation unit 107 without setting the output data to 0. In other embodiments, when the average number N is high (e.g. N=32 or 64), said delay time is longer and thedigital filter 10 may be arranged to set the output data to 0 when a predetermined number of successive input data is 0. It is appreciated that when the average number is adjusted, the delay number is adjusted as well. -
TABLE 1 Average number (N) Resolution range (DPI) 8 50-2,000 16 2,050-3,600 32 3,650-5,000 64 5,050-8,200 - Referring to
FIGS. 3 and 4 , they are flow charts of the track smoothing method of a navigation device according to some embodiments of the present disclosure. The track smoothing method includes the steps of: calculating a tap sum of every tap in a digital filter (Step S31); updating an accumulation value with the tap sum (Step S32); calculating a quotient and a remainder by dividing the accumulation value by an average number (Step S33); and outputting the quotient to be served as output data (Step S34). - Step S31: As shown in
FIG. 2 , thedelay buffer 10 is configured to output a tap sum of every tap therein (e.g. Tap1 to Tap8), and a tap number is determined according to the required smoothing strength. - Step S32: As mentioned above, an accumulation value is obtainable according to at least two ways. In one embodiment, when a quotient Q is smaller than 1 (e.g. equal to 0), the tap sum is directly added to the accumulation value stored in the accumulation buffer 105 (Step S321 shown in
FIG. 4 ) for updating, and a remainder R may be set to 0 or ignored; whereas, when the quotient Q is larger than 0, theadder 103 adds the remainder R to the tap sum (Step S322 shown inFIG. 4 ) to be served as a new accumulation value which is then stored in theaccumulation buffer 105. In other embodiments, the accumulation value is always a sum of the tap sum and the remainder R. - Step S33: Next, the
calculation unit 107 divides the accumulation value by the average number N so as to obtain the quotient Q and the remainder R. - Step S34: The
navigation device 10 outputs the quotient Q to be served as output data (i.e. smoothed displacement data). The remainder R may be stored in theremainder buffer 109, ignored or set to 0 according to different applications. - The track smoothing method of this embodiment may be referred to
FIGS. 1 to 2 and corresponding descriptions, and thus details thereof are not repeated herein. In addition, in order to increase the operation accuracy, the track smoothing method of this embodiment may also include the step: setting the output data to 0 when a predetermined number of successive input data is 0. - It should be mentioned that values in the above embodiment such as the average number, delay number, resolution, input data, output data, tap sum, accumulation value, remainder and quotient are only intended to illustrate but not to limit the present disclosure. In addition, although the values shown in
FIG. 2 are all integers, the present disclosure is not limited thereto, and the values inFIG. 2 may also be decimals. - As mentioned above, the track outputted by the conventional optical mouse may not be smooth due to the noise and operating conditions. Therefore, the present disclosure further provides a navigation device (
FIG. 1 ) and a track smoothing method thereof (FIGS. 3-4 ) that may smooth the track of a plurality of input data and the smoothing strength is adjustable according to the device resolution thereby increasing the adaptability thereof. - Although the disclosure has been explained in relation to its preferred embodiment, it is not used to limit the disclosure. It is to be understood that many other possible modifications and variations can be made by those skilled in the art without departing from the spirit and scope of the disclosure as hereinafter claimed.
Claims (11)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US16/446,699 US10511288B2 (en) | 2015-01-26 | 2019-06-20 | Navigation device with track smoothing |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/605,007 US10382012B2 (en) | 2015-01-26 | 2015-01-26 | Navigation device and track smoothing method thereof |
US16/446,699 US10511288B2 (en) | 2015-01-26 | 2019-06-20 | Navigation device with track smoothing |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/605,007 Continuation US10382012B2 (en) | 2015-01-26 | 2015-01-26 | Navigation device and track smoothing method thereof |
Publications (2)
Publication Number | Publication Date |
---|---|
US20190305759A1 true US20190305759A1 (en) | 2019-10-03 |
US10511288B2 US10511288B2 (en) | 2019-12-17 |
Family
ID=56432826
Family Applications (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/605,007 Active 2035-09-17 US10382012B2 (en) | 2015-01-26 | 2015-01-26 | Navigation device and track smoothing method thereof |
US16/446,699 Active US10511288B2 (en) | 2015-01-26 | 2019-06-20 | Navigation device with track smoothing |
Family Applications Before (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/605,007 Active 2035-09-17 US10382012B2 (en) | 2015-01-26 | 2015-01-26 | Navigation device and track smoothing method thereof |
Country Status (3)
Country | Link |
---|---|
US (2) | US10382012B2 (en) |
CN (1) | CN105824441B (en) |
TW (1) | TW201627828A (en) |
Families Citing this family (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10627518B2 (en) * | 2017-06-02 | 2020-04-21 | Pixart Imaging Inc | Tracking device with improved work surface adaptability |
CN107707178A (en) * | 2017-09-21 | 2018-02-16 | 上海辛格林纳新时达电机有限公司 | Filtering method, low pass filter and the servo-driver of low pass filter |
US10853106B2 (en) * | 2017-11-28 | 2020-12-01 | Bank Of America Corporation | Computer architecture for emulating digital delay nodes in a correlithm object processing system |
CN110110284B (en) * | 2019-05-22 | 2023-02-07 | 合肥奕斯伟集成电路有限公司 | Average number calculation method and device and electronic equipment |
US10929701B1 (en) * | 2019-10-03 | 2021-02-23 | Pixart Imaging Inc. | Navigation device with consistent output and 2-stage moving average filter thereof |
Family Cites Families (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH0736142B2 (en) * | 1991-10-10 | 1995-04-19 | インターナショナル・ビジネス・マシーンズ・コーポレイション | Method and information processing apparatus for recognizing movement stop of movement instruction means |
US5786804A (en) * | 1995-10-06 | 1998-07-28 | Hewlett-Packard Company | Method and system for tracking attitude |
KR100494470B1 (en) * | 2002-11-12 | 2005-06-10 | 삼성전기주식회사 | Image Data Processing apparatus of Optical Mouse and method thereof |
US7765251B2 (en) * | 2005-12-16 | 2010-07-27 | Cypress Semiconductor Corporation | Signal averaging circuit and method for sample averaging |
US7728816B2 (en) * | 2006-07-10 | 2010-06-01 | Cypress Semiconductor Corporation | Optical navigation sensor with variable tracking resolution |
US10548490B2 (en) * | 2012-03-01 | 2020-02-04 | Pixart Imaging Inc. | Physiological detection device and operating method thereof |
TWI524220B (en) * | 2012-03-01 | 2016-03-01 | 原相科技股份有限公司 | Displacement detection device and operating methof thereof |
TW201346651A (en) * | 2012-05-01 | 2013-11-16 | Pixart Imaging Inc | Optical navigation device and trace smoothing method thereof |
CN103389811B (en) * | 2012-05-11 | 2016-08-10 | 原相科技股份有限公司 | Optical navigator and smoothing track method thereof |
US9498710B2 (en) * | 2013-11-11 | 2016-11-22 | Pixart Imaging Inc. | 3D feedback mouse sensor |
-
2015
- 2015-01-26 US US14/605,007 patent/US10382012B2/en active Active
- 2015-09-15 TW TW104130565A patent/TW201627828A/en unknown
- 2015-09-28 CN CN201510628953.7A patent/CN105824441B/en active Active
-
2019
- 2019-06-20 US US16/446,699 patent/US10511288B2/en active Active
Also Published As
Publication number | Publication date |
---|---|
TW201627828A (en) | 2016-08-01 |
CN105824441A (en) | 2016-08-03 |
US20160218697A1 (en) | 2016-07-28 |
US10382012B2 (en) | 2019-08-13 |
US10511288B2 (en) | 2019-12-17 |
CN105824441B (en) | 2019-07-05 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10511288B2 (en) | Navigation device with track smoothing | |
US20090160774A1 (en) | Displacement detection apparatus and method | |
US7079116B2 (en) | Method of processing data of optical mouse | |
WO2019188348A1 (en) | Distance information acquisition device, multipath detection device, and multipath detection method | |
US7675630B2 (en) | System and method for selectively setting optical navigation resolution | |
US20160232684A1 (en) | Motion compensation method and apparatus for depth images | |
US20210080588A1 (en) | 3d image sensor and related 3d image sensing module and hand-held device | |
US10838514B2 (en) | Optical device and locus smoothing method thereof | |
US10928931B2 (en) | Navigation device with adjustable sample period | |
CN111507924B (en) | Video frame processing method and device | |
US20100103191A1 (en) | Locus smoothing method and navigation device using the same | |
US10628951B2 (en) | Distance measurement system applicable to different reflecting surfaces and computer system | |
KR20100088543A (en) | System and method for performing optical navigation using enhanced frames of image data | |
KR100728653B1 (en) | Method for computing the movement value of an optical pointing device and an optical pointing apparatus using it | |
US10331244B2 (en) | Navagation device with fast frame rate upshift and operating method thereof | |
JP2015087243A (en) | Image processor and image processing method | |
US10929701B1 (en) | Navigation device with consistent output and 2-stage moving average filter thereof | |
EP3439286B1 (en) | Calibration of pixels for producing super resolution images | |
US20210132198A1 (en) | Time-of-flight sensor and control method thereof | |
US11726586B1 (en) | Dynamic moving averaging method to suppress mouse stationary jitter | |
US10795454B2 (en) | Navigation device and calculation method of correlation search window thereof | |
US11128825B2 (en) | Navigation device with adaptive fixed pattern noise update and operating method thereof | |
US11803191B2 (en) | Moving robot with improved identification accuracy of step distance | |
RU2351983C1 (en) | Device for image input in computer of distortion correction | |
KR20190094705A (en) | Method and Apparatus for Noise Reduction |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: PIXART IMAGING INC., TAIWAN Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:PIXART IMAGING (PENANG) SDN. BHD.;REEL/FRAME:049532/0352 Effective date: 20190517 Owner name: PIXART IMAGING (PENANG) SDN. BHD., MALAYSIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHIN, KIAN- MING;SONG, WILLIE;REEL/FRAME:049532/0413 Effective date: 20140909 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |
|
MAFP | Maintenance fee payment |
Free format text: PAYMENT OF MAINTENANCE FEE, 4TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1551); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY Year of fee payment: 4 |