CN103135801A - Human-computer interface system and finger mouse system - Google Patents

Abstract

A human-machine interface system comprises a slave device and a master device. The slave device provides two different wavelengths of light to illuminate a finger surface, receives reflected light from the finger surface to generate a plurality of image frames, and calculates and outputs image frame information about a preset number of image frames. The master device calculates the contact state and displacement of the finger surface and the user's physiological characteristics based on the image frame information.

Description

Man-machine interface system and finger mouse system

technical field

The invention relates to a man-machine interface system, in particular to a man-machine interface system and a finger-mouse system capable of simultaneously detecting user's physiological characteristics, finger displacement and contact state.

Background technique

An optical finger mouse (OFM) is usually suitable for portable electronic devices due to its small size. A general optical finger mouse can detect the light intensity change of the reflected light on the surface of the user's finger to determine the contact state of the finger and the displacement of the finger relative to the touch surface. However, with the development of industrialization, the time for users to use portable electronic devices is gradually increasing, even exceeding the physical load without realizing it. Therefore, if a portable electronic device also has the function of detecting the user's physiological characteristics and giving a warning when necessary, the situation of overuse can be avoided.

It is known that the pulse oximeter uses a non-invasive method to detect the blood oxygen concentration and pulse rate of the user, which can generate red light beams (wavelength of about 660 nanometers) and infrared light beams (wavelength of about 910 nanometers) Penetrate the site to be measured, and use the characteristics of different absorption rates of oxyhemoglobin and deoxyhemoglobin for specific spectra to detect changes in the light intensity of the transmitted light, for example, refer to US Patent No. 7,072,701 No., entitled Method for Spectrophotometric Blood Oxygenation Monitoring. After detecting the light intensity changes of the two wavelengths of transmitted light, the blood oxygen concentration is calculated by the following formula

Blood oxygen concentration = 100% × [HbO ₂ ]/([HbO _{2 ]} + [Hb]);

Wherein, [HbO ₂ ] represents the concentration of oxygenated hemoglobin; [Hb] represents the concentration of deoxygenated hemoglobin.

The light intensity of the two wavelengths of penetrating light detected by the general oximeter will change as shown in Figure 1 with the heartbeat. The volume of blood passing through changes, which in turn changes the proportion of light energy absorbed. In this way, according to the constantly changing light intensity information, the absorption rate of blood to different spectra can be calculated to calculate the concentration information such as oxygenated hemoglobin concentration and deoxygenated hemoglobin concentration, and finally use the above blood oxygen concentration formula to calculate blood oxygen concentration. concentration.

However, because the blood oxygen saturation meter detects the light intensity change of the penetrating light, it will detect different light intensity signals with different parts to be tested; When moving, a violently changing chaotic waveform will be detected, and physiological characteristics cannot be correctly calculated based on it, so it is not suitable for portable electronic devices or devices that require mobile operations.

In view of this, the present invention proposes a human-machine interface system and a finger mouse system that can simultaneously detect the user's physiological characteristics, finger displacement and contact state, which can detect the user's physiological characteristics while detecting the finger displacement, and can Effectively eliminate signal interference caused by movement.

Contents of the invention

The object of the present invention is to provide a man-machine interface system and a finger-mouse system, which can simultaneously calculate finger displacement, contact state, and user physiological characteristics by analyzing reflected light signals of fingers.

Yet another object of the present invention is to provide a human-machine interface system and a finger mouse system, which can simultaneously detect finger displacement and contact status as well as user physiological characteristics, and have a mechanism to eliminate the influence of environmental light sources.

Another object of the present invention is to provide a human-machine interface system and a finger mouse system, which can simultaneously detect finger displacement and contact state and user's physiological characteristics, and have a noise reduction and bandwidth saving mechanism.

Another object of the present invention is to provide a man-machine interface system and a finger mouse system, which can simultaneously detect finger displacement and contact state and user's physiological characteristics, and have a system frequency correction mechanism.

Yet another object of the present invention is to provide a human-machine interface system and a finger mouse system, which can simultaneously detect finger displacement and contact status and user physiological characteristics, and enter into a sleep mode after being idle for a preset time.

Another object of the present invention is to provide a human-machine interface system and a finger-mouse system, which can simultaneously detect finger displacement and contact status as well as user physiological characteristics, and can discard or not respond to physiological characteristics when the displacement is too large.

Another object of the present invention is to provide a human-machine interface system and a finger mouse system, which can simultaneously detect finger displacement and contact state and user physiological characteristics, and have a mechanism for checking data transmission between master and slave devices.

In order to achieve the above purpose, the present invention provides a human-machine interface system, which includes a slave device and a master device. The slave device provides light of two different wavelengths to illuminate the finger, receives reflected light from the finger to generate a plurality of first image frames and a plurality of second image frames illuminated relative to the two different wavelengths of light. Two image frames, calculating and outputting the first image frame information about the first image frame of the preset sheet and the second image frame information about the second image frame of the preset sheet. The master device calculates the displacement and the physiological feature according to the first image frame information and the second image frame information.

According to another feature of the present invention, the present invention also provides a finger mouse system, which includes an optical detection device and a host. The optical detection device includes a first light source, a second light source, a light source control unit, an image sensor and a processing unit. The first light source emits light at a first wavelength to the finger. The second light source emits light at a second wavelength to the finger. The light source control unit controls turning on and off of the first light source and the second light source. The image sensor receives reflected light from the finger at a sampling frequency to generate a plurality of first image frames illuminated relative to the first light source and a plurality of second image frames illuminated relative to the second light source . The processing unit calculates and outputs first image frame information about a preset sheet of the first image frame and second image frame information about a preset sheet of the second image frame. The host computer receives and checks the first image frame information and the second image frame information, and calculates the displacement amount and the physiological characteristic according to them.

According to another feature of the present invention, the present invention also provides a human-machine interface system, which includes a finger mouse device and a host. The finger mouse device provides two different wavelengths of light to illuminate a finger, receives reflected light from the finger to generate a plurality of first image frames and a plurality of second image frames illuminated relative to the two different wavelengths of light. Two image frames, calculating and outputting the first image frame information about the first image frame of the preset sheet and the second image frame information about the second image frame of the preset sheet. The host includes a processing unit and a presentation unit. The processing unit calculates displacement and physiological features according to the first image frame information and the second image frame information. The display unit is used to respond to the displacement and the physiological characteristic.

In the man-machine interface system and finger mouse system of the present invention, the physiological characteristics include blood oxygen concentration and pulse rate. The invention separates mobile information and physiological information by using an independent component analysis method or a blind signal source separation method, and can effectively eliminate signal interference caused by movement.

In the man-machine interface system of the present invention, the slave device can be a mouse, a remote controller, a keyboard, an optical distance measuring device or other electronic peripheral devices; the master device can be a TV, a projection device, a game machine system or a computer system.

Description of drawings

FIG. 1 is a schematic diagram of light intensity changes of transmitted light detected by an oximeter.

FIG. 2A is a schematic diagram of a man-machine interface system according to an embodiment of the present invention.

FIG. 2B is a block diagram of a human-machine interface system according to an embodiment of the present invention.

FIG. 3 is a schematic diagram of an image frame captured by an image sensor of the man-machine interface system according to an embodiment of the present invention.

FIG. 4 is an image sensor of a man-machine interface system according to an embodiment of the present invention, which includes an optical filter disposed in front of part of the sensing surface.

5 is a schematic diagram of image acquisition and light source turning off in the man-machine interface system according to the embodiment of the present invention.

FIG. 6 is a schematic diagram of the separation of movement information and physiological information by the second processing unit of the man-machine interface system according to the embodiment of the present invention.

Fig. 7 is a flowchart of a physiological feature detection method according to an embodiment of the present invention.

Explanation of reference signs

1 slave device 111 first light source

112 Second light source 12 Light guide

13 touch elements 13S touch surface

14 image sensors 141 optical filters

14S sensing surface 15 first processing unit

16 Light source control unit 17 The first memory unit

18 transmission interface 19 clock generator

2 main device 21 second memory unit

22 Verification unit 23 Second processing unit

231 Movement/contact detection unit 232 Physiological characteristic detection unit

24 means unit S ₁₁ -S ₁₂ steps

9 fingers 9S finger surface

SI ₁ -SI _2N image frame information I ₁ -I ₆ image frame

B ₁ -B _2N , B ₁ ′-B _2N ′ average brightness

Detailed ways

In order to make the above and other objects, features and advantages of the present invention more apparent, the following will be described in detail with the accompanying drawings. In the description of the present invention, the same components are denoted by the same symbols, and will be described first.

FIG. 2A is a schematic diagram of a man-machine interface system according to an embodiment of the present invention, including a slave device (slavedevice) 1 and a master device (master device) 2. The slave device 1 is preferably an electronic peripheral device; the master device 2 is preferably an electronic device including a display unit for responding to the output of the slave device 1 (such as cursor control, software execution and/or physiological state display etc.); wherein the display unit may be a display, a light signal, a seven-character display and/or a sound device; the electronic device may be a portable electronic device or a general household electronic device. In one embodiment, the man-machine interface system may be a finger and mouse system, the slave device 1 is an optical detection device and the master device 2 is a host, and data transmission is performed with each other through wired or wireless methods. In another embodiment, the man-machine interface system may include a finger mouse device and a host, and the host computer further includes a display unit for responding to information output by the finger mouse device.

The man-machine interface system of the present invention is used to detect the displacement and contact state of the finger 9 of the user, as well as the physiological characteristics of the user, such as blood oxygen concentration and pulse rate. Generally speaking, the man-machine interface system starts to calculate the displacement and the physiological characteristics when it judges that the contact state is a contact state.

The slave device 1 includes two light sources 111 and 112, a light guide 12, a touch element 13, an image sensor 14, a first processing unit 15 and a light source control unit 16; in FIG. 2A, the spatial relationship of each component is only illustrative, It is not intended to limit the invention. The two light sources 111 and 112 can be, for example, light-emitting diodes or laser diodes, which respectively emit light of different wavelengths, and are preferably two wavelengths used by known oximeters, such as red light at 660 nanometers and 905, 910 Or 940 nm infrared light. It can be understood that the wavelength mentioned here refers to the central wavelength of the spectrum.

The light guide 12 is used to guide the light emitted by the light sources 111 and 112 to the touch element 13; The emitted light is transmitted to the touch element 13 in a penetrating manner, and there is no specific limitation. In other implementation manners, if the light emitted by the light sources 111 and 112 can be incident on the touch element 13 , the light guide element 12 can also be omitted.

The touch element 13 has a touch surface 13S for the finger 9 to manipulate on it, and the touch element 13 is preferably transparent to the light emitted by the light sources 111 and 112 . When the finger 9 approaches or touches the touch surface 13S, the light emitted by the light sources 111 and 112 is reflected. It can be understood that the area of the touch surface 13S may be larger or smaller than the finger surface 9S, and there is no specific limitation.

The image sensor 14 uses sampling parameters to receive reflected light from the touch control 13 (the finger surface 9S) to generate a plurality of image frames (each image frame has, for example, 16×16 pixels); wherein the The sampling parameters include exposure time, image gain, etc., but are not limited thereto. It can be understood that, FIG. 2A may additionally include other lenses for guiding reflected light to the image sensor 14 to increase the light-sensing efficiency of the image sensor 14 . The image sensor 14 is preferably an active array image sensor, such as a CMOS image sensor, but may also be other elements for sensing image frames.

The first processing unit 15 calculates and outputs image frame information about preset image frames, such as image frame sum, image frame average, normalized image frame sum or Normalized image frame average; that is, when the image sensor 14 outputs M image frames, the processing unit 15 calculates and outputs the image frame sum or image frame average of the M image frames , or further normalize the preferred image frame average of the image frame (described in detail later); in other words, an image frame information is a processed (addition, average, processing) image frame. The main device 2 calculates the displacement and contact state of the finger 9 relative to the touch surface 13S, as well as the physiological characteristics of the user according to the image frame information.

The light source control unit 16 is coupled to the first processing unit 15 to control the light sources 111 and 112 to be turned on and off according to the image frame acquisition of the image sensor 14 (details will be described later).

Referring to FIG. 2A and FIG. 2B , FIG. 2B is a block diagram of a human-machine interface system according to an embodiment of the present invention, which includes the slave device 1 and the master device 2 . The slave device 1 includes a first light source 111, a second light source 112, the image sensor 14, the first processing unit 15, the light source control unit 16, a first memory unit 17, a transmission interface 18 and a clock generator 19. The main device 2 includes a second memory unit 21, a checking unit 22, a second processing unit 23, and a display unit 24; wherein, the second processing unit 23 performs multifunctional calculations, and it may also include a movement/contact detection unit 231 used to calculate the displacement of the finger 9 relative to the touch surface 13S and the contact state, and the physiological feature detection unit 232 is used to calculate the physiological feature of the user; that is, the second The processing unit 23 can be a single component or divided into two components.

The first light source 111, for example, emits red light with a wavelength of about 660 nanometers; the second light source 112, for example, emits infrared light with a wavelength of about 905, 910 or 940 nanometers; broadly speaking, the first light source 111 and all The second light source 112 respectively emits two wavelengths of light used by known oximeters. The light source control unit 16 controls the lighting or extinguishing of the first light source 111 and the second light source 112, so that the image sensor 14 receives the reflected light from the finger 9 at a sampling frequency to generate light corresponding to the first light source 111. A plurality of first image frames illuminated by a light source 111 and a plurality of second image frames illuminated relative to the second light source 112 . The first processing unit 15 calculates the first image frame information of preset sheets of the first image frame, for example, the first image frame sum and first image frame average of the M first image frames 1. Normalize the first image frame and or normalize the average of the first image frame, and calculate the second image frame information of the preset second image frame, for example, M second image frames Second Image Frame Sum of Frames, Second Image Frame Average, Normalized Second Image Frame Sum, or Normalized Second Image Frame Average. The first memory unit 17 is, for example, a buffer for storing the first image frame information and the second image frame information obtained by the first processing unit 15 . The transmission interface 18 is used to transmit the first image frame information and the second image frame information stored in the first memory unit 17 to the main device 2 through wired or wireless transmission; Among them, wired and wireless transmission technologies are already known, so details are not repeated here. It must be noted that, if the adjustment of data transmission between the slave device 1 and the master device 2 is accurate enough, the first memory unit 17 may not be implemented. The clock generator 19 is used to provide the system frequency of the slave device 1 .

The second memory unit 21 is used for storing the first image frame information and the second image frame information received from the slave device 1 . The checking unit 22 is used to check whether the first image frame information and the second image frame information have oversampling or data loss. If there is oversampling in the first image frame information and the second image frame information, directly discard the oversampled information; if the first image frame information and the second image frame If there is data loss in the information, the lost data can be restored by means of interleaving; if the information does not have the above problems, then the first image frame information and the second image frame information are directly sent to The second processing unit 23 . It can be understood that since the checking unit 23 performs multiple functions, it can use a single component to perform all the functions or include multiple components to perform the functions of checking, discarding and interpolation respectively. The second processing unit 23 (or the movement/contact detection unit 231) is used to calculate the relative position of the finger 9 relative to the touch surface 13S according to the first image frame information and the second image frame information. The displacement amount and the contact state; the second processing unit 23 (or the physiological feature detection unit 232) is used to calculate the The physiological characteristics of the user. The display unit 24 is used for displaying and/or responding to the displacement, contact status and physiological characteristics calculated by the second processing unit 23 .

In one embodiment, the master device 2 is, for example, a television, a projection device, a computer system, a game system or other electronic devices with a display unit; the slave device 1 may be a remote controller, a mouse, a keyboard, an optical distance measuring device or other electronic peripherals. In other words, the slave device 1 and the master device 2 are wired or wirelessly coupled to form a single component (such as a portable device) or multiple components (such as a home appliance system); the slave device 1 Send the first image frame information that is lit relative to the first light source 111 and the second image frame information that is lit relative to the second light source 112, and the main device 2 and the second image frame information to calculate and/or respond to the displacement, contact status and physiological characteristics.

Therefore, the slave device 1 of the present invention can be matched with the master device 2 having the display unit 24, allowing the user to control the cursor displayed on the display unit 24 or the software executed by the master device 2 through the slave device 1. , indicating the user’s physiological characteristics for the user’s reference, and giving a warning to the user (according to the value of the physiological characteristics) when the physiological characteristics show a state of fatigue; where the physiological characteristics and the way of warning are displayed For example, it can be achieved by using software to perform screen display, light signal display or sound display.

In other embodiments, the slave device 1 can also use two image sensors to detect light of two different wavelengths respectively, wherein one image sensor or two image sensors can be provided with a bandpass filter (bandpass filter) to select The spectrum to be received.

sampling mechanism

The man-machine interface system of the present invention includes two light sources and performs two functions at the same time; among them, the detection functions of displacement and contact state are not limited to use image frames of specific wavelengths, while the detection of physiological characteristic functions must correspond to different wavelengths The image frames of are calculated separately. Firstly, the sampling mechanism of the image frame in the present invention will be described below.

In one embodiment, the light source control unit 16 controls the first light source 111 and the second light source 112 to emit light in turn, and the image sensor 14 is synchronized with a high-speed and fixed sampling frequency (for example, 3000 images per second). The lighting of the first light source 111 or the second light source 112 acquires an image frame, and outputs a plurality of image frames I ₁ -I ₆ ... as shown in FIG. 3 to the first processing unit 15, wherein the The image frames I ₁ -I ₆ ... include the first image frames I ₁ , I ₃ , I ₅ ..., which are for example illuminated relative to the first light source 111; the second image frames I ₂ , I ₄ , I ₆ . The first processing unit 15 respectively calculates the first image frame information of the M first image frames I ₁ , I ₃ , I ₅ ..., for example, the sum of the first image frames (I ₁ +I ₃ +I ₅ ...), first image frame average (I ₁ +I ₃ +I ₅ ...)/M, normalized first image frame sum (I ₁ +I ₃ +I ₅ ...)/(sampling parameter) or Normalized first image frame average (I ₁ +I ₃ +I ₅ ...)/(M×sampling parameter), and second image images of M second image frames I ₂ , I ₄ , I ₆ ... Frame information, such as second image frame sum (I ₂ +I ₄ +I ₆ ...), second image frame average (I ₂ +I ₄ +I ₆ ...)/M, normalized second image frame Sum (I ₂ +I ₄ +I _{6 .} . . )/(sampling parameter) or normalized second image frame average (I ₂ +I ₄ +I _{6 .} . . )/(M×sampling parameter). In one embodiment, the first processing unit 15 processes every 10 first and second image frames, that is, M=10; but it is not limited thereto.

The second processing unit 23 (or the movement/contact detection unit 231) can judge the contact state and calculate the displacement according to the first and second image frame information stored in the second memory unit 21, for example According to the comparison result of the brightness of the first image frame information and the second image frame information with at least one threshold, it is determined whether the finger 9 approaches or touches the touch surface 13S, wherein when the image frame When the brightness of the information is greater than or less than the at least one threshold value, it is determined to enter the contact state; after entering the contact state, the second processing unit 23 can use the frame information of two first images and the frame information of one first image The displacement amount is calculated based on a correlation (correlation) with the frame information of one second image or the frame information of two second images. It must be noted that although known methods can be used to determine the contact state and calculate the displacement, in the present invention, the image frame information corresponding to two different wavelengths of reflected light must be used for determination and calculation. And different from the traditional navigation device (navigation device).

The second processing unit 23 (or the physiological feature detection unit 232) must calculate brightness changes of a plurality of first image frame information according to the first image frame information, and calculate brightness changes according to the second image frame information. The frame information calculates the brightness changes of multiple second image frame information (detailed later), and calculates the absorption ratio of the two spectra according to it to obtain the oxygenated hemoglobin concentration [HbO 2 ] and deoxygenated hemoglobin concentration [HbO ₂ ] Hemoglobin concentration [Hb], and finally use the blood oxygen concentration formula to calculate the blood oxygen concentration; and compare the brightness change of the first image frame information and/or the second image frame information with at least one threshold Count the pulse rate.

In another embodiment, the light source control unit 16 controls the first light source 111 and the second light source 112 to be synchronized with the image frame acquisition of the image sensor 14 to emit light at the same time; that is, the image sensor 14 Reflected light of two wavelengths is received simultaneously. Therefore, in this embodiment, an optical filter 141 (as shown in FIG. 4 ) may also be provided in front of a part of the sensing surface 14S of the image sensor 14, wherein the optical filter 141 may be a bandpass filter to The part of the sensing surface behind the filter 141 can only sense the spectrum of the first light source 111 or the spectrum of the second light source 112, so that the second processing unit 23 (or the mobile/ The contact detection unit 231 and the physiological feature detection unit 232) can distinguish the first image frame information (relative to the partial image frame of the first light source 111) and the second image frame information (relative to the partial image frame of the second light source 112) part of the image frame). It can be understood that, in the present invention, the position and area of the optical filter 141 are not limited to those shown in FIG. 4 .

In this way, the second processing unit 23 (or the movement/contact detection unit 231 ) can also calculate the contact state and displacement according to the first image frame information and the second image frame information. The second processing unit 23 (or the physiological feature detection unit 232) can also calculate the brightness change of the first image frame information according to the first image frame information and calculate the brightness change according to the second image frame information. The brightness change of the frame information of the second image is obtained, and the blood oxygen concentration and the pulse rate are calculated according to the relationship between the two brightness changes.

It can be understood that since the image sensor 14 may have different photosensitive efficiencies for light of different wavelengths, or the luminance of the first light source 111 and the second light source 112 are not completely the same, it can be used in the optical Before the finger mouse 1 leaves the factory, the brightness of the image frame detected by the image sensor 14 is adjusted (such as adjusting sampling parameters such as exposure time and image gain relative to different wavelength image frames), so that the initial image sensor 14 obtains Image frames have roughly the same brightness to eliminate the possibility of misjudgment.

The spirit of this embodiment is to use the slave device 1 to provide light of two different wavelengths to illuminate the finger surface 9S, receive the reflected light from the finger surface 9S to generate a plurality of image frames, calculate and output information about the preset The image frame information of the image frame; the main device 2 calculates the contact state, displacement and physiological characteristics according to the image frame information.

Eliminate ambient light mechanism

In Fig. 2A, since the contact element 13 is transparent and the finger is transparent, the ambient light from the outside of the device 1 will be received by the image sensor 14 through the contact element 13 and the finger to affect all its components. The image quality of the obtained image frame. In the present invention, the light source control unit 16 can control the first light source 111 and the second light source 112 to not emit light during a part of the period.

FIG. 5 shows the image acquisition of the image sensor 14 and the turning off of the first light source 111 and the second light source 112 ; wherein, the solid line arrow indicates that the light source is on and the dotted line arrow indicates that the light source is off. Diagram A in FIG. 5 is that the image sensor 14 continuously acquires image frames at a fixed frequency. In Figure B in Figure 5, the first light source 111 and the second light source 112 are turned on and off at the same time, so the image sensor 14 can acquire bright image frames (when the light source is on) and Dark image frame (when light source is off). Graph C in FIG. 5 is a situation where the first light source 111 and the second light source 112 light up every second image frame at the same time, which generally has a relatively low displacement relative to the finger 9 . As mentioned above, when the first light source 111 and the second light source 112 are turned on at the same time (Figures B and C in Figure 5), the image sensor preferably includes a filter 141 to spatially separate Image frames of different light sources, so that a part of the image sensor 14 can sense the reflected light of the first light source 111 and another part can sense the reflected light of the second light source 112 .

When the finger 9 touches or approaches the touch surface 13S, the bright image frame obtained when the light source is turned on contains (finger reflected light+stray light+environmental light), and when the light source is not turned on The obtained dark image frame only contains (ambient light), so if the bright image frame is subtracted from the dark image frame, the influence of ambient light can be effectively eliminated. The first processing unit 15 can calculate the difference image frame information according to the difference image frame of the bright image frame and the dark image frame, for example, the difference image frame sum and the difference image frame average of M difference image frames , normalized difference image frame and or normalized difference image frame average; the second processing unit 23 can calculate displacement, contact status and physiological characteristics according to the difference image frame information.

Diagram D in FIG. 5 is an embodiment in which the first light source 111 and the second light source 112 are turned on in turn. In this embodiment, since the image sensor 14 needs to acquire a dark image frame, the light source control unit 16 controls the first light source 111 and the second light source 112 to be separated by one image frame to take turns. Bright, for example, at time t _d of D in Figure 5, both light sources are off. Thereby, the first processing unit 15 can calculate the differential first image frame (bright first image frame-dark image frame) and the differential second image frame (bright second image frame-dark image frame) frame), and calculate the differential first image frame information and the differential second image frame information; the second processing unit 23 calculates displacement, contact state and physiological characteristics according to the differential image information. As mentioned above, when the first light source 111 and the second light source 112 are turned on in turn, the image sensor 14 separates image frames corresponding to different light sources by time.

The spirit of this embodiment is to enable the light source control unit 16 to control the first light source 111 and the second light source 112 to emit light simultaneously or alternately, and to enable the image sensor 14 to acquire the Frame the dark image and remove the influence of ambient light by calculating the difference between the light and dark images. Therefore, the turning off of each light source shown in FIG. 5 is only illustrative, and not intended to limit the present invention.

Noise reduction and bandwidth saving mechanism

Since there will be interference in the image frame acquired by the image sensor 14, and the interference is usually randomly distributed in the acquired image frame, so the present invention utilizes the first processing unit 15 to calculate M image frames The sum of frames is used to improve the signal-to-noise ratio (SNR) to increase the accuracy of calculating physiological characteristics; for example, every 10 image frames are added, and two consecutive 10 image frames can be partially repeated or completely different. repeat. Therefore, one image frame sum can be obtained for every 10 images. In addition to improving the signal-to-noise ratio, since the image sensor 14 preferably has a high sampling frequency, it can also effectively save data transmission between the slave device 1 and the master device 2. Bandwidth; in other implementation manners, the image frame sum can also be averaged. It can be understood that when the first light source 111 and the second light source 112 are turned on in turn, the sum of the image frame in this embodiment is the first image (for example, I ₁ +I ₃ in FIG. 3 +I _{5 .} . . ) and the second image (eg I ₂ +I ₄ +I ₆ . However, when the first light source 111 and the second light source 112 are turned on at the same time, the sum of image frames in this embodiment is a continuous image frame (such as I ₁ +I ₂ +I ₃ +I in FIG. 3 ₄ +I ₅ +I ₆ ...), and distinguish the two groups of light intensity changes through post-processing. In addition, when the above-mentioned mechanism for eliminating ambient light is used, the frame sum of the first image is the frame sum of the first image, the frame average of the first image is the frame average of the first image, and the frame average of the second image The frame sum is the differential second image frame sum, and the second image frame average is the differential second image frame average; that is, the noise reduction process is performed after the ambient light elimination process is performed. In other embodiments, only noise reduction processing may be performed.

As mentioned above, the image sensor 14 may acquire images with different sampling parameters under different conditions. Sampling parameters such as exposure time and image gain are used to make the first image and the second image have initial image frames with approximately the same brightness, so that post-processing can be performed correctly according to the image frames, that is, relative to the first image The sampling parameters of an image frame and a second image frame may be different. In the present invention, in order to exclude the influence of different sampling parameters, M image frames and or image frames may be averaged and divided by sampling parameters to perform normalization processing, for example (M image frames and/sampling parameters) or ( M image frame average/sampling parameter); wherein, M is a positive integer. If the above-mentioned mechanism for eliminating ambient light is used, the normalized first image frame sum is the normalized difference first image frame sum, the normalized first image frame average is the normalized difference first image frame average, The normalized second image frame sum is the normalized difference second image frame sum, and the normalized second image frame average is the normalized difference second image frame average.

Data Verification Mechanism

In the present invention, since the calculation of physiological information requires accurate image frame data, the verification unit 22 of the master device 2 checks the image frame data transmitted by the slave device 1, and discards the over-sampling when over-sampling occurs. Image data; when the data is lost, interpolation operation is performed to restore the data, so as to avoid the situation of false detection.

Physiological characteristic calculation

When different light sources are turned on, the image frame acquired by the image sensor 14 includes physiological information and movement information at the same time. Therefore, in the present invention, the second processing unit 23 (or the physiological feature detection unit 232 ) needs to separate the two kinds of information before it can correctly calculate the physiological feature. In the present invention, the second processing unit 23, for example, adopts Independent Component Analysis (ICA) or Blind Source Separation (BSS) to separate the two kinds of information.

Referring to Fig. 3 and Fig. 6, first taking the first image I ₁ , I ₃ , I ₅ ... of Fig. 3 as an example, the multiple first image frames (the M original image frames and the eliminated environment The image frame information of the M first image frame or normalized first image frame) of the light mechanism is divided into at least two parts and the average brightness is obtained respectively. For example, the image frame information _S11 is divided into an average brightness of B ₁ and B ₁ ′; the image frame information SI ₃ is divided into two parts whose average brightness is B ₃ and B ₃ ′; ...; the image frame information SI _2N-1 is divided into two parts whose average brightness is B _2N-1 and B _{2N -1} ' two parts (in other implementations, there may be more than two parts); wherein, the image frame information SI ₁ is, for example, (I ₁ +I ₃ +...I ₁₉ ), (I ₁ +I ₃ +...I ₁₉ ) /10, (I ₁ +I ₃ +...I ₁₉ )/sampling parameter, (I ₁ +I ₃ +...I ₁₉₎ /(10×sampling parameter); image frame information SI ₃ is, for example, (I ₂₁ +I ₂₃ +...I ₃₉ ), (I ₂₁ +I ₂₃ +...I ₃₉ )/10, (I ₂₁ +I ₂₃ +...I ₃₉ )/sampling parameter, (I ₂₁ +I ₂₃ +...I ₃₉ )/(10 × sampling parameter);  …. Next, the first movement information and the first physiological information (as shown in FIG. 6 ) are separated by using the independent component analysis method or the blind signal source separation method, both of which are displayed as a brightness change line. In the present invention, the movement information is discarded and the physiological information is used to calculate the physiological characteristics. It can be understood that, since the sampling frequency of the image sensor 14 is far greater than the pulse rate, the separated physiological information can show a line pattern (similar to FIG. 1 ) in which the light intensity varies with the pulse; the separated movement information distribution and It is not limited to what is shown in FIG. 6 . In addition, the two parts of the image frame division are not limited to upper and lower parts. In addition, because the physiological information of light of two different wavelengths must be calculated separately, the above separation procedure is for the first image frame information SI ₁ , SI ₃ , SI ₅ ... (corresponding to the lighting of the first light source) and the second The image frame information SI ₂ , SI ₄ , SI ₆ ... (corresponding to the lighting of the second light source) is performed.

It must be emphasized that the displacement and contact state of the finger 9 are directly determined by the second processing unit 23 (or the movement/contact detection unit 231) according to the frame information of the first image and the second image. The frame information is obtained without calculation based on the separated movement information. Independent component analysis or blind signal source separation is mainly used to separate mixed signals to eliminate signal interference caused by movement.

In the present invention, the second processing unit 23 (or the movement/contact detection unit 231 ) also calculates the pulse rate according to the comparison result of at least one threshold value and the first brightness change and/or the second brightness change.

System Frequency Correction Mechanism

Generally, in order to reduce system cost, the clock generator 19 can use an RC oscillator circuit with low cost, and the precision of the RC oscillator circuit is low and its oscillation frequency will change with the process and operating temperature; A quartz oscillation circuit can be used without particular limitation. Since a more accurate system frequency is required when calculating physiological characteristics (for example, it must be accurate when calculating the pulse rate), before the slave device 1 leaves the factory, it is preferable to use an external light source with a precise flickering frequency to be close to the contact surface 13S of the touch element 13, so as to The image sensor 14 is made to sense the brightness change of the external light source as a reference for adjusting the oscillation frequency of the clock generator 19 . For example, the difference between the clock generator 19 at different temperatures and the precise frequency can be obtained in advance as a system frequency correction parameter, and stored in the first memory unit 17 . When the system frequency is used, the accurate oscillation frequency can be obtained only by using the system frequency correction parameters.

sleep mode

The man-machine interface system of the present invention can enter into a sleep mode after being idle for a preset time. For example, when the second processing unit 23 determines that the finger 9 is not approaching or touching the touch surface 13S at a preset time, it can enter the sleep mode.

Physiological trait discarding mechanism

The second processing unit 23 of the man-machine interface system of the present invention can calculate the displacement and the physiological characteristics at the same time, but the accurate calculation of the physiological characteristics is preferably in the case of low displacement. Therefore, the present invention can judge in advance whether the displacement is greater than a preset value. If the displacement is greater than the preset value, the image frame acquired by the image sensor 14 is only used to calculate the displacement or determine the contact. The state is not used to calculate the physiological characteristic, or even if the physiological characteristic is calculated, it is not echoed by the display unit 24 .

The present invention proposes a method for detecting physiological characteristics according to the reflected light on the surface of the finger, comprising the following steps: providing light of the first wavelength and a second wavelength from the device to the finger, acquiring the reflected light of the light of the first wavelength to generate a plurality of first image frames and acquire the reflected light of the light of the second wavelength to generate a plurality of second image frames, calculate and output the first image frame information of the first image frame and the first image frame The second image frame information of the second image frame (step S ₁₁ ); and calculate the contact state, displacement and Physiological characteristics (step S ₁₂ ). In the physiological feature detection method of this embodiment, the definitions of the first image frame information and the second image frame information are as described above. In addition, the detailed implementation of each step in this embodiment has been described in detail above, so it is not repeated here.

The physiological feature detection method of the present invention can calculate the physiological feature by detecting the skin surface of the human body to be tested; therefore, the detection function can also be performed by replacing the finger 9 in the above-mentioned embodiment of the present invention with other parts to be tested of the human body; the present invention The human-machine interface system has mechanisms such as noise reduction, bandwidth saving, error checking, ambient light elimination, sleep mode and system frequency correction. The site to be measured is, for example, a body part used by a known oximeter to detect blood oxygen concentration.

In summary, it is known that the optical finger mouse cannot detect the user's physiological characteristics, and the way the oximeter calculates the blood oxygen concentration cannot be compatible with the optical finger mouse due to factors such as the inability to judge the moving part to be measured. . Therefore, the present invention also provides a human-machine interface system (Figure 2A and 2B), which can detect the user's physiological characteristics while detecting the finger displacement, and can effectively eliminate signal interference caused by movement and eliminate the influence of environmental light sources , and has mechanisms for system frequency correction, error checking, bandwidth saving, sleep mode, and discarding physiological information.

Although the present invention has been disclosed with the aforementioned embodiments, this is not intended to limit the present invention. Any person skilled in the technical field to which the present invention belongs can make various modifications and changes without departing from the spirit and scope of the present invention. Revise. Therefore, the protection scope of the present invention should be defined by the appended claims.

Claims (20)

1. A man-machine interface system, used to detect displacement and physiological characteristics of fingers, said man-machine interface system comprising: From the device, light of two different wavelengths is provided to illuminate the finger, and reflected light from the finger is received to generate a plurality of first image frames and a plurality of second image frames illuminated relative to the two different wavelengths of light. image frame, calculating and outputting first image frame information about the first image frame of a preset sheet and second image frame information about the second image frame of a preset sheet; and The master device calculates the displacement and the physiological feature according to the first image frame information and the second image frame information. 2. The man-machine interface system according to claim 1, wherein the slave device is a mouse, a remote controller, a keyboard, an optical distance measuring device; the master device is a TV, a projection device, a game console system or a computer system. 3. The human-machine interface system according to claim 1, wherein the physiological characteristics include blood oxygen concentration and/or pulse rate. 4. The man-machine interface system according to claim 1, wherein said slave device further comprises: a first light source emitting light of a first wavelength to the finger; a second light source emitting light of a second wavelength to the finger; a light source control unit, controlling the turning on and off of the first light source and the second light source; an image sensor to receive reflected light from the finger at a sampling frequency to generate the first image frame and the second image frame; and The first processing unit calculates the first image frame information of each preset frame of the first image frame and the second image frame information of each preset frame of the second image frame. 5. The man-machine interface system according to claim 4, wherein the first image frame information is the first image frame sum, first image frame average, normalized Normalizing the first image frame sum, or normalizing the average of the first image frame; the second image frame information is the second image frame sum and second image of the second image frame preset Frame Average, Normalized Second Image Frame and, or Normalized Second Image Frame Average. 6. The man-machine interface system according to claim 4, wherein the light source control unit controls the first light source and the second light source to be turned off during a predetermined period, so that the image sensor acquires a plurality of bright first images frame, a plurality of bright second image frames and a plurality of dark image frames; the first processing unit is based on the difference between the bright first image frame and the dark image frame and the bright second image The difference between the frame and the dark image frame is used to calculate the first image frame information and the second image frame information. 7. The man-machine interface system according to claim 4, wherein the light source control unit controls the first light source and the second light source to turn on so that the image sensor receives the first light source and the second light source in turn. the reflected light of the second light source; or control the first light source and the second light source to be turned on at the same time so that the image sensor simultaneously receives the reflected light of the first light source and the second light source, and the The image sensor includes a filter covering a part of the sensing surface of the image sensor. 8. The human-machine interface system according to claim 1, wherein the main device further comprises: a memory unit for storing the first image frame information and the second image frame information received from the slave device; a checking unit, checking oversampling and data loss of the first image frame information and the second image frame information; and A second processing unit is configured to calculate the displacement and the physiological feature according to the first image frame information and the second image frame information. 9. The man-machine interface system according to claim 8, wherein the second processing unit: divides each piece of the first image frame information into at least two parts and calculates the average brightness of each part, and uses independent element analysis The method analyzes the average brightness of each part of the first image frame information to obtain the first brightness change; divides each second image frame information into at least two parts and calculates the average of each part Luminance, analyzing the average luminance of each part of the second image frame information by independent element analysis to obtain a second luminance change; and calculating according to the first luminance change and the second luminance change The physiological characteristics. 10. The human-machine interface system according to claim 9, wherein the second processing unit further calculates a pulse rate according to a comparison result of at least one threshold value with the first brightness change and/or the second brightness change. 11. The man-machine interface system according to claim 9, wherein the main device further comprises a display unit, which is used to respond to the displacement and the physiological characteristics; the display unit is a display, a light, Seven-byte display and/or sound device. 12. The human-machine interface system according to claim 1, the human-machine interface system enters the sleep mode after being idle for a preset time. 13. The human-machine interface system according to claim 1, wherein when the displacement amount is greater than a preset value, the host device discards the physiological feature. 14. The man-machine interface system according to claim 1, wherein the slave device further comprises a memory unit storing system frequency calibration parameters. 15. The man-machine interface system according to claim 1, wherein the main device further compares the brightness of the first image frame information and the second image frame information with at least one threshold to determine the contact state . 16. The man-machine interface system according to claim 1, wherein the main device is based on two pieces of the first image frame information, one piece of the first image frame information and one piece of the second image frame information The frame information and the frame information of the two second images are used to calculate the displacement. 17. A finger mouse system, the finger mouse system comprising: An optical detection device, the optical detection device includes: a first light source emitting light of a first wavelength to the finger; a second light source emitting light of a second wavelength to the finger; a light source control unit, controlling the turning on and off of the first light source and the second light source; an image sensor for receiving reflected light from the finger at a sampling frequency to generate a plurality of first image frames illuminated relative to the first light source and a plurality of second image frames illuminated relative to the second light source; and a processing unit, calculating and outputting first image frame information about the first image frame of a preset sheet and second image frame information about the second image frame of a preset sheet; and The host computer receives and checks the first image frame information and the second image frame information, and calculates the displacement amount and the second image frame information according to the first image frame information and the second image frame information physiological characteristics. 18. The finger mouse system according to claim 17, wherein when the host computer checks that the first image frame information and the second image frame information are oversampled, the oversampled image frame information is discarded; When the host computer detects that the frame information of the first image and the frame information of the second image are missing, an interpolation program is executed. 19. The finger mouse system according to claim 17, wherein said host computer: divides each piece of said first image frame information into at least two parts and calculates the average brightness of each part, and analyzes said The average brightness of each part of the first image frame information to obtain the first brightness change; divide each of the second image frame information into at least two parts and calculate the average brightness of each part, using independent analyzing the average luminance of each part of the second image frame information to obtain a second luminance change; and calculating the physiological characteristic according to the first luminance change and the second luminance change . 20. A man-machine interface system, the man-machine interface system comprising: A finger mouse device providing two different wavelengths of light to illuminate a finger, receiving reflected light from said finger to generate a plurality of first image frames and a plurality of second images illuminated relative to said two different wavelengths of light frame, calculating and outputting first image frame information about the first image frame of a preset sheet and second image frame information about the second image frame of a preset sheet; and host, which contains: a processing unit, calculating displacement and physiological characteristics according to the first image frame information and the second image frame information; and The representation unit is used to respond to the displacement and the physiological characteristic.

Images