TWI510062B - Portable device, computing device with user motion sensing pointing device and method of the same - Google Patents

Description

Portable device, arithmetic device and device thereof with user motion detection pointing device law

The present invention relates generally to portable devices, and more particularly to computing devices having user motion control modules and hybrid memory.

Mobile communication systems typically include multiple base stations for communicating with mobile stations in different geographic transmission areas. Each base station provides an interface between the mobile station and the telecommunications network. The mobile phone system currently in use or under development has its system geographic coverage divided into smaller, independent cells that communicate with the network through fixed stations located within the cell. Mobile phones belonging to the system are free to move from one cell to another. When a user in the same system or in an external system wants to call an active user in the system, the network must have actual location information of the mobile phone.

In recent years, the price of mobile phones has been greatly reduced, and the general public has become affordable. It is quite common for a person to have more than one mobile phone. Some people even replace their mobile phones with the frequency of replacing clothes or hairstyles. Mobile phone manufacturers must introduce new styles with different looks, functions and styles more frequently to attract buyers' attention and capture favorable market share. In addition, conventional liquid crystal displays have many disadvantages, such as liquid crystal displays requiring a backlight module that increases the thickness of the panel and is not transparent. Liquid crystal materials and color filters are also essential components of liquid crystal displays. Moreover, the conventional liquid crystal display requires a large number of thin film transistor arrays, and the process is complicated and the resolution is not high enough. In addition, the antenna of the conventional device is embedded in the device, and is surrounded by a plurality of integrated circuits. Or obscured by components. Therefore, its signal reception is interrupted by numerous devices and electromagnetic shielding effects. In addition, the mouse is inconvenient for the user, especially the user of the portable device. Therefore, it is necessary to remove or change the design of the mouse device. For portability, it is necessary to make the projector as thin as possible. However, the purpose of this downsizing is hampered by the existing factors mentioned above.

It is an object of the present invention to provide a device having a hybrid memory and a high level display having a thin transparent panel on which a transparent antenna can be formed to minimize shielding effects.

Another object of the present invention is to provide an arithmetic device having a user motion detection module.

Another object of the present invention is to provide a pointing device (such as a mouse) having a user motion detecting module.

The present invention provides a portable device including a control unit; a display coupled to the control unit; and a dual wireless module coupled to the control unit for wireless data transmission, wherein the dual wireless module includes the first And a second wireless data transmission module to allow the user to select a communication with the external device. The dual wireless module further includes a management unit for managing the first and second wireless data transmission modules according to a policy engine. The first and second wireless data transmission modules include Bluetooth (BLUETOOTH) standard, 802.11x standard, wireless fidelity (Wi-Fi) standard, global interoperability microwave access (WiMAX) standard, and third generation mobile communication technology (3G). ) Standards and their upgraded versions. The portable device also includes a hybrid memory, which is non-volatile It is composed of a memory and a hard disk drive, and is also composed of a NOR type flash memory and a NAND type flash memory.

In another aspect, the present invention provides a portable device including a control unit; a display coupled to the control unit; and a hybrid memory coupled to the control unit, wherein the hybrid memory includes the first type Memory and second type of memory. The portable device also includes an embedded system coupled to the control unit to determine or assign tasks of the first type of memory and the second type of memory. The first type of memory comprises a hard disk drive, and the second type of memory comprises a non-volatile memory. In addition, the first type of memory includes a NOR type flash memory, and the second type of memory includes a NAND type flash memory, wherein the first type of memory and the second type of memory share an address decoder.

In still another aspect, the present invention provides a portable device including a control unit; a memory coupled to the control unit to store data; and a substantially transparent panel to minimize electromagnetic shielding effects, wherein substantially transparent The panel comprises a first substrate having a transparent electrode coupled to the control unit, a stacking gate formed on the first substrate and formed adjacent to the emitter, the second substrate having a phosphor material and spaced apart from the first substrate Wherein, when the electrons emitted by the emitter contact the phosphor, the phosphor will emit green, blue or red light. The portable device also includes an antenna located at a position corresponding to the substantially transparent panel to minimize electromagnetic shielding effects to improve signal reception/transmission performance. In addition, the portable device further includes at least one transparent antenna attached to the substantially transparent panel to minimize electromagnetic shielding effects.

In still another aspect, the present invention discloses an arithmetic device including a display, a detecting device for detecting a user's motion, and a mobile information generating module responsive to the detecting to generate an output signal, thereby generating The mobile information; and the cursor control module responsive to the movement information to drive the cursor on the display to correspond to the movement information. The computing device also includes a touch screen module coupled to the control unit to generate at least one button graphic accompanying the cursor on the display. In addition, the computing device also includes an input interface to allow the user to initiate a click action. The computing device includes, but is not limited to, a notebook computer, a personal computer, a mobile phone, a personal digital assistant (PDA), and equivalent or similar devices thereof.

In another aspect, the present invention discloses a digital camera including a control unit and a display; a detecting device for detecting an action of a user being photographed; and a mobile information generating module responsive to the detecting An output signal is generated to generate movement information; and an indicator control module responsive to the movement information to drive the face indicator on the display to correspond to the movement information. The digital camera also includes a wireless data transfer module coupled to the control unit for data transfer with an external device.

In still another aspect, the present invention discloses a method for controlling a virtual object by a user action in an electronic device, including detecting a user action by a detecting device; generating a control signal in response to the user action Detecting; and controlling virtual objects displayed on the display in response to control signals. The user actions described above are detected by a complementary metal oxide semiconductor (CMOS) sensor, a charge coupled device (CCD) sensor, or a touch sensor array. The above user actions include facial movements and eye movements. Action or finger movement. The method also includes the step of analyzing the user action prior to generating the control signal. The above analysis includes image change analysis of user actions. In addition, the present invention discloses a method for indicating a virtual object by user activity in an electronic device, comprising: detecting a user activity by the detecting device; generating a control signal in response to the user activity detection; A virtual object displayed on the display is controlled to respond to the control signal to execute the instruction.

In still another aspect, the present invention discloses a method for browsing web pages, files, emails, or images on a display of an electronic device, including displaying the first portion of a web page, file, email, or image on the display The display provides a sensor array thereon; determining whether the user is touching the sensor array; determining a user motion based on a user's touch; generating a control signal based on the user motion; and displaying the display based on the control signal The second part of the web page, file, email or image, where the control signal indicates where the user wants to browse. The user action described above includes at least one finger motion. The first portion does not overlap the second portion. In addition, the first portion at least partially overlaps the second portion.

In still another aspect, the present invention provides a pointing device for an electronic device (a mouse is shown as an example), comprising a housing for housing an element therein; and a user motion detecting device attached to the housing The upper side is for detecting the action of the user's finger; and the controller is stored in the outer casing for generating a control signal in response to the user motion detection. The user motion detecting device comprises a complementary metal oxide semiconductor (CMOS) sensor, a charge coupled device (CCD) sensor or a touch sensor array. Above User actions include finger movements.

The pointing device also includes a communication interface coupled to the controller. The above communication interface includes Universal Serial Bus (USB), IEEE1394, Firewire, IEEE 802.11x, Wireless Fidelity (Wi-Fi), Infrared (IR), Worldwide Interoperability for Microwave Access (WiMAX) or Bluetooth (BLUETOOTH) . The present pointing device also includes a power supply coupled to the controller. The above power supply is provided by wired, wireless, battery or solar energy.

In still another aspect, the present invention provides a pointing device for an electronic device, comprising a housing for housing an element therein, and a display attached to the housing for display, wherein the display includes an image icon To implement a predetermined function; a user motion detecting device attached to the display for detecting the action of the user's finger; and a controller stored in the housing for generating a control signal in response to the user's motion detection Measurement.

The user motion detecting device includes a touch sensor array. The pointing device also includes a communication interface coupled to the controller. The above communication interface includes Universal Serial Bus (USB), IEEE1394, Firewire, IEEE 802.11x, Wireless Fidelity (Wi-Fi), Infrared (IR), Worldwide Interoperability for Microwave Access (WiMAX) or Bluetooth (BLUETOOTH) . The pointing device also includes a power supply coupled to the controller, the power supply being provided by wire, wireless, battery or solar energy. The above display includes a liquid crystal display (LCD), a field emission display (FED), an organic light emitting diode display (OLED), an electroluminescent display (ELD), electronic inks, or electronic paper. The user motion detecting device is based on capacitive sensing, resistance sensing, surface acoustic wave sensing or thermal sensing.

The invention generally relates to an arithmetic device or a portable device. The device includes but is not limited to a mobile phone, a personal digital assistant (PDA), a smart phone, a notebook computer, a digital camera, a digital camera, a media player (MP3 or MP4), a pointing device (GPS), and the like or similar devices. .

The first figure is a schematic diagram illustrating the main components of a portable communication device according to an embodiment of the present invention. The portable communication device utilizes a panel having a transmitter and a transparent substrate. In the present embodiment, as shown in the first and second figures, the portable device 20 includes a radio frequency (RF) module 190. As is well known to those skilled in the art, a radio frequency (RF) module 190 includes an antenna. The antenna is coupled to a transceiver for receiving and transmitting signals. As is well known to those skilled in the art, radio frequency (RF) module 190 also includes a codec, a digital signal processor (DSP), and an analog digital (A/D) converter. Since the radio frequency (RF) module 190 is not a technical feature of the present invention, a detailed description thereof will be omitted. The other main components between the portable device 10 and the portable device 20 are similar, so the same component symbol means similar components, but the version, level and performance may be different. The present invention includes a control unit 100, an input/output (I/O) unit 150, an operating system 145, a hybrid memory 165, and a motion detection module 185 (such as eyeballs, finger movements). The portable device 10 or the portable device 20 may include other memory 155 such as a read only memory (ROM), a random access memory (RAM), and a flash memory. The radio frequency (RF) module 190 can perform functions of signal transmission and reception, frequency synthesis, baseband processing, and digital signal processing. If the portable device is a mobile phone, it may have a User Identification Module (SIM) The card hardware interface is used to accept a Subscriber Identity Module (SIM) card. Finally, the signal is transmitted to the terminal actuators, namely the speaker and microphone 195 or the input/output (I/O) unit 150.

The present invention further includes a wireless data transmission/reception module 220 coupled to the control unit 100. The wireless data transmission/reception module 220 is compatible with the BLUETOOTH standard, the radio frequency (RF) standard, the home wireless network (Home-RF) standard, and the 802.11x (x=a, b, g, n) standard. , Wireless Fidelity (Wi-Fi) standards, Worldwide Interoperability for Microwave Access (WiMAX) standards, Third Generation Mobile Communications Technology (3G) standards or higher-end versions. Its transmission range (air) is not secure, so it is encrypted in the wireless transmission network. In one embodiment, pair-wise encryption/decryption between each of the neighboring wireless network devices of the wireless transmission network is well known in the art. The data frame leaves from one wireless device to the other end of the same network from one end of the wireless transmission network and may require some encryption and decryption before reaching its final destination. The micro-projection module 230 can be incorporated into a portable device. The micro-projection module 230 can be referred to the inventor's U.S. Patent Application Serial No. 7,178,735.

As shown in the third figure, the present invention can also be selectively provided in a single unit with a hard disk drive incorporating flash memory to have the advantages of both. The hard disk drive includes a control unit 300 coupled to a disk module 320 having a spindle drive, a head mechanism, and the like. Detailed descriptions are omitted because of known techniques in the art. Memory controller 340 is coupled to control unit 300 and non-volatile memory 330 is coupled to memory controller 340. Application 350 is coupled to control unit 300 to execute the instructions. Hybrid hard disk drive integrates flash memory (non-volatile memory) 330 The device has the advantage of producing high speed access and low power consumption. Hard drives can be incorporated into portable media players, mobile phones, laptops or personal digital assistants (PDAs). The memory controller 340 is coupled to a non-volatile memory (flash memory) 330 to operate in accordance with an instruction or command. The above modes include a read mode, a write mode, an erase mode, and a programming mode. In addition, the memory controller 340 can be integrated into the control unit 300 according to the knowledge and technology of the integrated circuit. The embedded system 370 is selectively stored in an erasable/programmable storage medium such as a flash memory, a read only memory (ROM) or a hard disk. It can be an application, software or computer executable instruction serial. Alternatively, a read only memory (ROM) 380 can be embedded in the control unit 300. The data compiled into the read only memory (ROM) 380 will not be erased by the user. The embedded system 370 can be used to cause a computer or device to perform predetermined functions, such as operating a number of applications, opening devices, automatically backing up, or automatically executing files. In general, after powering up the device, the basic input/output system (BIOS) will perform the boot sequence before calling the operating system (OS). The basic input/output system (BIOS) is stored in a read only memory (ROM) 380. The program or instructions can be stored in the embedded system 370 to determine what type of data is stored in which type of storage device, such as a flash memory or a hard disk. Different types of volatile memory having a NOR type flash memory 410 and a NAND type flash memory 400 can also be utilized, as shown in the fourth figure. The above-described NOR type flash memory 410 and NAND type flash memory 400 can share a address decoder in one embodiment, as shown in the fourth figure. In this configuration, an arithmetic device such as a personal computer or a notebook computer can omit a hard disk drive. This scheme utilizes NOR type flash memory The device 410 plays and replaces the role of a hard disk to store the operating system and produces the advantages of high speed operation and low power consumption, while the data is stored in the NAND type flash memory 400. Of course, only a single flash memory type can be used instead of a hard disk. In this way, the computing speed will be faster than using a hard disk. In one embodiment, embedded system 370 is coupled to control unit 300 to determine or distribute the tasks of disk module 320 and non-volatile memory 330. Flash memory 330 (non-volatile memory) facilitates high-speed performance and low power consumption, while hard disk drives produce lower cost and higher capacity. This storage architecture is called a Complementary Storage Device (CSD). This complementary storage device (CSD) has the advantages of two types of memory. The process can reduce cost and increase performance by combining the two types of memory in the best memory ratio between the flash memory and the magnetic memory.

Traditionally, almost all devices contain an antenna, which is typically located within the device. The electromagnetic shielding effect produced by the shields, circuits, circuit boards and other components of the above devices often results in weak signal reception. If the antenna is transparent to the signal, its performance will be improved. Accordingly, another aspect of the present invention is to provide an antenna on a substantially transparent panel to minimize electromagnetic shielding effects, thereby improving signal reception/transmission performance. At least one transparent antenna is preferably attached to the substantially transparent panel to minimize electromagnetic shielding effects. An embodiment of the present invention is shown in the fifth figure, which is a schematic cross-sectional view of a field emission device according to the present invention. As shown in the fifth figure, a transparent substrate 500 is provided, and a transparent electrode 520 is formed on the transparent substrate 500. The material of the substrate 500 may be tantalum, glass, quartz or the like. The transparent electrode 520 can be controlled by indium tin oxide (ITO), conductive polymer or nano carbon It can be used as a transmitting electrode. The stack gate 510 is formed on the substrate 500, and the stack gate 510 covers a portion of the transparent electrode 520. An electron-emitting emitter 560 is formed on a portion of the transparent electrode 520. Each stacking gate 510 includes a mask layer 540 that covers a portion of the transparent electrode 520 and is formed by ultraviolet light etching the mask. The mask layer 540 is preferably transparent to visible light but opaque to ultraviolet light and may be made of an amorphous germanium layer. The ruthenium containing layer will be transparent when its thickness is sufficiently thin. The structure of the stack gate 510 includes a first insulating layer, a gate, a second insulating layer, and a focus gate formed on the substrate 500 in sequence. The gate insulating layer is preferably a hafnium oxide film having a thickness of 2.mu.m or more, and the gate is made of chromium and has a thickness of about 0.25.mu.m. The gate is used to attract an electron beam from the emitter 560. The focus gate acts as a collector for collecting electrons emitted by the emitter 560, whereby electrons can reach the phosphor layer 580 disposed above the emitter 560. If the device is used as a display, the substrate 500 can be a germanium substrate or a transparent substrate. Referring to the fifth figure, a front panel (substrate) 550 is disposed above the stacking gate 510. A number of visual images are displayed on the substrate 550. A phosphor layer 580 is attached to the bottom surface of the substrate 550, the bottom surface facing the stack gate 510, and a DC voltage applied to the phosphor layer 580 to facilitate emission of color light for display. If the phosphor layer has red, green, and blue phosphors, the phosphor can emit color light by mixing the excited light. When the phosphor material is excited by an electron beam uniformly distributed on the phosphor layer 580, it is preferable to emit red, green, and blue visible light. The spacer separating the substrate 550 from the stacking gate 510 is a black matrix layer, and is not illustrated to avoid blurring of the technical features of the present invention. Since the above thin film display is formed with a thin thickness, and the power is eliminated The power consumption is lower than that of the liquid crystal display, so the present invention provides a device of smaller size and lighter weight. Battery life can also be extended. The field emission device does not require the complex and power consuming backlights and filters required for liquid crystal displays. In addition, the device does not require a large number of thin film transistor arrays, so the main reasons for the high cost and productivity problems of active matrix liquid crystal displays have also disappeared. The resolution of the device can be improved by using the focus grid to parallelize the electrons attracted from the tiny tips. The emitter 560 preferably includes a carbon nanotube emitter to further reduce device size. Furthermore, the display can omit the liquid crystal material. In addition, field emission displays do not require the source/drain regions required for thin film transistors of liquid crystal displays.

In another embodiment, the display of the present invention is as shown in the sixth figure. The above display includes a transparent electrode 610 on the transparent substrate 600. A fluorescent film or powder 620 is attached to the upper surface of the lower transparent electrode 610. The fluorescent material preferably emits color light. The present invention comprises three such devices that display images in red, green, and blue primary colors, respectively. Each device emits a single color of light. Different powders will emit different colors. The upper transparent electrode 630 is formed on the fluorescent film or powder 620. The second transparent substrate 640 is formed on the upper transparent electrode 630. A biasing system is applied to the electrodes to inject holes and electrons, whereby the phosphors are excited by a combination of electrons and holes to emit red, green or blue visible light depending on the compound of the phosphor. Its components can be referred to an electroluminescent display (ELP).

In another embodiment, the wireless data transmission/reception module 220 includes a dual wireless module. Referring to the seventh embodiment, in an embodiment, the wireless data transmission/reception module 220 includes first and second wireless data transmission modules 700a and 700b for wireless transmission. Dual modules 700a and 700b are coupled to management unit 710 to manage dual modules in accordance with policies stored in policy engine 720. For example, the policy in policy engine 720 includes at least a transmission priority policy to determine which module is the default module for receiving/transmitting data. Policy engine 720 also includes switching conditions for switching between dual modules. For example, the signal strength is one of the factors used in the switching conditions. It also allows the user to set or modify conditions through the user interface. The first and second wireless modules 700a and 700b may be a BLUETOOTH standard, a radio frequency (RF) standard, a home wireless network (Home-RF) standard, and an 802.11x (x=a, b, g, n). Standard, Wireless Fidelity (Wi-Fi) standards, Worldwide Interoperability for Microwave Access (WiMAX) standards, Third Generation Mobile Communications (3G) standards or higher-level (upgrade) versions. The first wireless module 700a is preferably a wireless fidelity (Wi-Fi) standard, and the second wireless module 700b is preferably a Worldwide Interoperable Microwave Access (WiMAX) standard. Therefore, the present invention can produce the advantages of both. For example, the wireless fidelity (Wi-Fi) standard has a shorter access distance than the Worldwide Interoperability for Microwave Access (WiMAX) standard, but it consumes less power. In the office area, a wireless fidelity (Wi-Fi) wireless base station (AP) can be used to prevent others outside the office from accessing the server of the office. In another aspect, if the user wants to access or remotely control a terminal device located several miles away, the Worldwide Interoperability for Microwave Access (WiMAX) standard can be selected. The Wireless Fidelity (Wi-Fi) standard facilitates security and low power consumption, while the Worldwide Interoperability for Microwave Access (WiMAX) standard produces long-range and high-bandwidth effects. This module structure is called a dual wireless module (DWM). Dual Wireless Modules (DWM) have the advantages of two types of modules. The first wireless module 700a is compatible with the first communication protocol. The second wireless module 700b is compatible with the second communication protocol. The process can increase its performance by combining the two types of wireless modules with optimal configuration. The above agreement can be applied to a wireless local area network (WLAN) or a wireless metro network (WMAN).

As described above, a thick and complicated liquid crystal display panel can be removed, thereby reducing the thickness of the portable device, and many components such as a color filter, a liquid crystal material, and a backlight module can also be removed from the present invention. Furthermore, since the substrates 500, 550 can be formed using glass, quartz, or the like, the present invention can provide an unpredictable and interesting visual effect. When the device is in the sleep mode, the standby mode or the off state, since the panel is substantially transparent, the panel can be used as a digital photo frame or album. In addition, the user can change the color and image style displayed on the panel when the device is in the sleep mode or the off state. The user can determine the style and color of the panel cover. Furthermore, the transparent antenna can be attached to the transparent panel to provide better signal transmission because the electromagnetic shielding effect of the antenna will be minimal when the antenna is not placed in the device. In this embodiment, the antenna may be made of a material comprising an oxide of a metal or an alloy, wherein the metal is preferably one or more metals selected from the group consisting of gold, zinc, silver, palladium, platinum, rhodium, ruthenium, copper, Iron, nickel, cobalt, tin, titanium, indium, aluminum, antimony, gallium, antimony and antimony. The partially transparent material comprises a zinc oxide doped with tri-alumina.

In addition, referring to the eighth figure, the wireless data transmission/reception module 220 and the hybrid memory 808 can be applied to a digital camera or a digital camera. The device preferably includes a main body having a control unit 802; a display 804 formed on the main body and coupled to the control unit 802; the image capture element A member 806, formed in the body and coupled to the control unit 802; a hybrid memory 808 coupled to the control unit 802; a lens 810 formed over the body and coupled to the control unit 802 and to the image capture component 806 Corresponding; and a wireless data transmission/reception module 220 coupled to the control unit 802 for data transmission with an external device.

If the present invention is applied to a media player such as an MP3 player or an MP4 player, the player includes an analog digital (A/D) converter for converting the analog audio signal into a digital audio signal. The analog audio signal can come from a source coupled to the player. A digital signal processor (DSP) or an audio and/or video driver module, such as an MP3, MP4 codec, is coupled to an analog digital (A/D) converter for receiving digital audio signals. In an embodiment, the MP3 or MP4 codec is a firmware that includes an MPEG audio layer (eg, MP3, MP2, or both) codec or video codec (eg, MP4), and a digital signal processor (DSP) ) Performs firmware that contains different types of audio codecs (eg WMA, ACC, or both). In one embodiment, the firmware for the digital signal processor (DSP) also includes a video codec for encoding and decoding video (eg, MPEG-4 Release 1 / 2nd Edition / 3rd Edition, DivX 3.11) Version / Version 4.0 / Version 5.0, XviD, AVI/ASF, or any combination thereof). An MP3 (or MP4) codec and digital signal processor (DSP) are coupled to the non-volatile memory, which stores the compressed audio material. The user can select an audio file from non-volatile memory. A digital signal processor (DSP) is coupled to an audio processor that processes the digital audio signals in accordance with preset settings or user commands. The audio processor is coupled to a digital analog (D/A) converter that converts the digital audio signal into a class for the user Than the audio signal.

The present invention also provides a user motion detection or control module to control the cursor without the need for a mouse or trackpad. The computing device includes a display and a detecting device for detecting a user's motion. The mobile information generating device generates an output signal in response to the detection, thereby generating mobile information. The cursor control module drives the cursor on the display to respond to the movement information in response to the movement information. Referring now to the ninth to twelfth drawings, a schematic diagram of the basic components of the user motion control module 18500 incorporating the eyeball control module (or the limb control module) is shown in accordance with an embodiment of the present invention. The invention includes the steps of detecting user actions. The detection site can be an eyeball, a face, a limb, and the like. Here, eyeball detection is taken as an example to illustrate the technical features of the present invention. The subject's face or eyeball is located opposite the sensor 18510, whereby the subject's gaze is aligned along the centerline toward the pupil excitation source and the gaze target. The user action control module 18500 includes a sensor 18510 and a control unit 18515 to detect eye movements and generate control signals. Facial motions can also be used to practice the invention. The detection source 18505 is provided, and the pupil of the eye is illuminated by a detection source 18505, such as an infrared (IR) or a light emitting diode (LED). Preferably, two or more source light emitting diodes (LEDs) are used to project two spatially separated spots on the pupil of the subject. A dual-source light-emitting diode (LED) is constructed by placing two light-emitting diodes (LEDs) side by side on a panel 500a of a portable device. Light returned from the subject's eyeball is detected directly by the sensor 18510 or after passing through other optical mirrors or lenses. Another method is to detect the user's face, hand motion or image by the sensor 18510. Sensor 18510 It can be an optical sensor such as a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor. The output of sensor 18510 is input to processor or control unit 18515 to generate a control signal to cursor control module 18520 for controlling cursor 18530 on the display or panel. The eyeball detection method can be referred to U.S. Patent Nos. 7,130,447 and 5,943,754, assigned to Boeing Company (Chicago, IL), and U.S. Patent No. 5,422,690, which discloses an eyeball recognition and blink detection system. The detection source 18505 or its equivalent is preferably a position to scan the pupil of the eye. In this step, the pupil is illuminated by the detection source 18505, whereby the geometry of the pupil can be clearly depicted on the sensor 18510. Alternatively, the user's image (face) changes can be detected by the present invention. With image processing, the pupil position information is estimated to determine the position of the eyeball gaze display. The control signal can be used by the cursor control module 18520 to drive the cursor 18530 to the position where the eyeball is looking. A button graphic (or button image) 18535 can be generated by the image generator accompanying cursor 18530. In one embodiment, the image generator can be a touch screen module 18525, which can generate a touch screen image through a known touch screen technology, so that the user can click the touch screen to select the virtual button. input the command. In addition, the click signal can be input from the input interface 18540, such as the left and right buttons of the mini keyboard, the voice control through the microphone, or the eye movement through the sensor 18510. In an embodiment of voice control, another software or hardware may be required to identify the hardware and/or software handling object selection steps through sound. For example, the action on the left eye means clicking the left button, and the action on the right eye means clicking the right button. If both eyes are on, it can mean selecting an item from the list. Above preset Features can be implemented using programs or software. It will be appreciated by those skilled in the art that the preferred embodiments of the present invention are intended to illustrate the invention and not to limit the invention. Artists in this field can make modifications to their associations. Under the method disclosed by the present invention, the user can automatically move the cursor 18530 to omit the mouse. Similarly, as shown in FIG. 11, when reading a file displayed on the screen, a control signal can be used to drive the reel 18555 to move up or down without having to click on the reel 18555. Thus, the control signals generated by control unit 18515 will be provided to reel control module 18550 to drive reel 18555 on the display to move up or down, with the mouse or keyboard being omitted. An eyeball control screen indicator is provided. The eye tracking signal is performed in an operation performed by a processing tool within the processor or control unit 18515 to generate a cursor 18530 on the screen.

The sensor 18510 is electrically coupled to the control unit 18515 via a line. In a preferred embodiment, control unit 18515 includes a semiconductor integrated circuit or wafer configured to receive, interpret, and process electronic signals and provide output electronic signals. The output signal of control unit 18515 includes a signal indicative of eye movement, the movement of which is oriented in a direction corresponding to the direction of movement of the actual display upstream of the user. This embodiment considers the tow situation that the user may encounter. Sometimes some users need to drag a graphic or other object from one area of the screen to another. For this purpose on some computers, the user must keep the left button and control the pointing device at the same time. If the pointing device is a touchpad and the object has to be dragged across the screen for a long distance, sometimes the user's finger may reach the edge of the touchpad. Such a situation can be easily handled by the present invention. In this case, the control order The element 18515 can repeatedly send a command, for example, when the drag is clicked, the left button is pressed until the user's finger leaves the keyboard, that is, the button is stopped. This allows dragging even after the user's finger leaves the trackpad. U.S. Patent Application Serial No. 7,165,225, the disclosure of which is incorporated herein by reference to the entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire all U.S. Patent Application Serial No. 7,137,068, the disclosure of which is incorporated herein by reference in its entirety in its entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire entire all

Accordingly, the present invention provides a method for pointing a marker, such as cursor 18530, reel 18555 on a screen. The method includes detecting an action of the user (eg, eyeball, face, limb movement), and the sensor 18510 generates an output signal in response to the eyeball detection, thereby generating eye movement information. The cursor control module 18520 responsive to the user movement information to drive the cursor 18530 on the display to correspond to the movement information.

As such, the above method can be used for face tracking in the field of digital cameras or digital cameras to track the subject's face. With almost the same structure, the face indication (or indicator) module 18545 responds to the control signals to mark the face on the screen, thereby tracking the face of the user for the digital camera. The digital camera includes a control unit and a display; a detection source 18505 for detecting the eyeball of the user being photographed; and a sensor 18510 that detects the light from the return of the eyeball to generate an output signal, thereby generating an eye movement The cursor control module 18520 responds to the eye movement information to drive a facial indication module (face indicator) 18545 on the display to correspond to the eye movement information. The digital camera also includes a wireless data transmission/reception module. It is coupled to the control unit for data transfer with an external device.

As described above, the present invention discloses a user motion control (or detection) module 18500 for a computer or portable device. The module can be incorporated into a device adjacent to a mini-keyboard or keyboard area. Thereby, it can detect the image action of the user to move the cursor 18530. In some embodiments, a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor can be used to detect user motion, including facial expressions, facial motions, or finger motions. In such an application, the sensor 18510 can capture images and the controller can analyze the image changes to determine the movement of the cursor 18530. It can also be used to monitor and respond to facial expressions of the user. For example, user actions can be monitored by camera or camera. It is different from the trackball and control panel used in traditional notebook computers. The sensitivity, resolution, and handling of the control panel are not good. It should be noted by those skilled in the art that in this embodiment, the user motion detection module can be disposed on a mini keyboard adjacent to a notebook computer or a keyboard of a personal computer. The user motion detection module detects a user's image motion by the above method using a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor. Complementary metal-oxide-semiconductor (CMOS) sensors can be up to several megapixels in resolution. It accurately reflects the user's finger or facial movements.

In addition, the computer cursor 18530 or items or functions (such as opening files, closing files, copying, cutting, pasting files, etc.) can be controlled by user activities, such as by measuring human brain activity. The electroencephalograph (EEG) records the voltage fluctuations in the brain, which can be measured using electrode pads attached to the scalp. The brain wave (EEG) signal is derived from the cerebral cortex, which is a layer of several centimeters thick. The height of the spiral neurons tissue. If the user is focused on simple mental isolation such as squinting on the eyes, the brain may produce Alpha waves (8-13 Hz). The Beta wave (14-30 Hz) is a state of alertness to the spirit. The Theta wave (4-7 Hz) is usually associated with the early stages of sleep and frustration or disappointment. The Delta wave (below 3.5 Hz) is about deep sleep. An electromyography (EMG) sensor can be attached to human skin to sense and translate muscle impulses. In addition, the electrooculogram (EOG) signal can be sensed from the movement of the eyeball. U.S. Patent Application Serial No. 7,153,279, issued to George Washington University, discloses a brain contraction sensor. U.S. Patent Application Serial No. 7,171,262, which is assigned to PCT, discloses a monitor of vital signs. The twelfth figure is a schematic view of an exemplary embodiment of the above invention. The neural activity is tracked by a neural activity detecting device. The tracked neural activity preferably includes brain wave (EEG), ocular (EOG), and myoelectric (EMG) activities. Electronic signals representing neural activity are transmitted to the control unit via wire or wireless. The same electroencephalograph (EEG) reading can be monitored if the predetermined signal is sensed by the detecting device. For example, if the user is focused on certain actions, an Alpha wave (8-13 Hz) can be generated. Therefore, if the focus mode is detected, the system responds to the above signals and issues an order to "open file", "close file", "copy file", "click", "paste", "delete", "space" Or "Enter text" and other actions. Those skilled in the art should note that the state mode of potential users can be monitored prior to utilizing the system. Control unit 18515 is coupled to a signal receiver (not shown) that receives neural signals from sensor 18510 by antenna or wire. The operating system is operated, controlled and coordinated by a central processing unit (CPU) The functions of the various components of the system and the application software 18560, and thus the control module 18570. Such programs include programs for converting received neural signals into computer actions on the display screen. With the above device, the user can control the computer action by inputting neural information to the system through the sensor 18510. The setting of the above program will be described below in accordance with the present invention to allow the user to control the computer using the sensed neural signals. The above programs are installed in a computer to control computer functions and/or functions controlled by a computer using electronic signals. A procedural step is provided for predetermining a level of neural activity (or mode) indicative of the degree of user focus. A sensor 18510 is provided for monitoring the user's neural activity to determine when a predetermined level of neural activity is reached. Thereby, the user's brain wave (EEG) mode is determined. The user's neural activity is converted into an electrical signal to give instructions to perform software functions. Monitor the user's facial motion using an image sensor (a complementary metal oxide semiconductor (CMOS) sensor or a charge-coupled device (CCD) sensor) before the user's brain wave (EEG) mode is determined (or Eye movements) to determine where the user is looking at the screen.

Therefore, the present invention discloses a method for controlling a cursor of a computing device by a user action, comprising: detecting a user action by the detecting device; generating a control signal in response to the user motion detection; and controlling the display The cursor is displayed in response to the above control signal. The user action is detected by a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor, and the user action includes a facial motion, an eye motion, or a finger motion. The method also includes the step of analyzing the user action prior to generating the control signal. The above analysis involves analyzing the shadow of the user's actions. Like change.

The method for indicating an object of the computing device by the user activity comprises: detecting a user activity by the detecting device; generating a control signal in response to the user activity detection; and controlling an object displayed on the display to respond to the control signal, In order to execute the instructions. User activity is detected by a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor, and user activity includes facial motion, eye motion, or finger motion. The method also includes the step of analyzing user activity prior to generating the control signal. The above analysis involves analyzing image changes in user activity. In addition, user activity is detected by brain wave (EEG), myoelectric (EMG), and EOG sensors. Control signals include cursor movement, text input, and software application commands.

The method for indicating an object of the computing device by the user activity comprises detecting a user activity by a detecting device such as a complementary metal oxide semiconductor (CMOS) sensor or a charge coupled device (CCD) sensor; generating a control signal In response to the user activity detection; controlling an object displayed on the display to respond to the control signal; and detecting brain waves by an electroencephalogram (EEG), an electromyography (EMG), an EOG sensor ( EEG), myoelectric (EMG), and EOG modes to execute instructions.

A thirteenth diagram is a simplified schematic diagram of a portable electronic device 1310 in accordance with an embodiment of the present invention. The portable electronic device 1310 can be, for example, a handheld electronic device such as a mobile phone, a personal digital assistant (PDA), a media player, and a positioning device (GPS), or a notebook computer, a tablet PC (Tablet PCs), and a game machine. The portable electronic device 1310 is connected to the display A sensor array 1320 is mounted on the 1330. The assembly sensor array 1320 is used to detect the presence of a target such as a finger, as well as its position and the pressure applied by the finger or palm on the panel surface. In an embodiment, the sensor array 1320 can be based on capacitive sensing, resistance sensing, surface acoustic wave sensing, thermal sensing, and/or its equivalent. The sensor array 1320 can also be pressure sensed, such as strain gauges, force sensitive resisters, load cells, pressure plates, piezoelectric transducers. Or its equivalent.

As shown in the thirteenth diagram, the portable electronic device 1310 includes a housing and a display 1330 that is located within the front surface of the housing. The portable electronic device 1310 also includes a sensor array 1320 that is located on the display 1330. The thirteenth diagram is a perspective view of a portable electronic device 1310 in accordance with an embodiment of the present invention. The portable electronic device 1310 includes a housing that houses different electronic components including integrated circuit wafers and the like therein. In an embodiment, the housing may include a microprocessor (such as a central processing unit (CPU)), a memory (such as a read only memory (ROM), a random access memory (RAM)), and a power supply (such as a battery). ), printed circuit board (PCB), hard disk drive or other memory (such as flash memory) and / or different input / output (I / O) support circuits. The portable electronic device 1310 also includes a display 1330 disposed in the housing and permeable to the opening of the housing. The display 1330 is typically placed on the front surface of the portable electronic device 1310. Display 1330 provides visual information in the form of text, symbols or patterns. In an embodiment, the display 1330 can be a liquid crystal display (LCD), a field emission display (FED), an organic light emitting diode display (OLED), an electroluminescent display (ELD), or A display based on electronic inks or electronic paper.

To generate user input, the portable electronic device 1310 can include a sensor array 1320 that is located on a transparent input panel in front of the display 1330. When a target, such as a finger, moves from a surface of the sensor array 1320 at a particular location on the sensor array 1320 across the surface of the sensor array 1320, for example, moving linearly, radially, rotationally, etc. and/or tapping the sensor array with a finger At 1320, sensor array 1320 produces an input signal. In most cases, the sensor array 1320 allows the user to initially move in the graphical user interface (GUI) by simply touching the display screen through the finger. For example, the sensor array 1320 recognizes the touch and touch positions on the display 1330, and the translation controller 1340 of the portable electronic device 1310 translates the touch, and then performs an action based on the touch event. According to an embodiment, the sensor array 1320 can be a multi-touch sensing device that has the function of sensing multiple touch points at the same time and notifying the multi-touch to the translation controller 1340 of the handheld electronic device. In one implementation, the sensor array 1320 is a multi-point capacitive touch screen that is divided into a plurality of independent and spatially distinguishable sensing points, nodes or regions distributed over the display 1330. The generally transparent sensing points are scattered around the sensor array 1320, with each sensing point representing a different display surface location. The sense points can be placed in a gate or array of pixels, wherein each pixelated sense point can produce a signal. A signal is generated whenever an object is placed at a sensing point. Multiple signals are generated when the object system is placed on multiple sensing points or when an object moves between or over multiple sensing points. Sensing point is usually touch The curtain plane is mapped into a coordinate system, such as a Cartesian coordinate system or a Polar coordinate system.

The portable electronic device 1310 can be designed to recognize the gesture applied to the sensor array 1320 and control the virtual object orientation of the portable electronic device 1310 based on the gesture described above. The above gestures can be formed by various specific finger movements. The portable electronic device 1310 can include a gesture program, which can be part of the operating system or a separate application.

In one embodiment, the sensing input device is mapped to the display. When mapping is performed, the point on the sensing input device overlaps with the point on the display, i.e., has the same coordinates (x and y). Therefore, when the user touches the surface of the sensing input device, it is like the image of the same position where the user is touching the display. As shown, the sensor array 1320 is segmented into a number of independent and spatially distinguishable sensing points (or regions) that are disposed in individual components. The sensing points are typically scattered around individual components, each sensing point representing a different component and surface location of the portable device 10. The sense points can be placed in a gate or array of pixels, wherein each pixelated sense point can produce a signal. The number and configuration of sensing points can vary widely. The number and configuration of sensing points typically depends on the desired resolution of the touch surface and can vary across the display to achieve any desired cost-to-function compromise. In this embodiment, a signal is generated each time a finger is placed at a position on a sensing point. Multiple signals are generated when the finger is placed over multiple sensing points or when the finger moves between or over multiple sensing points. Those skilled in the art should be able to appreciate that the number, combination, and frequency of signals during a known time period may be indicative of the size, position, orientation of the fingers or palms on the surface of the device, Speed, acceleration and pressure. In one example, the control system can be a microcontroller located within the housing of the portable electronic device 1310.

The signal generated at the sensing point can be used to determine how the user wants to move the web page displayed on the display. In one example, each portion of the hand that is in contact with the device produces a contact block region. Each contact block region covers a plurality of sensing points, thereby generating a number of signals. Several signals can be aggregated to form a signal that represents how the user is moving the web page. In an embodiment, the difference between the current signal and the previous hand signal may indicate that the user wants to implement the function of moving the webpage. Significant differences indicate that the user wants to implement a function. The change in the contact block interval is more representative of a particular mobile signal. In the mapping, the touch surface is divided into one or more button areas representing the device area, and when the button area is selected, the device area implements a specific button function associated with the button area. The button zone having the most significant contact block change between the first and second hand signals is the area that would normally be implemented. The position and size of the button area can also be customized, such as the previous page, the next page, and the like. Customization can be implemented by the user and/or device. However, as shown in Figure 15, the display of the portable device is too small to be displayed by the display of the entire web page (or file, image). The display of the personal computer can browse almost the entire webpage information page, but it cannot achieve the purpose of the portable device having a small screen. As shown in the fifteenth figure, due to the limitation of the size of the display of the portable device, the user can only browse one part of the actual webpage instead of the entire webpage information page. The user cannot view the area other than the actual display window indicated in the fifteenth figure. Traditionally, users have to click on a scroll or button to scroll up and down the page (or file, image, email). It is quite inconvenient for the user, traditional The method is not user friendly design. However, the present invention provides its solution. The user can move their finger on the screen to introduce user (or finger) movement information to indicate where the user wants to see. For example, as shown in one of the fifteenth figures, the user can move his or her finger, and the mobile system passes through the area B to the area C from the area A as indicated by the arrow to browse the web page of the part (area C). Therefore, as shown in Figure 15 and Figure 15 and Figure 15, when the display is too small, especially for a portable device, the user can move his finger on the display to view any part of the webpage (image or file).

Figure 14 is a schematic illustration of the method of operation in accordance with an embodiment of the present invention. The method 1400 generally begins at block 1422 where the device is in a standby state. Standby usually means that the device is in a ready state, waiting for an event to occur, that is, the user takes an initial action. Following block 1422, the process continues to block 1424 where a determination is made as to whether the user is touching the device. It is typically implemented with a touch sensing device that can generate a signal when the hand approaches the device, and the control system is configured to monitor the activity of the touch sensing device. If it is determined that the user has not touched the device, then the process proceeds to block 1422, thereby maintaining the device in a standby state. If it is determined that the user is touching the device, then the process continues to block 1426 where the user has determined.

In one embodiment, once the second position is determined, the step continues to block 1428, wherein at least two sense point signals are detected by the controller. Following block 1428, the process continues to block 1430 where the touch event is monitored and the control signal is generated based on the touch event. Connection block 1430. The process continues to block 1432, wherein the control signal is used to notify the application software in the device to move the webpage displayed on the screen instead of moving the webpage by clicking the scroll with a button, a cursor or a stylus.

The processor described above can be implemented on a single wafer, multiple wafers, or multiple electronic components. For example, different structures may be utilized on the processor, including dedicated or embedded processors, single purpose processors, controllers, special purpose integrated circuits (ASICs), and the like. In most cases, the processor operates with the operating system to execute computer code and generate and utilize data. The operating system can be a known operating system such as OS/2, DOS, Unix, Linux, and Palm OS. The memory system provides space for storing computer code. The memory may include read only memory (ROM), random access memory (RAM), hard disk drive, flash memory, and/or its equivalent. The display is typically configured to display a graphical user interface that provides an easy-to-use interface between the user of the electronic device and the operating system or the application running thereon. The electronic device also includes a touch screen that is operatively coupled to the processor. The touch screen is configured to transfer data from outside to the device. The electronic device also includes a sensing device operatively coupled to the processor. The sensing device can also be used to issue a webpage movement instruction.

Examples of handheld devices include personal digital assistants (PDAs), mobile phones, media players, game consoles, video cameras, pointing device receivers, and equivalents thereof. Thus, the user can move a web page, image or file displayed on the display by moving the finger directly over the sensor array. The user can directly move the webpage, file, image, and icon displayed on the display by his or her hand or finger. This can also be applied to a general personal computer.

From the above description and with reference to the following figures, the following description of the invention will be set forth in the accompanying drawings. However, the invention may be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein. Such devices preferably utilize, for example, the BLUETOOTH standard, the radio frequency (RF) standard, the home wireless network (Home-RF) standard, the 802.11x (x=a, b, g, n) standard, wireless protection. The Wi-Fi standard, the Worldwide Interoperability for Microwave Access (WiMAX) standard, and the Third Generation Mobile Communication Technology (3G) standard operate in the wireless technology mode. Peripherals can communicate with integrated wireless technology in a portable computer or personal computer. It may also contain security devices such as biometric devices such as fingerprint scanners. Fig. 16 is a schematic diagram showing a relatively thin pointing device (a mouse is exemplified) according to an exemplary embodiment of the present invention. The user motion detection pointing device 1600 is a flat or folded structure and can be stored in a slot or a connection port in the portable computer. This feature is convenient for collection and charging. The user motion detection pointing device 1600 can include a main body portion that can accommodate electronic components and functional areas for selecting functions in the computer, such as left and right keys, and scroll wheel functions. Since the device that can be mounted in the slot or the port is generally only 3 to 10 mm high. Those skilled in the art will appreciate that the present invention is not an optical mouse or mechanical mouse to detect changes in the surface of the table. The present invention is a user motion detection structure. It is different from the traditional method.

The upper surface of the user motion detection pointing device (the mouse is exemplified) 1600 includes a detector (the touch panel is exemplified). The user motion detection pointing device 1600 is configured with a sensor array 1620 on the device. The sensor array 1620 is configured to detect the presence of an object such as a finger, and its position or hand Finger or palm pressure applied to the surface of the panel. In one embodiment, the sensor array 1620 can be based on capacitive sensing, resistance sensing, surface acoustic wave sensing, thermal sensing, and/or its equivalent. The sensor array 1620 can also be pressure sensed, such as strain gauges, force sensitive resisters, load cells, pressure plates, piezoelectric transducers. , image sensor or its equivalent. The user motion detection pointing device (mouse) 1600 can also include a display 1630 disposed in the housing and viewable through the opening of the housing. Display 1630 provides visual information in the form of text, symbols or patterns. In an embodiment, the display 1630 can be a liquid crystal display (LCD), a field emission display (FED), an organic light emitting diode display (OLED), an electroluminescent display (ELD), or an electronic ink based electronic device. A display of electronic paper.

To generate user input, the user motion detection pointing device 1600 can include a sensor array 1620, which is a transparent input panel located in front of the display 1630, but the display is not necessary (the mouse usually does not need to have a display), for example The detector array 1620 can be disposed on the side of the display 1630 as shown in FIG. When a target, such as a finger, moves from a surface located at a particular location on the sensor array 1620 across the surface of the sensor array 1620, such as linear, radial, rotary, etc., and/or taps the sensor array by a finger At 1620, sensor array 1620 produces an input signal. In most cases, the sensor array 1620 allows the user to initiate movement in the graphical user interface (GUI) by simply touching the proximity sensor array 1620 through the finger. For example, sensor array 1620 is recognized on display 1630 The touch and touch position, and the translation controller 1640 of the user motion detection pointing device 1600 translates the touch, and then performs an action based on the touch event. According to an embodiment, the sensor array 1620 can be a multi-touch or image sensing device that senses multiple touch points at the same time and notifies the multi-touch to the user motion detection pointing device 1600. The capabilities of the controller 1640. In one implementation, the sensor array 1620 is a multi-point capacitive touch screen that is segmented into a plurality of independent and spatially distinguishable sensing points, nodes or regions that are spread over the display 1630. The generally transparent sensing points are scattered around the sensor array 1620, with each sensing point representing a different display surface location.

The sense points can be placed in a gate or array of pixels, wherein each pixelated sense point can produce a signal. The user motion detection pointing device 1600 can be designed to recognize gestures applied to the sensor array 1620 and control external devices 1610 such as virtual objects such as cursors, icons of the computer based on the gestures described above. The above gestures can be formed by various specific finger movements. The user motion detection pointing device 1600 can include a gesture program, which can be part of the operating system or an independent application.

Display 1630 can include virtual button graphics 1650 to function as left and right button functions of conventional physical buttons for selecting, clicking on virtual graphics on a computer display or performing certain functions. While the sixteenth embodiment illustrates the basic features of the pointing device of the present invention, the present invention provides additional features to the user. The virtual graphics described above may include a virtual middle button that provides the functionality of a "middle key/wheel" type. A virtual scroll wheel can also be provided that includes the ability to trigger a button when a downward force is applied to produce a scroll wheel that is similar to moving back and forth. The function. Optical detection can also be used. It also provides a scrolling function (for example on a computer screen) that lasts until the release touches. All virtual buttons perform the same functions as buttons on traditional standard mice. Tap the virtual button area to provide additional functionality. Communication interface 1660 is coupled to translation controller 1640 to transmit signals to external device 1610, such as a computer. The translation controller 1640 can be a wafer, a processor, or an integrated circuit. The communication interface 1660 can be a wired interface such as a universal serial bus (USB), IEEE 1394 or Firewire, or a wireless interface such as IEEE 802.11x (x=a, b, g, n), wireless fidelity (Wi-Fi) ), infrared (IR), global interoperability microwave access (WiMAX), Bluetooth (BLUETOOTH). Power supply 1670 is coupled to translation controller 1640 and may be provided by wire, wireless, battery or solar.

Another embodiment of the present pointing device utilizes a semiconductor device that captures a finger image at a rate of, for example, 1,500 per second instead of touching the sensing array. For example, a complementary metal-oxide-semiconductor (CMOS) sensor captures the action of a finger, and the sensor transmits each image to a digital signal processor (DSP) that detects changes in the image. A charge coupled device (CCD) sensor can also be used to handle this function. The processor determines how far the finger has moved on the surface of the pointing device based on the pattern change on the image sequence and transmits its coordinates to the external device 1610, such as a computer, and the external device 1610 moves the cursor based on the coordinate received from the pointing device. Or image. Thus, a complementary metal oxide semiconductor (CMOS) pointing device can provide improved tracking resolution without the need for moving parts and can be applied to a variety of different surfaces to perform all of the functions of the various embodiments described above.

Those skilled in the art will appreciate that such pointing devices can be separate devices or integrated into electronic devices such as personal computers, notebook computers, keyboards, mobile phones, personal digital assistants (PDAs), media players, and the like. Furthermore, the present invention can include a memory 1680 coupled to display 1630 and/or controller 1640 for storing data, applications (software) or information. If the application is a media player, it can play audio (eg MP3) or video (eg MP4) format files. If the application is a digital photo player or a digital photo album, it can display photos or videos. If memory 1680 contains non-volatile memory, it can function as a flash drive. The memory 1680 can be built into a pointing device (such as a mouse) or a memory card 1680 that is inserted from a memory card slot and coupled by a memory connector. The user can access the file or open the file from the memory card 1680. Therefore, it can be used as a photo display or a digital memo. The method can also be applied to a keyboard, that is, to a keyboard or a keyboard area.

The above description is a preferred embodiment of the invention. Those skilled in the art should be able to understand the invention and not to limit the scope of the patent claims claimed herein. The scope of patent protection is subject to the scope of the patent application and its equivalent fields. Any modification or refinement made by those skilled in the art without departing from the spirit or scope of the present invention is equivalent to the equivalent change or design made in the spirit of the present disclosure, and should be included in the following patent application scope. Inside.

10‧‧‧Portable device

20‧‧‧Portable device

100‧‧‧Control unit

145‧‧‧Operating system

150‧‧‧Output unit

155‧‧‧ memory

165‧‧‧Mixed memory

185‧‧‧Action Control Module

190‧‧‧Wireless RF Module

195‧‧‧Speaker/Microphone

220‧‧‧Wireless data transmission/reception module

230‧‧‧Micro Projection Module

300‧‧‧Control unit

320‧‧‧Disk Module

330‧‧‧Non-volatile memory

340‧‧‧ memory controller

350‧‧‧Application

370‧‧‧ embedded system

380‧‧‧Reading memory

400‧‧‧NAND type flash memory

410‧‧‧NOR type flash memory

500‧‧‧Substrate

500a‧‧‧ panel

510‧‧‧Stacking gate

520‧‧‧Transparent electrode

540‧‧‧mask layer

550‧‧‧Front panel (substrate)

560‧‧ ‧shoot

580‧‧‧Fluorescent layer

600‧‧‧Transparent substrate

610‧‧‧Transparent electrode

620‧‧‧Flame film or powder

630‧‧‧Upper transparent electrode

640‧‧‧Second transparent substrate

700a‧‧‧First wireless module

700b‧‧‧Second wireless module

710‧‧‧Management unit

720‧‧‧Strategy Engine

802‧‧‧Control unit

804‧‧‧ display

806‧‧‧Image capture component

808‧‧‧Mixed memory

810‧‧‧ lens

1310‧‧‧Portable electronic device

1320‧‧‧ Sensor array

1330‧‧‧Display

1340‧‧‧Translation Controller

1400‧‧‧ method

1422‧‧‧ square

1424‧‧‧ square

1428‧‧‧ square

1430‧‧‧ square

1432‧‧‧Box

1600‧‧‧User Action Detective Point pointing device

1610‧‧‧External devices

1620‧‧‧Sensor array

1630‧‧‧ display

1640‧‧‧Translation Controller

1650‧‧‧ button graphics

1660‧‧‧Communication interface

1670‧‧‧Power supply

1680‧‧‧ memory

18500‧‧‧User motion control module

18505‧‧‧Detection source

18510‧‧‧Sensor

18515‧‧‧Control unit

18520‧‧‧ cursor control module

18525‧‧‧Touch screen module

18530‧‧‧ cursor

18535‧‧‧Key graphics

18540‧‧‧Input interface

18545‧‧‧Face indicator module (face indicator)

18550‧‧‧Reel Control Module

18555‧‧‧ reel

18560‧‧‧Application software

18570‧‧‧ function module

The invention may be understood by the description of the preferred embodiments and the detailed description and the accompanying drawings. However, artisans in this field should be able to understand All of the preferred embodiments of the present invention are intended to illustrate and not to limit the scope of the claims of the present invention, in which:

The first figure is a schematic illustration of a portable device in accordance with the present invention.

The second figure is a schematic view of a portable device in accordance with the present invention.

The third figure is a schematic diagram of a hybrid memory in accordance with the present invention.

The fourth figure is a schematic diagram of a hybrid memory according to the present invention.

Figure 5 is a schematic illustration of a field emission device in accordance with the present invention.

Figure 6 is a schematic illustration of electroluminescent light (EL) in accordance with the present invention.

The seventh diagram is a schematic diagram of a dual wireless module in accordance with the present invention.

The eighth figure is a schematic illustration of a digital camera in accordance with the present invention.

9 through 12 are schematic views of a user motion control module in accordance with the present invention.

Figure 13 is a schematic illustration of a portable electronic device in accordance with the present invention.

Figure 14 is a flow chart of the method of operation in accordance with the present invention.

The fifteenth to fifteenth twoth drawings are schematic views of the flow according to the present invention.

Figure 16 is a schematic illustration of a pointing device in accordance with an embodiment of the present invention.

10‧‧‧Portable device

100‧‧‧Control unit

145‧‧‧Operating system

150‧‧‧Output unit

155‧‧‧ memory

165‧‧‧Mixed memory

185‧‧‧Action Control Module

220‧‧‧Wireless data transmission/reception module

230‧‧‧Micro Projection Module

500‧‧‧Substrate

500a‧‧‧ panel

560‧‧ ‧shoot

580‧‧‧Fluorescent layer

550‧‧‧Front panel (substrate)

Claims (9)

A method for controlling a virtual object by a user's eyeball image change in an electronic computing device having a display, comprising: providing at least one image sensor to the display of the electronic computing device to capture the display An image of the user, wherein the at least one image sensor comprises a complementary metal oxide semiconductor or a charge coupled device sensor, wherein the at least one image sensor is configured to detect an eyeball image change of the user Outputting a first control signal to a virtual object without touching the display, the user is located in front of the at least one image sensor and maintaining a distance from the display; and the at least one image sensor is used to capture the use The image of the eyeball does not need to touch the display; analyzes the image change of the eyeball; generates a first control signal in response to the image change of the eyeball; transmits a microphone and a voice control software through the electronic computing device to provide a Sounding and discriminating to output a second control signal; and controlling the virtual object displayed on the display In response to the first control signal, the second control signal or a combination thereof, without touching the display of the electronic computing device. A method for controlling a virtual object by an eye movement of a user in an electronic computing device having a display according to claim 1, further comprising an infrared detector to detect the user. A method for controlling a virtual object by an eye movement of a user in an electronic computing device having a display according to claim 1, wherein the first control signal or the second control signal comprises cursor movement, text input , application instructions, web page movement or image size changes. A method for controlling a virtual object by an eyeball image change and a limb motion of a user in an electronic computing device having a display, comprising: providing at least one image sensor to the display of the electronic computing device to capture the An image of the user in front of the display, wherein the at least one image sensor comprises a complementary metal oxide semiconductor or a charge coupled device sensor, wherein the at least one image sensor is configured to detect the eyeball of the user The image is changed to output a first control signal to a virtual object without touching the display, the user is located in front of the at least one image sensor and maintains a distance from the display; by the at least one image sensor Capturing the image of the eyeball of the user without touching the display; analyzing the image change of the eye movement; generating a first control signal to respond to the image change of the eyeball; and detecting the image sensor by the at least one image sensor Measuring the image of the user's movement of the limb without touching the display; analyzing the image change of the limb movement; generating a second Braking signal in response to the image change detection of the body movement; and The virtual object displayed on the display is controlled to respond to the first control signal, the second control signal, or a combination thereof without touching the display of the electronic computing device. A method for controlling a virtual object by an eye movement and a limb motion of a user in an electronic computing device having a display according to claim 4, further comprising an infrared detector to detect the user. A method for controlling a virtual object by an eye movement and a limb motion of a user in an electronic computing device having a display according to claim 4, wherein the first control signal or the second control signal includes a cursor movement , text input, application commands, web page movement or image size changes. A method for controlling a virtual object by a user's eyeball image change and finger motion in an electronic computing device having a display, comprising: providing at least one image sensor to the display of the electronic computing device to capture the An image of the user in front of the display, wherein the at least one image sensor comprises a complementary metal oxide semiconductor or a charge coupled device sensor, wherein the at least one image sensor is configured to detect the eyeball of the user The image is outputting a first control signal to a virtual object without touching the display, and the user is located in the at least one image sensing Before the device is kept at a distance from the display; the at least one image sensor is used to capture the image of the eyeball of the user without touching the display; analyzing the image change of the eye movement; generating a first control signal The image change in response to the eyeball detection; the image of the finger motion of the user is captured by the at least one image sensor without touching the display; analyzing the image change of the finger motion; generating a second Controlling the signal in response to the image change detected by the finger motion; and controlling the virtual object displayed on the display to respond to the first control signal, the second control signal, or a combination thereof without touching the electronic computing device The display. A method for controlling a virtual object by an eye movement and a finger movement of a user in an electronic computing device having a display according to claim 7, further comprising an infrared detector to detect the user. A method for controlling a virtual object by an eye movement and a finger movement of a user in an electronic computing device having a display according to claim 7, wherein the first control signal or the second control signal comprises a cursor movement , text input, application commands, web page movement or image size changes.