SE515238C2 - Mobile phone, includes microphone for converting noise from hand movement over phone surface into e.g. text, cursor movement or command key activation on display screen - Google Patents

Abstract

The phone (1) has a microphone (5) which picks up different noises created as e.g. a finger or pen is moved over different patterns and/or texturing on a phone surface (3) in order to write down information. This data is converted into an electronic format for sending to a processor that converts the data into digital format for e.g. moving a cursor, activating a command or bringing up text on the display screen (4). An Independent claim is also included for a method of recognizing hand movements in order to e.g. move a cursor, activate a command key or bring up text on a display screen of a portable communication device.

Description

25 30 35 -. 51 5 2:58; 'H'¿3;: ”n': _ =: u“ Lf 2000-10-26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ ILB74090551 .dtJC MS Yet another solution to feed messages in a device are described in US-A-3909785, wherein a plate or a data surface has a plurality of delimited character areas, each area having the plurality of delimited sectors corresponding to digital coordinates, which define different characters to be entered. These are fed in with the help of a needle or pen that emits sound energy pulses. The sound energy pulses are registered continuously, whereby the sectors passed by the pen are identified and give a respective sign.

The main problem with the first-mentioned solution is that the keypad and monitor are often very small. There are usually not enough buttons for the entire alphabet, and most buttons must be pressed to enter a single letter. The second solution with a touch-sensitive screen over the monitor provides a lower contrast of the monitor and makes the monitor thicker and more expensive.

Small monitors also make it difficult to achieve a high resolution when a touch screen is used with a portable device, and when using a micro screen, a touch screen cannot be used as an input device. The third solution uses an extended monitor, which increases the size and cost of the device. The fourth solution uses a large impractical display, and only a specific pen or needle can be used to enter the characters, thereby restricting access to the device for another user without this specific pen, and if the pen is forgotten, the plate or data surface is unusable. .

SUMMARY OF THE INVENTION The object of the present invention is to provide a communication device with the possibility of entering handwritten messages more easily and faster.

This object is achieved for a portable communication device, comprising a microphone and a processing unit ~ un> qo 10 15 20 25 30 35 o 515 258 2000-10-26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ 1B74090EN .dec MS 3 for receiving a message written by a user on a surface of the device, and converting the message into digital data. The processing unit is operatively connected to the microphone, so that different sounds generated by movement over different patterns and / or structures on a surface of the communication device, when the user writes by hand on said surface, can be acoustically detected by the microphone and conducted in electrical form to the treatment center.

The movement of e.g. a pencil or tip over different combinations of said patterns and / or structures on the surface corresponds to different text entries or commands.

A surface of the communication device preferably has a specific design, the surface being divided into the majority of areas, which have sub-patterns for creating specific sounds when e.g. the pen or tip in contact with the surface moves towards these sub-patterns.

By equipping the communication device with a lid, which has a specific surface, and the treatment unit according to the invention, the following advantages are achieved. Entering information is simplified by not having to use the small and impractical buttons, thus avoiding the time-consuming pressing of the buttons. Notes and messages can thus be entered into the device more easily and faster. Commands for different functions can also be entered and executed with sound recognition. In addition, there is no particularly large extra cost to include a processing device in a communication device that already has a microphone, since all the hardware is already installed. Any reasonably sharp object can also be used to enter messages.

The object of the invention has also been achieved by a method of detecting and identifying sound by means of the microphone together with the processing unit and converting the sound into digital data. The digital data can be routed to the display screen on the communication device or used for h. ... to execute commands that control the communication device.

Other objects of the present invention are described in the following detailed features and advantages of the foregoing description and drawings as well as the appended claims.

Brief Description of the Drawings The present invention will now be described in further detail with reference to the accompanying drawings, in which: Fig. 1 is a perspective view, as a display device; Fig. 2 is a perspective view showing a communicator showing a communication device in a position where information can be entered on a lid according to the invention; Fig. 3 is a front view showing a preferred embodiment of a surface of a lid according to the invention; Fig. 4 is a front view showing a preferred embodiment of a surface of a lid according to the invention; Fig. 5 is a sectional view showing a preferred embodiment of a pattern on a surface according to the invention; Fig. 6 is a sectional view, as an embodiment of a pattern on the ring; Fig. 7 is a sectional view, as an embodiment of a pattern on the opening; Fig. 8 is a sectional view showing another preferred lid according to the invention showing yet another preferred lid according to the invention showing another preferred embodiment of a pattern of a lid according to the invention; and Fig. 9 is a flow chart, detecting, showing a method of identifying, converting and entering data according to the invention in a communication device.

W U 20 25 30 35 515 ßßë fi 2000-10-26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ IB740905 !! Detailed Description of the Invention FIG. lid 3. 3 'on the lid 3. The communication device is preferably 1-2 shows a communication device 1 with a FIG. 3-4 show two different embodiments of a surface a handheld device for entering messages in it or commands for controlling it, such as a portable telephone or mobile telephone. the lid 3, The device comprises a housing 2, a monitor 4, at least one microphone 5, a keypad 6, an antenna 8, loudspeaker openings 9 and a processing unit, which is operatively connected to the screen 4, the microphone 5 and the keypad 6. More than a microphone 5 can be used for a more accurate detection of sound and position of a tip 7 of a pen, a pointed object or a fingernail when handwriting. (such as a (DSP) together with the Processing Unit comprises a control unit central processor (CPU), a digital signal processor (PLA)) (such as an electrically erasable programmable (EEPROM), and a set of program instructions stored in the memory, or a programmable logic circuit a memory read memory a flash memory or similar devices) which can be executed by the controller. The treatment unit can e.g.

ASICs, components or other components that meet the requirements.

In FIG. alternatively is realized only with hardware, as an application-specific integrated circuit discrete com 1 and 2, the communication device 1 has a movable door or lid 3 to protect the keypad 6 against damage and accidental key presses. The lid 3 can be opened and thereby supports a pen 7 or other object with a relatively sharp tip, such as a fingernail, when used for handwriting on the communication device 1.

The microphone 5 is preferably located in the cover 3 near a position where a user's mouth would be located when using the telephone function, but can also be placed on the cover 3 at point 5 ', alone or with any number of microphones, closer to the communication device 1 or 10 15 20 25 30 35 515 258 2000-l0 ~ 26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ IB7409OSE .dOC MS. . . ». n 6 t.o.m. in the housing 2 of the device 1 near point 5 ", or in any other position in the device 1. This provides a more accurate detection of the sound due to a triangularity effect between several cooperating microphones instead of just one microphone.

The user can either use a pen 7 or another object with a sharp tip when writing by hand. The sound generated by the user while moving the pen 7 towards the lid 3 is registered by the microphone 5 and is conducted in electrical form to the treatment unit. The sound in electrical form is identified with the help of the treatment unit and suitable software.

The sound is converted to digital data, corresponding to e.g. one and is led to the monitor 4 on the communication device by means of processing-cursor movement or entered text. The sound in the form of digital data can also be used as commands to perform desired functions and control the communication device 1. To improve the accuracy of the recording of the sound, the lid 3 can be made slightly rough and / or made in relief in specific areas, so that different patterns produce different and distinct sounds, making the sound easier to identify.

In FIG. 3 shows a preferred embodiment of a surface 3 'on the lid 3. The surface 3' is divided into the plurality of areas 3a, 3 and 4. The surface 3 'an area 3a in the middle and at least one outer sub-area 3b as shown in FIG. consists of at least 3b surrounding the central area 3a. Sub-areas 3a and 3b are intended to define which characters are entered when writing a message by hand. This is done by arranging a specific sub-pattern 3 "for each sub-area 3b between the middle area 3a and its adjacent sub-area 3b. This means that the respective characters, corresponding to each sub-area 3b and its sub-pattern 3", are entered by touching the tip 7 on a relatively sharp object 2 times over the sub-pattern 3 "in two different directions. The surface 3 'can also be arranged on any other separate and substantially flat surface, ._ -vv- -. 10 15 20 25 30 35 515 258 2000- 10-26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ l874OSOSE.doc MS 7 operatively connected to the processing unit of the communication device 1, such as a separate tablet, table or any other handheld device for entering text or commands .

The most important thing during the entry of a handwritten message in the device 1 is that the movement of the tip 7 must begin and end in the center area 3a. The passage of the various sub-patterns 3 "defines the desired character to be entered into the communication device 1, thanks to the specific sound generated when the tip 7, in contact with the surface 3 ', passes over the respective sub-patterns 3". The initial movement out of the center area 3a past the specific sub-pattern 3 "and into the first outer sub-area 3b defines from which character group a desired character is to be selected. The second movement from the first sub-area 3b to another sub-area and past its sub-pattern 3 "back to the center area 3a defines the position of the desired character to be entered from the selected character group. This means that a character can be entered with a continuous movement without lifting the tip 7.

The tip 7 can be moved at varying speed and pressed against the surface 3 'with varying force by different users. The tip 7 can also be made of different materials, ie. harder or softer. The treatment unit has an additional function, which can identify sound and provide the correct information, regardless of the type of tip 7 used or the force with which the user presses the tip 7 against the surface 3 '. This function is a learning setting, which is selected by pressing a key 6. The learning of the treatment unit is performed, after initiating this setting, by moving the tip 7 towards a specific sub-pattern 3 "on the surface 3 '. generated by this movement over the specific sub-pattern 3 "is detected, recorded and stored for later use as a reference sound.

NOW 20 25 30 35 515 238 ¿¶: I§ff = _ '=. ¿_, F 2000-10-26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ l874090EN .dDC MS 8 About a character to be fed cannot be found in any of the sub-areas 3b, other character groups can be selected by moving the tip 7 in and out of the middle area 3a, ie. back and forth over one or two specific sub-areas 3 ", or by tapping the cover 3 or more times, or alternatively by pressing a button on the keypad 6. This change of character groups can be performed by using the processing unit for identification The sound can be indicated by a symbol or symbols on the screen 4 of the device 1 and can show which character group is to be used for entering the desired information. Any sub-pattern can be defined for controlling the change between different groups of characters by means of symbols on the surface of the cover 3, and these symbols can also be displayed simultaneously on the screen 4 of the communication device 1.

These symbols can be designed in many ways, e.g. such as circles, squares, triangles, stars, punctuation marks or have any shape and can be defined for different functions, e.g. changing the selected character groups from uppercase letters to lowercase letters, or to numbers, or alternatively to parentheses or to execute various commands. A preferred symbol group is a square that initiates a stop, a circle, a triangle pointing upward and initiating activation of an uppercase or lowercase input setting, a triangle pointing to the right for entering spaces, and a third triangle that point to the left to delete the last character entered.

The surface 3 'preferably has eight outer sub-areas 3b as shown in FIG. 3 and 4. Each sub-area 3b has a sub-pattern 3 ", which consists of at least one groove or a ridge for generating a specific sound, when the tip 7 passes over it / it. This sound must be unique for each sub-area 3b, whereby each of the sub-patterns 3 "10 15 20 25 30 515 23s ..» i .. »«, vov ~. J I * Ü '_:. . . . ,. : z I '_ j' 2000-10-26 \\ cuRRz-: N'r \ ns \ PuBL1c \ noc \ p \ 1sv4ososs.dos Ms - - '* *' "9 have different patterns consisting of different numbers of tracks or elevations arranged in different combinations with different mutual distances, preferably eight different patterns with elevations A, B, C, D, E, F, G and H, shown in greater detail in Figs. 5, 6, 7 and 8. Each of the sub-areas 3b may also have a surface which is rough, i.e. a surface with a large number of very small grooves or ridges, similar to the small ridges shown in Fig. 7 in the form of patterns C 'and E'. of the patterns C and E, which are smaller than the said sub-patterns 3 "to produce a basic sound, when the tip 7 in contact with the sub-areas 3b moves over them. This basic sound improves the sound recognition by means of a more accurate detection of the position of the tip 7. For the same reason, the center area 3a should be as smooth as possible in relation to the sub-areas 3b.

A sequence of movements is shown for the sake of clarity in the form of thick lines on the surface 3 'for inputting a character. The movement starts in the middle area 3a at point s, then the sub-pattern A is passed by the tip 7 and defines from which group the desired sign, e.g. the lowercase letter f, must be selected. The tip 7 is then moved from the first sub-area after passing the sub-pattern A into the second sub-area and past the second sub-pattern H back into the middle area 3a at position e. The second sub-pattern H defines the position of the lower case letter f the selected group, first subarea 3b. Another character is entered by ie. in the second position in the upper row of the moving tip 7 in another sequence, e.g. the lowercase character X is entered by moving the tip 7 from the center area 3a at point s 'past the sub-pattern A into the sub-area in the upper right corner of the surface 3', and into a second sub-area followed by a third sub-area 3b located in the lower right corner of the surface 3 ', and back into the middle area 3a at point e' past the sub-pattern C. 10 15 20 25 30 35 515 238 2 '==. "=' = ___-" 2000-10-26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ l874090EN.doc MS: '' Another sequence of movements is shown in FIG.

The first movement from position s "in the middle area 3a past the sub-pattern H and into the first sub-area defines the part in a change of character group. The movement from this first area, past the sub-pattern H a thread to and back the middle area 3a at position e "defines a change from lowercase to uppercase, activating the symbol in the middle of the three symbols above the H pattern. The subsequent "second start" movement from the sub-area 3a at position s ^, past the sub-pattern G and into the sub-area 3b, defines from which character group the character is to be selected. The movement from this sub-area back into the center area 3a at position e ”and past the sub-pattern G again, defines the middle position in the upper row for the letter A. If the character number 4 is to be entered after the capital letter A above, the tip is moved 7 from point s * in the middle area 3a past the sub-pattern D into the lower middle sub-area, this defines a change of character group. The movement from this first sub-area past the next adjacent sub-area at the lower right corner, and into the right middle sub-area past the sub-pattern C back into the middle area 3a at point e *, defines the position of number 4, i.e. two steps from the middle square symbol in the first selected character group in the lower subarea.

Fig. 5 shows a preferred embodiment of a pattern 3 "with eight sub-patterns A, B, C, D, E, F, G, H with different combinations of elevations, whereby each sub-pattern can be combined with which sub-area 3b The illustration in cross section is made with reference to only a sub-pattern F along line xx in Figs. 3 and 4. Other designs of the pattern instead of elevations can be used, for example grooves or any other shape. , as shown in FIG. 6. The ridges can also be round, flat or of any other shape, as shown in WU 20 25 30 35 _. u nu 0. o; u ». n _,,. Q-. c - 'f ",' I ... . .. n; 'U I. 1 v I' '' l. | 2000-10-26 \\ CURREN'I '\ DB \ PUBLIC \ DOC \ P \ 1B74OSOSE.GOC MS 1 au ll a larger scale in FIG 8. The elevations can also have different sizes which creates sounds with different frequencies, such as and E in FIG. 7.

Fig. 5 is explained by means of combinations of straight arrows shown by pattern C 'The different sub-patterns in (T) pointing upwards. Sub-pattern A consists of an elevation (T), and sub-pattern B consists of two elevations (TT) with a mutual distance d. Sub-pattern C has three elevations, a pair of the elevations having a mutual distance d and the third elevation is arranged on a distance more than 2d from the first pair (TT T). Sub-pattern D also includes three elevations, where one elevation is located near the center area 3a and the other two elevations are located at a distance more than 2d from the first elevation (T TT). pair of elevations with a mutual distance d placed on Sub-pattern E includes four elevations with two a distance more than 2d from each other (TT TT). Sub-pattern F consists of three elevations with a mutual distance d (TTT). Sub-pattern G has four elevations with a mutual distance d (TTTT), and sub-pattern H consists of two elevations with a mutual distance of more than 2d from each other (T T).

Fig. 6 shows a preferred embodiment of a pattern 3 "with eight sub-patterns A, B, C, D, E, F, G, H with different combinations of grooves instead of elevations as in Fig. 5. only one sub-pattern F along line xx in FIG.

The illustration in cross section is made with reference to 3 and 4. The mutual distances between the grooves are the same as in FIG.

The grooves can have any other design or size, such as a shallower and / or wider shape or have a more rounded bottom.

Fig. 7 shows another preferred embodiment of a 3 "with eight sub-patterns A, B, C, D, E, F, G, H with different combinations of elevations as in Fig. 5. The illustration in cross-section is made with reference to only a sub-pattern F along line xx in FIGS. 3 and 4. Distance and size 10 15 20 25 30 35 515 238 ¿E_j == ¿jj¿ = .j: .- '2000-10-26 \\ CURRENT \ DB. a pair of elevations having a mutual distance d and a third elevation being arranged at a distance more than 2d from the first pair.The area between the single elevation and the pair of elevations is provided with a plurality of small elevations similar to the basic pattern in the subareas. 3b to create a type of basic sound (TT T) _ with a mutual distance d placed at a distance more than the sub-pattern E comprises two larger elevations 2d from va randra. The area between the elevation pairs has a number of smaller elevations that create a pattern similar to the basic pattern in the subareas 3b (TT TT).

Fig. 8 shows yet another preferred embodiment of a sub-pattern 3 "with eight sub-patterns A, B, C, D, E, F, G, H, as shown in Figs. 5, 6 and 7. The sub-patterns will be explained by using the same arrows (T) as in the above description of Fig. 5. The sub-patterns are illustrated as arrows (T) in Fig. 8. For the sake of simplicity, the sub-patterns are shown with reference to only one sub-pattern F along line xx in Figs. 3 and 4. Sub-pattern A has three elevations with a mutual distance of more than 2d from each other (TTT).

The sub-pattern B has four elevations consisting of a pair of elevations with a mutual distance d and two other elevations with a distance of more than 2d from each other and the first pair (TT T T). Sub-pattern C includes three pairs of elevations, ie. a total of six elevations, with a mutual of more than 2d between each pair, the arrows in each pair having a mutual distance d (TT TT TT). Sub-pattern D has five elevations (TT TT T). The sub-pattern E includes four elevations with a mutual distance of more than 2d (T T T T). The sub-pattern F has five elevations but in another combination with an elevation placed at a distance more than 2d from the other four elevations with a mutual distance d (T TTTT). The sub-pattern G consists of five elevations. 13 in a combination similar to F (TTTT T). The sub-pattern H has six elevations with a first pair of elevations with a mutual distance d placed at a distance more than 2d from the other four elevations with a mutual distance d (TT TTTT). Fig. 8 also shows six magnifications, which describe different possible designs and combinations of elevations or grooves.

In the preferred embodiment of the cover 3, shown in FIG. 2, the cover 3 is used to enter a handwritten message, move a cursor on the screen 4 or execute commands to control the device 1. This can be accomplished by the following steps shown in FIG. 9.

The first step 910 consists in opening the lid 3 followed by a registration in step 920 of whether only tea step 925, tion with sound recognition in step 930. If sound recognition of the lefon function is to be used, or in combination handwriting on the lid 3 is to be used, a tap with the pen 7 in the middle of the cover 3 or a key press on the keypad 6 must be made to initiate the sound recognition function. In the next step, the tip 7 is moved on the surface 3 'of the lid 3, whereby a sound is detected which is detected in step 930 by means of the microphone 5, converted to electrical form in a step 940, led to the treatment unit in a step 950 and identified by appropriate software in a step 960. The identified information is then converted to digital data using the processing unit, shown in a step 970, and identified as text or commands in a step 980. If the digital data is intended to move a cursor or enter text, they are passed to the monitor 4 as in step 985, or, alternatively, if the digital data is a command, this is done to control a specific function in the communication device in the last step 990. Mouse-like functionality can be achieved by combining the sound with one or more keys on the keypad 6. -ou- - 0-.u ..--. .-- vn 0 .-- Q. u ao ..-.- 5 1 5 2 5 8. . .,. . .

I n = ',. «. i, ». .g: 2000e10 ~ 26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ IB74OBOSE.doc MS ß '14 The present invention has been described above with reference to a few embodiments, but this description is not intended to be a limitation of the invention. Embodiments other than those specifically described are also possible within the scope of the invention, as defined by the appended independent claims.

Claims (11)

1. N H 20 25 30 35 51 238 - .- = = I.:. 1 i: z I:: 2 z,: .y a t i- --- u u fJ *. 2000-10-26 \\ CURREN'1 '\ DB \ PUBLIC \ DOC \ P \ 18740QÛSEJÅOC MS I a n. ~ - 15 CLAIMS 1. Portable communication device (1) with a microphone (5) and a processing unit for receiving a handwritten message from a user and converting the message into digital data, characterized in that the processing unit is operatively connected to the microphone (5), so that different sounds generated by movement over different patterns and / or structures on a surface (3 ') of the communication device (1), when the user writes by hand on said surface (3') electrical form to the processing unit . can be acoustically detected by the microphone (5) and conducted in Device according to claim 1, wherein the movement of e.g. a pen (7) or tip over different combinations of said patterns and / or structures on the surface (3 ') corresponds to different text entries or commands. Device according to claim 2, wherein the surface (3 ') is divided into the plurality of sub-areas and specific sub-patterns (3 ") for generating specific sounds when, for example, the pen (7) or the tip is in contact with the surface (3'). ) moves towards these sub-patterns (3 "). Device according to claim 3, wherein the surface (3 ') consists of at least one central area (3a) surrounded by at least one outer sub-area (3b) with associated sub-pattern (3 "). Device according to claim 4, wherein the surface (3 ') consists of a central area (3a) surrounded by at least two, preferably (3b), (3b) has an associated sub-pattern, each sub-area (3 ") (3a) and the respective sub-area (3b), also eight sub-areas arranged between the middle areas Device according to claim 5, wherein the surface (3 ') has several sets of character groups, preferably more 20 20 25 30 35 15 238 2000-10-26 MS 16 than two sets, for each sub-area (3b), comprising different combinations of lowercase and uppercase letters, numbers, punctuation or any other character needed to write a message. Device according to claim 6, wherein the surface (3 ') comprises more than one combination of sub-patterns (3 "), preferably a combination for each sub-area (3b), each of the sub-patterns consisting of elevations or grooves of different sizes, preferably elevations, which are arranged in different numbers and with different mutual distances to form a specific pattern, which produces a specific sound for each sub-pattern, when it is passed by, for example, the tip of a pen (7) or any other tip at any time. Device according to claim 7, wherein the surface (3 ') is placed on a cover (3) of the communication device (1) or on some other separate and substantially flat surface of an external accessory, which is operatively connected to the device. - the action unit of the communication device (1), such as a separate plate, a table surface or a portable device for entering text or commands. An apparatus according to any one of the preceding claims, wherein the processing unit comprises: at least one central processor (CPU), a digital signal processor (DSP) or a programmable logic circuit (PLA); together with such as an electrically erasable and programmable (EEPROM), a memory, readable memory a flash memory or similar devices; and a set of program instructions stored in the memory, which can be executed by the control unit. Device according to any one of the preceding claims, wherein the communication device (1) is a portable device. . u onøov *,. 2000-10-26 \\ CURRENT \ DB \ PUBLIC \ DOC \ P \ 1874090EN.d0C MS l7, preferably a portable telephone or mobile telephone, for entering text or commands therein. 11. ll. Method for detecting, identifying, converting and entering data in a communication device (1) such as a mobile telephone, characterized by the steps of detecting (930) different sounds with a microphone (5), brought about by movement over different patterns and / or structures - on a surface (3 ') of the communication device (1), when the user writes by hand on said surface (3'), convert (940) the sound into electrical form by means of the microphone (5), conduct (950) the sound in electrical form to a processing unit of the communication device (1), perform (960) a software routine in the processing unit for identifying the sound, convert (970) the identified sound into digital data in the processing unit, and direct (985) the digital data to a monitor (4) on the communication device (1), or use the sound, in the form of digital data, to execute commands (990) which control desired functions in the communication device (1).