WO2009044214A1

WO2009044214A1 - System and method of picture communication between mobile communication devices

Info

Publication number: WO2009044214A1
Application number: PCT/HR2008/000033
Authority: WO
Inventors: Natasa Vlahovic Klecina
Original assignee: Natasa Vlahovic Klecina
Priority date: 2007-10-01
Filing date: 2008-09-30
Publication date: 2009-04-09
Also published as: HRPK20070458B3; HRP20070458A2; WO2009044214A4; HRP20070458A9

Abstract

Invention refers to system and method of visual creation, sending and receiving of messages in form of picture symbol sequences between two mobile communication devices. Invention allows fast and simple creation of picture messages by using visual navigator customized for persons with disabilities and health problems, and sending and receiving of picture messages. Furthermore invention allows all users new way of creating, sending and receiving picture messages through mobile phone devices as a substitution for standard text messages like SMS in a funny, fast and simple way. Solution reduces time to create message and number of necessary 'clicks' per message and creates new way of visual communication through mobile phones, and at the end it allows conversion of picture messages into only text messages in desired language if receiver does not have installed application on mobile phone.

Description

SYSTEM AND METHOD OF PICTURE COMMUNICATION BETWEEN MOBILE

COMMUNICATION DEVICES

FIELD OF THE INVENTION

[0001] Invention refers to enhancement in communication with picture messages in form of SMS₃ MMS, EMS and/or e-mail between mobile communication devices. Creation and sending of picture messages is customized for needs of disabled people that could not communicate with standard SMS massages. Also invention enables all users new way of creating, sending and receiving messages as substitution of text messages, combination of text and pictures or combination of text and hyperlinks in a new, funny, quick and simple way.

TECHNICAL PROBLEM

[0002] Many persons could not till now use standard mobile phone functionalities, to be more precise to communicate through SMS. As example people with bad seeing like old people, people with fine motoric problems, persons with autism, Down syndrome, mental retardation and etc., could not use SMS because buttons for writing text are too small, procedure of writing is too complicated or text is not understood as media of communication. There was no solution that could enable quick and simple way of creating SMS that could be used for named groups of potential users for mutual communication or toward other persons. STATE OF THE ART

[0003] There are solutions for direct personal communication like "Communicator" for persons with autistic spectrum disorders that is based on visual communication picture exchange system, but it is only direct communication between persons and not through mobile phone devices with visual functionalities like SMS, EMS, MMS and/or e-mail. There are also SMS solutions that combine pictures and text, text and hyperlinks or similar, but named solutions are technically customized for fun and advertising only. The most similar patent to innovation is EP 1 300 770 A2 that describes method of saving and editing pictures for SMS communication, but this pictures can not replace textual massage and do not allow simple and visual creation of massage as this innovation. None of named solutions do not support for named customer segments easy usage of SMS functionalities nor sending messages with picture symbols that totally replace text, nor support text functionalities on many languages.

DISCLOSURE OF THE INVENTION

[0004] Innovation refers to picture messages in form of SMS, EMS, MMS and e-mail via mobile communication devices customized for persons with disabilities, motoric disabilities and or visual problems, who till now could not communicate with classic SMS. Furthermore innovation refers to system and method for creating, sending and receiving messages in form of picture symbols from one mobile telecommunication terminal to another and e-mail. Innovation expands possibilities of using SMS, MMS, EMS and e-mail. In detail, presented innovation allow by using visual navigation customized for persons with disabilities and health problems, creation, sending and receiving of pictorial message. In this manner innovation is also suitable for "healthy" population without named disabilities. Innovation enables conversion of picture messages into Multilanguage text messages. Specification and figures present technical solution for picture messaging (could also support photographs) via mobile phones (SMS, MMS, EMS services) and/or e-mail.

BRIEF DESCRIPTION OF THE DRAWINGS

[0005] For better understanding of named functionalities of innovation there are several drawings enclosed, in which: Fig 1 is drawing of mobile communication device with elements of solution on display.

Fig 2 is block diagram of solution.

Fig 3 is example of creating picture message step by step.

Fig 4 is example of using advanced text search.

Fig 5 is a scheme of all components necessary for picture messaging via mobile phone.

Fig 6 is matrix presentation of picture organization within innovation.

Fig 7 is a scheme of implemented solution in mobile communication network for sender and receiver of named message.

DETAILED DESCRIPTION

[0006] Communication system via picture exchange symbols " Picture Exchange Communication System- PECS" was initially designed for disabled persons (especially visual persons) with communication deficiencies as alternative way of communication, using simplicity and unique means of picture symbols. Experts estimated for basic communication at least 200 symbols is needed. Population that has problem with expressing words and sentences, understanding speech, writing, reading or memorizing in general have communication problem. PECS was initially designed as only or helping way of communication for persons with autistic spectrum disorder, and as helping communication for persons with not completely developed speech such as Down Syndrome, types of mental retardation, problems with word synthesis, memorizing, concentration and multifactor disabilities. Persons with communication disabilities when using PECS take desired symbols from the book (in example symbols are magnetically attached to pages) and gives it to other person. Communication partner looks at symbol and executes desired service or gives desired product. In next phases communication develops with more and more picture symbols that are placed on communication tray and delivered to communication partner as request, hi more developed phases of communication response can be published on communication tray, furthermore repeat of questions and answers can lead into discussion.

[0007] Invention is a solution that applies PECS communication into mobile services SMS, EMS, MMS and e-mail where are instead of words used communication symbols. Pictures as symbols have functionalities of word, part of sentence or complete sentence. Presented innovation allows creation of complete message via single or more picture symbols, photography's, text with pictures or similar combinations. Picture symbols could be linked with describing text.

[0008] Invention does not only allow communication for persons with communication disabilities of standard language but also allows usage of named mobile phone pictorial messaging for persons with motoric and visual handicaps. Into named groups of users, group of old people can be also included to use this innovation on mobile phones. In addition innovation allows safety and simplicity of usage for parents and toddlers that do not know how to use standard text messaging. At the last, presented innovation allows to everybody, especially youngsters, new way of fun, faster and attractive, Multilanguage and meaningful way of exchange of picture messages.

[0009] Next innovation in invention is usage of visual navigator when creating and sending picture communication messages, navigating through pictures (or photographs) structured in visual category and subcategory vocabulary (for example table 3x4 with picture categories: food, activities, questions, emotions, business and etc.). Each category has one or more picture tables with symbols. Table categories could be also navigated with horizontal or vertical scroll bar navigation.

[0010] Message tray is placed on display, and is permanently present during complete message creation and it is filled with picture symbols. Content of message tray is picture message. More advanced users with bigger amounts of pictures in categories can also use level of subcategories have tables with pictures. Message tray has link through shortcut into telephone directory that is also structured as table with pictures or photographs of possible receivers of messages like mother, father and etc. Photos could be connected with telephone numbers from a mobile standard telephone book.

[0011] Navigation and selection of picture symbols (or photos) could be realized with cursor and/or touch screen features if mobile devices support it and at the last by navigating via buttons. Named functionalities extend precision of selection procedure (especially important for persons with fine motoric skill).

[0012] Innovation in picture messaging is in significant time reduction of message creation and reduction of number of clicks (events) to create it comparing to standard textual message in SMS, EMS, MMS and/or e-mail. For example to create standard SMS text at least 10 clicks are needed and to create picture message 5 to 6 clicks are needed for simplest message.

[0013] Picture symbols can be used without description text, with description embedded into picture or description enclosed to picture. Text as part of picture could be on one or two pre selected languages. User in advance chooses between pictures without description text or picture with one or two languages in description. Sender and receiver can have selected different description languages and variations of pictures with or without descriptions. Each picture is encrypted and sent via mobile communication system. If receiver does not have installed application for picture messages will receive only equivalent textual message based upon description text. Advantage of picture symbol is its unique meaning and linked with description text that it presents and could be translated more precisely than only words.

[0014] Next innovation for advanced search (not necessary for basic functionality for mobile picture messaging) is search by keywords or parts of keywords. For advanced application new tray for entering keywords is nested on display together with picture tray and icon gallery. Entering keywords into try for search will start new form with resulting icons related to searched keyword and descriptions. From the resulting search form can be selected pictures as already described in previous text. Search can be done on previously defined language, and result is picture symbols with or without text descriptions (one or two languages)

[0015] Invention enables customization of picture symbols (animated pictures symbols, icons, photos) in mane category form subcategory form and list of receivers form. Named types of pictures are possible to import into application and use them for picture messaging. User can create by itself pictures in picture editor panel and download it to local memory on mobile device. In order to keep solution dynamic and attractive certain variants of solution could contain animated pictures (animated gif for example) to describe activities or verbs.

[0016] For basic picture messages only two basic modules are necessary: module for creating and module for sending picture messages. Additional functionalities such as advanced search might be supported with separate modules (for example Multilanguage or translation module). [0017] Basic application should contain at least 200 pictures (recommendation of experts). Invention could be diversified into two basic variants: version for disabled people (that could be also used by healthy population) and version with advanced search, Multilanguage descriptions, big stock of picture symbols and similar for advanced users.

[0018] Basic organisation of picture symbols is on two levels, level of category and level of tables with customizing number of rows and columns for displaying picture symbols. Advanced solution with more picture symbols could have three levels: level of category, level of subcategory and level of tables with picture symbols.

[0019] To create and send picture messages following items should be used: display with picture table categories - first level, display of tables with pictures - second level. For displaying more pictures horizontal or vertical scroll bars could be used. Basically more pictures usually appear at tables with pictures (lowest level of display) or on form with receivers list.

[0020] Figure 1 describes mobile phone with implemented solution for picture messaging - main panel for creating messages. Solution consists of: visual navigator 10, message tray 12, shortcut to receivers list 14 and frame for advanced search by keywords 16.

[0021] Figure 2 represents block diagram for creating and sending picture messages by using category. After start of picture messaging system one of following categories is chosen in main form 22. Table with pictures is started by choosing category on main form. If table contains wanted symbol, symbol is chosen 24 and deposited into message tray 28. If wanted symbol is not in displayed part of table vertical and horizontal scroll bars can be used to navigate through other tables in selected category until desired symbol is found. After picture symbol is found and chosen application returns user to starting main panel .User can repeat steps from 22 to 28 to expand message or simply end message creation by choosing receiver list shortcut 30. Picture message procedure of creation ends with choosing picture symbol of receiver 32, application then automatically sends message.

[0022] Figure 3 represents example of simple message creating and sending in real situation based on flow diagram described in figure 2. First one category has to chosen on main form by simple clicking on picture symbol. In this example category "Questions" is chosen, rounded with circle 40. Then opens category of questions with table of pictures that represent words like where, when, who and similar, where user chooses picture "when" rounded with black circle 42. Picture "when" is nested into message tray 44. User is rerouted automatically to main panel 46 with categories and picture "when" stays in message tray. User then chooses category picture "Locations" marked with circle 46. Next step of user is choosing picture symbol "shopping" which is added to message tray 48 and user is once again being rerouted to main panel and message tray will contain two picture symbols "when" and "shopping" 50. Message is finished and user chooses receivers list shortcut. By opening receiver picture symbols list user chooses picture of whom to send picture and message "When shopping" is being send 54.

[0023] Figure 4 describes advanced picture symbols search by typing keywords or part of keywords into search tray 16. In this example text "house" is inserted into search tray. Result of search is only one picture symbol in this example. Search engine checks all symbols in all categories. By typing text in search tray 16, main form is being removed and new result form is being open with adequate picture symbols.

[0024] Figure 5 is functional scheme showing interaction of different invention segments. Inside mobile device, pictures symbols are represent by visual display menu 10 and representing structure of picture symbols is managed by picture symbol selector 71. Picture seleptor allows structuring of visual display menu and managing pictures from picture symbol repository 72. Adding pictures to picture repository can be done from web server 74 or manually directly with importing function into mobile phone. Software application could be implemented on web server 74 plus on mobile phone by installing download from web server. Application might be also preinstalled only on mobile phones without web server. In visual display menu is possible to make advanced textual searching with module for advanced search 16 and with each picture symbol is possible to make connection with text descriptions via multilingual text repository 73. Pictures are chosen in visual display menu 10 and stored in message tray 12. After message is finished its being sent to receiver 77 that is chosen from receivers list 79. Sending can be done in two ways: sending code through web server and server sends MMS message (pictures and text) to receiver 83 or by sending code directly to receiver 82 where the code is decrypted 80 and published on receiver's screen 81. If receiver does not have installed application will receive only textual message correspondent to text descriptions attached to picture symbols with separators between each description. Picture symbol coding includes description and for example separator #. According to example of message from figure 3 combination of two pictures will give following textual message on receiver screen: When to go #shopping#

[0025] In figure 6 pictures are organized in two or three levels depending on total number of pictures that are in application and on preferences of the user. User can choose between two or three level of picture organization. Organization of pictures on two levels consists of: main form with category symbols 90 and tables with picture symbols (symbols go in more detail for each category) 91. Organization of picture symbols in three levels consist of (main level with category symbols 90, subcategory level with more detailed subgroups of categories 93 and level of tables with picture symbols with very detailed symbols 91). Each of named levels might have horizontal or vertical scroll bar if needed to search through larger quantity of pictures in one display screen. Three level of picture organization is more convenient to avoid usage of scroll bars and to organize bigger amount of picture symbols.

User chooses on main form symbol of category 91. If user does not find wanted symbol on presented table user can switch to another symbol "car" will have k(l),(xl I,yl3) in case of two level organization 92 or k(12),(xl 12,ylO3) in case of three level of organization. Each showed combination of categories and positions is unique for every picture symbol within application;

[0026] Figure 7 describes implemented solution in mobile communication network for sender and receiver of picture messages. Senders 100 can use web page with application for writing messages 110, can have preinstalled J2EE application 111 or preinstalled application made with Symbian, J2ME or similar 112. Pre installation can be made in production factory or can be downloaded from web server and installed. Messages from sender via GPRS 106 or GPRS/EDGE/UMTS 105 through web server 101 are directed to mobile operator of receiver. Although sending of messages can be made directly without contribution of web server 108. From web server to mobile operators 102 are sent MMS messages 107. Receivers 103 receive MMS message, MMS with link to web page, SMS, EMS or e-mail 113.

SCOPE OF THE INVENTION

[0027] Invention allows fast and simple usage of picture symbols, as substitution for text messages like SMS, for persons that were not capable of using text messaging or for persons that standard text messaging was too heavy, too complicated or too slow. Invention is possible to implement for usage on market in short time period and with smaller financial investment. Marketing experts will instantly spot benefits of this invention as possible upgrades and changes without loosing scope and basic definition of invention.

Claims

1. A System of picture communication between mobile communication devices, including mobile phone device of sender, mobile phone device of receiver and/or web server with application and picture repository comprising application on receivers side consist of visual display menu with gallery of picture symbols (10), picture selector (71), message tray (12), shortcut to receivers list, visual table of receivers represented by photos or picture symbols (79) and search tray for advanced textual search (16), that visual display menu is linked with multilingual text repository (73) and with picture symbols from picture repository (72) or picture repository placed on web server (74), that picture symbols are organized in level of category, level of subcategory types of groupings, that application includes transmitter (77) and message receiver (80) and finally application on receivers side shows on screen picture message as sequence of picture symbols (81).

2. A Method of picture communication between mobile communication devices comprising the steps of: opening visual display menu with picture symbols of categories (10), choosing category (40), upload with pictures from picture repository (72) or web server repository (74) , showing category in visual display menu, choosing picture in category (42), opening tables with picture symbols, choosing picture in table level, placing picture into message tray (12), choosing list of receivers form (52), choosing picture symbol of receiver (54) from repository of receivers (79) with also other necessary data of receiver , automatic sending picture message (77), that receiver views picture message (80) on its screen (81) of the mobile phone device.

3. The method of claim 2, comprising the steps of choosing category, subcategory and table with picture symbols can be done repeatedly several times until message consisting of more pictures is finally created and sent.

4. The method of claim 2, comprising that message tray for messages includes automatic inclusion of pictures from picture repository after picture symbol has been chosen on visual navigation menu and that selected picture will be placed on first empty space starting from the left or right side of message tray.

5. The method of claim 2, comprising that message tray contains shortcut to receivers (14) and by choosing shortcut form with photos or symbols will be opened, that each picture is linked with receivers phone number, that receivers list is pre defined by attaching to users name pictures and phone numbers and that picture message is sent by choosing one of the receiver.

6. The method of claim 2 to 5, comprising that picture symbols could contain text descriptions as part of picture, which picture symbols are linked with text descriptions, and that picture symbols could be any type of picture, illustration, photography or animation.

7. The method of claim 6, comprising the steps of: picture symbol could contain text descriptions on more languages, language selection allows users also to communicate by choosing between pictures with or without text description on chosen languages that are deposited in Multilanguage repository (73), on receivers side message is decrypted and receiver gets same picture message as created and sent by sender or receiver gets only text descriptions with separator characters, receiver can select to receive picture symbols with or without text descriptions on selected language, and picture symbol could contain maximum two text descriptions on different languages.

8. The method of claim 2 to 7, comprising application contains: advanced text search (16) of picture symbols via keywords or parts of keywords, part of screen for entering text to search for (16), activation of search engine, publishing results in form of related picture symbols in new result form, selection of desired symbol and search is made on pre selected language.

9. The method of claim 8, comprising that in case result of search might contains more then one picture symbol tables for further selection horizontal and vertical scroll bars are used.

10. The method of claim 2 to 9, comprising the steps of picture symbols on two or three levels of organization are: menu of categories, subcategory and tables with picture symbols organized in customable number of rows or columns, visual search with horizontal or vertical scrollbars of picture symbols in each of categories if symbols do not fit into screen.