WO2022172272A1 - Apparatus system and method of interacting with a web page - Google Patents

Apparatus system and method of interacting with a web page Download PDF

Info

Publication number
WO2022172272A1
WO2022172272A1 PCT/IL2022/050170 IL2022050170W WO2022172272A1 WO 2022172272 A1 WO2022172272 A1 WO 2022172272A1 IL 2022050170 W IL2022050170 W IL 2022050170W WO 2022172272 A1 WO2022172272 A1 WO 2022172272A1
Authority
WO
WIPO (PCT)
Prior art keywords
interaction
web page
code
commands
graphic
Prior art date
Application number
PCT/IL2022/050170
Other languages
French (fr)
Inventor
Mati HOROVITZ
Original Assignee
Elementor Ltd.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Elementor Ltd. filed Critical Elementor Ltd.
Publication of WO2022172272A1 publication Critical patent/WO2022172272A1/en

Links

Classifications

    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00Speech recognition
    • G10L15/22Procedures used during a speech recognition process, e.g. man-machine dialogue
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/954Navigation, e.g. using categorised browsing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/951Indexing; Web crawling techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/16Sound input; Sound output
    • G06F3/167Audio in a user interface, e.g. using voice commands for navigating, audio feedback
    • GPHYSICS
    • G10MUSICAL INSTRUMENTS; ACOUSTICS
    • G10LSPEECH ANALYSIS OR SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING; SPEECH OR AUDIO CODING OR DECODING
    • G10L13/00Speech synthesis; Text to speech systems
    • G10L13/02Methods for producing synthetic speech; Speech synthesisers
    • G10L13/027Concept to speech synthesisers; Generation of natural phrases from machine-based concepts

Definitions

  • Some embodiments described herein generally related to a web page and, more specifically, to a markup language for building web pages.
  • the way to interact with a web page is by interacting with visual elements such as, for example, links, buttons, and input fields.
  • visual elements such as, for example, links, buttons, and input fields.
  • the interaction is done by clicking or pressing on the visual elements, e.g., images, text on the web page, via a mouse and/or a keyboard.
  • Some solutions for accessibility may include adding a markup code to existing HyperText Markup Language (HTML) code of the web page to allow "Screen Reader” software to read and speak the web page content.
  • HTML HyperText Markup Language
  • TTS text to speech
  • STT speech to text
  • One exemplary embodiment may include a product comprising one or more tangible computer-readable non-transitory storage media comprising program instructions for generating one or more commands for interacting with a web page, wherein execution of the program instructions by one or more processors comprising: receiving a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a first markup language configured to display the web page content to a user, and a non-graphic interaction code created by a second markup language, configured to provide the non-graphic interaction functionality to the web page; introducing the one or more interaction commands to the user; performing a non graphic interaction between the user and the web page by receiving a non-graphic interaction command of the one or more interaction commands and providing a response based on the non-graphic interaction command.
  • One other exemplary embodiment may include a product comprising one or more tangible computer-readable non-transitory storage media comprising program instructions for generating one or more commands for interacting with a web page, wherein execution of the program instructions by one or more processors comprising: receiving a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a markup language, configured to display the web page content to a user, and a non-graphic interaction code created by a web command markup language (WCML); configured to provide the non-graphic interaction functionality to the web page; introducing the one or more interacting commands to the user; performing a non-graphic interaction between the user and the web page by receiving a non-graphic interaction command of the one or more interaction commands; and providing a response based on the non-graphic interaction command.
  • WCML web command markup language
  • the execution of the program instructions by one or more processors comprising: with the use of the non-graphic interaction code, parsing the web page code and generate a parsed code; and identifying in the parsed code one or more components configured to provide the graphic interaction functionality to the web page.
  • the execution of the program instructions by one or more processors comprising: generating a command console wherein the command console is configured to run the one or more non-graphic interaction commands.
  • the second markup language comprises one of an Extensible Markup Language (XML), a JavaScript Object Notation (JSON), a web command markup language (WCML).
  • XML Extensible Markup Language
  • JSON JavaScript Object Notation
  • WCML web command markup language
  • first language and the second language comprise the same language.
  • first code is written on the body of the web page and the second code is written on the header of the web page.
  • the second markup language comprises a parser configured to parse the first code and identify one or more interaction components in a parsed code.
  • the processing circuitry is configured to control with the second markup language comprises an interaction engine, and execution of the program instructions of the interaction engine comprises: identifying the one or more interaction commands based on the one or more interaction components.
  • the interaction engine comprises a text to speech engine
  • execution of the program instructions of the text to speech engine comprises: converting the one or more interaction commands into one or more voice commands to be used by the user and/or by a machine.
  • the interaction engine comprises a speech-to-text engine
  • execution of the program instructions of the speech-to-text engine comprises: converting the one or more voice commands into text code to be used by the web page for interacting with the user by texting.
  • Some other exemplary embodiment disclosed an apparatus and/or a program for building a web page comprising a processing circuitry, wherein the processing circuitry is configured to receive a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a first markup language configured to display the web page content to a user, and a non-graphic interaction code created by a second markup language, configured to provide the non-graphic interaction functionality to the web page; introduce the one or more interaction commands to the user; perform a non-graphic interaction between the user and the web page by receiving a non-graphic interactive command of the one or more interactive commands and provide a response based on the non-graphic interaction command.
  • the processing circuitry is configured to: with the use of the second code, parse a non-graphic interaction code of the first markup language and generate a parsed code; and identify one or more components in the parsed code configured to provide a visual interaction functionality to interact with the web page.
  • the processing circuitry is configured to: generate a command console wherein the command console is configured to run the one or more non-graphic interaction commands.
  • the second markup language comprises one of an Extensible Markup Language (XML), a JavaScript Object Notation (JSON), a web command markup language (WCML).
  • the first language and the second language comprises a same language.
  • the processing circuitry comprises a parser configured to parse the web page code and identify in a parsed code one or more interaction components.
  • the processing circuitry comprises an interaction engine configured to identify one or more user interaction commands based on the one or more interaction components.
  • the interaction engine comprises a text to speech engine configured to convert the one or more interacting commands into one or more voice commands for interacting with the user and/or a machine.
  • the interaction engine comprises a speech-to-text engine configured to convert the one or more voice commands into the text to be used for interaction with the user and/or machine.
  • FIG. 1 illustrates a block diagram of an apparatus to interact with a web page, according to some demonstrative embodiments.
  • FIG. 2 illustrates a flow chart of a method to generate commands for interacting with a web page, according to some demonstrative embodiments.
  • FIG. 3 illustrates a block diagram of a system to interact with a web page, according to some demonstrative embodiments.
  • FIG. 4 illustrates a product of manufacture, according to some demonstrative embodiments.
  • Discussions made herein utilizing terms such as, for example, "processing,” “computing,” “calculating,” “determining,” “establishing,” “analyzing,” “checking,” or the like, may refer to operation(s) and/or process(es) of a computer, a computing platform, a computing system, or other electronic computing devices, that manipulate and/or transform data represented as physical (e.g., electronic) quantities within the computer's registers and/or memories into other data similarly represented as physical quantities within the computer's registers and/or memories or other information storage medium that may store instructions to perform operations and/or processes.
  • processing may refer to operation(s) and/or process(es) of a computer, a computing platform, a computing system, or other electronic computing devices, that manipulate and/or transform data represented as physical (e.g., electronic) quantities within the computer's registers and/or memories into other data similarly represented as physical quantities within the computer's registers and/or memories or other information storage medium that may store instructions to perform operations and/or processes.
  • references to "one embodiment,” “an embodiment,” “demonstrative embodiment,” “various embodiments,” etc., indicate that the embodiment(s) so described may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Further, repeated use of the phrase “in one embodiment” does not necessarily refer to the same embodiment, although it may.
  • circuitry may refer to, be part of, or include, an Application Specific Integrated Circuit (ASIC), an integrated circuit, an electronic circuit, a processor (shared, dedicated, or group), and/or memory (shared, dedicated, or group), that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable hardware components that provide the described functionality.
  • ASIC Application Specific Integrated Circuit
  • the circuitry may be implemented in, or functions associated with the circuitry may be implemented by one or more software or firmware modules.
  • the circuitry may include logic, at least partially operable in hardware.
  • logic may refer, for example, to computing logic embedded in the circuitry of a computing apparatus and/or computing logic stored in a memory of a computing apparatus.
  • the logic may be accessible by a processor of the computing apparatus to execute the computing logic to perform computing functions and/or operations.
  • logic may be embedded in various types of memory and/or firmware, e.g., silicon blocks of various chips and/or processors.
  • Logic may be included in and/or implemented as part of various circuitry, e.g., radio circuitry, receiver circuitry, control circuitry, transmitter circuitry, transceiver circuitry, processor circuitry, and/or the like.
  • logic may be embedded in volatile memory and/or nonvolatile memory, including random access memory, read-only memory, programmable memory, magnetic memory, flash memory, persistent memory, and the like.
  • Logic may be executed by one or more processors using memory, e.g., registers, stuck, buffers, and/or the like, coupled to the one or more processors, e.g., as necessary to execute the logic.
  • markup language is a system for annotating a document in a way that is syntactically distinguishable from the text. For example, when the document is processed for display, the markup language is not shown and is used to format the text. With embodiments of the below disclosure, the markup language can be used to build a web page.
  • Extensible Markup Language is a markup language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable.
  • JSON JavaScript Object Notation
  • JSON is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute-value pairs and array data types.
  • JSON may serve as a replacement for XML in AJAX systems.
  • JSON is a language-independent data format. It was derived from JavaScript, but many modern programming languages include code to generate and parse JSON-format data.
  • HTML Hypertext Markup Language
  • CSS Cascading Style Sheets
  • JavaScript scripting languages
  • Web browsers may receive HTML documents from a web server and/or from local storage.
  • the web server may render the documents into multimedia web pages.
  • HTML elements may be the building blocks of HTML pages.
  • images and other objects such as, for example, interactive forms may be embedded into the rendered page.
  • HTML may embed programs written in a scripting language such as, for example, JavaScript, which may affect the behavior and content of web pages.
  • web server can be implemented in software and/or hardware and/or as a combination of software and hardware.
  • the web server may be dedicated to running software that can satisfy client requests on the World Wide Web.
  • a web server may include one or more websites.
  • a web server may process incoming network requests over HTTP and several other related protocols.
  • a web server For example, the primary function of a web server is to store, process, and deliver web pages to clients. The communication between client and server takes place using, for example, HTTP. Pages delivered are most frequently HTML documents, which may include images, style sheets, and/or scripts in addition to the text content.
  • commands as used hereinbelow is a means that the user and or a machine may use to interact with the web page. For example, a command is a directive to a computer program to perform a specific task supported by a web page.
  • visual interaction is related to interaction with a web page by using a visual element, e.g., a graphic element, on the web page, that the user can click on, press with a keyboard and/or use a voice command that is represented by the visual element.
  • a visual element e.g., a graphic element
  • non-graphic interaction is related to interaction with a web page using one or more interaction commands that not be represented by the graphic element on the web page and execute supported actions by the web page.
  • WML Web Command Markup Language
  • the markup language may include a flow of commands that depend on each other, a set of accepted arguments by each command, and a reference to operational code to be executed.
  • the WCML may add non-graphic functionality to the web page, such as, for example, enabling full-screen video playing; connect and/or enable registration to the web page through external web pages, such as, for example, Facebook, Google, etc.; generate a communication form that includes the user name and email address which saved with the client; may interact with voice assistance; save the parameters that were used to the next use with indication to the place of where those parameters are saved, may interact with the web page by gesture and the like.
  • the WCML may include but is not limited to code that enables to perform authentication by voice.
  • the middleware may offer to save once a pair of a username and password and/or obtain an access token from a 3 rd party authentication service in order to offer login again with the saved user credentials.
  • the WCML may include, but not be limited to, code that enables a required token to avoid force command.
  • the middleware may request to include a unique token generated for the current session in the command arguments.
  • CSRF Cross-site request forgery
  • the WCML may include, but not be limited to, code that enables saving a current order for re-order later. Because WCML is a command-oriented language, the final command with the final arguments may be saved for re-execution later. For example, save the current order details for re-order later with the same arguments.
  • the WCML may include, but not be limited to, commands that can be used to automate a process of re-taking screenshots of a webpage, even after the content and/or the functionality code of the web page has been changed while keeping the original meaning of the screenshot.
  • the WCML code may be on the header part of the web page, and the HTML code may be on the body part of the web page. [060] In some demonstrative embodiments, the WCML code may be on another URL referenced from the first page, e.g., link to another page and/or file.
  • Embodiments of the discloser may include a markup language and a protocol that is designed for non-graphic interaction with a web page.
  • a user may run these actions by voice, voice assistants, and/or other bots that may interact with the web page without the need for a software development kit (SDK) embedded in the web page.
  • SDK software development kit
  • the user may run action by a set of visual gestures, for example, waving the hand in front of the device camera.
  • the WCLM code below may demonstrate a dialog between an end- user, e.g., a visitor, and the web page.
  • embodiments of the disclosure enable a dialog between the web page and the user, e.g., a visitor, by the use of non-graphical communication and/or commands, such as, for example, voice, text, gesture and etc.
  • the use of non-graphical communication and/or commands may enable men, women, and/or children with disabilities to interact with the web page, for example, heavy-sighted.
  • the web page may be presented to the visitor in one way and for heavy-sighed visitors in another way.
  • the web page may be adapted to heavy-sited visitors.
  • embodiments of the disclosure enable a dialog between the web page and the user, e.g., a visitor, without the need to see the web page, e.g., when the communication device is hidden.
  • Figure 1 is an illustration of a block diagram of an apparatus 100 to interact with a web page 140, according to some demonstrative embodiments.
  • apparatus 100 may be configured to build the web page 140.
  • Apparatus 100 may include processing circuitry 110, an instructor 120 configured to interact with one or more web pages, for example, web page 140, and a web page generator 130 configured to generate the one or more web pages.
  • web page 140 may include a first web page code 150.
  • the first web page code 150 may be the web page code of web page 140 and may include one or more visual and/or graphic commands that a user and/or a machine may use to interact with the web page by clicking on them with, for example, a keyboard, a mouse, a pen, a finger, or the like.
  • the one or more visual and/or graphic commands may be images, text, logos, video, or the like.
  • web page 140 may include a second web page code 160, e.g., WCML markup as a second web code 160.
  • the second web page code 160 e.g., additional web code, may include a parser 162, an interaction engine 166, and one or more non-graphic commands 164.
  • parser 162 may be configured to receive the first web page code 140, e.g., the web page code, and to generate the one or more non-graphic commands 164.
  • the one or more non-graphic commands 164 may include voice commands, text commands, number commands, and the like.
  • interaction engine 166 may be configured to generate one or more none-graphic commands based on the one or more interaction components.
  • interaction engine 166 may include a gesture engine 123 operably coupled to a camera 125, e.g., a video camera.
  • gesture engine 123 may detect in one or frames received from camera 125 one or more gestures and may convert the one or more gestures to a non-graphic command, e.g., text command.
  • interaction engine 166 may include a speech-to-text (STT) engine 126 operably coupled to a microphone 124.
  • STT engine 126 may convert a voice command of a user to a text command 122 for web page 140.
  • interaction engine 166 may include a text- to-speech (TTS) engine 128 operably coupled to a speaker 127.
  • TTS engine 128 may convert a text of the web page to a voice signal in order to respond to the user command, if desired.
  • interactor 120 may be used to manage the interaction between the user and/or the machine with web page 140.
  • interactor 120 may be implemented by software and/or by hardware and/or a combination of software and hardware.
  • apparatus 100 may include a memory 170 configured to store software and/or a markup language 175, which may be used by a web page generator to build the web page 140.
  • memory 170 may include any type of memory, such as, for example, RAM, DRAM, ROM, programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), Flash memory, a hard disk drive (HDD), a solid-state disk drive (SDD), fusion drive, a nonvolatile memory, volatile memory and the like.
  • the markup language 175 may include a plurality of markup languages, such as, for example, HTML, XML, JSON, WCML, and the like.
  • web page generator 130 may be implemented by software and/or by hardware and/or a combination of software and hardware.
  • the processing circuitry 110 may be configured to add a non-graphic interaction functionality to the web page 140 by adding to the first code 150 created by a first markup language a second code 160 created by a second markup language.
  • the first markup language may include HTML, XML, JSON, Javascript, and the like.
  • the second markup language may include WCML and the like. It should be understood that the second markup language may include similar WCML languages.
  • the processing circuitry 110 may be configured to parse a code, e.g., the first code 150, of the first markup language and generate a parsed code.
  • the processing circuitry 110 may be configured to identify in the parsed code one or more components, e.g., visual commands 155, configured to provide a visual interaction functionality to interact with the web page 140.
  • parser 162 may be configured to parse the first code and identify in a parsed code one or more interaction components.
  • the processing circuitry 110 may be configured to generate one or more commands, e.g., non-graphic commands 164, configured to provide the non-graphic interaction functionality to web page 140 based on the one or more components, e.g., non-graphic commands 164.
  • commands e.g., non-graphic commands 164
  • the processing circuitry 110 may be configured to initiate an interaction with the web page 140, e.g., order pizza, when receiving a command of the one or more commands, e.g., non-graphic commands 164.
  • the processing circuitry 110 may be configured to generate, for example, a command console (not shown), wherein the command console may include the one or more commands, e.g., non-graphic commands 164.
  • a user may type the one or more commands directly on the web page 140. It should be understood that other non-graphic methods may be used to interact with web page 140.
  • interaction engine 166 may include the TTS engine 128.
  • the TTS engine 128 may be configured to convert the one or more commands into one or more voice commands, e.g., audio signals, to be used by a user and/or a machine.
  • voice commands e.g., audio signals
  • interaction engine 166 may include STT engine 126.
  • the STT engine 126 may be configured to convert the one or commands into one or more text codes to be used by web page 140 to perform the one or more voice commands.
  • the voice command "call 199" may be converted to a text code that causes web page 140 to initiate a phone call to 199.
  • FIG. 2 illustrates a flow chart of a method 200 to generate commands for interacting with a web page, according to some demonstrative embodiments.
  • method 200 may be executed by a markup language client, for example, by a WCML client.
  • the markup language client may receive a web page code of a selected web page (text box 210).
  • the markup language client may include a parser, e.g., parser 162 (FIG. 1).
  • the parser may parse the web page code and may provide a parsed code (text box 220).
  • the parser may identify one or more commands components, e.g., graphic, e.g., visual and non-graphic commands components, in the parsed code (text box 230).
  • the markup language client may include a TTS engine, e.g., TTS engine 128 (FIG.l), which may be configured to identify one or more non-graphic commands (text box 240).
  • TTS engine 128 (FIG.l)
  • the one or more non-graphic commands may be used to interact with the web page, e.g., web page 140 (FIG. 1) (text box 250).
  • the non-graphic HTML code may be located on the header of the web page and the HTML code may be on the body of the web page, e.g., web page 140 (FIG. 1).
  • the above exemplary code on the header of the web page may enable one user to interact with the page by using, for example, a mouse, and another user may interact with the web page by using, for example, voice, text, gesture and the like.
  • system 300 may include a website HTML 301.
  • the website HTML 301 may include WCML markup module 302.
  • the WCML markup module 302 may include, for example, Javascript operational code 308, which may be configured to operate code on the webpage 301.
  • system 300 may further include a markup language client 320, for example, a WCML client.
  • the markup language client 320 may be a browser, voice assistance and/or a test runner, and the like. It should be understood that a markup language client 320 is not limited to the above examples, and other use cases may be implemented to the markup language client 320.
  • the markup language client 320 may include a parser 322, e.g., WCML parser, an output module 324, e.g., WCML output module, and an input/runner module 326, e.g., WCML input/runner module.
  • the output module 324 may include a TTS engine 325, and the input/runner module may include an STT engine 328.
  • the output module 324 may output a visual instruction and/or a voice instruction to a user 330, and the input/runner module 326 may receive from user 330 a mouse click, a keyboard typing, a finger touch, a pen touch, a voice command and/or the like.
  • system 300 may further include a server 310.
  • Server 300 may include a processing circuitry 312.
  • Processing circuitry 312 may include a Central Processing Unit (CPU) and a memory.
  • server 310 may further include an application execution module 314, e.g., a software code 314, one or more databases 316, etc.
  • application execution module 314 e.g., a software code 314, one or more databases 316, etc.
  • markup language 306, e.g., WCML markup may send the JSON code and/or XML code 306 to parser 322, e.g., WCML parser.
  • Parser 322 may be configured to receive JSON code and/or XML code 306 and to generate the one or more non-graphic commands.
  • the one or more non graphic commands may include voice commands, text commands, number commands, and the like.
  • output module 324 may receive from parser 322 the non-graphic commands, e.g., voice and/or visual commands.
  • TTS engine 325 may convert the voice command and/or voice instructions to speech.
  • User 330 may select what he wants to do on the web page based on the voice command and/or voice instructions.
  • User 330 may say a voice command.
  • the STT engine 328 may convert the user voice into text.
  • the input/runner module may transfer the command to the application execution module 314 at server 310.
  • the application execution module 314 may use DB 316 in order to execute the user 330 commands.
  • the voice command may be configured to perform a phone call.
  • the input/runner module 326 may dial phone 340 to make the call.
  • the voice command may be configured to perform a call to a phone center 350.
  • the input/runner module 326 may dial phone center 350 to make the call.
  • Product 400 may include one or more tangible computer-readable non-transitory storage media 410, which may include computer-executable instructions 430, implemented by processing device 420, operable to, when executed by at least one computer processor, enable at least one processing circuitry 110 (FIG. 1) to implement one or more program instructions for generating a web page code which enables a user to interact with the web page by using non-graphic commands and/or to perform, trigger and/or implement one or more operations, communications and/or functionalities as described above with reference to Figs 1-3.
  • the phrase " non-transit ory machine-readable medium” is directed to include all computer-readable media, with the sole exception being a transitory propagating signal.
  • product 400 and/or machine-readable storage medium 410 may include one or more types of computer-readable storage media capable of storing data, including volatile memory, nonvolatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and the like.
  • machine-readable storage medium 410 may include any type of memory, such as, for example, RAM, DRAM, ROM, programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), Flash memory, a hard disk drive (HDD), a solid-state disk drive (SDD), fusion drive, and the like.
  • the computer-readable storage media may include any suitable media involved with downloading or transferring a computer program from a remote computer to a requesting computer carried by data signals embodied in a carrier wave or other propagation medium through a communication link, e.g., a modem, radio, or network connection.
  • processing device 420 may include logic.
  • the logic may include instructions, data, and/or code, which, if executed by a machine, may cause the machine to perform a method, process, and/or operations as described herein.
  • the machine may include, for example, any suitable processing platform, computing platform, computing device, processing device, a computing system, processing system, computer, processor, or the like, and may be implemented using any suitable combination of hardware, software, firmware, and the like.
  • processing device 420 may include or may be implemented as software, firmware, a software module, an application, a program, a subroutine, instructions, an instruction set, computing code, words, values, symbols, and the like.
  • Instructions 740 may include any suitable types of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. Instructions may be implemented according to a predefined computer language, manner or syntax, for instructing a processor to perform a specific function.
  • the instructions may be implemented using any suitable high-level, low- level, object-oriented, visual, compiled, and/or interpreted programming languages, such as markup language, HTML, XML, JSON, WCML, C, C++, C#, Java, Python, BASIC, Mat lab, assembly language, machine code, and the like.
  • one or more computer programs, modules, and/or applications that, when executed, perform methods of the present invention need not reside on the single computer or processor but can be distributed in a modular fashion amongst a number of different computers or processors to implement various aspects of the systems and methods disclosed herein.
  • each block in the flowchart or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s).
  • each block in the flowchart or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s).

Abstract

An apparatus, system, and method of interacting with a web page are disclosed. The apparatus includes a processing circuitry which is configured to provide a non-graphic interaction functionality to the web page by adding to the web page code a code. The code is created by a web command markup language.

Description

APPARATUS SYSTEM AND METHOD OF INTERACTING WITH A WEB
PAGE
TECHNICAL FIELD
[001] Some embodiments described herein generally related to a web page and, more specifically, to a markup language for building web pages.
BACKGROUND
[002] The way to interact with a web page is by interacting with visual elements such as, for example, links, buttons, and input fields. The interaction is done by clicking or pressing on the visual elements, e.g., images, text on the web page, via a mouse and/or a keyboard.
[003] Some solutions for accessibility may include adding a markup code to existing HyperText Markup Language (HTML) code of the web page to allow "Screen Reader" software to read and speak the web page content.
[004] However, in this type of solution, the user can not know which interactions, such as making a phone call, ordering food, creating a post, and purchasing goods, are supported on the web page.
[005] Voice interaction with the web page requires special and custom development and integration with text to speech (TTS) and/or speech to text (STT) engines.
[006] The way to interact with the web page is by using visual elements. Thus, people with disabilities, e.g., blind men, cannot interact with the web page.
SUMMARY
[007] Embodiments related to a system, a method, and a product of interacting with a web page are described hereinbelow by the ways of example only.
[008] One exemplary embodiment may include a product comprising one or more tangible computer-readable non-transitory storage media comprising program instructions for generating one or more commands for interacting with a web page, wherein execution of the program instructions by one or more processors comprising: receiving a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a first markup language configured to display the web page content to a user, and a non-graphic interaction code created by a second markup language, configured to provide the non-graphic interaction functionality to the web page; introducing the one or more interaction commands to the user; performing a non graphic interaction between the user and the web page by receiving a non-graphic interaction command of the one or more interaction commands and providing a response based on the non-graphic interaction command.
[009] One other exemplary embodiment may include a product comprising one or more tangible computer-readable non-transitory storage media comprising program instructions for generating one or more commands for interacting with a web page, wherein execution of the program instructions by one or more processors comprising: receiving a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a markup language, configured to display the web page content to a user, and a non-graphic interaction code created by a web command markup language (WCML); configured to provide the non-graphic interaction functionality to the web page; introducing the one or more interacting commands to the user; performing a non-graphic interaction between the user and the web page by receiving a non-graphic interaction command of the one or more interaction commands; and providing a response based on the non-graphic interaction command.
[010] For example, the execution of the program instructions by one or more processors comprising: with the use of the non-graphic interaction code, parsing the web page code and generate a parsed code; and identifying in the parsed code one or more components configured to provide the graphic interaction functionality to the web page.
[011] For example, the execution of the program instructions by one or more processors comprising: generating a command console wherein the command console is configured to run the one or more non-graphic interaction commands.
[012] For example, wherein the second markup language comprises one of an Extensible Markup Language (XML), a JavaScript Object Notation (JSON), a web command markup language (WCML).
[013] For example, wherein the first language and the second language comprise the same language. [014] For example, wherein the first code is written on the body of the web page and the second code is written on the header of the web page.
[015] For example, the second markup language comprises a parser configured to parse the first code and identify one or more interaction components in a parsed code. [016] For example, the processing circuitry is configured to control with the second markup language comprises an interaction engine, and execution of the program instructions of the interaction engine comprises: identifying the one or more interaction commands based on the one or more interaction components.
[017] For example, the interaction engine comprises a text to speech engine, and execution of the program instructions of the text to speech engine comprises: converting the one or more interaction commands into one or more voice commands to be used by the user and/or by a machine.
[018] For example, the interaction engine comprises a speech-to-text engine, and execution of the program instructions of the speech-to-text engine comprises: converting the one or more voice commands into text code to be used by the web page for interacting with the user by texting.
[019] Some other exemplary embodiment disclosed an apparatus and/or a program for building a web page comprising a processing circuitry, wherein the processing circuitry is configured to receive a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a first markup language configured to display the web page content to a user, and a non-graphic interaction code created by a second markup language, configured to provide the non-graphic interaction functionality to the web page; introduce the one or more interaction commands to the user; perform a non-graphic interaction between the user and the web page by receiving a non-graphic interactive command of the one or more interactive commands and provide a response based on the non-graphic interaction command.
[020] For example, the processing circuitry is configured to: with the use of the second code, parse a non-graphic interaction code of the first markup language and generate a parsed code; and identify one or more components in the parsed code configured to provide a visual interaction functionality to interact with the web page.
[021] For example, the processing circuitry is configured to: generate a command console wherein the command console is configured to run the one or more non-graphic interaction commands. [022] For example, wherein the second markup language comprises one of an Extensible Markup Language (XML), a JavaScript Object Notation (JSON), a web command markup language (WCML).
[023] For example wherein the first language and the second language comprises a same language.
[024] For example, wherein the first code is written on the body of the web page and the second code is written on the header of the web page.
[025] For example, the processing circuitry comprises a parser configured to parse the web page code and identify in a parsed code one or more interaction components.
[026] For example, the processing circuitry comprises an interaction engine configured to identify one or more user interaction commands based on the one or more interaction components.
[027] For example, the interaction engine comprises a text to speech engine configured to convert the one or more interacting commands into one or more voice commands for interacting with the user and/or a machine.
[028] For example, the interaction engine comprises a speech-to-text engine configured to convert the one or more voice commands into the text to be used for interaction with the user and/or machine.
[029] It is understood that the present disclosure described a solution for shortcomings in the field of the art. More specifically, the embodiments described herein enable interaction with a web page by using non-graphic commands.
BRIEF DESCRIPTION OF THE DRAWING
[030] FIG. 1 illustrates a block diagram of an apparatus to interact with a web page, according to some demonstrative embodiments.
[031 ] FIG. 2 illustrates a flow chart of a method to generate commands for interacting with a web page, according to some demonstrative embodiments.
[032] FIG. 3 illustrates a block diagram of a system to interact with a web page, according to some demonstrative embodiments.
[033] FIG. 4 illustrates a product of manufacture, according to some demonstrative embodiments. DETAILED DESCRIPTION
[034] In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of some embodiments. However, it will be understood by persons of ordinary skill in the art that some embodiments may be practiced without these specific details. In other instances, well-known methods, procedures, components, units, and/or circuits have not been described in detail so as not to obscure the discussion.
[035] Discussions made herein utilizing terms such as, for example, "processing," "computing," "calculating," "determining," "establishing," "analyzing," "checking," or the like, may refer to operation(s) and/or process(es) of a computer, a computing platform, a computing system, or other electronic computing devices, that manipulate and/or transform data represented as physical (e.g., electronic) quantities within the computer's registers and/or memories into other data similarly represented as physical quantities within the computer's registers and/or memories or other information storage medium that may store instructions to perform operations and/or processes.
[036] The terms "plurality" and "a plurality," as used herein, include, for example, "multiple" or "two or more." For example, "a plurality of items" includes two or more items.
[037] References to "one embodiment," "an embodiment," "demonstrative embodiment," "various embodiments," etc., indicate that the embodiment(s) so described may include a particular feature, structure, or characteristic, but not every embodiment necessarily includes the particular feature, structure, or characteristic. Further, repeated use of the phrase "in one embodiment" does not necessarily refer to the same embodiment, although it may.
[038] As used herein, unless otherwise specified, the use of the ordinal adjectives "first," "second," "third," etc., to describe a common object merely indicate that different instances of like objects are being referred to and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or any other manner.
[039] As used herein, the term "circuitry" may refer to, be part of, or include, an Application Specific Integrated Circuit (ASIC), an integrated circuit, an electronic circuit, a processor (shared, dedicated, or group), and/or memory (shared, dedicated, or group), that execute one or more software or firmware programs, a combinational logic circuit, and/or other suitable hardware components that provide the described functionality. In some demonstrative embodiments, the circuitry may be implemented in, or functions associated with the circuitry may be implemented by one or more software or firmware modules. In some demonstrative embodiments, the circuitry may include logic, at least partially operable in hardware.
[040] The term "logic" may refer, for example, to computing logic embedded in the circuitry of a computing apparatus and/or computing logic stored in a memory of a computing apparatus. For example, the logic may be accessible by a processor of the computing apparatus to execute the computing logic to perform computing functions and/or operations. In one example, logic may be embedded in various types of memory and/or firmware, e.g., silicon blocks of various chips and/or processors. Logic may be included in and/or implemented as part of various circuitry, e.g., radio circuitry, receiver circuitry, control circuitry, transmitter circuitry, transceiver circuitry, processor circuitry, and/or the like. In one example, logic may be embedded in volatile memory and/or nonvolatile memory, including random access memory, read-only memory, programmable memory, magnetic memory, flash memory, persistent memory, and the like. Logic may be executed by one or more processors using memory, e.g., registers, stuck, buffers, and/or the like, coupled to the one or more processors, e.g., as necessary to execute the logic.
[041] The term "markup language," as used hereinbelow, is a system for annotating a document in a way that is syntactically distinguishable from the text. For example, when the document is processed for display, the markup language is not shown and is used to format the text. With embodiments of the below disclosure, the markup language can be used to build a web page.
[042] The term "Extensible Markup Language (XML)" as used hereinbelow is a markup language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable.
[043] The term "(JavaScript Object Notation (JSON)," as used hereinbelow, is an open standard file format and data interchange format that uses human-readable text to store and transmit data objects consisting of attribute-value pairs and array data types. In some embodiments, JSON may serve as a replacement for XML in AJAX systems. [044] Furthermore, JSON is a language-independent data format. It was derived from JavaScript, but many modern programming languages include code to generate and parse JSON-format data.
[045] The term "Hypertext Markup Language (HTML) code," as used hereinbelow is the standard markup language for documents designed to be displayed in a web browser. For example, HTML may be assisted by technologies such as Cascading Style Sheets (CSS) and/or scripting languages such as, for example, JavaScript.
[046] For example, Web browsers may receive HTML documents from a web server and/or from local storage. The web server may render the documents into multimedia web pages.
[047] In some demonstrative embodiments, HTML elements may be the building blocks of HTML pages. With HTML constructs, images and other objects such as, for example, interactive forms may be embedded into the rendered page. For example, HTML may embed programs written in a scripting language such as, for example, JavaScript, which may affect the behavior and content of web pages.
[048] The term "web server" as used hereinbelow can be implemented in software and/or hardware and/or as a combination of software and hardware. The web server may be dedicated to running software that can satisfy client requests on the World Wide Web. A web server may include one or more websites. A web server may process incoming network requests over HTTP and several other related protocols.
[049] For example, the primary function of a web server is to store, process, and deliver web pages to clients. The communication between client and server takes place using, for example, HTTP. Pages delivered are most frequently HTML documents, which may include images, style sheets, and/or scripts in addition to the text content. [050] The term "command(s)" as used hereinbelow is a means that the user and or a machine may use to interact with the web page. For example, a command is a directive to a computer program to perform a specific task supported by a web page.
[051 ] The term "visual interaction, " as used hereinbelow, is related to interaction with a web page by using a visual element, e.g., a graphic element, on the web page, that the user can click on, press with a keyboard and/or use a voice command that is represented by the visual element.
[052] The term "non-graphic interaction," as used hereinbelow, is related to interaction with a web page using one or more interaction commands that not be represented by the graphic element on the web page and execute supported actions by the web page.
[053] The term "Web Command Markup Language (WCML)," as used hereinbelow, is a markup language that defines at least a set of none-graphic commands supported by a webpage. The markup language may include a flow of commands that depend on each other, a set of accepted arguments by each command, and a reference to operational code to be executed. The WCML may add non-graphic functionality to the web page, such as, for example, enabling full-screen video playing; connect and/or enable registration to the web page through external web pages, such as, for example, Facebook, Google, etc.; generate a communication form that includes the user name and email address which saved with the client; may interact with voice assistance; save the parameters that were used to the next use with indication to the place of where those parameters are saved, may interact with the web page by gesture and the like.
[054] For example, the WCML may include but is not limited to code that enables to perform authentication by voice. The middleware may offer to save once a pair of a username and password and/or obtain an access token from a 3rd party authentication service in order to offer login again with the saved user credentials.
[055] Furthermore, the WCML may include, but not be limited to, code that enables a required token to avoid force command. For example, to avoid automatic and not authenticated commands, the middleware may request to include a unique token generated for the current session in the command arguments. Thus, web-bots and/or a not authenticated user will be forbidden, e.g., Cross-site request forgery (CSRF) attack. [056] The WCML may include, but not be limited to, code that enables saving a current order for re-order later. Because WCML is a command-oriented language, the final command with the final arguments may be saved for re-execution later. For example, save the current order details for re-order later with the same arguments. [057] The WCML may include, but not be limited to, commands that can be used to automate a process of re-taking screenshots of a webpage, even after the content and/or the functionality code of the web page has been changed while keeping the original meaning of the screenshot.
[058] For example, to re-take an updated screenshot of a web application in a specific situation, e.g., after adding a new task with the status "In Progress."
[059] In some demonstrative embodiments, the WCML code may be on the header part of the web page, and the HTML code may be on the body part of the web page. [060] In some demonstrative embodiments, the WCML code may be on another URL referenced from the first page, e.g., link to another page and/or file.
[061] Advantageously, in the current state of the art, in order to automate a process that takes a webpage screenshot in a specific state (not as loaded by the URL), it's needed that the HTML or the Javascript of the web application will be the same as the time the original screenshot was taken. But with the WCML commands method, the automation system can re-run the same commands and re-take a screenshot that presents the same situation, although the HTML and the JavaScript functionality were changed.
[062] It should be understood that WCML is a name and/or a title that is used to describe the web command markup language has it described above. Other names and/or titles may be used for the web command markup language described above. [063] Embodiments of the discloser may include a markup language and a protocol that is designed for non-graphic interaction with a web page. A method to create web pages that include a markup language to describe supported actions and interactions on a specific web page, a method to extend the browser functionality in order to read these actions and make them be accessible in many ways for the end-user, a browser extension and/or an independent software that may read the content of the web page to a user and ask him what to do, without reading the full web page.
[064] For example, a user may run these actions by voice, voice assistants, and/or other bots that may interact with the web page without the need for a software development kit (SDK) embedded in the web page.
[065] In some demonstrative embodiments, the user may run action by a set of visual gestures, for example, waving the hand in front of the device camera.
[066] For example, the WCLM code below may demonstrate a dialog between an end- user, e.g., a visitor, and the web page.
<wcml>
<command name="contact" label="Contact " callback="sendContactDetails ">
<input name="name" label="What is your name?" allow-saved-reply=" true">
<input name="phone" type="tel" Label="To which phone number do you want us to call you?" allow-saved-reply=" true"> <input name="email" type="email" Label="What is your email?" allow-saved-reply="true"> <input name="message" type="long-text" label="Tell me your message, I'm listening, to stop say twice the stop keyword" stop- keyword="End Message">
</command?
<command name="quick-booking" label="Quick Booking" callback="sendQuickBookingDetails ">
<input name="name" label="What is your name?" allow-saved-reply="true">
<input name="email" type="email" Label="What is your email?" allow-saved-reply="true">
<input name="people" type="number " Label="How many people?" min="l" max="3">
<input name="from" type="date" Label="From which date?" min-validator="today() ">
<input name="to" type="date" Label="Until which date? ">
</command?
<command name="create" label="Create">
<select name="content-type" label="What type of content to create?"?
<command name="create-post" label="Post" callback="createPost ">
<input name="content" type="long- text" label="Tell me your post, I'm listening, to stop say twice the stop keyword" stop-keyword="End Post">
</command?
<command name="create-checkin" label="Check In" callback="createCheckin"> <input name="location" type="location" label="Where are you checking in?">
</command>
</select>
</command>
</wcml>"
[067] Advantageously, embodiments of the disclosure enable a dialog between the web page and the user, e.g., a visitor, by the use of non-graphical communication and/or commands, such as, for example, voice, text, gesture and etc. Furthermore, the use of non-graphical communication and/or commands may enable men, women, and/or children with disabilities to interact with the web page, for example, heavy-sighted. [068] For example, the web page may be presented to the visitor in one way and for heavy-sighed visitors in another way. For example, the web page may be adapted to heavy-sited visitors. In addition, embodiments of the disclosure enable a dialog between the web page and the user, e.g., a visitor, without the need to see the web page, e.g., when the communication device is hidden.
[069] Referring first to Figure 1, which is an illustration of a block diagram of an apparatus 100 to interact with a web page 140, according to some demonstrative embodiments.
[070] In some demonstrative embodiments, apparatus 100 may be configured to build the web page 140. Apparatus 100 may include processing circuitry 110, an instructor 120 configured to interact with one or more web pages, for example, web page 140, and a web page generator 130 configured to generate the one or more web pages.
[071] In some demonstrative embodiments, web page 140 may include a first web page code 150. For example, the first web page code 150 may be the web page code of web page 140 and may include one or more visual and/or graphic commands that a user and/or a machine may use to interact with the web page by clicking on them with, for example, a keyboard, a mouse, a pen, a finger, or the like. For example, the one or more visual and/or graphic commands may be images, text, logos, video, or the like.
[072] In some demonstrative embodiments, web page 140 may include a second web page code 160, e.g., WCML markup as a second web code 160. For example, the second web page code 160, e.g., additional web code, may include a parser 162, an interaction engine 166, and one or more non-graphic commands 164.
[073] In some demonstrative embodiments, parser 162 may be configured to receive the first web page code 140, e.g., the web page code, and to generate the one or more non-graphic commands 164. For example, the one or more non-graphic commands 164 may include voice commands, text commands, number commands, and the like.
[074] In some demonstrative embodiments, interaction engine 166 may be configured to generate one or more none-graphic commands based on the one or more interaction components.
[075] In some demonstrative embodiments, interaction engine 166 may include a gesture engine 123 operably coupled to a camera 125, e.g., a video camera. For example, gesture engine 123 may detect in one or frames received from camera 125 one or more gestures and may convert the one or more gestures to a non-graphic command, e.g., text command.
[076] In some demonstrative embodiments, interaction engine 166 may include a speech-to-text (STT) engine 126 operably coupled to a microphone 124. For example, STT engine 126 may convert a voice command of a user to a text command 122 for web page 140.
[077] In some demonstrative embodiments, interaction engine 166 may include a text- to-speech (TTS) engine 128 operably coupled to a speaker 127. For example, TTS engine 128 may convert a text of the web page to a voice signal in order to respond to the user command, if desired.
[078] In some demonstrative embodiments, interactor 120 may be used to manage the interaction between the user and/or the machine with web page 140.
[079] In some demonstrative embodiments, interactor 120 may be implemented by software and/or by hardware and/or a combination of software and hardware.
[080] In some demonstrative embodiments, apparatus 100 may include a memory 170 configured to store software and/or a markup language 175, which may be used by a web page generator to build the web page 140.
[081] For example, memory 170 may include any type of memory, such as, for example, RAM, DRAM, ROM, programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), Flash memory, a hard disk drive (HDD), a solid-state disk drive (SDD), fusion drive, a nonvolatile memory, volatile memory and the like. [082] For example, the markup language 175 may include a plurality of markup languages, such as, for example, HTML, XML, JSON, WCML, and the like.
[083] In some demonstrative embodiments, web page generator 130 may be implemented by software and/or by hardware and/or a combination of software and hardware.
[084] In some demonstrative embodiments, the processing circuitry 110 may be configured to add a non-graphic interaction functionality to the web page 140 by adding to the first code 150 created by a first markup language a second code 160 created by a second markup language. For example, the first markup language may include HTML, XML, JSON, Javascript, and the like. The second markup language may include WCML and the like. It should be understood that the second markup language may include similar WCML languages.
[085] In some demonstrative embodiments, with the use of the second code 160, the processing circuitry 110 may be configured to parse a code, e.g., the first code 150, of the first markup language and generate a parsed code.
[086] In some demonstrative embodiments, with the use of the second code 160, the processing circuitry 110 may be configured to identify in the parsed code one or more components, e.g., visual commands 155, configured to provide a visual interaction functionality to interact with the web page 140.
[087] For example, parser 162 may be configured to parse the first code and identify in a parsed code one or more interaction components.
[088] In some demonstrative embodiments, with the use of the second code 160, the processing circuitry 110 may be configured to generate one or more commands, e.g., non-graphic commands 164, configured to provide the non-graphic interaction functionality to web page 140 based on the one or more components, e.g., non-graphic commands 164.
[089] In some demonstrative embodiments, with the use of the second code 160, the processing circuitry 110 may be configured to initiate an interaction with the web page 140, e.g., order pizza, when receiving a command of the one or more commands, e.g., non-graphic commands 164.
[090] In some demonstrative embodiments, the processing circuitry 110 may be configured to generate, for example, a command console (not shown), wherein the command console may include the one or more commands, e.g., non-graphic commands 164. [091] In some other demonstrative embodiments, a user may type the one or more commands directly on the web page 140. It should be understood that other non-graphic methods may be used to interact with web page 140.
[092] In some demonstrative embodiments, interaction engine 166 may include the TTS engine 128. The TTS engine 128 may be configured to convert the one or more commands into one or more voice commands, e.g., audio signals, to be used by a user and/or a machine. For example, "Pizza," "Olives," "Cola" and the like [093] In some demonstrative embodiments, interaction engine 166 may include STT engine 126. The STT engine 126 may be configured to convert the one or commands into one or more text codes to be used by web page 140 to perform the one or more voice commands. For example, the voice command "call 199" may be converted to a text code that causes web page 140 to initiate a phone call to 199.
[094] Reference is now made to FIG. 2, which illustrates a flow chart of a method 200 to generate commands for interacting with a web page, according to some demonstrative embodiments.
[095] In some demonstrative embodiments, method 200 may be executed by a markup language client, for example, by a WCML client. The markup language client may receive a web page code of a selected web page (text box 210).
[096] In some demonstrative embodiments, the markup language client may include a parser, e.g., parser 162 (FIG. 1). The parser may parse the web page code and may provide a parsed code (text box 220). The parser may identify one or more commands components, e.g., graphic, e.g., visual and non-graphic commands components, in the parsed code (text box 230).
[097] In some demonstrative embodiments, the markup language client may include a TTS engine, e.g., TTS engine 128 (FIG.l), which may be configured to identify one or more non-graphic commands (text box 240). The one or more non-graphic commands may be used to interact with the web page, e.g., web page 140 (FIG. 1) (text box 250).
[098] In some demonstrative embodiments, the non-graphic HTML code may be located on the header of the web page and the HTML code may be on the body of the web page, e.g., web page 140 (FIG. 1).
[099] For example, the below code may be presented on the header of the web page:
<wcml> <command name="help" trigger="on-load" callback- 'actionsList" label="Welcome to the website of the Israel Patent Office, we will immediately present you with the options for action on the website. What is your selection?" />
<command name="call" label="Call Us" action-label- 'Call" callback="call"/>
<command name=" contact" label="We can call you" action-label="Call Me" callb ack= " sendContactD etail s " >
<input name="name" label- 'What is your name?" allow-saved- reply="true"/>
<input name="phone" type="tel" Label="To which phone number do you want us to call you?" allow-saved-reply="true"/>
</command>
</wcml>
[0100] With the above example of HTML code, the visitor of the web page may not see the effect of this code on the web page, but he may hear the web saying:
"Welcome to the website of the Israel Patent Office, and we will immediately present you with the options for action on the website. What is your selection?"
"Callus"
"We can call you"
"What is your name?"
"To which phone number do you want us to call you?"
[0101] Another example of WCML code:
<wcml>
<command name="create" label="Create">
<select name="content-type" label="What type of content to create?">
<command name="create-post" label="Post" callback="createPost ">
<input name="content" type="long- text" label="Tell me your post, I'm listening, to stop say twice the stop keyword" stop-keyword="End Post">
</command>
<command name="create-checkin" label="Check In" callback="createCheckin"> <input name="location" type="location" label="Where are you checking in?">
</command>
</select>
</command>
</wcml>
[0102] The above exemplary code on the header of the web page, e.g., web page 140, may enable one user to interact with the page by using, for example, a mouse, and another user may interact with the web page by using, for example, voice, text, gesture and the like.
[0103] Reference is now made to FIG. 3, which illustrates a block diagram of a system 300 to interact with a web page, according to some demonstrative embodiments. [0104] In some demonstrative embodiments, system 300 may include a website HTML 301. The website HTML 301 may include WCML markup module 302. The WCML markup module 302 may include, for example, Javascript operational code 308, which may be configured to operate code on the webpage 301.
[0105] In some demonstrative embodiments, system 300 may further include a markup language client 320, for example, a WCML client. For example, the markup language client 320 may be a browser, voice assistance and/or a test runner, and the like. It should be understood that a markup language client 320 is not limited to the above examples, and other use cases may be implemented to the markup language client 320. [0106] In some demonstrative embodiments, the markup language client 320 may include a parser 322, e.g., WCML parser, an output module 324, e.g., WCML output module, and an input/runner module 326, e.g., WCML input/runner module. The output module 324 may include a TTS engine 325, and the input/runner module may include an STT engine 328. The output module 324 may output a visual instruction and/or a voice instruction to a user 330, and the input/runner module 326 may receive from user 330 a mouse click, a keyboard typing, a finger touch, a pen touch, a voice command and/or the like.
[0107] In some demonstrative embodiments, system 300 may further include a server 310. Server 300 may include a processing circuitry 312. Processing circuitry 312 may include a Central Processing Unit (CPU) and a memory.
[0108] In some demonstrative embodiments, server 310 may further include an application execution module 314, e.g., a software code 314, one or more databases 316, etc.
[0109] In some demonstrative embodiments, markup language 306, e.g., WCML markup may send the JSON code and/or XML code 306 to parser 322, e.g., WCML parser. Parser 322 may be configured to receive JSON code and/or XML code 306 and to generate the one or more non-graphic commands. For example, the one or more non graphic commands may include voice commands, text commands, number commands, and the like.
[0110] In some demonstrative embodiments, output module 324, e.g., WCML output, may receive from parser 322 the non-graphic commands, e.g., voice and/or visual commands. TTS engine 325 may convert the voice command and/or voice instructions to speech. User 330 may select what he wants to do on the web page based on the voice command and/or voice instructions. User 330 may say a voice command. The STT engine 328 may convert the user voice into text.
[0111] For example, the input/runner module may transfer the command to the application execution module 314 at server 310. The application execution module 314 may use DB 316 in order to execute the user 330 commands.
[0112] In another example, the voice command may be configured to perform a phone call. The input/runner module 326 may dial phone 340 to make the call.
[0113] In a further example, the voice command may be configured to perform a call to a phone center 350. The input/runner module 326 may dial phone center 350 to make the call.
[0114] Reference is now made to FIG 4, which is a schematic illustration of a product of manufacture 400, according to some demonstrative embodiments. Product 400 may include one or more tangible computer-readable non-transitory storage media 410, which may include computer-executable instructions 430, implemented by processing device 420, operable to, when executed by at least one computer processor, enable at least one processing circuitry 110 (FIG. 1) to implement one or more program instructions for generating a web page code which enables a user to interact with the web page by using non-graphic commands and/or to perform, trigger and/or implement one or more operations, communications and/or functionalities as described above with reference to Figs 1-3. The phrase " non-transit ory machine-readable medium" is directed to include all computer-readable media, with the sole exception being a transitory propagating signal.
[0115] In some demonstrative embodiments, product 400 and/or machine-readable storage medium 410 may include one or more types of computer-readable storage media capable of storing data, including volatile memory, nonvolatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and the like.
[0116] For example, machine-readable storage medium 410 may include any type of memory, such as, for example, RAM, DRAM, ROM, programmable ROM (PROM), erasable programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), Flash memory, a hard disk drive (HDD), a solid-state disk drive (SDD), fusion drive, and the like. The computer-readable storage media may include any suitable media involved with downloading or transferring a computer program from a remote computer to a requesting computer carried by data signals embodied in a carrier wave or other propagation medium through a communication link, e.g., a modem, radio, or network connection.
[0117] In some demonstrative embodiments, processing device 420 may include logic. The logic may include instructions, data, and/or code, which, if executed by a machine, may cause the machine to perform a method, process, and/or operations as described herein. The machine may include, for example, any suitable processing platform, computing platform, computing device, processing device, a computing system, processing system, computer, processor, or the like, and may be implemented using any suitable combination of hardware, software, firmware, and the like.
[0118] In some demonstrative embodiments, processing device 420 may include or may be implemented as software, firmware, a software module, an application, a program, a subroutine, instructions, an instruction set, computing code, words, values, symbols, and the like. Instructions 740 may include any suitable types of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, and the like. Instructions may be implemented according to a predefined computer language, manner or syntax, for instructing a processor to perform a specific function. The instructions may be implemented using any suitable high-level, low- level, object-oriented, visual, compiled, and/or interpreted programming languages, such as markup language, HTML, XML, JSON, WCML, C, C++, C#, Java, Python, BASIC, Mat lab, assembly language, machine code, and the like.
[0119] It is to be understood that the system and/or the method for generating a web page code that enables a user to interact with the web page by using non-graphic commands is described hereinabove by way of example only. Other embodiments may be implemented based on the detailed description and the claims that followed.
[0120] It is to be understood that numerals in the drawings represent like elements through the several figures and that not all components and/or steps described and illustrated with reference to the figures are required for all embodiments or arrangements.
[0121] It should also be understood that the embodiments, implementations, and/or arrangements of the systems and methods disclosed herein can be incorporated as a software algorithm, application, program, module, or code residing in hardware, firmware, and/or on a computer useable medium (including software modules and browser plug-ins) that can be executed in a processor of a computer system or a computing device to configure the processor and/or other elements to perform the functions and/or operations described herein.
[0122] It should be appreciated that according to at least one embodiment, one or more computer programs, modules, and/or applications that, when executed, perform methods of the present invention need not reside on the single computer or processor but can be distributed in a modular fashion amongst a number of different computers or processors to implement various aspects of the systems and methods disclosed herein.
[0123] Thus, illustrative embodiments and arrangements of the present systems and methods provide a computer-implemented method, computer system, and computer program product for processing code(s). The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments and arrangements. In this regard, each block in the flowchart or block diagrams can represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). [0124] It should also be noted that, in some alternative implementations, the functions noted in the block can occur out of order noted in the figures. For example, two blocks shown in succession may be executed substantially concurrently, or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by particular purpose hardware-based systems that perform the specified functions or acts or combinations of specialized purpose hardware and computer instructions.
[0125] The terminology used herein is to describe particular embodiments only and is not intended to be limiting of the invention. As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
[0126] Also, the phraseology and terminology used herein are for the purpose of description and should not be regarded as limiting. The use of "including," "comprising," or "having," "containing," "involving," and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items.
[0127] The subject matter described above is provided by way of illustration only and should not be construed as limiting. Various modifications and changes can be made to the subject matter described herein without following the example embodiments and applications illustrated and described and without departing from the true spirit and scope of the present invention, which is set forth in the following claims.

Claims

CLAIMS What is claimed is:
1. A product comprising one or more tangible computer-readable non-transitory storage media comprising program instructions for generating one or more commands for interacting with a web page, wherein execution of the program instructions by one or more processors comprising: receiving a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a first markup language configured to display the web page content to a user, and a non-graphic interaction code created by a second markup language, configured to provide the non-graphic interaction functionality to the web page; introducing the one or more interaction commands to the user; performing a non-graphic interaction between the user and the web page by receiving a non-graphic interaction command of the one or more interaction commands; and providing a response based on the non-graphic interaction command.
2. A product comprising one or more tangible computer-readable non-transitory storage media comprising program instructions for generating one or more commands for interacting with a web page, wherein execution of the program instructions by one or more processors comprising: receiving a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a markup language, configured to display the web page content to a user, and a non-graphic interaction code created by a web command markup language (WCML); configured to provide the non-graphic interaction functionality to the web page; introducing the one or more interacting commands to the user ; performing a non-graphic interaction between the user and the web page by receiving a non-graphic interaction command of the one or more interaction commands; and providing a response based on the non-graphic interaction command.
3. The product of claim 1 or 2, wherein execution of the program instructions by one or more processors comprising: with the use of the non-graphic interaction code, parsing the web page code and generating a parsed code; and identifying in the parsed code one or more components configured to provide the graphic interaction functionality to the web page.
4. The product of claim 1 or 2, wherein execution of the program instructions by one or more processors comprising: generating a command console wherein the command console is configured to run the one or more non-graphic interaction commands.
5. The product of claim 1 , wherein the second markup language comprises one of an Extensible Markup Language (XML), a JavaScript Object Notation (JSON), a web command markup language (WCML).
6. The product of claim 1 wherein the first language and the second language comprises a same language.
7. The product of claim 2 wherein the markup language comprises WCML.
8. The product of claim 1, wherein the processing circuitry is configured to control with the second markup language a parser configured to parse the code and identify in a parsed code one or more interaction components.
9. The product of claim 1, wherein the processing circuitry is configured to control with the second markup language, comprises an interaction engine and execution of the program instructions of the interaction engine comprises: identifying the one or more interaction commands based on the one or more interaction components.
10. The product of claim 11, wherein the interaction engine comprises a text to speech engine and execution of the program instructions of the text to speech engine comprises: converting the one or more interaction commands into one or more voice commands to be used by the user.
11. The product of claim 12, wherein the interaction engine comprises a speech to text engine and execution of the program instructions of the speech to text engine, comprises: converting the one or more voice commands into text commands to be used by the web page for interaction with the user by texting.
12. An apparatus for interacting with a web page comprising a processing circuitry, wherein the processing circuitry is configured to: receive a web page with a non-graphic interaction functionality, wherein the web page comprises a code created by a first markup language configured to display the web page content to a user, and a non-graphic interaction code created by a second markup language, configured to provide the non-graphic interaction functionality to the web page; introduce the one or more interaction commands to the user; perform a non-graphic interaction between the user and the web page by receiving a non-graphic interactive command of the one or more interactive commands; and provide a response based on the non-graphic interaction command.
13. The apparatus of claim 14, wherein the processing circuitry is configured to: with the use of the second code, parse the non-graphic interaction code and generate a parsed code; and identify one or more components in the parsed code which are configured to provide the graphic interaction functionality to the web page.
14. The apparatus of claim 14, wherein the processing circuitry is configured to: generate a command console wherein the command console is configured to run the one or more non-graphic interaction commands.
15. The apparatus of claim 14, wherein the second markup language comprises one of an Extensible Markup Language (XML), a JavaScript Object Notation (JSON), a web command markup language (WCML).
16. The apparatus of claim 14, wherein the first language and the second language comprise the same language.
17. The apparatus of claim 14, wherein the first code is written on the body of the web page and the second code is written on the header of the web page.
18. The apparatus of claim 12, wherein the processing circuitry comprises a parser configured to parse the web page code and identify in a parsed code one or more interaction components.
19. The apparatus of claim 12, wherein the processing circuitry comprises an interaction engine which is configured to identify one or more interaction commands based on the one or more interacting interaction components.
20. The apparatus of claim 19, wherein the interaction engine comprises a text to speech engine configured to convert the one or more interaction commands into one or more voice commands to be used by a user.
21. The apparatus of claim 19, wherein the interaction engine comprises a speech to text engine configured to convert the one or more voice commands into text to be used by the web page for interaction with the user.
PCT/IL2022/050170 2021-02-14 2022-02-13 Apparatus system and method of interacting with a web page WO2022172272A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
IL280842 2021-02-14
IL280842A IL280842A (en) 2021-02-14 2021-02-14 Apparatus system and method of interacting with a web page

Publications (1)

Publication Number Publication Date
WO2022172272A1 true WO2022172272A1 (en) 2022-08-18

Family

ID=82611050

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IL2022/050170 WO2022172272A1 (en) 2021-02-14 2022-02-13 Apparatus system and method of interacting with a web page

Country Status (2)

Country Link
IL (1) IL280842A (en)
WO (1) WO2022172272A1 (en)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140040722A1 (en) * 2012-08-02 2014-02-06 Nuance Communications, Inc. Methods and apparatus for voiced-enabling a web application
WO2014189987A1 (en) * 2013-05-21 2014-11-27 Microsoft Corporation Method for finding elements in a webpage suitable for use in a voice user interface (disambiguation)
US20170269816A1 (en) * 2016-03-18 2017-09-21 Audioeye, Inc. Modular Systems and Methods For Selectively Enabling Cloud-Based Assistive Technologies

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6587822B2 (en) * 1998-10-06 2003-07-01 Lucent Technologies Inc. Web-based platform for interactive voice response (IVR)
US20050229048A1 (en) * 2004-03-30 2005-10-13 International Business Machines Corporation Caching operational code in a voice markup interpreter
US8413047B2 (en) * 2011-05-12 2013-04-02 Qualcomm Incorporated Concurrent parsing and processing of HTML and JAVASCRIPT®
US9288321B2 (en) * 2014-03-07 2016-03-15 Paypal, Inc. Interactive voice response interface for webpage navigation

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140040722A1 (en) * 2012-08-02 2014-02-06 Nuance Communications, Inc. Methods and apparatus for voiced-enabling a web application
WO2014189987A1 (en) * 2013-05-21 2014-11-27 Microsoft Corporation Method for finding elements in a webpage suitable for use in a voice user interface (disambiguation)
US20170269816A1 (en) * 2016-03-18 2017-09-21 Audioeye, Inc. Modular Systems and Methods For Selectively Enabling Cloud-Based Assistive Technologies

Also Published As

Publication number Publication date
IL280842A (en) 2022-07-01

Similar Documents

Publication Publication Date Title
US10649826B2 (en) Flexible scripting platform for troubleshooting
US9584504B2 (en) Auto login method and device
US9807224B2 (en) Method and apparatus for accessing services of a device
US9122848B2 (en) Authentication of user interface elements in a web 2.0 environment
US20110202854A1 (en) Metadata Capture for Screen Sharing
US9916128B2 (en) Visual and voice co-browsing framework
US9026902B2 (en) Handling unexpected responses to script executing in client-side application
CN107622135A (en) Method and apparatus for display information
US20140325323A1 (en) Online video playing method and apparatus and computer readable medium
CN108416021B (en) Browser webpage content processing method and device, electronic equipment and readable medium
WO2015043442A1 (en) Method, device and mobile terminal for text-to-speech processing
WO2014176906A1 (en) Online video playing method and apparatus and computer readable medium
CN113382083B (en) Webpage screenshot method and device
CN103384244A (en) Method and apparatus for configuring keyboard for web browser
US9805181B1 (en) Messaging channel for web pages and web applications
CN113076294A (en) Information sharing method and device
CN112015383A (en) Login method and device
US10142446B2 (en) Dialog server
CN107193819B (en) Method and equipment for webpage pre-reading, browser and client equipment
US8041839B2 (en) Method and system of providing active web user interface
GB2478767B (en) Method and apparatus for accessing services of a device
US20190304336A1 (en) Editing tool for math equations
WO2022172272A1 (en) Apparatus system and method of interacting with a web page
US10742639B1 (en) System and method for improving login and registration efficiency to network-accessed data
CN111968630B (en) Information processing method and device and electronic equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22752460

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22752460

Country of ref document: EP

Kind code of ref document: A1