KR20200086175A - Method for augmentative and alternative communication, and computer program recorded on record-medium for executing method therefor - Google Patents

Abstract

According to the present invention, provided is an augmentative and alternative communication method. The augmentative and alternative communication method comprises the steps of: loading a keyboard arrangement pre-registered corresponding to a user account; configuring and outputting virtual keys in accordance with the loaded keyboard arrangement, and selecting any one of the keys on the basis of information received from an auxiliary input device connected for augmentative and alternative communication; outputting a symbol corresponding to the selected key to a text box; and allowing modification of the symbol output to the text box until input completion is received from the auxiliary input device, and transmitting the symbol output to the text box to a predetermined application when the input completion is received. Accordingly, a user selects a symbol to be input without limitation, and is provided with an opportunity for modifying an unintentionally selected symbol.

Description

Method for augmentative and alternative communication, and computer program recorded on record-medium for executing method therefor}

The present invention relates to special education. More specifically, it relates to a method for performing augmentative and alternative communication (AAC) and a computer program recorded on a recording medium to implement the same.

Complementary Alternate Communication (AAC) refers to various forms of communication used to promote the language ability of a person who has difficulty communicating through speech or writing. For example, complementary alternative communication (AAC) supports communication by augmenting the speech of a person who is late in speech development or has problems with articulation, or surgery or disability on articulation organs. This may include supporting other alternative means of communication for those who are unable to pronounce.

In general, the complementary alternative communication (AAC) is a method of directly selecting a symbol (symbol) that the user wants to input or a method of indirectly selecting a symbol that the user wants to input.

Specifically, a method of directly selecting a symbol indicates a symbol that a user wants to input with a part of his body (eg, finger, toe, nose, eye, elbow, etc.), or attempts to input it using auxiliary means. It is to point out the sign to be directly or indirectly.

Here, the auxiliary means for directly pointing out the sign may include head sticks, hand splints, and mouth sticks. In addition, auxiliary means that can indirectly indicate a symbol may include a touch pad, a light beam pointer, joysticks, and an ocular eyegaze monitor.

A method of indirectly selecting a symbol is to select a symbol to be input by a user in a process in which the communication device scans for various symbols. Here, the scanning may be classified into automatic scanning, step scanning, inverse scanning, group scanning, and row-column scanning.

Automatic scanning is a method in which a user selects a symbol by entering a switch when focus reaches a symbol to be input in the process of scanning various symbols. Here, the focus is an indicator indicating a symbol that can be selected by a user among a plurality of symbols in the scanning process.

Sequential scanning is a method in which the user repeatedly inputs a switch to move the focus to a symbol to be input, and then inputs another switch to select the symbol.

Reverse scanning is a method in which the user presses a switch until the focus reaches the symbol to be input, and when the focus reaches the symbol to be input, the user presses the switch to select the symbol.

Group scanning is a method that primarily performs scanning on largely classified groups, and when a specific group is selected by a user, scanning is performed on subclassified symbols belonging to the selected group.

Then, matrix scanning preferentially scans rows among various symbols arranged in a two-dimensional matrix, and when a specific row is selected by the user, scans symbols belonging to the selected row. This is how it works.

However, according to the conventional complementary alternative communication methods, the user was able to select a symbol to be input only within a limited kind of symbols. In addition, the user had to search for a symbol to be input among symbols arranged at a predetermined location.

In addition, other symbols are unintentionally selected in the process of directly selecting a symbol to be input (ie, the user points out using a part of the body or assistive means), or indirectly selecting a symbol to be input in the scanning process. It is very likely to choose. However, according to the conventional complementary alternative communication methods, it is intended to output the unintentionally selected symbol (ie, a typo) in the form of a screen or sound directly, so that the user is experiencing difficulty in communication. There is a problem of exposing directly from the communication partner.

Accordingly, there is a need for a solution that can provide an opportunity to select a symbol to be input by the user without limitation, and to modify it even if an unintentionally selected other symbol is selected.

Republic of Korea Patent Publication No. 10-2005-0092293,'Complementary and alternative communication terminal for the disabled', (2005.09.21. public)

One object of the present invention is to provide a complementary alternative communication method, which provides an opportunity to select a symbol to be input by a user without limitation, and to modify an unintentionally selected symbol.

Another disclosure of the present invention provides a computer program recorded on a recording medium in order to implement a complementary alternative communication method, which provides an opportunity to select a symbol to be input by a user without limitation and to modify an unintentionally selected symbol. The purpose is to do.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

The present invention proposes a complementary alternative communication method for achieving the technical problems as described above. The complementary alternative communication method includes loading a keyboard layout registered in advance corresponding to a user's account; Based on the information received from the auxiliary input device connected for augmentative and alternative communication, the virtual keys are configured and output according to the loaded keyboard layout, and any one of the keys is selected. Selecting; Outputting a symbol corresponding to the selected key to a text box; And allowing modification of a symbol output to the text box until input completion is received from the auxiliary input device, and when the input completion is received, passing the symbol output to the text box to a previously designated application. It may include.

The application may be determined according to the priority of a process, task, or thread managed by an operating system. As such, the application may be any one of a web browser, a social network service, or a messenger, but is not limited thereto.

The loading of the keyboard layout may determine a keyboard layout determined according to the type of the application among a plurality of keyboard arrangements registered in correspondence to the user's account.

The selecting step may be received from the auxiliary input device when the auxiliary input device is any one of a touch pad, a light beam pointer, joysticks, and an ocular eyegaze monitor. It is possible to select a key positioned to correspond to the input coordinates. Alternatively, when the auxiliary input device is a switch, the selecting step receives a selection signal from the auxiliary input device while sequentially moving focus with respect to the keys arranged according to the arrangement of the keyboard. You can select the key where the focus is placed at the point in time.

Each key constituting the keyboard layout may correspond to any one of Hangeul consonants, Hangeul gatherings, one or more words, and sentences composed of a plurality of words. Then, when the selected key corresponds to a Korean vowel, the outputting step may change the color of the key corresponding to the Hangul consonant among the outputted keys according to the keyboard layout and output the changed key.

The step of outputting identifies a misspelled symbol in a string consisting of symbols being output to the text box based on a spelling dictionary, and of the misspelled symbol among the symbols being output to the text box. You can change the color and print it out.

The outputting step enqueues a symbol corresponding to the selected key into a queue, and the delivering step dequeues the symbols inserted into the queue one by one until the queue is empty. By doing so, it can be inserted into the message input queue of the application. In this case, the delivering step may insert a key input message (WM_KEYDOWN) having a virtual key code corresponding to the symbol extracted from the queue as a word parameter (wParam) into the message input queue. .

In the transmitting, when the input completion is received, a word or sentence composed of symbols output to the text box may be output as a sound using a text to speech (TTS) library.

In order to achieve the technical problems as described above, the present invention proposes a computer program recorded on a recording medium to implement a complementary alternative communication method. The program includes loading a keyboard layout registered in advance corresponding to a user's account; Constructing and outputting virtual keys according to the loaded keyboard layout, and selecting any one of the keys based on information received from an auxiliary input device connected for complementary alternative communication (AAC). ; Outputting a symbol corresponding to the selected key to a text box; And allowing modification of a symbol output to the text box until input completion is received from the auxiliary input device, and when the input completion is received, passing the symbol output to the text box to a predetermined application. Can.

Details of other embodiments are included in the detailed description and drawings.

According to embodiments of the present invention, as well as Hangul consonants, Hangul vowels, English alphabets, numbers, and special symbols, as well as frequently used words and sentences are provided to provide a customized keyboard that can be freely arranged, making communication difficult. The range of intentions that a user can experience can be expanded.

In particular, according to embodiments of the present invention, since the opportunity to modify the symbol without promptly outputting the symbol selected by the user is provided, the communication counterpart may not expose that the user is having difficulty communicating.

The effects of the present invention are not limited to the above-mentioned effects, and other effects not mentioned will become apparent to those skilled in the art from the description of the claims.

1 and 2 are exemplary views showing some examples of a conventional complementary alternative communication method.
3 is an exemplary view showing an example of a complementary alternative communication method according to an embodiment of the present invention.
4 is a configuration diagram of a complementary alternative communication system according to an embodiment of the present invention.
5 is a logical configuration diagram of a communication device according to an embodiment of the present invention.
6 is a hardware configuration diagram of a communication device according to an embodiment of the present invention.
7 is a flowchart illustrating a process in which a communication device performs a complementary alternative communication method according to an embodiment of the present invention.
8 is a flowchart illustrating a process of delivering a symbol to an input target application according to an embodiment of the present invention.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the present invention. In addition, technical terms used in this specification should be interpreted as meanings generally understood by a person having ordinary knowledge in the technical field to which the present invention belongs, unless defined otherwise. It should not be interpreted as a meaning or an excessively reduced meaning. In addition, when the technical term used in this specification is a wrong technical term that does not accurately represent the spirit of the present invention, it should be understood as being replaced by a technical term that can be correctly understood by those skilled in the art. In addition, the general terms used in the present invention should be interpreted as defined in the dictionary or in context before and after, and should not be interpreted as an excessively reduced meaning.

In addition, a singular expression used in this specification includes a plural expression unless the context clearly indicates otherwise. In the present application, terms such as "consist of" or "have" should not be construed as including all of the various components, or various steps described in the specification, including some of the components or some steps It may or may not be construed as further comprising additional components or steps.

Further, terms including ordinal numbers such as first and second used in the present specification may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from other components. For example, the first component may be referred to as a second component without departing from the scope of the present invention, and similarly, the second component may also be referred to as a first component.

When an element is said to be "connected" or "connected" to another component, it may be directly connected to or connected to the other component, but other components may exist in the middle. On the other hand, when a component is said to be "directly connected" or "directly connected" to another component, it should be understood that no other component exists in the middle.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the accompanying drawings, but the same or similar elements will be given the same reference numbers regardless of the reference numerals, and redundant descriptions thereof will be omitted. In addition, in the description of the present invention, when it is determined that detailed descriptions of related known technologies may obscure the subject matter of the present invention, detailed descriptions thereof will be omitted. In addition, it should be noted that the accompanying drawings are only for easy understanding of the spirit of the present invention and should not be interpreted as limiting the spirit of the present invention by the accompanying drawings. The spirit of the present invention should be interpreted to be extended to all changes, equivalents, and substitutes in addition to the accompanying drawings.

Prior to the description of the present invention, some terms repeatedly described herein are described.

Augmentative and Alternative Communication (AAC) is a form of communication that is used to promote the language ability of those who have difficulty communicating through speech or writing. The complementary alternative communication device is a device that supports communication by outputting a screen (ie, text) or sound (ie, speech) based on a signal input from a user.

A symbol is a unit of intention that a user enters into a complementary alternative communication device. That is, the symbol is a component of a doctor who wants the user to express to the other party using a complementary alternative communication device. Such symbols may include Hangul consonants, Hangul vowels, English alphabets, numbers, special symbols, words, and sentences, but are not limited thereto.

The key is a component of the keyboard of the complementary alternative communication device. Each key constituting the keyboard of the complementary alternative communication device may correspond to each symbol supported by the complementary alternative communication device. A keyboard according to an embodiment of the present invention is not composed of a physical button key, but a virtual keyboard consisting of a virtual key that outputs a symbol through a screen and selects a symbol based on an input signal. ).

A text box is a graphical user interface (GUI) object through which a user can enter or modify symbols. The text box may be implemented using an application programming interface (API) or library provided by an operating system (OS), but is not limited thereto.

Hereinafter, features of the complementary alternative communication method proposed by the present invention will be described in comparison with the conventional complementary alternative communication method.

1 and 2 show some examples of conventional complementary alternative communication methods It is an example .

In general, complementary alternative communication methods may be implemented using fixed computing devices or mobile computing devices. Here, the fixed computing device may include a desktop, a workstation, or a server, but is not limited thereto. In addition, mobile computing devices include smart phones, laptops, tablets, phablets, portable multimedia players (PMPs), and personal digital assistants (PDAs). ) Or an e-book reader, but is not limited thereto.

In addition, the conventional complementary alternative communication method supports either a symbol in the unit of “sentence” or a symbol in the unit of “Korean consonants, Korean vowels, English letters, numbers, or special symbols”.

1 illustrates an example of a conventional complementary alternative communication method implemented using a fixed computing device and supporting sentence-based symbols.

According to the conventional complementary alternative communication method as illustrated in FIG. 1, when symbols in sentence units classified as groups are output on the screen 10, the user preferentially taps according to the group to which the symbol to be input belongs. After selecting (tap, 11), it was possible to select the key 13 of the symbol to be input from among the symbols belonging to the group. Here, the tab 11 is a GUI object for selecting a group.

On the other hand, as services such as web searching, social network services (SNS), or messengers become popular, people with language disabilities also have a need to express their intentions freely. It is getting stronger.

However, using the conventional complementary and alternative communication method illustrated in FIG. 1, the user can input only a predetermined symbol in a sentence unit, and thus cannot freely express his or her intention.

2 illustrates an example of a conventional complementary alternative communication method that is implemented using a mobile computing device and supports symbols in Hangul consonants, Hangul vowels, English characters, numbers, or special symbol units.

According to the conventional complementary alternative communication method as shown in FIG. 2, when all the user-selectable symbols are displayed on the screen 20, the user searches for a symbol he or she wants to input among a number of displayed symbols After that, the key 21 of the searched symbol could be selected.

Therefore, using the conventional complementary alternative communication method illustrated in FIG. 2, the user had a difficulty in searching for a symbol to be input among the many symbols output on the screen 20.

If the user has an intellectual disability in addition to the language disability, the location of the symbols cannot be memorized, so that whenever a single symbol is selected, the symbol to be entered is searched again among the numerous symbols displayed on the screen 20 Had to.

In the case of a user who indirectly selects a symbol to be input from among symbols that are scanned using a switch or the like, the number of symbols to be scanned is too large, and the time for focus to move to the symbol to be input is long. Had to wait.

In addition, in the case of a user who is unfamiliar with using a switch or the like, there is a problem in that other symbols are unintentionally entered (that is, typos). Particularly, in the case of a user using an auxiliary means such as an ocular eyegaze monitor, if a user searches for a sign that he or she wants to input while reading many signs displayed on the screen 20 with eyes, he/she looks to read the sign. The sign was chosen unintentionally at the moment of this stay.

In addition, the unintentionally entered symbol is immediately output in the form of a screen or a sound, and there is a problem in that the communication counterpart exposes that the user is having difficulty in communication.

The present invention has been derived to solve the problems of the conventional complementary alternative communication method as described above. In particular, the complementary alternative communication method according to an embodiment of the present invention was derived from the viewpoint of inputting a user's expression of expression in an application providing a specific service to a user, rather than communicating with a counterpart in an adjacent space.

Figure 3 is one of the present invention Example Examples of complementary alternative communication methods It is an example .

As shown in FIG. 3, according to the complementary alternative communication method according to an embodiment of the present invention, when the user-customized keyboard 30 is output, the user selects the key 31 of the keyboard 30 to select a text box If the word or sentence to be entered in (33) is completed, the input object of the application (40) that the user wants to use the completed word or sentence in the text box (33) by entering the input completion (35) (41).

In order to solve the problems of the conventional complementary alternative communication methods as described above, the complementary alternative communication method according to an embodiment of the present invention is the type of the clasp 31 constituting the keyboard 30 by the user or the guardian of the user And positions can be freely arranged. In addition, the symbols of the key 31 constituting the keyboard 30 are not limited to Hangul consonants, Hangul vowels, English alphabets, numbers, and special symbols, and words frequently used by users and sentences frequently used by users may also be applicable. have.

Particularly, in the complementary alternative communication method according to an embodiment of the present invention, after the user freely corrects (37) the word or sentence to be entered in the text box 33, the completed word or sentence is integrated. It can be delivered to the input object 41 of the application 40. Therefore, even if the application 40 that the user wants to use is a counterpart such as a social network service (SNS) or a messenger, it is difficult for the counterpart to notice that the user is having difficulty communicating.

Hereinafter, a configuration of a system for performing a complementary and alternative communication method of the present invention having the characteristics as described above will be described.

Figure 4 is one of the present invention Example This is a schematic diagram of the complementary alternative communication system.

As illustrated in FIG. 4, a complementary alternative communication system according to an embodiment of the present invention may include an auxiliary input device 100, a communication device 200, and a communication support server 300. . Since the components of the complementary alternative communication system are only functionally divided elements, any one or more components can be implemented by being integrated with each other in an actual physical environment.

Describe each component of the complementary alternative communication system.

The auxiliary input device 100 is a device that is connected to the communication device 200 for complementary alternative communication, and is used to directly or indirectly select a symbol to be input by a user.

Although the auxiliary input device 100 is shown in FIG. 4 in the form of a switch used for scanning, the auxiliary input device 100 includes a touch pad, a light beam pointer, joysticks, and the like. It may be any one of an ocular eyegaze monitor, but is not limited thereto.

If the auxiliary input device 100 corresponds to any one of a touch pad, an optical pointer, a joystick, and a gaze tracking monitor, the auxiliary input device 100 transmits input coordinates according to the user's manipulation to the communication device 200 Can transmit. In addition, when the auxiliary input device 100 corresponds to a switch, the auxiliary input device 100 may transmit a selection signal according to a user's input to the communication device 200.

In addition, the auxiliary input device 100 may transmit input completion to the communication device 200 according to a user's operation. Here, the input completion is a signal informing that the user has completed a word or sentence to be input.

In the following configuration, the communication device 200 is a device that performs a complementary alternative communication method according to an embodiment of the present invention, based on a signal received from the auxiliary input device 100.

The specific configuration and operation of the communication device 200 will be described in more detail with reference to FIGS. 5 to 8 later.

The auxiliary input device 100 and the communication device 200 may transmit and receive data using a directly connected cable or short-range wireless communication. Here, the short-range wireless communication may include, but is not limited to, Bluetooth (Bluetooth), Bluetooth Low Energy (BLE), Radio Frequency Identification (RFID), Near Field Communication (NFC), or ZigBee. .

In addition, according to another embodiment of the present invention, the auxiliary input device 100 and the communication device 200 may be implemented as one device.

With the following configuration, the communication support server 300 is a device that enables the communication device 200 to smoothly perform a complementary alternative communication method according to an embodiment of the present invention.

Specifically, the communication support server 300 may receive a user's account and type of application from the communication device 200. The communication support server 300 may determine one keyboard layout according to the type of the received application among a plurality of keyboard layouts registered in response to the user's account. Then, the communication support server 300 may transmit the determined arrangement of the keyboard to the communication device 200.

The communication support server 300 may receive a character string from the communication device 200. The communication device 200 may perform a spell check on the received character string based on a spelling dictionary. In addition, the communication support server 300 may transmit the spelling test result to the communication device 200.

The communication device 200 and the communication support server 300 may transmit and receive data using either a public wired communication network or a mobile communication network. Here, the common wired communication network may include ethernet, digital subscriber line (xDSL), hybrid fiber coax (HFC), and fiber to the home (FTTH). , But is not limited thereto. In addition, mobile communication networks include code division multiple access (CDMA), wide band code division multiple access (CDMA), high speed packet access (HSPA), and long term evolution (Long Term Evolution). LTE) may be included, but is not limited thereto.

Hereinafter, the configuration of the communication device 200 as described above will be described in more detail.

Figure 5 is one of the present invention Example It is a logical diagram of the communication device.

As illustrated in FIG. 5, the communication device 200 according to an embodiment of the present invention includes a communication unit 205, an input/output unit 210, a keyboard layout management unit 215, an AAC control unit 220, and an additional function management unit 225 and an input completion processing unit 230. Since the components of the communication device 200 are only functionally divided elements, any one or more components may be implemented by being integrated with each other in an actual physical environment.

Each component of the communication device 200 will be described.

The communication unit 205 may transmit and receive data to and from the auxiliary input device 100 or the communication support server 300.

Specifically, the communication unit 205 may receive an input coordinate or a selection signal from the auxiliary input device 100. Here, the input coordinates may be received when the auxiliary input device 100 corresponds to any one of a touch pad, an optical pointer, a joystick, and a gaze tracking monitor. The selection signal may be received when the auxiliary input device 100 is a switch.

The communication unit 205 may transmit the user's account, application type, and character string to the communication support server 300. Then, the communication unit 250 may receive a keyboard layout and a spelling test result from the communication support server 300.

In the following configuration, the input/output unit 210 uses input/output devices that do not correspond to the auxiliary input device 100 of the communication device 200, and data necessary for the operation of the communication device 200 with the user or the guardian of the user Can input and output.

Specifically, the input/output unit 210 may receive a user's account from a user or guardian. In addition, the input/output unit 210 may receive arrangement information of a key for configuring a custom keyboard layout from a user or guardian. Here, the arrangement information of the lock is information regarding the type and location of one or more locks constituting the keyboard.

The input/output unit 210 may output a screen of an application in which a complementary alternative communication method according to an embodiment of the present invention is implemented. The input/output unit 210 may output a screen of an application to be used by the user. The input/output unit 210 may output a symbol input by a user through a screen or sound. In addition, the input/output unit 210 may output a completed word or sentence to the user through a screen or sound.

With the following configuration, the keyboard layout management unit 215 may manage a user-specified keyboard layout. The keyboard layout management unit 215 may register a new keyboard layout or load an existing registered keyboard layout.

First, the process of registering the new keyboard layout by the keyboard layout management unit 215 will be described.

The keyboard layout management unit 215 may identify the user's account input through the input/output unit 210. The keyboard layout management unit 215 may generate a new keyboard layout of the user based on the arrangement information of the key input through the input/output unit 210.

The symbol according to an embodiment of the present invention may be any one of a Korean consonant, a Korean vowel, an alphabetic alphabet, a number, a special symbol, a word frequently used by a user, and a sentence frequently used by a user. Accordingly, each key constituting the keyboard layout may correspond to any one of a sentence consisting of a Korean consonant, a Korean vowel, an alphabetic alphabet, numbers and special symbols, words of one or more letters, and a plurality of words.

When the user inputs a symbol using the new keyboard layout, the keyboard layout management unit 215 may identify the type of the application that the user is running at the same time.

The keyboard layout management unit 215 may correspond to the user's account and application type, and store the new keyboard layout in an internal database. In addition, the keyboard layout management unit 215 may transmit information corresponding to the type of the user's account and application to the new keyboard layout to the communication support server 300.

Next, the keyboard layout management unit 215 describes a process of loading an existing registered keyboard layout.

The keyboard layout management unit 215 may identify the user's account information input through the input/output unit 210. The keyboard layout management unit 215 may identify the type of the input target application.

Here, the input target application may be set in advance by a user or guardian. Alternatively, the input target application may be determined according to a priority of a process, task, or thread managed by an operating system (OS). The target application may be any one of a web browser, a social network service (SNS), or a messenger, but is not limited thereto.

The keyboard layout management unit 215 may determine a keyboard layout according to a type of an input target application among a plurality of keyboard arrangements registered in an internal database corresponding to the identified user account. Then, the keyboard layout management unit 125 may recall the determined one keyboard layout.

Alternatively, the keyboard layout management unit 215 may transmit the identified user's account and the type of the input target application to the communication support server 300. Also, the keyboard layout management unit 215 may receive the keyboard layout from the communication support server 300.

With the following configuration, the AAC control unit 220 may control complementary alternative communication through the auxiliary input device 100.

Specifically, the AAC control unit 220 may configure and output a virtual key according to the keyboard layout registered or loaded by the keyboard layout management unit 215. The AAC control unit 220 may select any one of the keys constituting the keyboard layout, based on the information received from the auxiliary input device 100. The AAC control unit 220 may output a symbol corresponding to the selected key in the text box. Then, the AAC control unit 220 may allow modification of the symbol output in the text box until input completion is received from the auxiliary input device 100.

If the auxiliary input device 100 is any one of a touch pad, an optical pointer, a joystick, and a line-of-sight monitor, the AAC control unit 220 is selected from the auxiliary input devices 100 among the keys constituting the keyboard layout. It is possible to select a key positioned to correspond to the received input coordinates.

If the auxiliary input device 100 is a switch, the AAC control unit 220 sequentially selects a focus signal from the auxiliary input device 100 while sequentially moving focus with respect to the keys arranged according to the keyboard arrangement. You can select the key where the focus is placed at the time of receipt. Here, the focus is an indicator indicating a symbol that can be selected by a user among a plurality of symbols in the scanning process.

The AAC control unit 220 may change and output the color of the next user-selectable key according to the type of the selected key. For example, when the selected key corresponds to a Korean vowel, the AAC control unit 220 may change the color of the key corresponding to the Hangul consonant among the outputted keys according to the keyboard layout and output the text. Alternatively, when the selected key corresponds to a Hangul consonant, the AAC control unit 220 changes the color of the key corresponding to the Korean vowel, period, question mark, and exclamation mark among the outputted fonts according to the keyboard layout. You may.

In addition, the AAC control unit 220 may store symbols (ie, words or sentences being completed by the user) output in the text box. Specifically, when the key is selected based on the information received from the auxiliary input device 100, the AAC control unit 220 may enqueue the selected symbol into the queue. Here, the queue is a data structure having first-in and first-out characteristics.

With the following configuration, the additional function management unit 225 may manage additional functions that may be provided in a complementary and alternative communication process.

The add-on management unit 225 may perform a spell check. Specifically, the add-on management unit 225 may identify a misspelled symbol in a string composed of symbols being output to a text box based on a built-in spelling dictionary. In addition, the additional function management unit 225 may change the color of a misspelled symbol among symbols being displayed in the text box and output the changed color.

Alternatively, the add-on management unit 225 may transmit a string composed of symbols being displayed in the text box to the communication support server 300. The add-on management unit 225 may receive a spelling test result from the communication support server 300. In addition, the additional function management unit 225 may change the color of a misspelled symbol among symbols being displayed in the text box and output the result based on the received spell check result.

The add-on management unit 225 may perform text to speech (TTS). Specifically, the additional function management unit 225 may output a sound of a symbol corresponding to a key selected by the AAC control unit 220 using the built-in TTS library. Alternatively, the supplementary function management unit 225 may output a word or sentence composed of symbols output in a text box as a sound when input completion is received from the auxiliary input device 100 using the TTS library.

With the following configuration, when input completion is received from the auxiliary input device 100, the input completion processing unit 230 may deliver a symbol (ie, a completed word or sentence) output to a text box to an input target application.

Specifically, the input completion processing unit 230 may determine whether input completion is received from the auxiliary input device 100. When the input completion processing unit 230 receives the input completion, the AAC control unit 220 may identify the queue in which the symbol is inserted. The input completion processing unit 230 may dequeue the symbols inserted in the queue one by one until the identified queue is empty, and insert them into the message input queue of the input target application. Here, the message input queue is a queue allocated to deliver a message on the generated event to an input target application when an event related to input occurs in an operating system (OS) operating in a message-driven manner.

For example, when the operating system (OS) corresponds to Windows of Microsoft, the input completion processing unit 230 may insert a key input message (WM_KEYDOWN) into the message input queue of the input target application. . In this case, the key input message has a virtual key code corresponding to the symbol extracted from the queue as a word parameter (wParam), and information about the number of repetitions of the input symbol, the status before the input of the symbol, etc. Can have a long parameter (lParam).

Hereinafter, hardware for implementing a logical configuration of the communication device 200 as described above will be described in more detail.

Figure 6 is one of the present invention Example It is a hardware configuration diagram of the communication device.

6, the communication device 200 includes a processor (Processor, 250), a memory (Memory, 255), a transceiver (Transceiver, 260), an input/output device (265) and a data bus ( Bus, 270) and storage (Storage, 275).

The processor 250 may implement operations and functions related to the complementary alternative communication method based on the instructions according to the software 280a in which the supplemental alternative communication (AAC) method resident in the memory 255 is implemented. In the memory 255, a software 280a in which a complementary alternative communication method is implemented may be loaded. The transceiver 260 may transmit and receive data to and from the auxiliary input device 100 or the communication support server 300. The input/output device 265 may receive data necessary for performing the complementary alternative communication method, and output the input symbols. The data bus 270 is connected to the processor 250, the memory 255, the transceiver 260, the input/output device 265, and the storage 275, so that each component communicates with each other to transfer data to each other. Can play a role.

The storage 275 may store an application programming interface (API), a library file, and a resource file required for the execution of the software 280a in which the complementary alternative communication method is implemented. The storage 275 may store the software 280b in which the complementary alternative communication method is implemented. Further, the storage 275 may store a database 285 necessary for performing the insurance recommendation method. Here, the registered keyboard layout may be stored in the database 285, but is not limited thereto.

The software 280a, 280b for implementing a complementary alternative communication method resident in the memory 255 or stored in the storage 275 includes loading a keyboard layout registered in advance corresponding to a user's account; Configuring and outputting virtual keys according to the loaded keyboard layout, and selecting any one of the keys based on information received from the connected auxiliary input device 100; Outputting a symbol corresponding to the selected key to a text box; And allowing modification of the symbol output to the text box until input completion is received from the auxiliary input device 100, and when the input completion is received, passing the symbol output to the text box to a pre-designated application. In order to execute, it can be a computer program recorded on a recording medium.

More specifically, the processor 250 may include an application-specific integrated circuit (ASIC), other chipsets, logic circuits, and/or data processing devices. The memory 255 may include read-only memory (ROM), random access memory (RAM), flash memory, a memory card, a storage medium, and/or other storage devices. The transceiver 260 may include a baseband circuit for processing wired and wireless signals. The input/output device 265 includes an input device such as a keyboard, a mouse, and/or a joystick, and a liquid crystal display (LCD), a light emitting diode (LED), and an organic device. It may include a printing device such as a printer, a plotter, and the like, an image output device such as an organic LED (OLED) and/or an active matrix OLED (AMOLED).

When the embodiment included in the present specification is implemented in software, the above-described method may be implemented as a module (process, function, etc.) performing the above-described function. The module resides in the memory 255 and can be executed by the processor 250. The memory 255 may be internal or external to the processor 250, and may be connected to the processor 250 by various well-known means.

Each component illustrated in FIG. 6 may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. For implementation by hardware, one embodiment of the present invention includes one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), FPGAs ( Field Programmable Gate Arrays), processors, controllers, microcontrollers, microprocessors, and the like.

In addition, in the case of implementation by firmware or software, an embodiment of the present invention is implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above, and is recorded in a record carrier readable through various computer means. Can be recorded. Here, the recording medium may include program instructions, data files, data structures, or the like, alone or in combination. The program instructions recorded on the recording medium may be specially designed and configured for the present invention, or may be known and available to those skilled in computer software. For example, recording media include magnetic media such as hard disks, floppy disks and magnetic tapes (Magnetic Media), compact disk read only memory (CD-ROM), optical media such as DVD (Digital Video Disk), and optical media. It includes a magneto-optical media such as a disk (Floptical Disk), and a hardware device specially configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions may include high-level language code that can be executed by a computer using an interpreter, etc., as well as machine language codes made by a compiler. Such a hardware device can be configured to operate as one or more software to perform the operation of the present invention, and vice versa.

Hereinafter, the complementary alternative communication method as described above will be described in detail.

Figure 7 is one of the present invention Example Accordingly, the communication device represents the process of performing complementary alternative communication methods. Is a flow chart .

Referring to FIG. 7, the communication device 200 may load an existing registered keyboard layout (S100). Specifically, the communication device 200 may determine one keyboard layout from among a plurality of keyboard arrangements registered in the database in response to a user's account. Then, the communication device 200 may load the determined keyboard layout.

The communication device 200 may configure and output a virtual key according to the keyboard of the loaded array (S200).

The communication device 200 may select any one of the pieces constituting the keyboard layout based on the information received from the auxiliary input device 100 (S300). Specifically, when the auxiliary input device 100 is any one of a touch pad, an optical pointer, a joystick, and a line-of-sight monitor, the communication device 200 includes, among the brackets constituting the keyboard layout, the auxiliary input device 100 ) May select a key positioned corresponding to the input coordinate received from. Alternatively, when the auxiliary input device 100 is a switch, the communication device 200 selects from the auxiliary input device 100 while sequentially moving focus with respect to the keys arranged according to the arrangement of the keyboard You can select the key on which the focus is placed at the time the signal is received.

The communication device 200 may output a symbol corresponding to the selected key in the text box. Then, the communication device 200 may store the symbol (ie, the word or sentence being completed by the user) output in the text box (S400).

The communication device 200 may determine whether input completion is received from the auxiliary input device 100 in operation S500. As a result of the determination, if input completion has not yet been received from the auxiliary input device 100, the communication device 100 may allow modification of the symbol output in the text box.

As a result of the determination, when input completion is received from the auxiliary input device 100, a symbol (ie, a completed word or sentence) output in a text box may be transmitted to an input target application (S600 ). The process of passing the symbol to the input target application will be described with reference to FIG. 8.

Figure 8 is one of the present invention Example Follows the process of passing the symbol to the input target application Is a flow chart .

Referring to FIG. 8, the communication device 200 may determine whether a queue in which a symbol is inserted is empty (S610). As a result of the determination, if the queue is empty, the communication device 200 ends without performing the following steps.

As a result of the determination, if the queue is not empty, the communication device 200 may extract symbols one by one from the queue (S620). The communication device 200 may insert the extracted symbol into the message input queue of the input target application (S630).

After inserting the extracted symbol into the message input queue of the input target application, the communication device 200 may determine whether the queue is empty (S610).

As described above, in the present specification and drawings, preferred embodiments of the present invention have been disclosed, but it is possible in the technical field to which the present invention pertains that other modified examples based on the technical spirit of the present invention may be implemented in addition to the embodiments disclosed herein. It is obvious to those with ordinary knowledge. In addition, although specific terms are used in the present specification and drawings, they are merely used in a general sense to easily describe the technical contents of the present invention and to understand the present invention, and are not intended to limit the scope of the present invention. Therefore, the above detailed description should not be construed as limiting in all respects and should be considered illustrative. The scope of the invention should be selected by rational interpretation of the appended claims, and all changes within the equivalent scope of the invention are included in the scope of the invention.

In addition, the device or terminal according to the present invention can be driven by instructions that cause one or more processors to perform the functions and processes described above. For example, such instructions may include interpreted instructions such as script instructions such as JavaScript or ECMAScript instructions, executable code, or other instructions stored on a computer-readable medium. Furthermore, the device according to the present invention may be implemented in a distributed manner over a network, such as a server farm, or may be implemented in a single computer device.

In addition, a computer program (also known as a program, software, software application, script or code) mounted on the device according to the present invention and executing the method according to the present invention includes a compiled or interpreted language or a priori or procedural language. It can be written in any form of a programming language, and can be deployed in any form, including standalone programs or modules, components, subroutines, or other units suitable for use in a computer environment. Computer programs do not necessarily correspond to files in the file system. The program is in a single file provided to the requested program, or in multiple interactive files (e.g., one or more modules, files storing subprograms or parts of code), or part of a file holding other programs or data (Eg, one or more scripts stored in a markup language document). The computer program can be deployed to be executed on one computer or on multiple computers that are located at one site or distributed across multiple sites and interconnected by a communication network.

In addition, in describing the embodiment according to the present invention, operations are depicted in the drawings in a specific order, but it is necessary to perform such operations in the specific order or sequential order shown in order to obtain a desired result or all illustrated actions They should not be understood as to be performed. In certain cases, multitasking and parallel processing may be advantageous. Also, the separation of various system components of the above-described embodiments should not be understood as requiring such separation in all embodiments, and the described program components and systems are generally integrated together into a single software product or packaged in multiple software products. You should understand that you can.

Auxiliary input device: 100 Communication device: 200
Communication Support Server: 300
Communication unit: 205 Input/output unit: 210
Keyboard layout management: 215 AAC control: 220
Additional function management unit: 225 Input complete processing unit: 230

Claims (10)

Loading a keyboard layout registered in advance corresponding to the user's account;
Based on the information received from the auxiliary input device connected for augmentative and alternative communication, the virtual keys are configured and output according to the loaded keyboard layout, and any one of the keys is selected. Selecting;
Outputting a symbol corresponding to the selected key to a text box; And
Allowing the modification of the symbol output to the text box until input completion is received from the auxiliary input device, and when the input completion is received, passing the symbol output to the text box to a predetermined application. Complementary alternative methods of communication, including. The method of claim 1, wherein the application
It is determined according to the priority of a process, task, or thread managed by an operating system,
The step of loading the keyboard layout is
Compensation alternative communication method, characterized in that for determining the one of the plurality of keyboard layouts registered in response to the user's account, determined according to the type of the application. The method of claim 1, wherein the selecting step
When the auxiliary input device is one of a touch pad, a light beam pointer, joysticks, and an ocular eyegaze monitor, it corresponds to input coordinates received from the auxiliary input device Select a key that is located very well, or
When the auxiliary input device is a switch, focus is positioned at a time when a selection signal is received from the auxiliary input device while sequentially moving focus with respect to the keys arranged according to the arrangement of the keyboard. Complementary alternative communication method, characterized by selecting a key. According to claim 1, Each key constituting the arrangement of the keyboard
Hangul consonants, Hangeul gatherings, one or more words, and a sentence consisting of multiple words.
The step of outputting
When the selected key corresponds to the Korean vowel, a complementary alternative communication method, characterized in that the color of the key corresponding to the Hangul consonant is changed and output among the outputted keys according to the keyboard layout. The method of claim 1, wherein the step of outputting
Based on a spelling dictionary, a misspelled symbol is identified in a string consisting of symbols being displayed in the text box, and the color of the misspelled symbol among the symbols being displayed in the text box is changed and output. Complementary alternative communication method, characterized in that. The method of claim 1, wherein the step of outputting
Insert a symbol corresponding to the selected key into a queue, and
The step of delivering
Complementary alternative communication method, characterized in that by extracting the symbols inserted into the queue one by one until the queue is empty (empty) and inserting them into the message input queue of the application. The method of claim 6, wherein the step of delivering
Complementary alternative communication method, characterized in that a key input message (WM_KEYDOWN) having a virtual key code corresponding to the symbol extracted from the queue as a word parameter (wParam) is inserted into the message input queue. . The method of claim 1, wherein the step of delivering
When the input completion is received, a complementary alternative communication method characterized by outputting a word or sentence composed of symbols output to the text box as a sound using a text to speech (TTS) library. The method of claim 1, wherein the application
Complementary alternative communication method, characterized in that any one of a web browser (web browser), a social network service (Social Network Service), or a messenger (messenger). An output device;
Memory; And
Combined with a computing device configured to include a processor (processor) for processing instructions resident (loading) in the memory,
Loading, by the processor, a keyboard layout registered in advance corresponding to a user's account;
The processor configures virtual keys according to the loaded keyboard layout and outputs them through the output device, based on information received from an auxiliary input device connected for augmentative and alternative communication. , Selecting any one of the brackets;
The processor outputting a symbol corresponding to the selected key to a text box; And
The processor permits modification of the symbol output to the text box until input completion is received from the auxiliary input device, and when the input completion is received, the symbol output to the text box is delivered to a predetermined application. A computer program recorded on a recording medium in order to execute the steps.

Images