KR20120048966A - Device and methodf for providing virtual keyboard - Google Patents

Abstract

PURPOSE: A virtual keyboard supplying apparatus and a method thereof are provided to intuitively recognize an input language by displaying the input language in an application program on an input key of a virtual keyboard. CONSTITUTION: A controlling unit(130) monitors a keyboard state in an active application program and obtains the keyboard state. The controlling unit obtains an active language value in the keyboard state and extracts a primary and sub scheme. The controlling unit compares the primary and sub scheme with a primary and sub scheme of a virtual keyboard. An image processing unit(160) outputs a layout of the virtual keyboard based on a comparison result.

Description

DEVICE AND METHODF FOR PROVIDING VIRTUAL KEYBOARD}

The present invention relates to a method of providing a language of a corresponding window on a user interface to match a language of a window activated in a virtual keyboard. Specifically, when a user attempts to input in a currently activated window, By providing the appropriate language on the user interface to suit your needs, you can intuitively see the status of the active language in the current window.

A virtual keyboard, or on-screen keyboard, is a virtual extension designed to secure the shortcomings of devices such as computers, computing devices, PDAs, smartphones, mobile phones, laptops, etc. that have fewer input interfaces than traditional keyboards have. It is a computer program that plays a role. In the world of desktop computers, the virtual keyboard provided in Windows XP or the like may be a virtual machine or emulator that acts as a virtual keyboard moving with a real keyboard or a computer mouse.

 In the conventional window system illustrated in FIG. 1A, the language that can be input in the currently activated window 30 can be known through the IME window 20 on the lower right side. However, when using the virtual keyboard 10, it was difficult to intuitively grasp what language is input in the currently activated window. As shown in FIG. 1A, the language available on the window 20 is Korean, but the virtual keyboard 10 does not reflect this at all.

In addition, even if the IM recognizes the language that can be input in the currently activated window through the window 20, if the inputable language is Japanese as shown in FIG. 1B, the virtual keyboard is not displayed to the user who is not familiar with the keyboard input state of the Japanese mode. (10) can be meaningless. In the case of the virtual keyboard according to the related art, the language of the user interface in which the user writes on the keyboard other than Korean or English, for example, Japanese or Russian, is English single, or the keyboard of the computer of the country is written only in a single language. There was a problem that was difficult to actually input.

The present invention provides a method for displaying on the input key of the virtual keyboard by reflecting a language that can be input in the activated application program to solve the above problems.

A method for providing a virtual keyboard based on an activated application program in a display device according to the present invention includes: monitoring a keyboard state of the activated application program; Obtaining the keyboard state; Obtaining an activated language value in the keyboard state; Extracting a primary language and a sub language from the activated language value; Comparing the extracted primary language and the sub language with the primary method or sub method output from the virtual keyboard; And outputting a layout of the virtual keyboard based on the comparison result.

A display device that provides a virtual keyboard based on an activated application in a display device according to the present invention, monitors the keyboard state in the activated application, obtains the keyboard state, and activates the language in the keyboard state. A control unit for obtaining a value, extracting a primary method and a sub method from the activated language value, and comparing the extracted primary method and the sub method with a primary method and a sub method of a virtual keyboard, and based on the comparison result It includes an image processor for outputting the layout of the virtual keyboard.

A computer-readable storage medium including command code for providing a virtual keyboard based on an application program activated in a display device according to the present invention, wherein the command code is controlled by the controller of the display device. Monitor a keyboard state, obtain the keyboard state, obtain an activated language value in the keyboard state, extract a primary way and a sub way of the activated language value, and extract the primary way and the sub way Is compared with a primary method and a sub method of the virtual keyboard, and includes a code for outputting the layout of the virtual keyboard based on the comparison result.

According to the present invention, the language that can be input in the activated application program is reflected and displayed on the input key of the virtual keyboard so that the user can intuitively grasp the language that can be input.

Further, according to the present invention, the language that can be input and the keyboard layout corresponding thereto according to the position and operation state of the input interface in the activated application program are displayed on the input keys of the virtual keyboard to reconsider convenience in document work.

1 shows a conventional virtual keyboard.
2A, 2B, 2C, and 2D illustrate a virtual keyboard according to one embodiment of the invention.
3 illustrates a virtual keyboard according to an embodiment of the present invention.
4 illustrates a virtual keyboard according to an embodiment of the present invention.
5 illustrates a virtual keyboard according to another embodiment of the present invention.
6 illustrates a virtual keyboard according to an embodiment of the present invention.
FIG. 7 is a block diagram illustrating a configuration of a display apparatus for providing a virtual keyboard according to an embodiment of the present invention.
8 is a flowchart of a method of providing a virtual keyboard according to an embodiment of the present invention.

Hereinafter, with reference to the drawings will be described the present invention in more detail.

The suffix "module" and " part "for components used in the following description are given merely for ease of description, and the" module "and" part "

On the other hand, the display device described herein may be, for example, a device or a set-top box having an intelligent display with the addition of computer support. In addition, the display device is faithful to the broadcast reception function, but the Internet function is added, it is possible to have a more convenient interface, such as a handwritten input device, a touch screen or a space remote control. In addition, by being connected to the Internet and a computer with the support of a wired or wireless Internet function, it is possible to perform functions such as e-mail, web browsing, banking or gaming. A standardized general-purpose OS can be used for these various functions.

Therefore, in the display device described in the present invention, various applications can be freely added or deleted, for example, on a general-purpose OS kernel, so that various user-friendly functions can be performed. More specifically, the display device may be, for example, a PC, a laptop, a netbook, a network TV, an HBBTV, a smart TV, or the like, and may be applicable to a smartphone in some cases.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG.

As used herein, terms used in the present invention are selected from general terms that are widely used in the present invention while taking into account the functions of the present invention, but these may vary depending on the intention or custom of a person skilled in the art or the emergence of new technologies. In addition, in certain cases, there is a term arbitrarily selected by the applicant, and in this case, the meaning will be described in the corresponding description of the invention. Therefore, it is intended that the terminology used herein should be interpreted based on the meaning of the term rather than on the name of the term, and on the entire contents of the specification.

2A illustrates a virtual keyboard according to an embodiment of the invention. Referring to FIG. 2A, the virtual keyboard 100 is displayed at the bottom of the screen of the display device according to the present invention. The window 31 is activated on the screen displayed by the display apparatus 100. In the activated window 31, English can be input. The task bar 20 displays English as the language 21 that can be input in the activated window 31. "K0" and "A" are marked in the input language field 21. That is, the keyboard state of the display device shown in FIG. 2A is "KO", indicating a Korean keyboard state. For the Korean keyboard status, language values other than Korean can be entered. For example, in the Windows IME system, language values of Korean and English can be input in a Korean keyboard state. In FIG. 2A, a language value of English is inputtable and is indicated in an inputtable language field 21. That is, in the operating system of the display device illustrated in FIG. 2A, the keyboard state and the inputtable language value of the activation window 31 are displayed on the part of the task bar 20 so that the user can grasp the inputtable language. .

Accordingly, in the virtual keyboard system according to the present invention, the keyboard state displayed on the task bar 20 may be periodically monitored and the keyboard state may be obtained. After determining the keyboard state of the Korean input mode, a language value that can be input in the currently activated window 31, that is, the activated application program, is obtained. The inputtable language value may be an activated language value. In FIG. 2A, English is obtained as an activated language value. The inputtable language value may be an activated language value. In FIG. 2A, it is detected that 'KO' is designated, the Korean language is acquired as an activated language value, and a Qwerty type keyboard layout is displayed on the screen using a Korean keyboard as a virtual keyboard.

The activated language value may include a primary method and a sub method.

2B to 2D, the case of another language will be described.

FIG. 2B shows an arrangement of a French French keyboard, FIG. 2C shows a Belgian French keyboard, and FIG. 2D shows an arrangement of a Swiss French keyboard.

For Microsoft's computer OS system, the input language can be one of six languages: French (Luxembourg), French (Monaco), French (Belgium), French (Switzerland), French (Canada), and French (France). have. Where French is the primary language and the country in parentheses is the sub language.

같은 심볼의 위치는 프랑스의 자판과 위치가 상이하다. For example, when the user selects French (Belgium) as the input language, the input language 21 is displayed as "FR". However, French keyboards in France and Belgium have slightly different layout positions for input keys. The arrangement of several input keys is different in Belgian French keyboard keyboard. I.e.!, @,-, _, +, =, And

The location of the same symbol is different from that of the French keyboard.

Also, it can be seen that the arrangement of the Swiss keyboard shown in FIG. 2D is different from that of the French keyboard of FIG. 2B.

That is, even though the activated language values are the same, since the keyboard layout for the plurality of key input values provided on the virtual keyboard may vary, the language values may include a keyboard layout method for the plurality of key input values. Therefore, the virtual keyboard system according to the present invention needs to adjust the layout of the virtual keyboard by extracting a suitable method according to the keyboard layout method.

In FIG. 2A, the virtual keyboard of the English language QWERTY system of the Hangul keyboard outputs "z" as the input key 51 located at the lower left of the character input keys.

In addition to the input key 51, the virtual keyboard system according to the present invention may change and output all of a plurality of different key input values provided by the virtual keyboard according to the changed language value.

Hereinafter, a method of displaying a virtual keyboard in a case where there is a switching or other change of an activated application program will be described.

3 illustrates a virtual keyboard according to an embodiment of the present invention. Referring to FIG. 3, the activated application program is switched from the window 31 to the window 32. In the input language column 21 of the task bar 20, a Korean language value is indicated as the input language. That is, it can be seen that the Korean language can be input in the activated application program window 32. Accordingly, the virtual keyboard system according to the present invention can obtain a Korean value as the activated language value. In addition, the virtual keyboard system can extract the 2 beol and 3 beol methods as the primary and sub methods as Korean language values. The virtual keyboard system may compare the extracted value with the primary and sub-language languages output from the virtual keyboard. If the extracted primary method or sub method is different from the primary method and sub method output from the virtual keyboard, the layout data of the virtual keyboard may be changed. In addition, the primary method value and the sub method value of the virtual keyboard 50 may be stored as an extracted value and updated. As shown in FIG. 3, the virtual keyboard 50 according to the present invention layouts and outputs a Korean-language type of a second type that is a modified primary method.

That is, in the virtual keyboard system according to the present invention, the keyboard state information can be obtained by monitoring the keyboard state information provided by the operating system of the display apparatus, for example, the input language field 21 of the task bar for a predetermined period, for example, 200 ms. Can be. In addition, the layout of the virtual keyboard can be updated by comparing the acquired keyboard state and language value with the currently displayed virtual keyboard.

Meanwhile, in the virtual keyboard system according to the present invention, in addition to the method for monitoring the keyboard state information provided by the operating system of the display device, the keyboard state may be obtained when the activated application program is changed. That is, when the user switches the activated application program from the window 31 to the window 32, the operating system of the display device detects this and may call a thread in the corresponding switching window 32. The virtual keyboard system according to the invention from the called thread may obtain the keyboard state.

4 illustrates a virtual keyboard according to an embodiment of the present invention. Referring to FIG. 4, an activated window 33 is displayed on the screen of the display device according to the present invention. The window 33 shows the Japanese consulate homepage and is a language in which Japanese can be input. In the input language field 21 of the task bar, "JP" is displayed as an activated language value. The virtual keyboard system according to the present invention can obtain the Japanese language value by monitoring the input language field 21 of the task bar. The keyboard layout method can be extracted from the corresponding language value, and the layout of the virtual keyboard can be changed since the keyboard layout is different from that previously output on the virtual keyboard. It can be seen that the input key 51 of the virtual keyboard 50 is changed and output. In addition, the keyboard state and language value, primary and sub way of the displayed virtual keyboard can be stored as new values.

5A illustrates a virtual keyboard according to another embodiment of the present invention. Referring to FIG. 5A, a virtual keyboard 50 according to a Hangul language value is provided, and the user changes the corresponding language value to, for example, English by placing the pointer 22 in the input language field and selecting "ga". have. That is, the language value activated by the user's selection is changed without changing the activated application program. The operating system of the display apparatus may detect such an operation and receive and process a thread regarding a request for changing an activated language value. The virtual keyboard system according to the present invention may detect the thread and acquire at least one of a keyboard state or an activated language value to be changed through a request for language value change.

5B illustrates that the layout is changed and output on the virtual keyboard after a request for changing the activated language value by the user's selection in FIG. 5A is performed. The layout of the virtual keyboard 50 is output according to the requested language value 'English'.

As described with reference to FIGS. 5A and 5B, even when there is a change in the activation language value by the user's selection without changing or switching the activated application program, the virtual keyboard system according to the present invention senses the value of the virtual keyboard. Reflected in layout.

6 illustrates a virtual keyboard according to another embodiment of the present invention. Referring to FIG. 6, an MS word window 34 is illustrated as an activated application program.

The activated application program according to the present invention may include a user interface for inputting letters or numbers, such as a text input device. In this case, the virtual keyboard 50 may obtain the language value based on the position of the pointer on the input menu of the text editor. For example, when the user places the pointer, that is, the cursor at the location 341, Korean characters can be input at the location 341, and thus, the activated input language value at the location 341 may be Korean. The virtual keyboard according to the present invention can detect this and obtain a Hangul language value. Accordingly, the layout of the virtual keyboard may be output in Korean.

The input language of the task bar provided by the display device detects this and displays Korean.

In the same way, when the user places the pointer, i.e., the cursor, at position 342, English can be entered at that position, so the activated input language value at that position 342 can be English. The virtual keyboard according to the present invention can detect this to obtain an English language value, change the layout of the keyboard, and output it.

As illustrated in FIG. 6, in the virtual keyboard system according to the present invention, an activated language value changed according to the position of a pointer, a cursor, and a mouse may be acquired on the same activated application program.

FIG. 7 is a block diagram illustrating a configuration of a display apparatus for providing a virtual keyboard according to an embodiment of the present invention. Referring to FIG. 7, the display apparatus 100 according to the present invention may include a network interface unit 110, an external device interface unit 120, a controller 130, a storage unit 140, a user interface unit 150, and an image. The processor 160 and the audio output unit 170 may be included. In FIG. 7, the input keyboard 190 and the display 180 are separately illustrated, but may be included in the display apparatus 100.

The network interface unit 110 receives the packets received from the network and transmits the packets to the network. That is, the network interface unit 110 may receive a service, content, and the like from a service provider through a network.

The external device interface unit 120 may connect the external device to the display device 100. To this end, the external device interface unit 120 may include an A / V input / output unit (not shown) or a wireless communication unit (not shown).

The controller 130 may generate and output a signal for outputting an image or an audio by demultiplexing an input stream or processing the demultiplexed signals through the external device interface unit 120. In addition, the controller 130 may receive and process a signal from the user interface unit. The controller 130 according to the present invention may generate layout data of the virtual keyboard and transmit the layout data to the image processor 160.

The control unit 130 according to the present invention monitors the keyboard state in the activated application, obtains the keyboard state, obtains the activated language value in the keyboard state, and the primary method from the activated language value. And a sub method, and the extracted primary method and sub method can be compared with the primary method and sub method of the virtual keyboard.

In addition, when the extracted primary method and sub method are different from the existing primary method and sub method of the virtual keyboard, the controller 130 may change the layout data of the previously stored virtual keyboard. The controller 130 may monitor the keyboard state information provided by the operating system of the display device for a predetermined period, for example, 200 ms. In addition, the controller 130 may obtain at least one of the keyboard state and the activated language value when a language value change request is made from the activated application program. In addition, the controller 130 may change a plurality of key input values provided by the virtual keyboard according to the changed language value.

The storage 140 may store each input value corresponding to a plurality of keys provided by the virtual keyboard. In addition, even when the activated language value and the primary or sub system are changed, the corresponding changed value may be stored.

In addition, the storage 140 according to the present invention may store a keyboard state for each application program executable in the display apparatus 100, language values according to each keyboard state, a keyboard layout method, and the like.

The user interface unit 150 transmits a signal input by the user to the controller 130 or transmits a signal from the controller 130 to the user.

For example, the user interface unit 150 may be powered on / off from a remote control device such as a remote controller according to various communication methods, such as a radio frequency (RF) communication method and an infrared (IR) communication method. Control signals such as selection and screen setting may be received and processed, or may be processed to transmit control signals from the controller 170 to the remote controller.

Also, for example, the user interface 150 may transmit a control signal input from a local key (not shown) such as a power key, a channel key, a volume key, and a set value to the controller 130.

In addition, the user interface 150 according to the present invention may receive a request to start or change an activated application program.

In addition, the user interface 150 according to the present invention may receive a request for changing the activated language value. When the display unit 180 is configured as a touch screen, a request for changing a language value may be received by a touch method, and in this case, the display unit 180 may be included in the user interface unit 150.

The image processor 160 converts an image signal, a data signal, an OSD signal processed by the controller 130, or an image signal, data signal, etc. received from the external device interface unit 120 into R, G, and B signals, respectively. A drive signal can be generated.

In addition, the image processor 160 according to the present invention may receive layout data of the virtual keyboard received from the controller 130, and may change and output the image to suit the display unit 180.

The audio output unit 170 receives a signal processed by the controller 130, for example, a stereo signal, a 3.1 channel signal, or a 5.1 channel signal, and outputs a voice signal.

The display unit 180 may display image data received by the image processor 160. The display unit 180 may be a PDP, an LCD, an OLED, a flexible display, a 3D display, or the like.

The display unit 180 may be configured as a touch screen and used as an input device in addition to the output device.

8 is a flowchart of a method of providing a virtual keyboard according to an embodiment of the present invention.

In step S11, the keyboard status in the activated application program is monitored.

In step S12, a keyboard state is obtained.

In step S13, the activated language value in the obtained keyboard state is obtained. The language value in the keyboard state may include a list of inputable languages, and the activated language value may be one of a list of inputable languages.

In step S14, the primary language and the sub language may be extracted from the activated language value.

In step S15, it is determined whether the extracted primary language and sub language are the same or different by comparing the primary methods or sub methods output from the virtual keyboard.

In step S16, when the extracted primary method or sub method is different from the primary method and sub method output from the virtual keyboard, the layout data of the virtual keyboard is changed and the layout data of the virtual keyboard is updated with the extracted values. Save it.

In step S17, the layout of the virtual keyboard is output. When the primary language and the sub language extracted in the step S15 are determined to be the same by comparing the primary method or the sub system output from the virtual keyboard, the output is output as an existing virtual keyboard layout.

In the above, the method and apparatus for providing the virtual keyboard according to an embodiment of the present invention have been described.

The virtual keyboard providing method and display apparatus according to the present invention are not limited to the configuration and method of the embodiments described as described above, but the embodiments are all or part of each embodiment so that various modifications can be made. May be optionally combined.

On the other hand, the virtual keyboard providing method of the present invention can be implemented as a processor-readable code on a processor-readable recording medium provided in the computer. The processor-readable recording medium includes all kinds of recording devices that store data that can be read by the processor. Examples of the processor-readable recording medium include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like, and may also be implemented in the form of a carrier wave such as transmission over the Internet. . The processor-readable recording medium can also be distributed over network coupled computer systems so that the processor-readable code is stored and executed in a distributed fashion.

In addition, although the preferred embodiment of the present invention has been shown and described above, the present invention is not limited to the specific embodiments described above, but the technical field to which the invention belongs without departing from the spirit of the invention claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be individually understood from the technical spirit or the prospect of the present invention.

10: conventional virtual keyboard
20: taskbar
21: input language
50: virtual keyboard

Claims (21)

In the method for providing a virtual keyboard based on the application activated on the display device,
Monitoring keyboard status in an activated application;
Obtaining the keyboard state;
Obtaining an activated language value in the keyboard state;
Extracting a primary language and a sub language from the activated language value;
Comparing the extracted primary language and the sub language with the primary method or sub method output from the virtual keyboard; And
And outputting a layout of the virtual keyboard based on the comparison result. The method of claim 1,
If the extracted primary method or sub method is different from the primary method and the sub method output from the virtual keyboard, change the layout data of the virtual keyboard,
A virtual keyboard providing method of storing a primary method value and a sub method value of the virtual keyboard as an extracted value. The method of claim 1,
The monitoring may include monitoring keyboard state information provided by an operating system of the display device for a predetermined period. The method of claim 3,
And said predetermined period comprises 200ms. The method of claim 1,
And obtaining a keyboard state when the activated application is changed. The method of claim 1,
And obtaining at least one of the keyboard state and the activated language value when there is a request for changing a language value from the activated application. The method of claim 1,
And changing a plurality of key input values provided by the virtual keyboard according to the changed language value. The method of claim 1,
The activated application provides a virtual keyboard comprising an inputable user interface. The method of claim 1,
The activated application includes a text editor, and obtains the language value based on the position of the pointer on the input menu of the text editor. The method of claim 1,
The language value may include a keyboard layout method for a plurality of key input values provided by the virtual keyboard. A display device for providing a virtual keyboard based on an application activated on a display device, the display device comprising:
Monitor the keyboard state in an activated application, obtain the keyboard state, obtain an activated language value in the keyboard state, extract primary and sub-methods from the activated language value, and extract the A controller for comparing the primary method and the sub method with the primary method and the sub method of the virtual keyboard;
And an image processor configured to output the layout of the virtual keyboard based on the comparison result. The method of claim 11,
If the extracted primary method and sub method is different from the existing primary method and sub method of the virtual keyboard, the controller changes the layout data of the virtual keyboard,
And a storage unit which stores the primary method and the sub method of the virtual keyboard as extracted values. The method of claim 11,
The control unit monitors the keyboard state information provided by the operating system of the display device for a predetermined period. The method of claim 13,
And the predetermined period includes 200 ms. The method of claim 11,
The control unit obtains a keyboard state when the activated application is changed. The method of claim 11,
And the controller is configured to obtain at least one of the keyboard state and the activated language value when the language value change request is made from the activated application program. The method of claim 11,
The control unit changes the plurality of key input values provided by the virtual keyboard according to the changed language value. The method of claim 11,
The activated application program includes an inputtable user interface. The method of claim 11,
The activated application program includes a text editor, and obtains the language value based on a position of a pointer on an input menu of the text editor. The method of claim 11,
The language value may include a keyboard layout method for a plurality of key input values provided by a virtual keyboard. A computer-readable storage medium comprising command code for providing a virtual keyboard based on an application activated on a display device, the computer-readable storage medium comprising:
The command code is a control unit of the display device,
Monitor keyboard status in enabled applications,
Obtain the keyboard state,
Acquire an activated language value in the keyboard state,
Extracting a primary method and a sub method of the activated language value,
Compare the extracted primary method and sub method with the primary method and sub method of the virtual keyboard,
And code for outputting a layout of the virtual keyboard based on the comparison result.

Images