KR20200117140A - Method and system for virtual input on the web - Google Patents

Abstract

Disclosed are a method and system for a virtual input on web. According to one embodiment of the present invention, the method for a virtual input performed by an electronic device implemented by a computer may comprise the steps of: configuring at least one layer to which attribute information for data edition is assigned; defining a data model for storing input information recognized in the configured layers; and rendering interaction information related to the data edition modeled through the defined data model based on an environment of browser.

Description

Virtual input method and system on the web {METHOD AND SYSTEM FOR VIRTUAL INPUT ON THE WEB}

The description below relates to techniques for editing data on the web.

Use HTML5's contentEditable to implement the Wiziwick web editor. HTML5's contentEditable is a good way to create a simple web editor because it automatically controls the Wizwick editing at the browser level. There are technical limitations due to various problems to add customization functions such as making DOM elements composed of complex structures instead of single DOM elements behave like single objects or providing fixed layout functions to such web editors. Due to the incomplete contentEditable specification, it is difficult to control the cursor because the operation method is different for each browser, and because there is no division of the input/output area, it is difficult to limit the selection area. In addition, since the final result is in HTML format, it is difficult to recycle data, so a separate purification process is required.

On the other hand, Korean Laid-Open Patent Publication No. 10-2011-0076311 relates to'a multimedia web editor apparatus and method supporting tailoring', and such web editors allow users to use computer languages such as HTML (Hypertext Markup Language) and the Internet. Even if you do not have a lot of knowledge, it supports the creation of various multimedia data such as texts and images in a certain format as easily as using a document writing software. However, the prior art only focuses on the user's ability to use the web editor more conveniently and conveniently from the user's point of view. Accordingly, from the developer's point of view, there is a need for a technology that enables the developer to conveniently expand and implement using a web editor.

It is possible to provide a virtual input method and system for structuring data through a coupling relationship between a layer and a data model as user interaction information occurs.

The input area and the output area are separated to provide a virtual input method and system that implements various templates for edit data.

It is possible to provide a virtual input method and system that provides a consistent editing interface irrespective of a browser environment and edits data through the provided editing interface.

A virtual input method performed by a computer-implemented electronic device may include: configuring at least one layer to which attribute information for data editing is provided; Defining a data model for storing input information recognized in the configured at least one or more layers; And rendering interaction information related to data editing modeled through the defined data model based on an environment of a browser.

The configuring of the at least one layer may include: an input buffer layer in charge of inputting data, an editing area layer in charge of outputting data through an interaction with an input device, and a virtual cursor displaying the input position information and the selected area. It may include the step of configuring a layer.

The defining of the model includes an editing data model for expressing an input element including text data and image data through the at least one configured layer, and a selection information model for expressing a region selected based on the edited data model. It may include the step of defining a data model including a.

The editing data model may include dividing the text image and the input element including the image data into an expanded or reduced form.

The selection information model consists of a start point and an end point of the selected area, and the start point and the end point are composed of a component, a unit, a paragraph, a text node, and an offset. It can be distinguished by (offset) information.

In the rendering based on the environment of the browser, as a frame having an input element is generated, the selected key input information is received in the editing area exposed to the browser through the input device, and the received key input information is handled. It may include moving the focus of the browser to the input buffer layer.

The rendering based on the browser environment may include transmitting key input information received from the input device in the input buffer layer to a DOM object in the editing area layer.

In the rendering based on the environment of the browser, the editing area layer receives key input information input through an input device from the input buffer layer, and edits the interaction information generated through the received key input information into an edit data model. And rendering an edit area layer from the modeled edit data model.

Rendering based on the environment of the browser may include adding a class selector to an element corresponding to an edit data model for location information where the interaction information occurs.

In the rendering based on the browser environment, the virtual cursor layer detects interaction information occurring in the editing area layer and updates the position information being edited through a selection information model, and a virtual cursor from the updated selection information model It may include the step of rendering the layer.

In the rendering based on the browser environment, the editing location information is calculated as screen location information corresponding to the browser screen, and cursor information is exposed on an editing area layer corresponding to the calculated screen location information. Can include.

In the rendering based on the environment of the browser, when the starting point of the selection information model and the same point coincide, the cursor information is exposed in a caret form, and whether the starting point of the selection information model and the same point coincide with each other. Otherwise, the step of exposing the cursor information in the form of a block area may be included.

In the rendering based on the environment of the browser, the cursor information in the form of the caret is expressed to blink, and the area occupied between the start point and the end point of the block area cursor information is calculated to provide transparency and color information. It may include expressing the DOM element including a to be exposed to the editing area layer.

The rendering based on the browser environment may include converting and storing interaction information related to data editing modeled through the defined data model into a JSON-type structure.

The rendering based on the browser environment may include converting the interaction information data in the structure of the stored JSON format according to the browser environment and rendering it to the editing area layer.

Rendering based on the browser environment may include configuring a template by modeling the interaction information based on the edit data model.

The rendering based on the browser environment may include modeling interaction information related to key input information selected in the editing area layer exposed to the browser through the input device in the defined data model.

A computer program stored in a computer-readable storage medium to execute a virtual input method performed by a computer-implemented electronic device includes: configuring at least one layer to which attribute information for data editing is provided; Defining a data model for storing input information recognized in the configured at least one or more layers; And rendering interaction information related to data editing modeled through the defined data model based on an environment of a browser.

The computer-implemented virtual input system of an electronic device includes: a layer configuration unit configuring at least one layer to which attribute information for data editing is assigned; A data model definition unit defining a data model for storing input information recognized in the configured at least one or more layers; And a rendering unit for rendering interaction information related to data editing modeled through the defined data model based on an environment of a browser.

Various templates can be implemented by providing complex element expressions for interaction information through the data model.

Even in a browser environment with each other, interaction information can be extracted as structured data.

Through the structured data format, it can operate in various types of browser environments, and according to the browser environment, interaction information can be rendered without a separate refinement process.

1 is a diagram illustrating an example of a network environment according to an embodiment.
2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an exemplary embodiment.
3 is a block diagram illustrating an example of components that may be included in a processor of an electronic device according to an exemplary embodiment.
4 is a flowchart illustrating a virtual input method performed in an electronic device according to an exemplary embodiment.
5 is an example for explaining a layer structure for a virtual input according to an embodiment.
6 is an example for explaining a virtual input operation through a correlation between a layer and a data model when interaction information occurs through an input device, according to an embodiment.
7 to 9 are examples for explaining rendering of a data model in a browser according to an embodiment.

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

1 is a diagram illustrating an example of a network environment according to an embodiment.

The network environment of FIG. 1 shows an example including a plurality of electronic devices 110, 120, 130, and 140, a plurality of servers 150 and 160, and a network 170. 1 is an example for explaining the present invention, and the number of electronic devices or servers is not limited as in FIG. 1.

The plurality of electronic devices 110, 120, 130, and 140 may be a fixed terminal implemented as a computer device or a mobile terminal. Examples of the plurality of electronic devices 110, 120, 130, 140 include smart phones, mobile phones, navigation, computers, notebook computers, digital broadcasting terminals, personal digital assistants (PDAs), portable multimedia players (PMPs). ), tablet PC, etc. For example, the electronic device 1 110 may communicate with other electronic devices 120, 130, and 140 and/or the servers 150 and 160 through the network 170 using a wireless or wired communication method. In an embodiment, the electronic device may be a client.

The communication method is not limited, and short-range wireless communication between devices as well as a communication method using a communication network (for example, a mobile communication network, a wired Internet, a wireless Internet, a broadcasting network) that the network 170 may include may be included. For example, the network 170 includes a personal area network (PAN), a local area network (LAN), a campus area network (CAN), a metropolitan area network (MAN), a wide area network (WAN), and a broadband network (BBN). , Internet, and the like. In addition, the network 170 may include any one or more of a network topology including a bus network, a star network, a ring network, a mesh network, a star-bus network, a tree or a hierarchical network, etc. Not limited.

Each of the servers 150 and 160 is a computer device or a plurality of computers that communicates with a plurality of electronic devices 110, 120, 130, and 140 through a network 170 to provide commands, codes, files, contents, services, etc. It can be implemented with devices.

For example, the server 160 may provide a file for installing an application to the electronic device 1 110 accessed through the network 170. In this case, the electronic device 1 110 may install an application using a file provided from the server 160. In addition, a service provided by the server 150 by accessing the server 150 under the control of an operating system (OS) included in the electronic device 1 110 and at least one program (for example, a browser or the installed application). I can receive content. For example, when the electronic device 1 110 transmits a service request message to the server 150 through the network 170 under the control of the application, the server 150 transmits a code corresponding to the service request message to the electronic device 1 It may be transmitted to 110, and electronic device 1 110 may provide content to a user by configuring and displaying a screen according to a code according to the control of an application.

2 is a block diagram illustrating an internal configuration of an electronic device and a server according to an exemplary embodiment.

In FIG. 2, the internal configuration of the electronic device 1 110 as an example of one electronic device and the server 150 as an example of one server is described. The same or similar internal components may be applied not only to the other electronic devices 120, 130, 140 or the server 160, but also to other electronic devices or other servers that may be included in the network environment described with reference to FIG. 1.

The electronic device 1 110 and the server 150 may include memories 211 and 221, processors 212 and 222, communication modules 213 and 223, and input/output interfaces 214 and 224. The memories 211 and 221 are computer-readable recording media, and may include a permanent mass storage device such as a random access memory (RAM), a read only memory (ROM), and a disk drive. In addition, the memory 211 and 221 may store an operating system and at least one program code (for example, a browser installed and driven in the electronic device 1 110 or a code for the above-described application). These software components may be loaded from a computer-readable recording medium separate from the memories 211 and 221 using a drive mechanism. Such a separate computer-readable recording medium may include a computer-readable recording medium such as a floppy drive, a disk, a tape, a DVD/CD-ROM drive, and a memory card. In another embodiment, software components may be loaded into the memories 211 and 221 through communication modules 213 and 223 other than a computer-readable recording medium. For example, at least one program is a program installed by files provided by a file distribution system (for example, the server 160 described above) for distributing the installation files of developers or applications (example It may be loaded into the memories 211 and 221 based on the above-described application).

The processors 212 and 222 may be configured to process instructions of a computer program by performing basic arithmetic, logic, and input/output operations. The instructions may be provided to the processors 212 and 222 by the memories 211 and 221 or the communication modules 213 and 223. For example, the processors 212 and 222 may be configured to execute a command received according to a program code stored in a recording device such as the memories 211 and 221.

The communication modules 213 and 223 may provide a function for the electronic device 1 110 and the server 150 to communicate with each other through the network 170, and other electronic devices (for example, electronic device 2 120) Alternatively, a function for communicating with another server (for example, the server 160) may be provided. For example, a request generated by the processor 212 of the electronic device 1 110 according to a program code stored in a recording device such as a memory 211 (for example, a request for a streaming service for content) is controlled by the communication module 213 Depending on the network 170 may be transmitted to the server 150 through the. Conversely, control signals, commands, contents, files, etc. provided under the control of the processor 222 of the server 150 are transmitted via the communication module 223 and the network 170 to the communication module of the electronic device 1 110 ( 213) through the electronic device 1 (110). For example, a control signal or command of the server 150 received through the communication module 213 may be transmitted to the processor 212 or the memory 211, and the electronic device 1110 may transmit contents or files. It may be stored in a storage medium that may further contain.

The input/output interfaces 214 and 224 may be means for interfacing with the input/output device 215. For example, the input device may include a device such as a keyboard or a mouse, and the output device may include a device such as a display for displaying a communication session of an application. As another example, the input/output interface 214 may be a means for interfacing with a device in which input and output functions are integrated into one, such as a touch screen. As a more specific example, the processor 212 of the electronic device 1 110 is configured by using data provided by the server 150 or the electronic device 2 120 in processing a command of a computer program loaded in the memory 211 The service screen or content to be displayed may be displayed on the display through the input/output interface 214.

In addition, in other embodiments, the electronic device 1 110 and the server 150 may include more components than those of FIG. 2. However, there is no need to clearly show most of the prior art components. For example, the electronic device 1 110 may be implemented to include at least some of the input/output devices 215 described above, or other components such as a transceiver, a global positioning system (GPS) module, a camera, various sensors, and a database. It may also contain more elements.

The embodiments define a data model that stores input information recognized in at least one layer to which attribute information for data editing is assigned, and based on the browser environment, interaction information related to data editing modeled through the defined data model It relates to a technology for implementing a virtual input to be rendered. For example, the electronic device 1 110 may be operated by downloading a program for virtual input from the server 150 on the web.

3 is a block diagram illustrating an example of components that may be included in a processor of an electronic device according to an exemplary embodiment, and FIG. 4 is a flowchart illustrating a virtual input method performed in an electronic device according to an exemplary embodiment. .

The processor 222 of the electronic device 1 110 may include a layer configuration unit 310, a data model definition unit 320, and a rendering unit 330. Components of the processor 212 may be expressions of different functions performed by the processor 212 according to a control command provided by a program code stored in the server. The processor 212 and components of the processor 212 may control the server to perform steps S410 to S430 included in the virtual input method of FIG. 4. In this case, the components of the processor 212 and the processor 212 may be implemented to execute an instruction according to the code of the operating system included in the memory and the code of at least one program.

The processor 212 may load a program code stored in a program file for a virtual input method into a memory. For example, when a program is executed in the electronic device 1 110, the processor may control the electronic device 1 110 to load a program code from a program file into a memory under control of the operating system. At this time, each of the layer configuration unit 310, the data model definition unit 320, and the rendering unit 330 included in the processor 212 and the processor 212 execute a command of a corresponding part of the program code loaded into the memory. Thus, different functional expressions of the processor 222 for executing the subsequent steps S410 to S430 may be used.

In step S410, the layer configuration unit 310 may configure at least one layer to which attribute information for data editing is assigned. The layer configuration unit 310 configures an input buffer layer in charge of inputting data, an editing area layer in charge of outputting data through an interaction with an input device, and a virtual cursor layer displaying input possible position information and a selected area. I can.

In step S420, the data model definition unit 320 may define a data model for storing input information recognized in at least one configured layer. The data model definition unit 320 includes an editing data model for expressing an input element including text data and image data through at least one configured layer, and a selection information model for expressing an area selected based on the editing data model. You can define a data model to do. In this case, the editing data model may divide the text image and the input element including the image data into an expanded or reduced form. For example, the editing data model may classify a text image and an input element including image data into a component, a unit, a paragraph, and a text node. The selection information model consists of a start point and an end point related to the selected area, and the start point and end point are component, unit, paragraph, text node, and offset information. It can be classified as

In step S430, the rendering unit 330 may render the interaction information related to data editing modeled through the defined data model based on the browser environment. Referring to FIG. 6, a virtual input operation is described through a correlation between a layer and a data model when interaction information occurs through an input device. Referring to FIG. 6A, a key input event may be generated in the editing area layer 520 through an input device. For example, the user may generate a key input event (eg, a mouse event) to touch or select a browser screen through a mouse. In this case, the key input event input by the user on the browser screen may be recognized in the editing area layer 520. A frame (eg, iframe) having an input element may be generated in the input buffer layer 530. In this case, in order to minimize malfunction due to user interaction interference in the editing area layer 520, an iframe may be generated. As a frame having an input element is generated in the input buffer layer 530, the rendering unit 330 may receive a key input event selected from the editing area layer 520 exposed on the browser screen through the input device.

The rendering unit 330 updates the interaction information related to the key input event in the virtual cursor layer 510 to the selection information model as the key event generated by selecting the editing area layer 520 is transmitted to the virtual cursor layer 510 can do. If the interaction information stored in the selection information model does not exist, the rendering unit 330 may reflect the interaction information related to the mouse event transmitted from the editing area layer 520, and the interaction information stored in the selection information model is If present, the interaction information related to the mouse event transmitted from the editing area layer 520 may be updated. Here, the interaction information may be information generated in connection with a key input event.

The rendering unit 330 may render the cursor information by updating the interaction information in the selection information model in the virtual cursor layer 510. The rendering unit 330 may display cursor information on a point where a key input event occurs on a browser screen. The rendering unit 330 may move the focus to the input buffer layer 530 as the virtual cursor layer 510 updates the interaction information in the selection information model. The rendering unit 330 may forcibly move the actual browser focus to the input buffer layer 530. Forcibly moving the focus is called the still focus function. In this case, since the input buffer layer 530 is not an actual input area, the user cannot input a key, and thus the focus of the actual browser must be taken so that the user's key input information can be handled. When the focus is taken, if the block area was selected, the HTML of the selected block area can be copied to the input area layer. This is to allow HTML to be copied to the clipboard when the user copies it.

Referring to FIG. 6B, a key input event may occur based on a text area hidden under the input buffer layer 530. For example, the user may select a browser screen through an input device (eg, a mouse) and then input text data through another input device (eg, a keyboard). A frame (eg, iframe) having an input element may be generated in the input buffer layer 530. In this case, in order to minimize malfunction due to user interaction interference in the editing area layer 520, an iframe may be generated. As a frame having an input element is generated in the input buffer layer 530, the rendering unit 330 may receive a key input event selected from the editing area layer 520 exposed on the browser screen through the input device.

The rendering unit 330 may update the interaction information related to the keyboard event received in the input buffer layer 510 to the editing data model of the editing area layer 520. If there is no interaction information in the edit data model in the editing area layer 520, the rendering unit 330 may reflect the interaction information related to the keyboard event received from the input buffer layer 510, and the edit data When interaction information exists in the model, interaction information related to a keyboard event received from the input buffer layer 510 may be updated.

The rendering unit 330 may render the updated or reflected interaction information through the edit data model. Accordingly, text data input from the user may be displayed on the browser screen. After updating the editing data model of the editing area layer 520, the rendering unit 330 may update the position information being edited in relation to the last input position of the interaction information in the selection information model of the virtual cursor layer 510. I can. The rendering unit 330 may display the cursor information by rendering the updated location information in the selection information model of the virtual cursor layer 510.

5 is an example for explaining the structure of a layer for virtual input according to an embodiment. A virtual input device web editor configured in a form in which such layers are overlapped may be provided. For example, a user may edit text data or image data through a virtual input device web editor, and the edited data may be displayed on a browser screen. In fact, the user edits data through the editing area layer in the virtual input device web editor.

The input buffer layer 530 may perform processing related to key input information input through an input device (eg, mouse, keyboard, digitizing, etc.). The input buffer layer 530 may exist in a hidden area that is not visible to the user. The input buffer layer configures a text area hidden below the input buffer layer to receive a user's keyboard input and convert it into data. In addition, since cursor selection information is included when text is input in the text area, the cursor information can be transferred to the virtual cursor layer and rendered. The input buffer layer 530 may perform a process of converting data by receiving a user's key input as an input element/textarea element. In this case, the input buffer layer 530 may use a contentEditable Element to freely control HTML when copy/paste is performed.

The input buffer layer 530 may forcibly move the focus of the actual browser as interaction information is generated from the user through the editing area layer 520. Also, a key input event input through the input device may be transmitted to the DOM object of the editing area layer. Delivering key input events to the editing area layer is called a relay event function. Since the key input event occurs only in the input buffer layer 530, an external application cannot register a key event listener like a general input element. The rendering unit 330 duplicates the key input event generated in the input buffer layer 530 and transmits it to the editing area layer, so that an external developer can handle it like a normal DOM object. The rendering unit 330 may transmit a custom event so that the input buffer layer 530 may partially update the edit data model when there is a change in an input value for input information generated through an input device.

The editing area layer 520 may perform processing related to data output and interaction information input through an input device. The editing area layer 520 may model the input data transmitted through the input buffer layer as an editing data model, and perform a process of rendering the editing area layer from the modeled editing data model.

The editing area layer 520 may be an editing area of any DOM element. The editing area layer 520 may model interaction information generated in relation to the input device transmitted through the input buffer layer through the editing data model. In this case, the editing area layer 520 may directly implement the rendering of the modeled data in order to increase the degree of freedom of customization. The editing area layer 520 may add a class selector including a class and an id to an element corresponding to the editing data model with location information where interaction information generated through a key input event occurs.

The virtual cursor layer 530 may perform a process of expressing inputtable position information and a selection area. The virtual cursor layer 530 may display a blinking caret or block area on the editing area layer as cursor information. The virtual cursor layer 530 may recognize location information related to a point at which the user is actually inputting input information in the editing area layer 530. The virtual cursor layer 530 may model a selection information model as interaction information is generated through an input device in the editing area layer 520.

The virtual cursor layer 510 may be displayed using an actual DOM element. By detecting the interaction information occurring in the editing area layer 520, the position information being edited may be modeled through the selection information model. The virtual cursor layer 510 may calculate the position information being edited as screen position information corresponding to the browser screen, and expose the cursor information by overlapping the editing area layer corresponding to the calculated screen position information. The virtual cursor layer 510 exposes cursor information in the form of a caret when the starting point of the selection information model and the same point coincide, and in the form of a black area when the starting point of the selection information model and the same point do not coincide. I can make it. In this case, in the case of a caret-type cursor, the DOM element may be periodically or aperiodically show/hide to be expressed as a blinking cursor. The block area may express a DOM element including transparency and color information to be exposed on the editing area layer by calculating an area occupying the block area-type cursor information between the start point and the end point of the selection information model.

7 to 9 are examples for explaining rendering of a data model in a browser according to an embodiment.

Before describing the rendering of the data model according to the browser environment, the editing data model will be described.

The editing data model may divide a text image and an input element including the image data into an expanded or reduced form. For example, the edited data model may consist of 4depth of edited data up to an actual unit text for expressing a complex element. From the top, the editing data model can be divided into component -> unit -> paragraph -> textnode. A component may mean one unit for expressing complex things including text data and image data. In other words, a component can be defined as a top-level object of a composite element, and a plurality of components can be created. A unit is a lower-level division object of a component, and may be a text area or an image area. For example, as an image having a caption is defined as one component, the image and the caption may each be a unit. Paragraphs are objects that separate paragraphs, and are used to separate paragraphs within a unit corresponding to text. The text node is a unit text entity and can distinguish unit texts having different styles.

In the editing data model, the caption image component may be defined as the DATA part of FIG. 7. In this way, the interaction information stored in the editing data model may be converted into data in a JSON format and stored in an electronic device. The user can convert and render the data stored in the JSON format according to the browser environment. For example, a caption image component defined as a DATA part may be converted to HTML and rendered on a browser screen. Accordingly, a browser screen may be exposed as shown in the drawing on the far right.

Also, when text nodes have different styles, text nodes having different styles may be expressed as shown in FIG. 8. In other words, you can edit the style of text data such as thickness, size, color, and font (font). For example, as shown in the DATA part of FIG. 8, text data may be set to be displayed in bold and red in the edit data model based on the expression of the complex element defined in the edit data model. The interaction information stored in the editing data model may be converted into data in a JSON format and stored in an electronic device. For example, the expression of another text node, such as a DATA part, may be converted to HTML and rendered on the browser screen. Accordingly, a browser screen in which text nodes to which different styles are applied, as shown in the rightmost drawing, may be exposed.

In this way, a separate conversion process is performed according to the browser environment by using the structured data in the JSON format edit data model stored in the electronic device, so that the interaction information may be rendered on the browser screen.

The final edit data of the edit data model may be completed as an array of components in the form of DomainData=[component1, component2, ...];. Since a template can be determined according to how a component is defined in the edit data model, various templates can be configured according to a combination of components, and the configured template can be rendered on the browser screen.

The selection information model is a model for expressing an area selected based on the editing data model. It may be composed of location information including a starting point and an ending point being edited in relation to the interaction information. Each of these points may include component, unit, paragraph, text node, and offset information.

The selection information model including cursor information present in the caption of the caption image component described in FIG. 7 may be expressed as shown in FIG. 9. Referring to FIG. 9, this is an example for explaining cursor information when a caret type, a text area is selected, and a component is selected. For example, the virtual cursor layer may detect interaction information occurring in the editing area layer, update position information being edited through the selection information model, and render cursor information from the updated selection information model. In this case, the location information related to the interaction information stored in the selection information model may be converted into data in a JSON format and stored in the electronic device.

Specifically, location information being edited may be calculated as screen location information corresponding to a browser screen, and cursor information may be exposed on an editing area layer corresponding to the calculated screen location information. A browser screen in which cursor information is exposed at the front of the caption image component described in FIG. 7 may be rendered. In this case, when the starting point and the same point of the selection information model coincide on the browser screen, cursor information may be exposed in the form of a caret. At this time, cursor information in the form of caret blinks periodically or aperiodically at the front of the caption image component. For example, a blinking cursor can be expressed by repeatedly operating show/hide cursor information in the form of caret through CSS adjustment.

Also, when the starting point and the same point of the selection information model do not coincide, the cursor information may be exposed in the form of a block area. For example, a DOM element including transparency and color information may be expressed to be exposed to the editing area layer by calculating an area occupying the block area type cursor information between the start point and the end point. For example, the block area may be displayed in translucent blue. As shown in FIG. 9, when a text area labeled “Image Caption” is selected on a browser screen, cursor information in the form of a block area may be generated in the text area labeled “Image Caption”. In this case, when cursor information in the form of a block area is generated in the text area, a partial area corresponding to the text area may be displayed on the browser screen as cursor information in the form of a block area.

Also, when a component is selected on the browser screen, cursor information in the form of a block area may be generated in the component area. In this case, when cursor information in the form of a block area is generated in the component area, the entire area corresponding to the component may be displayed on the browser screen as cursor information in the form of a block area.

The apparatus described above may be implemented as a hardware component, a software component, and/or a combination of a hardware component and a software component. For example, the devices and components described in the embodiments are, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA). , A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions, such as one or more general purpose computers or special purpose computers. The processing device may execute an operating system (OS) and one or more software applications executed on the operating system. In addition, the processing device may access, store, manipulate, process, and generate data in response to the execution of software. For the convenience of understanding, although it is sometimes described that one processing device is used, one of ordinary skill in the art, the processing device is a plurality of processing elements and/or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of these, configuring the processing unit to behave as desired or processed independently or collectively. You can command the device. Software and/or data may be interpreted by a processing device or to provide instructions or data to a processing device, of any type of machine, component, physical device, virtual equipment, computer storage medium or device. Can be embodyed in The software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like alone or in combination. The program instructions recorded on the medium may be specially designed and configured for the embodiment, or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -A hardware device specially configured to store and execute program instructions such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of the program instructions include not only machine language codes such as those produced by a compiler, but also high-level language codes that can be executed by a computer using an interpreter or the like.

As described above, although the embodiments have been described by the limited embodiments and drawings, various modifications and variations are possible from the above description by those of ordinary skill in the art. For example, the described techniques are performed in a different order from the described method, and/or components such as a system, structure, device, circuit, etc. described are combined or combined in a form different from the described method, or other components Alternatively, even if substituted or substituted by an equivalent, an appropriate result can be achieved.

Therefore, other implementations, other embodiments, and claims and equivalents fall within the scope of the claims to be described later.

Claims (19)

In the virtual input method performed by an electronic device implemented as a computer,
Configuring at least one layer to which attribute information for data editing is assigned;
Defining a data model for storing input information recognized in the configured at least one or more layers; And
Rendering interaction information related to data editing modeled through the defined data model based on the browser environment
Virtual input method comprising a. The method of claim 1,
The step of configuring the at least one layer,
A step of configuring an input buffer layer in charge of inputting data, an editing area layer in charge of outputting data through interaction with an input device, and a virtual cursor layer displaying the input position information and the selected area.
Virtual input method comprising a. The method of claim 1,
The step of defining the model,
Defining a data model including an editing data model for expressing an input element including text data and image data through the configured at least one or more layers and a selection information model for expressing an area selected based on the editing data model step
Virtual input method comprising a. The method of claim 3,
The edited data model,
Dividing the text image and the input element including the image data into an expanded or reduced form
Virtual input method. The method of claim 3,
The selection information model,
It consists of a start point and an end point of the selected area, and divides the start point and the end point into component, unit, paragraph, text node, and offset information. doing
Virtual input method comprising a. The method of claim 1,
Rendering based on the environment of the browser,
As a frame having an input element is generated, receiving selected key input information in an editing area exposed to a browser through an input device, and moving the focus of the browser for handling the received key input information to the input buffer layer
Virtual input method comprising a. The method of claim 6,
Rendering based on the environment of the browser,
Transmitting key input information received from the input device in the input buffer layer to a DOM object of the editing area layer
Virtual input method comprising a. The method of claim 7,
Rendering based on the environment of the browser,
Receives key input information input through an input device from the input buffer layer in the edit area layer, models interaction information generated through the received key input information as an edit data model, and edits from the modeled edit data model Steps to render an area layer
Virtual input method comprising a. The method of claim 8,
Rendering based on the environment of the browser,
Adding a class selector to an element corresponding to the edit data model for the location information where the interaction information occurs
Virtual input method comprising a. The method of claim 8,
Rendering based on the environment of the browser,
Detecting interaction information occurring in the editing area layer in a virtual cursor layer, updating position information being edited through a selection information model, and rendering a virtual cursor layer from the updated selection information model
Virtual input method comprising a. The method of claim 10,
Rendering based on the environment of the browser,
Calculating the position information being edited as screen position information corresponding to the browser screen, and exposing cursor information to an editing area layer corresponding to the calculated screen position information
Virtual input method comprising a. The method of claim 10,
Rendering based on the environment of the browser,
When the starting point of the selection information model and the same point coincide, the cursor information is exposed in a caret form, and when the starting point of the selection information model and the same point do not coincide, the cursor information is exposed in the form of a block area. step
Virtual input method comprising a. The method of claim 12,
Rendering based on the environment of the browser,
The cursor information in the form of the caret is expressed to blink periodically, and the area occupying the cursor information in the block area form between the start point and the end point is calculated, and a DOM element including transparency and color information is added to the editing area layer. Expressing to be exposed
Virtual input method comprising a. The method of claim 1,
Rendering based on the environment of the browser,
Converting and storing interaction information related to data editing modeled through the defined data model into data in a JSON format structure
Virtual input method comprising a. The method of claim 14,
Rendering based on the environment of the browser,
Converting the interaction information converted into the saved JSON format according to the browser environment and rendering it to the editing area layer
Virtual input method comprising a. The method of claim 8,
Rendering based on the environment of the browser,
Configuring a template by modeling the interaction information based on the edited data model
Virtual input method comprising a. The method of claim 1,
Rendering based on the environment of the browser,
Modeling interaction information related to key input information selected in the editing area layer exposed to the browser through the input device in the defined data model
Virtual input method comprising a. A computer program stored in a computer-readable storage medium to execute a virtual input method performed by an electronic device implemented as a computer,
Configuring at least one layer to which attribute information for data editing is assigned;
Defining a data model for storing input information recognized in the configured at least one or more layers; And
Rendering interaction information related to data editing modeled through the defined data model based on the browser environment
A computer program stored in a computer-readable storage medium comprising a. In the virtual input system of an electronic device implemented by a computer,
A layer construction unit configuring at least one layer to which attribute information for data editing is assigned;
A data model definition unit defining a data model for storing input information recognized in the configured at least one or more layers; And
Rendering unit that renders interaction information related to data editing modeled through the defined data model based on the browser environment
Virtual input system of an electronic device comprising a.

Images