KR102515372B1 - System and Method for Providing Electronic Service Implementing Output Remote Screen and Program, Computer Readable Recording Medium - Google Patents

Abstract

Provided are an electronic service providing system, method, program, and computer readable recording medium for implementing remote screen output, which perform real-time image processing on connection between a drawing pad and a computer without delay when communication is connected between a low-end drawing pad and a computer, so that remote collaboration between the drawing pad and a PC is possible by linking the low-end drawing pad and the PC without delay.

Description

System and Method for Providing Electronic Service Implementing Output Remote Screen and Program, Computer Readable Recording Medium}

The present invention relates to an electronic service providing system, and more particularly, when a low-spec drawing pad and a computer are connected through communication, real-time image processing is performed on the connection between the drawing pad and the computer without delay, so that the drawing work on the drawing pad is output from the computer. It relates to an electronic service providing system, method, program, and computer readable recording medium that implements remote screen output capable of cooperating with each other.

User interaction with devices such as computers and other electronic computing devices has increased significantly in recent years.

Thanks to the development of electronic technology, various types of electronic products are being developed and supplied. In recent years, electronic devices having touch screen displays, such as smart phones and/or tablet PCs, have been increasingly popular.

A touch input to a touch screen display of an electronic device may be performed by a user's finger. For more precise input, an electronic pen such as a stylus may be utilized.

Illustrators, cartoonists, and designers perform drawing work using a drawing pad and a pressure pen, but purchase expensive tablet equipment and connect it to a computer by wire.

Low-end drawing pads can be purchased easily and cheaply, but a significant delay occurs in the communication method with the computer, making it difficult to link the drawing pad and computer that the operator works with. There was an inconvenience of not being able to collaborate with a remote PC.

Korean Patent Registration No. 10-2386480

In order to solve this problem, the present invention outputs the drawing work on the drawing pad from the computer and collaborates with each other by processing images in real time without delay when connecting the low-end drawing pad and the computer through communication. An object of the present invention is to provide an electronic service providing system, method, program, and computer readable recording medium that implements this possible remote screen output.

An electronic service providing system implementing remote screen output according to the features of the present invention for achieving the above object includes a first electronic device having a screen output function and a data transmission/reception function; and

A second electronic device having a lower specification than the first electronic device that performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions;

The second electronic device detects a touch input where a pen touches a point on the screen of the second display unit as a pressure value, and outputs an event signal including a pressure value and coordinate values according to the detection result. And, receiving the event signal from the second pen sensor, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating screens of various layouts including objects based on the calculated attribute values and a second graphic card that captures the screen of the generated layout and compresses and encodes the video, and generates second screen display information including pre-stored hardware information, and compresses the generated second screen display information and the encoded video. a second processor transmitting screen information to the first electronic device;

When the first electronic device analyzes the hardware information of the second electronic device included in the second screen display information and determines that the second electronic device has low-end hardware that cannot transmit real-time screens, the second screen display information enables real-time screen transmissions. At least one image quality information among a resolution, a bit rate, and a frame rate is changed, and the decoded image information is output to the first display unit based on the changed image quality information.

The second screen display information is hardware information of the second electronic device, and includes header information such as IP information, frame start time, resolution related to image quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, and video data. It includes the screen size, CPU information, and graphic card information.

The first processor stores the encoded video compression information received through the first communication unit and the second screen display information, which is a hardware device specification, in a first data storage unit, and the second screen display information is stored in the first data storage unit. The IP information extracted from is identified, the encoded video compression information and the second screen display information are matched and stored in IP information, and the frame reception time at which the second screen display information is received is determined as the second screen display information. Add to the field and save.

The first processor calculates a difference time between a frame start time and a frame reception time of the field of the second screen display information, and when the calculated difference time is greater than or equal to a predetermined real-time screen transmission time, the real-time screen transmission is possible. 2 At least one image quality information among resolution, bit rate, and frame rate of the screen display information is changed, and image information decoded based on the changed image quality information is output to the first display unit.

In the electronic service providing method for realizing remote screen output performed between a first electronic device and a second electronic device according to the features of the present invention,

The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,

detecting, by the second electronic device, a touch input in which a point on the screen of the second display unit is touched with a pen as a pressure value, and outputting an event signal including a pressure value and coordinate values according to the detection result;

receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating screens of various layouts including objects based on the calculated attribute values;

The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;

The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among a resolution, a bit rate, and a frame rate; and

The first electronic device includes outputting the decoded image information to a first display unit based on the changed image quality information.

A computer-readable recording medium recording a program for executing an electronic service providing method for realizing remote screen output performed between a first electronic device and a second electronic device according to the features of the present invention, comprising:

The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,

The electronic service providing method implementing the remote screen output,

detecting, by the second electronic device, a touch input in which a point on the screen of the second display unit is touched with a pen as a pressure value, and outputting an event signal including a pressure value and coordinate values according to the detection result;

receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating screens of various layouts including objects based on the calculated attribute values;

The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;

The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among a resolution, a bit rate, and a frame rate; and

The first electronic device includes outputting the decoded image information to a first display unit based on the changed image quality information.

As a program stored in a computer-readable recording medium for executing an electronic service providing method for implementing a remote screen output performed between a first electronic device and a second electronic device according to the features of the present invention,

The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,

The electronic service providing method implementing the remote screen output,

detecting, by the second electronic device, a touch input in which a point on the screen of the second display unit is touched with a pen as a pressure value, and outputting an event signal including a pressure value and coordinate values according to the detection result;

receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating screens of various layouts including objects based on the calculated attribute values;

The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;

The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among a resolution, a bit rate, and a frame rate; and

The first electronic device includes outputting the decoded image information to a first display unit based on the changed image quality information.

According to the configuration described above, the present invention has an effect of enabling remote collaboration between the drawing pad and the PC by linking the low-end drawing pad and the PC without delay.

There is an effect of performing a joint drawing operation with a PC using a low-end drawing pad without using an expensive drawing pad or tablet of the present invention.

1 is a diagram showing the configuration of an electronic service providing system implementing remote screen output according to a first embodiment of the present invention.
2 is a block diagram schematically illustrating an internal structure of a first electronic device and a second electronic device according to a first embodiment of the present invention.
3 is a diagram showing an example of resolution, frame rate, and bit rate according to the first embodiment of the present invention.
4 is a diagram illustrating a state in which a first electronic device and a second electronic device simultaneously perform a drawing operation on one image object according to the first embodiment of the present invention.
5 is a diagram illustrating a state in which a first electronic device and a second electronic device perform a drawing operation on two image objects according to the first embodiment of the present invention.
6 is a diagram showing the configuration of an electronic service providing system implementing remote screen output according to a second embodiment of the present invention.
7 is a block diagram briefly illustrating an internal configuration of a fourth electronic device according to a second embodiment of the present invention.
8 is a block diagram briefly showing the internal configuration of a virtual machine according to a second embodiment of the present invention.
9 is a diagram showing the configuration of an electronic service providing system implementing remote screen output according to a third embodiment of the present invention.
10 is a diagram showing a connection relationship between a virtual machine and an electronic device according to a third embodiment of the present invention.

Since the present invention can make various changes and have various embodiments, specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, and should be understood to include all modifications, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals have been used for like elements throughout the description of each figure.

Terms such as first, second, A, and B may be used to describe various components, but the components should not be limited by the terms. These terms are only used for the purpose of distinguishing one component from another. For example, a first element may be termed a second element, and similarly, a second element may be termed a first element, without departing from the scope of the present invention. The term “and/or” includes any combination of a plurality of related listed items or any of a plurality of related listed items.

It is understood that when an element is referred to as being "connected" or "connected" to another element, it may be directly connected or connected to the other element, but other elements may exist in the middle. It should be. On the other hand, when an element is referred to as “directly connected” or “directly connected” to another element, it should be understood that no other element exists in the middle.

Terms used in this application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this application, the terms "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art to which the present invention belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

Hereinafter, with reference to the accompanying drawings, preferred embodiments of the present invention will be described in more detail. In order to facilitate overall understanding in the description of the present invention, the same reference numerals are used for the same components in the drawings, and redundant descriptions of the same components are omitted.

1 is a diagram showing the configuration of an electronic service providing system implementing remote screen output according to a first embodiment of the present invention, and FIG. 2 is a first electronic device and a second electronic device according to the first embodiment of the present invention. It is a block diagram briefly showing the internal configuration of , and FIG. 3 is a diagram showing an example of resolution, frame rate, and bit rate according to the first embodiment of the present invention.

In the electronic service providing system 100 implementing remote screen output according to an embodiment of the present invention, a first electronic device 110 and a second electronic device 120 are connected to each other in a wired manner to transmit and receive data in both directions.

The first electronic device 110 may be a PC (Personal Computer) having a screen output function and a data transmission/reception function, in addition to a smart phone, mobile phone, navigation, digital It may be a broadcasting terminal, Personal Digital Assistants (PDA), Portable Multimedia Player (PMP), tablet PC, or the like.

The second electronic device 120 has lower specifications than the first electronic device 110, transmits and receives screen data to and from a remote host PC of the first electronic device 110, and is a drawing pad or a drawing pad that performs a drawing task such as a webtoon. It may be an electronic pad.

The first electronic device 110 and the second electronic device 120 install a bidirectional data control app and are connected to each other through the installed bidirectional data control app to transmit and receive data in both directions.

When the first electronic device 110 is connected to the second electronic device 120 by wire and performs a drawing operation on the second electronic device 120, the first electronic device 110 receives drawing data of the second electronic device 120, and the second electronic device 110 receives drawing data from the second electronic device 120. Drawing data of the device 120 can be worked. That is, the first electronic device 110 and the second electronic device 120 may perform drawing work together by exchanging drawing data.

The first electronic device 110 stores the IP address information of the second electronic device 120 in the address storage 112 . Accordingly, when the second electronic device 120 runs the interactive data control app, it generates an access request signal including IP address information and transmits it to the first electronic device 110 .

A first electronic device 110 according to an embodiment of the present invention includes a first memory 111, an address storage unit 112, a first processor 113, a first display unit 114, and a first communication unit 115. , a first data storage unit 116, a first graphic card 117, and a first pen detection unit 118.

The first electronic device 110 can receive touch input through the first pen sensor 118 and may include other input means such as a keyboard and a mouse.

The first memory 111 stores an interactive data control app.

When receiving an access request signal from the second electronic device 120 through the first communication unit 115, the first processor 113 analyzes the received connection request signal to obtain IP address information of the second electronic device 120. extract

The first processor 113 determines whether the extracted IP address information is already stored in the address storage unit 112, and if the extracted IP address information is stored in the address storage unit 112, the second electronic device 120 A connection response signal corresponding to the connection request signal of is generated and transmitted to the second electronic device 120 through the first communication unit 115 . As a result, the communication channels of the first electronic device 110 and the second electronic device 120 are connected.

The first graphic card 117 may create a screen including various objects such as icons, images, and text by using a calculation unit and a rendering unit. The calculation unit calculates attribute values such as coordinate values, shapes, sizes, and colors of each object to be displayed according to the layout of the screen.

The rendering unit generates screens of various layouts including objects based on the attribute values calculated by the calculation unit. The screen created by the rendering unit is provided to the first display unit 114 and displayed within the area of the first display unit 114 .

The first pen detection unit 118 is a module for parsing a trajectory drawn by the user on the surface of the first display unit 114 and recognizing the contents.

When a point on the screen of the first display unit 114 is touched by a finger or a pen, the first pen sensor 118 generates event signals of x and y coordinate values of the touch point and transmits them to the graphic card.

The first pen sensor 118 detects a proximity input or touch input of a pen as a pressure value, and outputs an event signal including a pressure value and coordinate values according to the detection result.

When the touch point is moved in a touched state, the first pen sensor 118 generates in real time an event signal including the touch coordinate value of the changed trajectory, the attribute value of the shape and size, and the pressure value to generate the first graphic Transfer to card 117.

The first graphic card 117 calculates the attribute values received from the first pen sensor 118 and creates screens of various layouts including objects based on the calculated attribute values.

The first graphic card 117 displays the screen of the created layout within the area of the first display unit 114 .

The second electronic device 120 according to an embodiment of the present invention includes a second memory 121, a second processor 122, a second display unit 123, a second communication unit 124, a second data storage unit ( 125), a second graphic card 126, and a second pen detection unit 127.

The second memory 121 stores interactive data control apps.

The second processor 122 controls the operation of the pre-installed interactive data control app so that the bi-directional data control app is already activated.

The second communication unit 124 receives a connection response signal permitting device access from the first electronic device 110 through the interactive data control app.

The second graphic card 126 may create a screen including various objects such as icons, images, and text by using a calculation unit and a rendering unit. The calculation unit calculates attribute values such as coordinate values, shapes, sizes, and colors of each object to be displayed according to the layout of the screen.

The rendering unit generates screens of various layouts including objects based on the attribute values calculated by the calculation unit. The screen generated by the rendering unit is provided to the second display unit 123 and displayed within the area of the second display unit 123 .

The second pen detection unit 127 is a module for parsing a trajectory drawn by the user on the surface of the second display unit 123 and recognizing the contents.

When a point on the screen of the second display unit 123 is touched by a finger or a pen, the second pen sensor 127 generates an event signal of the x and y coordinate values of the touch point to detect the second graphic card 126 ) is sent to

The second pen sensor 127 detects a proximity input or touch input of a pen as a pressure value, and outputs an event signal including a pressure value and coordinate values according to the detection result.

When the touch point is moved in a touched state, the second pen sensor 127 generates in real time an event signal including the touch coordinate value of the changed trajectory, the attribute value of the shape and size, and the pressure value to generate the second graphic Send to card 126.

The second graphic card 126 receives an event signal from the second pen sensor 127, calculates attribute values, and creates screens of various layouts including objects based on the calculated attribute values.

The second graphic card 126 displays the screen of the created layout within the area of the second display unit 123 .

The second graphics card 126 performs screen capture at predetermined screen capture times (eg, at least 33 ms) when there is a screen change, such as the screen of the created layout, compresses and encodes the captured screen, and then encodes the screen. The compressed video screen information is transmitted to the second processor 122.

The second graphic card 126 compresses and encodes the captured screen as a video by continuously compressing a frame composed of a plurality of pixels to a predetermined size. A still image is composed of one frame, and a moving image is composed of a plurality of frames.

The second graphics card 126 compresses a plurality of 2×2 blocks into one pixel value, that is, compresses four pixels included in one block into one pixel value to generate a compressed pixel value. .

The second graphics card 126 may compress four pixels into one pixel value using a general data compression technique.

The second processor 122 generates second screen display information including pre-stored hardware information. Here, the second screen display information is hardware information of the second electronic device 120, and is header information, which includes IP information, frame start time, resolution related to picture quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, It includes the size of the screen on which the image data is displayed, CPU information, and graphic card information. Encoder settings are encoding protocol, video codec, frame rate, and bit rate.

The second processor 122 transmits the encoded video compression screen information and the second screen display information to the first electronic device 110 through the second communication unit 124 .

The first processor 113 stores the encoded video compression information received through the first communication unit 115 and second screen display information that is a hardware device specification in the first data storage unit 116 . In the first data storage unit 116, IP information extracted from the second screen display information is identified, video compression information encoded in the IP information, and second screen display information are matched and stored. At this time, the frame reception time at which the second screen display information is received is added to the field of the second screen display information and stored.

The first processor 113 restores the compressed pixel values of the encoded video compression information into 4 pixels of 2×2 size using a standard reconstruction technique to generate decoded video information.

The first data storage unit 116 pre-stores first screen display information capable of displaying the decoded image information on the first display unit 114 .

Here, the first screen display information is hardware information of the first electronic device 110, and includes the size of the screen on which the video data is displayed, resolution and bit rate related to image quality, frame rate, bit rate, encoder setting, decoder setting, CPU information, including graphics card information.

The first processor 113 analyzes the hardware information of the second electronic device 120 included in the second screen display information, and analyzes the hardware information of the second electronic device 120 and the hardware of the first electronic device 110. Compare with information.

When the first processor 113 determines that the hardware information of the second electronic device 120 is low-end hardware that cannot transmit real-time screen to the first electronic device 110, the first processor 113 transmits the second screen display information to enable real-time screen transmission. At least one image quality information among resolution, bit rate, and frame rate is changed, and decoded image information is output to the first display unit 114 based on the changed image quality information.

Specifically, the first processor 113 adds 500 Kbps to the bit rate of the second electronic device 120 in the case of a quality priority notice, and in the case of real-time screen transmission, 1000 Kbps in the bit rate of the second electronic device 120 to 1500 Kbps.

For example, when the CPU and graphic card information of the second electronic device 120 is of low specifications in real-time screen transmission, the bit rate must be increased due to low computing power.

The first processor 113 is real-time screen transmission, and when the CPU information and the graphic card information are low specifications, 1000 Kbps to 1500 Kbps are added to the bit rate of the second electronic device 120 to cover the declining computing power with the amount of data per second. .

In terms of picture quality, the effect of resolution is greater, resolution is related to the sharpness of the image, frame rate is related to the smoothness of the image movement, and bit rate is related to the image deterioration symptoms as the amount of data per second increases.

Bit rate represents the size of data in units of bits constituting an image of 1 second. The unit is bps (Bit Per Second). Since the bit rate directly affects the size of an image file, the higher the bit rate, the better the image quality, and the lower the bit rate, the lower the image quality.

A video is a collection of still images that are continuously displayed, and each image is called a frame. The frame rate represents the number of images reproduced for one second in a video, and uses fps (Frames Per Second) as a unit. It can be seen that the image quality improves as the frame rate of 5 fps, 15 fps, 20 fps, and 30 fps increases.

When the bit rate needs to be lowered, lowering the frame rate at the same time can reduce picture quality degradation.

The first processor 113 transmits a screen in real time, and when the CPU information and the graphic card information are low specifications, the first display unit 114 of the first electronic device 110 at the bit rate of the second electronic device 120 A minimum bit rate and a maximum bit rate are generated to satisfy the displayable image quality.

The first processor 113 outputs the decoded image information to the first display unit 114 while sequentially changing from the minimum bit rate to the maximum bit rate. The first processor 113 may select a bit rate with the least deterioration in image quality or may select a bit rate of image information selected through the first pen sensor 118 .

The first processor 113 outputs the decoded image information based on the image quality information including the selected bit rate to the first display unit 114, and the first data storage unit 116 outputs the changed information matching the IP address information. Stores image quality information.

The first electronic device 110 detects a touch input as a pressure value through the first pen sensor 118 while the decoded image information is displayed on the first display unit 114, and the pressure value according to the detection result. Outputs an event signal including and coordinate values.

When the touch point is moved in a touched state, the first pen sensor 118 generates in real time an event signal including the touch coordinate value of the changed trajectory, the attribute value of the shape and size, and the pressure value to generate the first graphic Transfer to card 117.

The first graphic card 117 calculates the attribute values received from the first pen sensor 118 and creates screens of various layouts including objects based on the calculated attribute values.

The first graphic card 117 displays the screen of the created layout within the area of the first display unit 114 .

The first graphic card 117 performs screen capture at predetermined screen capture times when there is a screen change, such as the screen of the created layout, compresses and encodes the captured screen, and then transmits the encoded video compression screen information. It is transmitted to the first processor 113.

The first processor 113 transmits encoded video compression screen information to the second electronic device 120 through the first communication unit 115 .

The second processor 122 restores the compressed pixel values of the encoded video compression screen information into four pixels of 2×2 size using a standard restoration technique to generate decoded image information, and the generated decoded image information It is output through the second display unit 123.

The first electronic device 110 and the second electronic device 120 may draw an image object together.

The first processor 113 calculates a difference time between the frame start time and the frame reception time of the field of the second screen display information, determines whether the calculated difference time is equal to or less than a predetermined real-time screen transmission time, and determines that the calculated difference time is When the predetermined real-time screen transmission time is longer, at least one image quality information among the resolution, bit rate, and frame rate of the second screen display information is changed to enable real-time screen transmission, and the decoded image information based on the changed image quality information Output to the first display unit 114.

When the first processor 113 stores the decoded image information in the first data storage unit 116 for each time period and receives a signal requesting the decoded image information including the time information from the second electronic device 120 , Decoded image information may be extracted based on corresponding time information and IP information, and the extracted decoded image information may be encoded and transmitted to the second electronic device 120 .

When a drawing work is deleted by mistake, the second electronic device 120 may request and receive decoded image information of a desired time period from the first electronic device 110 .

FIG. 4 is a diagram illustrating a state in which a drawing operation of one image object is simultaneously performed in the first electronic device and the second electronic device 120 according to the first embodiment of the present invention.

When the second electronic device 120 performs a drawing task, image information of the drawing task is displayed on the first display unit 114 of the first electronic device 110 .

At the same time, if another part of the image object is drawn on the first electronic device 110, it is displayed on the second display unit 123 of the second electronic device 120. Simultaneous drawing may be possible for one image object in the first electronic device 110 and the second electronic device 120 .

5 is a diagram illustrating a state in which a first electronic device and a second electronic device perform a drawing operation on two image objects according to the first embodiment of the present invention.

The first electronic device 110 and the second electronic device 120 may each perform a drawing operation on two different image objects, and allow the other electronic device to view the drawing image objects that are working with each other.

6 is a diagram showing the configuration of an electronic service providing system implementing remote screen output according to a second embodiment of the present invention, and FIG. 7 is a simplified internal configuration of a fourth electronic device according to a second embodiment of the present invention. 8 is a block diagram briefly showing the internal configuration of a virtual machine according to a second embodiment of the present invention.

In the electronic service providing system 100 implementing remote screen output according to the second embodiment of the present invention, the third electronic device 130 and the fourth electronic device 140 are connected to each other in a wired or wireless manner to transmit and receive data in both directions. do.

The third electronic device 130 is a PC (Personal Computer) having a screen output function, and the fourth electronic device 140 is a drawing pad or electronic pad that performs a drawing task such as a webtoon.

The third electronic device 130 according to the second embodiment of the present invention includes a third memory 131, an address storage unit 132, a third processor 133, a third display unit 134, and a communication module 135. ), a third data storage unit 136, a third graphic card 137, a third pen detection unit 138, a virtual machine control unit 139a, a virtual machine emulator generation unit 139b, a virtual machine and peripheral devices ( 10).

The third electronic device 130 and the fourth electronic device 140 install a bidirectional data control app and are connected to each other through the installed bidirectional data control app to transmit and receive data in both directions.

The third electronic device 130 stores the IP address information of the fourth electronic device 140 in the address storage unit 132 . Accordingly, when the interactive data control app is driven, the fourth electronic device 140 generates an access request signal including IP address information and transmits it to the third electronic device 130 .

The third memory 131 stores interactive data control apps.

When receiving an access request signal from the fourth electronic device 140 through the communication module 135, the third processor 133 analyzes the received connection request signal to obtain IP address information of the fourth electronic device 140. extract

The third processor 133 determines whether the extracted IP address information is already stored in the address storage unit 132, and if the extracted IP address information is stored in the address storage unit 132, the fourth electronic device 140 A connection response signal corresponding to the connection request signal of is generated and transmitted to the fourth electronic device 140 through the communication module 135 . As a result, the communication channel between the third electronic device 130 and the fourth electronic device 140 is connected.

When the communication channel between the third electronic device 130 and the fourth electronic device 140 is connected, the third processor 133 generates a virtual machine request signal of the fourth electronic device 140 and uses the virtual machine control unit 139a send to

When a point on the screen of the third display unit 134 is touched by a finger or a pen, the third pen sensor 138 generates an event signal of x and y coordinate values of the touch point to detect the third graphic card 137 ) is sent to

The third pen sensor 138 detects a proximity input or touch input of a pen as a pressure value, and outputs an event signal including a pressure value and coordinate values according to the detection result.

When the touch point is moved in a touched state, the third pen sensor 138 generates in real time an event signal including touch coordinate values, shape, and size attribute values of a changed trajectory, and a pressure value to generate a third graphic Transfer to card 137.

The third graphic card 137 calculates attribute values received from the pen sensor, and creates screens of various layouts including objects based on the calculated attribute values.

The third graphic card 137 displays the screen of the created layout within the area of the third display unit 134 .

The third graphics card 137 performs screen capture at predetermined screen capture times when there is a screen change, such as the screen of the created layout, compresses and encodes the captured screen, and converts the encoded video compression screen information to It is transmitted to the third processor 133.

The third processor 133 generates third screen display information including pre-stored hardware information, and stores the encoded video compression screen information and the third screen display information in the third data storage unit 136 . Here, the third screen display information is hardware information of the third electronic device 130, and includes resolution related to image quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, screen size on which image data is displayed, CPU information, including graphics card information.

The third processor 133 transmits encoded video compression screen information and third screen display information to the fourth electronic device 140 through the communication module 135 .

Upon receiving the virtual machine request signal from the third processor 133, the virtual machine control unit 139a generates a control signal for generating the virtual machine 139c including the IP information of the fourth electronic device 140 to create a virtual machine emulator. It is transmitted to the generator 139b. The virtual machine 139c is a logical operating system or application capable of independent execution of multiple units using one physical device through virtualization technology. The virtual machine 139c can execute different operating systems or applications by creating a virtual execution environment by being allocated hardware resources of a physical machine.

When the virtual machine emulator generation unit 139b receives a control signal for generating the virtual machine 139c including the IP information of the fourth electronic device 140 from the virtual machine control unit 139a, the ID of the virtual machine 139c Information is generated, and a virtual machine link is generated using the generated ID information and the IP information of the fourth electronic device 140 and transmitted to the fourth electronic device 140 through the communication module 135 .

The virtual machine emulator generator 139b receives hardware information of the fourth electronic device 140 from the third processor 133 and generates resource allocation information based on the received hardware information of the fourth electronic device 140. do. Here, the hardware information includes the resolution, bit rate, frame rate, CPU information, graphic card information, and the like of the fourth electronic device 140 .

The virtual machine emulator generating unit 139b generates a virtual machine 139c corresponding to the fourth electronic device 140 based on the resource allocation information, and generates a communication between the created virtual machine 139c and the fourth electronic device 140. Connect the access link. Through this connection link, the virtual machine 139c can copy the functional characteristics of the fourth electronic device 140 or function as an emulator designed to execute the same.

In other words, the virtual machine 139c is an emulator of the fourth electronic device 140 and operates as if all tasks occurring in the fourth electronic device 140 are equally executed.

The virtual machine emulator generating unit 139b electrically connects the third processor 133 and the communication module 135.

The fourth electronic device 140 according to an embodiment of the present invention includes a fourth memory 141, a fourth processor 142, a fourth display unit 143, a fourth communication unit 144, and a fourth data storage unit ( 145), a fourth graphic card 146, and a fourth pen detection unit 147.

The fourth electronic device 140 is the same device as the second electronic device 120 of the first embodiment described above, and descriptions of overlapping components may be omitted.

The fourth memory 141 stores interactive data control apps.

The fourth processor 142 controls the operation of the pre-installed interactive data control app to make the bi-directional data control app already activated.

The fourth communication unit 144 receives a connection response signal permitting device access from the third electronic device 130 through the interactive data control app.

The fourth graphic card 146 may create a screen including various objects such as icons, images, and text by using a calculation unit and a rendering unit.

The fourth pen detection unit 147 is a module for parsing a trajectory drawn by the user on the surface of the fourth display unit 143 and recognizing the contents.

When a point on the screen of the second display unit 123 is touched by a finger or a pen, the fourth pen sensor 147 generates an event signal of x and y coordinate values of the touch point to detect the fourth graphic card 146 ) is sent to

The fourth pen sensor 147 detects a proximity input or touch input of a pen as a pressure value, and outputs an event signal including a pressure value and coordinate values according to the detection result.

When the touch point is moved in a touched state, the fourth pen sensor 147 generates in real time an event signal including the touch coordinate value of the changed trajectory, the attribute value of the shape and size, and the pressure value to generate the fourth graphic. Send to card 146.

The fourth graphic card 146 calculates attribute values received from the pen sensor, and creates screens of various layouts including objects based on the calculated attribute values.

The fourth graphic card 146 displays the screen of the created layout within the area of the fourth display unit 143 .

The fourth graphics card 146 performs screen capture at a predetermined screen capture time (eg, at least 33 ms) when there is a screen change, such as the screen of the created layout, compresses and encodes the captured screen, and then encodes the screen. The compressed video screen information is transmitted to the fourth processor 142.

The fourth processor 142 generates fourth screen display information including previously stored hardware information. Here, the fourth screen display information is hardware information of the fourth electronic device 140, and is header information, which includes IP information, frame start time, resolution related to picture quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, It includes the size of the screen on which the image data is displayed, CPU information, and graphic card information.

The fourth processor 142 transmits the encoded video compression screen information and the fourth screen display information to the third electronic device 130 through the fourth communication unit 144 .

The third processor 133 of the third electronic device 130 stores encoded video compression information received through the third communication unit and fourth screen display information that is a hardware device specification in the third data storage unit 136 . In the third data storage unit 136, IP information extracted from the fourth screen display information is identified, video compression information encoded in the IP information, and fourth screen display information are matched and stored. At this time, the frame reception time at which the fourth screen display information is received is added to the field of the fourth screen display information and stored.

The third processor 133 restores the compressed pixel values of the encoded video compression information into 4 pixels of 2×2 size using a standard reconstruction technique to generate decoded video information.

The third data storage unit 136 pre-stores third screen display information capable of displaying the decoded image information on the third display unit 134 .

Here, the third screen display information is hardware information of the third electronic device 130, and includes the size of the screen on which the image data is displayed, resolution and bit rate related to image quality, frame rate, bit rate, encoder setting, decoder setting, CPU information, including graphics card information.

The third processor 133 analyzes the hardware information of the fourth electronic device 140 included in the fourth screen display information, and the hardware information of the fourth electronic device 140 and the hardware of the fourth electronic device 140. Compare with information.

When the third processor 133 determines that the hardware information of the fourth electronic device 140 is low-end hardware that cannot transmit real-time screens to the fourth electronic device 140, the third processor 133 transmits the screen display information in real time to the fourth screen display information. At least one image quality information among resolution, bit rate, and frame rate is changed, and image information decoded based on the changed image quality information is output to the third display unit 134 .

The third processor 133 outputs the decoded image information based on the image quality information including the selected bit rate to the third display unit 134, and the third data storage unit 136 outputs the changed information matching the IP address information. Stores image quality information.

The third electronic device 130 detects the touch input as a pressure value through the third pen sensor 138 while the decoded image information is displayed on the third display unit 134, and the pressure value according to the detection result. Outputs an event signal including and coordinate values.

When the touch point is moved in a touched state, the third pen sensor 138 generates in real time an event signal including touch coordinate values, shape, and size attribute values of a changed trajectory, and a pressure value to generate a third graphic Transfer to card 137.

The third graphic card 137 calculates attribute values received from the third pen sensor 138 and creates screens of various layouts including objects based on the calculated attribute values.

The third graphic card 137 displays the screen of the created layout within the area of the third display unit 134 .

The third graphics card 137 performs screen capture at predetermined screen capture times when there is a screen change, such as the screen of the created layout, compresses and encodes the captured screen, and converts the encoded video compression screen information to It is transmitted to the third processor 133.

The third processor 133 transmits encoded video compression screen information to the fourth electronic device 140 through the communication module 135 .

The fourth processor 142 of the fourth electronic device 140 restores compressed pixel values to four pixels of 2×2 size using a standard reconstruction technique for encoded video compression screen information to generate decoded video information, , and outputs the generated decoded image information through the fourth display unit 143.

The third electronic device 130 and the fourth electronic device 140 may draw an image object together.

The virtual machine 139c is basically equipped with an operating system of the fourth electronic device 140 and includes a hard disk driver 139d and a device management unit 139e.

When a virtual disk image emulated as a hard disk is allocated to the virtual machine 139c, the hard disk driver 139d recognizes the allocated virtual disk image, and the device management unit 139e configures the hardware of the fourth electronic device 140. Indicates that the device has been attached.

When the device management unit 139e receives notification from the hard disk driver 139d that the hardware device of the fourth electronic device 140 is attached, the fourth processor 142 of the fourth electronic device 140, the fourth display unit ( 143), the fourth graphic card 146, the fourth pen sensor 147, the fourth data storage unit 145, and the fourth memory 141 may be controlled and managed.

For example, when the fourth pen detection unit 147 of the fourth electronic device 140 outputs an event signal including a pressure value and a coordinate value, the virtual machine 139c detects the fourth pen from the device management unit 139e. An event signal of unit 147 is generated.

That is, control signals and event signals generated in the hardware of the fourth electronic device 140 are equally generated in the device management unit 139e of the virtual machine 139c so that the third processor 133 can detect them in real time. .

The virtual machine emulator generating unit 139b determines a resource allocation ratio in advance according to the hardware information of the fourth electronic device 140 . Here, the resource allocation ratio is preset, such that more resources are allocated as the resolution, frame rate, and bit rate are higher, and fewer resources are allocated as the resolution, frame rate, and bit rate are smaller.

The fourth electronic device 140 is connected to the peripheral devices (printer, mouse, keyboard, sound equipment, etc.) 10 connected to the third electronic device 130 through the virtual machine 139c and functions as the peripheral device 10. is available.

9 is a diagram showing the configuration of an electronic service providing system implementing remote screen output according to a third embodiment of the present invention, and FIG. 10 shows a connection relationship between a virtual machine and an electronic device according to a third embodiment of the present invention. is the drawing shown.

In the electronic service providing system 100 implementing remote screen output according to the third embodiment of the present invention, the third electronic device 130 and the fourth electronic device 140 are wirelessly connected to each other to transmit and receive data in both directions. And, the third electronic device 130 and the fifth electronic device 150 are wirelessly connected to each other to transmit and receive data in both directions.

The third electronic device 130 is a PC (Personal Computer) having a screen output function, and the fourth electronic device 140 and the fifth electronic device 150 are drawing pads or electronic pads that perform drawing tasks such as webtoons. .

The third electronic device 130 according to an embodiment of the present invention includes a third memory 131, an address storage unit 132, a third processor 133, a third display unit 134, a communication module 135, A third data storage unit 136, a third graphic card 137, a third pen detection unit 138, a virtual machine controller 139a, a virtual machine emulator generator 139b, a first virtual machine 20, It includes a second virtual machine 30 and a peripheral device 10 .

Like the first and second embodiments, a drawing operation may be performed between the third electronic device 130 and the fourth electronic device 140, and the third electronic device 130 and the fifth electronic device 150 ) can perform drawing work between them. transmission of image information between the third electronic device 130 and the fourth electronic device 140 and between the third electronic device 130 and the fifth electronic device 150, encoding, decoding, simultaneous drawing, etc. The same principle as in the second embodiment is applied.

The third electronic device 130 of the third embodiment is composed of a device having a higher specification than the connected fourth electronic device 140 and fifth electronic device 150, such as a PC or a server.

The third electronic device 130 and the fourth electronic device 140 install a bidirectional data control app and are connected to each other through the installed bidirectional data control app to transmit and receive data in both directions.

The third electronic device 130 and the fifth electronic device 150 install a bi-directional data control app and are connected to each other through the installed bi-directional data control app to transmit and receive data in both directions.

The third electronic device 130 stores the IP address information of the fourth electronic device 140 in the address storage unit 132 . Accordingly, when the interactive data control app is driven, the fourth electronic device 140 generates an access request signal including IP address information and transmits it to the third electronic device 130 .

The third electronic device 130 stores the IP address information of the fifth electronic device 150 in the address storage unit 132 . Accordingly, when the interactive data control app is driven, the fifth electronic device 150 generates an access request signal including IP address information and transmits it to the third electronic device 130 .

The third memory 131 stores interactive data control apps.

When receiving an access request signal from the fourth electronic device 140 through the communication module 135, the third processor 133 analyzes the received connection request signal to obtain IP address information of the fourth electronic device 140. extract

The third processor 133 determines whether the extracted IP address information is already stored in the address storage unit 132, and if the extracted IP address information is stored in the address storage unit 132, the fourth electronic device 140 A connection response signal corresponding to the connection request signal of is generated and transmitted to the fourth electronic device 140 through the communication module 135 . As a result, the communication channel between the third electronic device 130 and the fourth electronic device 140 is connected.

When the communication channel between the third electronic device 130 and the fourth electronic device 140 is connected, the third processor 133 generates a virtual machine request signal of the fourth electronic device 140 and uses the virtual machine control unit 139a send to

When receiving an access request signal from the fifth electronic device 150 through the communication module 135, the third processor 133 analyzes the received connection request signal to obtain IP address information of the fifth electronic device 150. extract

The third processor 133 determines whether the extracted IP address information is already stored in the address storage unit 132, and if the extracted IP address information is stored in the address storage unit 132, the fifth electronic device 150 A connection response signal corresponding to the connection request signal of is generated and transmitted to the fifth electronic device 150 through the communication module 135 . As a result, the communication channel between the third electronic device 130 and the fifth electronic device 150 is connected.

When the communication channel between the third electronic device 130 and the fifth electronic device 150 is connected, the third processor 133 generates a virtual machine request signal of the fifth electronic device 150 and uses the virtual machine control unit 139a send to

When a point on the screen of the third display unit 134 is touched by a finger or a pen, the third pen sensor 138 generates an event signal of x and y coordinate values of the touch point to detect the third graphic card 137 ) is sent to

The third pen sensor 138 detects a proximity input or touch input of a pen as a pressure value, and outputs an event signal including a pressure value and coordinate values according to the detection result.

When the touch point is moved in a touched state, the third pen sensor 138 generates in real time an event signal including touch coordinate values, shape, and size attribute values of a changed trajectory, and a pressure value to generate a third graphic Transfer to card 137.

The third graphic card 137 calculates attribute values received from the pen sensor, and creates screens of various layouts including objects based on the calculated attribute values.

The third graphic card 137 displays the screen of the created layout within the area of the third display unit 134 .

The third graphics card 137 performs screen capture at predetermined screen capture times when there is a screen change, such as the screen of the created layout, compresses and encodes the captured screen, and converts the encoded video compression screen information to It is transmitted to the third processor 133.

The third processor 133 generates third screen display information including pre-stored hardware information, and stores the encoded video compression screen information and the third screen display information in the third data storage unit 136 . Here, the third screen display information is hardware information of the third electronic device 130, and includes resolution related to image quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, screen size on which image data is displayed, CPU information, including graphics card information.

The third processor 133 transmits the encoded video compression screen information and the third screen display information to the fourth electronic device 140 or the fifth electronic device 150 through the communication module 135 .

When receiving a virtual machine request signal from the third processor 133, the virtual machine control unit 139a generates a control signal for generating a virtual machine including the IP information of the fourth electronic device 140 to create a virtual machine emulator ( 139b). A virtual machine is a logical operating system or application capable of independent execution of multiple units using one physical device through virtualization technology. These virtual machines can execute different operating systems or applications by creating a virtual execution environment by being allocated hardware resources of a physical machine.

When receiving a control signal for generating a virtual machine including IP information of the fourth electronic device 140 from the virtual machine controller 139a, the virtual machine emulator generating unit 139b generates and generates ID information of the virtual machine. A virtual machine link is generated using one ID information and IP information of the fourth electronic device 140 and transmitted to the fourth electronic device 140 through the communication module 135 .

When receiving a control signal for generating a virtual machine including the IP information of the fifth electronic device 150 from the virtual machine control unit 139a, the virtual machine emulator generating unit 139b generates and generates ID information of the virtual machine. A virtual machine link is generated using one ID information and IP information of the fifth electronic device 150 and transmitted to the fifth electronic device 150 through the communication module 135 .

Since the third embodiment is the same as the internal components of the third electronic device 130 of the second embodiment, descriptions of overlapping components will be omitted and description will focus on the differences.

The virtual machine emulator generation unit 139b may generate the first virtual machine 20 and the second virtual machine 30, and may also generate two or more virtual machines in addition.

When receiving a control signal for generating a virtual machine including IP information of the fourth electronic device 140 from the virtual machine controller 139a, the virtual machine emulator generating unit 139b generates and generates ID information of the virtual machine. A virtual machine link is generated using one ID information and IP information of the fourth electronic device 140 and transmitted to the fourth electronic device 140 through the communication module 135 .

In addition, when the virtual machine emulator generating unit 139b receives a control signal for generating a virtual machine including the IP information of the fifth electronic device 150 from the virtual machine control unit 139a, it generates ID information of the virtual machine, , A virtual machine link is generated using the generated ID information and the IP information of the fifth electronic device 150 and transmitted to the fifth electronic device 150 through the communication module 135 .

The virtual machine emulator generating unit 139b receives hardware information of the fourth electronic device 140 from the third processor 133 and receives hardware information of the fifth electronic device 150 from the third processor 133. . Here, the hardware information includes the resolution, bit rate, frame rate, CPU information, graphics card information, and the like of the corresponding electronic device.

The virtual machine emulator generating unit 139b derives a first resource allocation weight by substituting the hardware information into Table 1 below.

The third processor 133 derives the second resource allocation weight by using the real-time screen transmission time, which is the difference between the frame start time and the frame reception time. More specifically, the third processor 133 calculates the difference time between the frame start time and the frame reception time of the field of the fourth screen display information of the fourth electronic device 140, and calculates the difference time as shown in Table 2 below. Substituting in to derive the second resource allocation weight.

The third processor 133 determines whether the calculated difference time is less than or equal to the preset real-time screen transmission time, and if the calculated difference time is greater than or equal to the preset real-time screen transmission time, the resolution of the fourth screen display information to enable real-time screen transmission. At least one image quality information among , bit rate, and frame rate is changed, and decoded image information is output to the third display unit 134 based on the changed image quality information.

The third processor 133 calculates the difference between the frame start time and frame reception time of the fifth screen display information field of the fifth electronic device 150, and substitutes the calculated difference time into Table 2 below to obtain 2 Derive resource allocation weights.

The third processor 133 determines whether the calculated difference time is equal to or less than the predetermined real-time screen transmission time, and if the calculated difference time is greater than or equal to the preset real-time screen transmission time, the resolution of the fifth screen display information to enable real-time screen transmission. At least one image quality information among , bit rate, and frame rate is changed, and decoded image information is output to the third display unit 134 based on the changed image quality information.

The third processor 133 calculates a priority index using the derived first resource allocation weight and the second resource allocation weight using Equation 1 below, and calculates the priority index through the virtual machine emulator generator 139b. send to

Here, R1 is a first resource allocation weight, and R2 is a second resource allocation weight.

The priority index is preset by matching resource allocation information.

The virtual machine emulator generating unit 139b extracts resource allocation information matching the priority index, and generates a first virtual machine 20 corresponding to the fourth electronic device 140 based on the extracted resource allocation information, A connection link between the created first virtual machine 20 and the fourth electronic device 140 is connected. Through this connection link, the first virtual machine 20 may copy the functional characteristics of the fourth electronic device 140 or function as an emulator designed to execute the same.

In other words, the first virtual machine 20 is an emulator of the fourth electronic device 140 and operates as if all tasks occurring in the fourth electronic device 140 are equally executed.

The first virtual machine 20 is basically equipped with an operating system of the fourth electronic device 140 and includes a first hard disk driver 21 and a first device management unit 22 .

When a virtual disk image emulated as a hard disk is allocated to the first virtual machine 20, the first hard disk driver 21 recognizes the allocated virtual disk image and sends the first device manager 22 a fourth electronic disk image. Indicates that the hardware device of device 140 is attached.

When the first device manager 22 receives notification from the first hard disk driver 21 that the hardware device of the fourth electronic device 140 is attached, the fourth processor 142 of the fourth electronic device 140, Input/output of the 4 display unit 143, the fourth graphic card 146, the fourth pen sensor 147, the fourth data storage unit 145, and the fourth memory 141 can be controlled and managed.

For example, when the fourth pen sensor 147 of the fourth electronic device 140 outputs an event signal including a pressure value and a coordinate value, the first virtual machine 20 operates in the first device manager 22. An event signal of the fourth pen sensor 147 is generated.

That is, control signals and event signals generated in the hardware of the fourth electronic device 140 are identically generated in the first device management unit 22 of the first virtual machine 20 and detected in real time by the third processor 133. You can do it.

The virtual machine emulator generating unit 139b electrically connects the third processor 133 and the communication module 135.

The virtual machine emulator generating unit 139b extracts resource allocation information matching the priority index, and generates a second virtual machine 30 corresponding to the fifth electronic device 150 based on the extracted resource allocation information, A connection link between the created second virtual machine 30 and the fifth electronic device 150 is connected. Through this connection link, the second virtual machine 30 can copy the functional characteristics of the fifth electronic device 150 or function as an emulator designed to execute the same.

In other words, the second virtual machine 30 is an emulator of the fifth electronic device 150 and operates as if all tasks occurring in the fifth electronic device 150 are equally executed.

The second virtual machine 30 is basically equipped with an operating system of the fifth electronic device 150 and includes a second hard disk driver 31 and a second device management unit 32 .

When a virtual disk image emulated as a hard disk is allocated to the second virtual machine 30, the second hard disk driver 31 recognizes the allocated virtual disk image and sends the second device manager 32 a fourth electronic disk image. Indicates that the hardware device of device 140 is attached.

When the second device manager 32 receives notification from the second hard disk driver 31 that the hardware device of the fifth electronic device 150 is attached, the fifth processor and the fifth display unit of the fifth electronic device 150 , the fifth graphic card, the fifth pen detection unit, the fifth data storage unit, and the input/output of the fifth memory may be controlled and managed.

Since the internal components of the fifth electronic device 150 are the same as those of the fourth electronic device 140, a detailed description thereof will be omitted.

For example, when the fifth pen sensor of the fifth electronic device 150 outputs an event signal including a pressure value and a coordinate value, the second virtual machine 30 detects the fifth pen in the second device manager 32. Generates an event signal of the sensing unit.

That is, control signals and event signals generated in the hardware of the fifth electronic device 150 are equally generated in the second device management unit 32 of the second virtual machine 30 and detected in real time by the third processor 133. You will be able to do it.

The virtual machine emulator generating unit 139b may generate a plurality of virtual machines, and creates the virtual machines using resource allocation information matching the priority index.

The fourth electronic device 140 and the fifth electronic device 150 may run different operating systems and may have different hardware information.

In the electronic service providing method for realizing remote screen output performed between a first electronic device and a second electronic device according to the features of the present invention,

The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,

detecting, by the second electronic device, a touch input in which a point on the screen of the second display unit is touched with a pen as a pressure value, and outputting an event signal including a pressure value and coordinate values according to the detection result;

receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating screens of various layouts including objects based on the calculated attribute values;

The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;

The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among a resolution, a bit rate, and a frame rate; and

The first electronic device includes outputting the decoded image information to a first display unit based on the changed image quality information.

A computer-readable recording medium recording a program for executing an electronic service providing method for realizing remote screen output performed between a first electronic device and a second electronic device according to the features of the present invention, comprising:

The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,

The electronic service providing method implementing the remote screen output,

detecting, by the second electronic device, a touch input in which a point on the screen of the second display unit is touched with a pen as a pressure value, and outputting an event signal including a pressure value and coordinate values according to the detection result;

receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating screens of various layouts including objects based on the calculated attribute values;

The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;

The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among a resolution, a bit rate, and a frame rate; and

The first electronic device includes outputting the decoded image information to a first display unit based on the changed image quality information.

As a program stored in a computer-readable recording medium for executing an electronic service providing method for implementing a remote screen output performed between a first electronic device and a second electronic device according to the features of the present invention,

The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,

The electronic service providing method implementing the remote screen output,

detecting, by the second electronic device, a touch input in which a point on the screen of the second display unit is touched with a pen as a pressure value, and outputting an event signal including a pressure value and coordinate values according to the detection result;

receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating screens of various layouts including objects based on the calculated attribute values;

The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;

The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among a resolution, a bit rate, and a frame rate; and

The first electronic device includes outputting the decoded image information to a first display unit based on the changed image quality information.

The electronic service providing system 100 implements a remote screen output in which functions and operations of components of a first electronic device 110 and a second electronic device 120 are performed between the first electronic device and the second electronic device. It may represent or implement a program stored in a computer readable recording medium recording a program for performing an electronic service providing method and executing the electronic service providing method, and a computer readable recording medium for executing the electronic service providing method.

The recording medium and the program are connected to or downloaded from the first electronic device 110, the second electronic device 120, the third electronic device 130, the fourth electronic device 140, and the fifth electronic device 150. In this case, an operation of setting whether the first electronic device 110, the second electronic device 120, the third electronic device 130, the fourth electronic device 140, or the fifth electronic device 150 may be added. there is.

Accordingly, the recording medium and the program can perform the above-described electronic service providing method in the corresponding electronic device.

Operations according to the embodiments of the present specification can be implemented as computer-readable programs or codes on a computer-readable recording medium. A computer-readable recording medium includes all types of recording devices in which data that can be read by a computer system is stored. In addition, computer-readable recording media may be distributed to computer systems connected through a network to store and execute computer-readable programs or codes in a distributed manner.

When the embodiment is implemented as software, the above-described technique may be implemented as a module (process, function, etc.) that performs the above-described functions. A module can be stored in memory and executed by a processor. The memory may be internal or external to the processor, and may be coupled with the processor in a variety of well-known means.

In addition, the computer-readable recording medium may include hardware devices specially configured to store and execute program instructions, such as ROM, RAM, and flash memory. The program command may include high-level language codes that can be executed by a computer using an interpreter or the like as well as machine code generated by a compiler.

Although some aspects of the present invention have been described in the context of an apparatus, it may also represent a description according to a corresponding method, where a block or apparatus corresponds to a method step or feature of a method step. Similarly, aspects described in the context of a method may also be represented by a corresponding block or item or a corresponding feature of a device. Some or all of the method steps may be performed by (or using) a hardware device such as, for example, a microprocessor, programmable computer or electronic circuitry. In some embodiments, one or more of the most important method steps may be performed by such an apparatus.

In embodiments, a programmable logic device (eg, a field programmable gate array) may be used to perform some or all of the functions of the methods described herein. In embodiments, a field programmable gate array may operate in conjunction with a microprocessor to perform one of the methods described herein. Generally, methods are preferably performed by some hardware device.

Although the above has been described with reference to preferred embodiments of the present invention, those skilled in the art will variously modify and change the present invention within the scope not departing from the spirit and scope of the present invention described in the claims below. You will understand that it can be done.

100: electronic service providing system 110: first electronic device
111: first memory 112: address storage unit
113: first processor 114: first display unit
115: first communication unit 116: first data storage unit
117: first graphic card 118: first pen detection unit
120: second electronic device 121: second memory
122: second processor 123: second display unit
124: second communication unit 125: second data storage unit
126: second graphic card 127: second pen detection unit
130: third electronic device 131: third memory
132: address storage unit 133: third processor
134: third display unit 135: communication module
136: third data storage unit 137: third graphic card
138: third pen detection unit 139a: virtual machine control unit
139b: virtual machine emulator generation unit 139c: virtual machine
140: fourth electronic device 141: fourth memory
142: fourth processor 143: fourth display unit
144: fourth communication unit 145: fourth data storage unit
146: 4th graphic card 147: 4th pen detection unit
150: fifth electronic device

Claims (7)

A first electronic device having a screen output function and a data transmission/reception function; and
A second electronic device having a lower specification than the first electronic device that performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions;
The second electronic device detects a touch input in which a point on the screen of the second display unit of the second electronic device is touched with a pen as a pressure value, and outputs an event signal including the pressure value and the coordinate value of the point. A second pen sensing unit that receives the event signal from the second pen sensing unit and calculates property values such as coordinate values, shape, and size of a changing trajectory, and based on the calculated property values To include an object A screen of the layout is created, a second graphic card that captures the screen of the created layout to compress and encode a video, and second screen display information including pre-stored hardware information is generated, and the generated second screen display information and A second processor for transmitting the encoded video compression screen information to the first electronic device;
When the first electronic device analyzes the hardware information of the second electronic device included in the second screen display information and determines that the second electronic device has low-end hardware that cannot transmit real-time screens, the second screen display information enables real-time screen transmissions. Changing at least one image quality information among a resolution, bit rate, and frame rate of , compressing and encoding a video according to the changed image quality information to the second electronic device, and transmitting a signal to be transmitted to the first electronic device; outputting image information obtained by decoding the encoded image information based on the changed image quality information to a first display unit of the first electronic device;
The second screen display information is hardware information of the second electronic device, which includes IP information, frame start time, resolution related to picture quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, and video data as header information. It includes the size of the displayed screen, CPU information, and graphics card information,
The first processor of the first electronic device stores the encoded video compression information received through the first communication unit of the first electronic device and the second screen display information that is a hardware device specification in a first data storage unit, The first data storage unit recognizes the IP information extracted from the second screen display information, matches the encoded video compression information with the IP information and stores the second screen display information, and receives the second screen display information. Adding and storing one frame reception time to a field of the second screen display information,
The first processor calculates a difference time between a frame start time and a frame reception time of the field of the second screen display information, and when the calculated difference time is greater than or equal to a predetermined real-time screen transmission time, the real-time screen transmission is possible. Changing the image quality information to the changed image quality information;
The first electronic device generates ID information of the virtual machine based on the control signal and a virtual machine controller that generates a control signal for generating a virtual machine including the IP information of the second electronic device, and the ID information and A virtual machine emulator generating unit generating a virtual machine link using IP information, transmitting the virtual machine link to the second electronic device through the first communication unit, and deriving a first resource allocation weight based on hardware information of the second electronic device. do,
The first processor derives a second resource allocation weight using a time difference between the frame start time and the frame reception time, and calculates a priority index based on the first resource allocation weight and the second resource allocation weight. and transmits the calculated priority index to the virtual machine emulator generation unit;
The virtual machine emulator generating unit generates the virtual machine corresponding to the second electronic device based on resource allocation information matching the priority index, and connects a connection link between the virtual machine and the second electronic device ,
An electronic service delivery system that implements remote screen output. delete delete delete An electronic service providing method for implementing remote screen output performed between a first electronic device and a second electronic device,
The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,
The second electronic device detects a touch input in which a point on the screen of the second display unit of the second electronic device is touched with a pen as a pressure value, and outputs an event signal including the pressure value and the coordinate value of the point. doing;
receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating a layout screen including an object based on the calculated attribute values;
The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;
The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among resolution, bit rate, and frame rate of the image quality information, compressing and encoding a video according to the changed image quality information to the second electronic device, and transmitting a signal to be transmitted to the first electronic device. ; and
outputting, by the first electronic device, image information obtained by decoding the encoded image information based on the changed image quality information to a first display unit of the first electronic device;
The second screen display information is hardware information of the second electronic device, which includes IP information, frame start time, resolution related to picture quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, and video data as header information. It includes the size of the displayed screen, CPU information, and graphics card information,
The first processor of the first electronic device stores the encoded video compression information received through the first communication unit of the first electronic device and the second screen display information that is a hardware device specification in a first data storage unit, The first data storage unit recognizes the IP information extracted from the second screen display information, matches the encoded video compression information with the IP information and stores the second screen display information, and receives the second screen display information. Adding and storing one frame reception time to a field of the second screen display information,
The first processor calculates a difference time between a frame start time and a frame reception time of the field of the second screen display information, and when the calculated difference time is greater than or equal to a preset real-time screen transmission time, the real-time screen transmission is possible. Changing the image quality information to the changed image quality information;
The first electronic device generates a control signal for generating a virtual machine including the IP information of the second electronic device, generates ID information of the virtual machine based on the control signal, and combines the ID information and the IP information. A used virtual machine link is generated and transmitted to the second electronic device through the first communication unit, and a first resource allocation weight is derived based on hardware information of the second electronic device;
The first processor derives a second resource allocation weight using a time difference between the frame start time and the frame reception time, and calculates a priority index based on the first resource allocation weight and the second resource allocation weight. do,
The first electronic device creates the virtual machine corresponding to the second electronic device based on resource allocation information matching the priority index, and connects an access link between the virtual machine and the second electronic device. ,
An electronic service delivery method that implements remote screen output. A computer-readable recording medium recording a program for executing an electronic service providing method for realizing remote screen output performed between a first electronic device and a second electronic device, comprising:
The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,
The electronic service providing method implementing the remote screen output,
The second electronic device detects a touch input in which a point on the screen of the second display unit of the second electronic device is touched with a pen as a pressure value, and outputs an event signal including the pressure value and the coordinate value of the point. doing;
receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating a layout screen including an object based on the calculated attribute values;
The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;
The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among resolution, bit rate, and frame rate of the image quality information, compressing and encoding a video according to the changed image quality information to the second electronic device, and transmitting a signal to be transmitted to the first electronic device. ; and
outputting, by the first electronic device, image information obtained by decoding the encoded image information based on the changed image quality information to a first display unit of the first electronic device;
The second screen display information is hardware information of the second electronic device, which includes IP information, frame start time, resolution related to picture quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, and video data as header information. It includes the size of the displayed screen, CPU information, and graphics card information,
The first processor of the first electronic device stores the encoded video compression information received through the first communication unit of the first electronic device and the second screen display information that is a hardware device specification in a first data storage unit, The first data storage unit recognizes the IP information extracted from the second screen display information, matches the encoded video compression information with the IP information and stores the second screen display information, and receives the second screen display information. Adding and storing one frame reception time to a field of the second screen display information,
The first processor calculates a difference time between a frame start time and a frame reception time of the field of the second screen display information, and when the calculated difference time is greater than or equal to a preset real-time screen transmission time, the real-time screen transmission is possible. Changing the image quality information to the changed image quality information;
The first electronic device generates a control signal for generating a virtual machine including the IP information of the second electronic device, generates ID information of the virtual machine based on the control signal, and combines the ID information and the IP information. A used virtual machine link is generated and transmitted to the second electronic device through the first communication unit, and a first resource allocation weight is derived based on hardware information of the second electronic device;
The first processor derives a second resource allocation weight using a time difference between the frame start time and the frame reception time, and calculates a priority index based on the first resource allocation weight and the second resource allocation weight. do,
The first electronic device creates the virtual machine corresponding to the second electronic device based on resource allocation information that matches the priority index, and connects an access link between the virtual machine and the second electronic device. ,
A computer-readable recording medium. A program stored in a computer-readable recording medium for executing an electronic service providing method for implementing a remote screen output performed between a first electronic device and a second electronic device,
The first electronic device has a screen output function and a data transmission/reception function, and the second electronic device performs a drawing operation and is connected to the first electronic device to transmit and receive data in both directions. has,
The electronic service providing method implementing the remote screen output,
The second electronic device detects a touch input in which a point on the screen of the second display unit of the second electronic device is touched with a pen as a pressure value, and outputs an event signal including the pressure value and the coordinate value of the point. doing;
receiving, by the second electronic device, the event signal, calculating coordinate values, shape, and size attribute values of the changing trajectory, and generating a layout screen including an object based on the calculated attribute values;
The second electronic device generates second screen display information including a second graphic card that captures the screen of the created layout and compresses and encodes a video, and pre-stored hardware information, and generates the second screen display information and the second screen display information. Transmitting encoded video compression screen information to the first electronic device;
The first electronic device analyzes the hardware information of the second electronic device included in the second screen display information to determine whether it is a low-end hardware that cannot transmit real-time screens, and transmits the screens in real time to the second screen display information. Changing at least one image quality information among resolution, bit rate, and frame rate of the image quality information, compressing and encoding a video according to the changed image quality information to the second electronic device, and transmitting a signal to be transmitted to the first electronic device. ; and
outputting, by the first electronic device, image information obtained by decoding the encoded image information based on the changed image quality information to a first display unit of the first electronic device;
The second screen display information is hardware information of the second electronic device, which includes IP information, frame start time, resolution related to picture quality, bit rate, bit rate, frame rate, encoder setting, encoding compression rate, and video data as header information. It includes the size of the displayed screen, CPU information, and graphics card information,
The first processor of the first electronic device stores the encoded video compression information received through the first communication unit of the first electronic device and the second screen display information that is a hardware device specification in a first data storage unit, The first data storage unit recognizes the IP information extracted from the second screen display information, matches the encoded video compression information with the IP information and stores the second screen display information, and receives the second screen display information. Adding and storing one frame reception time to a field of the second screen display information,
The first processor calculates a difference time between a frame start time and a frame reception time of the field of the second screen display information, and when the calculated difference time is greater than or equal to a preset real-time screen transmission time, the real-time screen transmission is possible. Changing the image quality information to the changed image quality information;
The first electronic device generates a control signal for generating a virtual machine including the IP information of the second electronic device, generates ID information of the virtual machine based on the control signal, and combines the ID information and the IP information. A used virtual machine link is generated and transmitted to the second electronic device through the first communication unit, and a first resource allocation weight is derived based on hardware information of the second electronic device;
The first processor derives a second resource allocation weight using a time difference between the frame start time and the frame reception time, and calculates a priority index based on the first resource allocation weight and the second resource allocation weight. do,
The first electronic device creates the virtual machine corresponding to the second electronic device based on resource allocation information matching the priority index, and connects an access link between the virtual machine and the second electronic device. ,
A program stored on a computer readable recording medium.

Images