KR102091137B1 - System and method for rpoviding image - Google Patents

Abstract

An image providing system and method is provided. The image providing method includes: transmitting a first video having an original resolution to an external device at a first resolution; Receiving, from the external device, area information on a region of interest of the first video displayed at the first resolution in the external device; Determining an area corresponding to the region of interest in the first video of the original resolution based on the region information; Converting a part of the first video of the original resolution into a second video; And transmitting the second video to the external device at the first resolution.

Description

Video providing system and method {SYSTEM AND METHOD FOR RPOVIDING IMAGE}

The present invention relates to an image providing system and method for providing an image including an object corresponding to a part of an image photographed on a device to another device.

In recent years, technologies for transmitting and receiving images between devices having multimedia functions have been activated, in particular, as mobile communication infrastructures have been built, and smartphones, mobile phones, PDAs (Personal Digital Assistance), etc. have become popular. Various services have been provided outside the calling function.

In addition, the video call using the device has been rapidly expanded because users on both sides can talk while facing each other directly.

In general, the transmitting device acquires a user's video for a video call, encodes the acquired video according to a predetermined video and video encoding format, and transmits the video to another device.

In addition, the receiving device receives the encoded video data from the mobile communication network, decodes it, and displays it using a display device built in the receiving device, such as a liquid crystal display (LCD). Therefore, the user of the receiving side can perform a call while viewing the image photographed by the device of the transmitting side in real time.

However, since the resolution of the display apparatus included in the devices is different, and the streaming resolution of data transmitted between devices is different according to a communication method, a part of a video captured by a transmitting device is effectively provided to a receiving device There is a need for a technology that can do it. In addition, there is a demand for a technique capable of effectively selecting and enlarging a portion of an image photographed by a transmitting device during a video call and providing it to a user.

One embodiment of the present invention provides an image providing system and method for tracking an object included in an image in which the device is being photographed, and providing some images including the tracked object to other devices.

As a technical means for achieving the above-described technical problem, a first aspect of the present invention includes: transmitting a first video to an external device based on an image being photographed; Receiving area information on a partial area of the first video from the external device; Identifying and tracking an object related to the area information from the captured image; And providing a second video including the tracked object to the external device based on the captured image.

Further, the area information is information on a partial area selected by the external device, and may be generated by the external device.

In addition, when the screen of the external device is touched, an area including an object including the touched part may be selected as the partial area.

In addition, when a screen of the external device is touched, an area in a preset range from the touched portion may be selected as the partial area.

In addition, when the screen of the external device is touched, receiving coordinate information for the touched point from the external device; including, identifying and tracking the object is based on the received coordinate information , It is possible to identify and track an object corresponding to the touched point from the captured image.

In addition, when the screen of the external device is touched, a user interface for selecting the partial area may be displayed on the screen of the external device.

In addition, the first video and the second video are videos for a video call with the external device, and the resolution of the video being captured may be greater than the streaming resolution of the video call.

Also, a range of the partial region may be determined according to the type of communication method with the external device.

Also, at least one resolution of the first video and the second video may be determined according to the type of communication method with the external device.

The method may further include changing a resolution of at least one of the first video and the second video as a communication method with the external device changes.

In addition, the method, as the communication method with the external device is changed, changing the area corresponding to the second video; And providing a video of the changed area to the external device.

In addition, the method, if the object disappears from the image being captured, changing the area corresponding to the second video; And providing a video of the changed area to the external device.

In addition, the method, when the disappeared object appears on the image being photographed, providing a video of a partial area including the object to the external device among the entire area of the image being captured; You can.

In addition, the second aspect of the present invention, receiving a first video from an external device; Providing area information on a partial area of the received first video to the external device; And receiving a second video including the object related to the provided partial area from the external device, wherein the first video and the second video are based on the video being recorded on the external device. It is provided from an external device, and the object is identified and tracked by the external device from the video being captured.

In addition, the method may include selecting the partial region based on user input; And generating the area information on the selected partial area.

Also, in the selecting of the partial area, when the screen of the external device is touched, the area including the object including the touched part may be selected as the partial area.

In addition, when the screen of the external device is touched, the selecting of the partial region may select a region of a preset range from the touched portion as the partial region.

In addition, in the step of providing the area information, when the screen of the external device is touched, the coordinate information for the touched point is provided to the external device, and the provided coordinate information includes the external device being photographed. From the image, it can be used to identify and track the object corresponding to the touched point.

The method may further include displaying a user interface for selecting the partial area when the screen of the external device is touched.

In addition, the first video and the second video are videos for a video call with the external device, and the resolution of the video being captured may be greater than the streaming resolution of the video call.

Also, a range of the partial region may be determined according to the type of communication method with the external device.

Also, according to the type of communication method with the external device, at least one resolution of the first video and the second video may be determined in the external device.

In addition, as the communication method with the external device changes, at least one resolution of the first video and the second video may be changed by the external device.

Also, as the communication method with the external device is changed, the partial area may be changed.

In addition, when the object disappears from the image being photographed, an area corresponding to the second video is changed, and the method may further include receiving a video of the changed area from the external device. .

The method may further include receiving, when the disappeared object appears in the image being captured, a video including the displayed object from the external device.

In addition, a third aspect of the present invention, a memory in which at least one program is stored; And a processor that provides a video to an external device by executing the at least one program, wherein the at least one program comprises: transmitting a first video to an external device based on the image being captured; Receiving area information on a partial area of the first video from the external device; Identifying and tracking an object related to the area information from the captured image; And providing a second video including the tracked object to the external device, based on the image being captured.

Further, the area information is information on a partial area selected by the external device, and may be generated by the external device.

In addition, when the screen of the external device is touched, an area including an object including the touched part may be selected as the partial area.

In addition, when a screen of the external device is touched, an area in a preset range from the touched portion may be selected as the partial area.

In addition, when the screen of the external device is touched, the device further includes instructions for executing; receiving coordinate information for the touched point from the external device, and identifying and tracking the object, , Based on the received coordinate information, an object corresponding to the touched point can be identified and tracked from the image being captured.

In addition, when the screen of the external device is touched, a user interface for selecting the partial area may be displayed on the screen of the external device.

In addition, the first video and the second video are videos for a video call with the external device, and the resolution of the video being captured may be greater than the streaming resolution of the video call.

Also, a range of the partial region may be determined according to the type of communication method with the external device.

Also, at least one resolution of the first video and the second video may be determined according to the type of communication method with the external device.

In addition, the device may further include instructions for executing; changing a resolution of at least one of the first video and the second video according to a change in a communication method with the external device.

In addition, the device, as the communication method with the external device is changed, changing the area corresponding to the second video; And providing a video of the changed area to the external device.

In addition, the device, if the object disappears from the image being captured, changing the area corresponding to the second video; And providing a video of the changed area to the external device.

In addition, the device, when the disappeared object appears in the image being photographed, providing a video of a portion of the entire area of the image being captured, including the object, to the external device. It may further include.

In addition, a fourth aspect of the present invention, a memory in which at least one program is stored; And a processor configured to receive a video from an external device by executing the at least one program, wherein the at least one program comprises: receiving a first video from the external device; Providing area information on a partial area of the received first video to the external device; And receiving, from the external device, a second video including an object related to the provided partial region, wherein the first video and the second video are videos being recorded on the external device. Based on the provided from the external device, the object can be provided that the device, which is identified and tracked by the external device from the video being captured.

Also, the device may include selecting the partial area based on a user input; And generating the region information on the selected partial region.

Also, in the selecting of the partial area, when the screen of the external device is touched, the area including the object including the touched part may be selected as the partial area.

In addition, when the screen of the external device is touched, the selecting of the partial region may select a region of a preset range from the touched portion as the partial region.

In addition, in the step of providing the area information, when the screen of the external device is touched, the coordinate information for the touched point is provided to the external device, and the provided coordinate information includes the external device being photographed. From the image, it can be used to identify and track the object corresponding to the touched point.

Further, the device may further include instructions to execute; when a screen of the external device is touched, displaying a user interface for selecting the partial area.

In addition, the first video and the second video are videos for a video call with the external device, and the resolution of the video being captured may be greater than the streaming resolution of the video call.

Also, a range of the partial region may be determined according to the type of communication method with the external device.

Also, according to the type of communication method with the external device, at least one resolution of the first video and the second video may be determined in the external device.

In addition, as the communication method with the external device changes, at least one resolution of the first video and the second video may be changed by the external device.

Also, as the communication method with the external device is changed, the partial area may be changed.

In addition, when the object disappears from the image being captured, an area corresponding to the second video is changed, and the device receives the video about the changed area from the external device. It can contain.

In addition, the device may further include instructions for executing; when the disappeared object appears on the image being photographed, receiving a video including the displayed object from the external device.

1 is an overall schematic diagram of an image providing system according to an embodiment of the present invention.
2 is a diagram illustrating an example in which an image including an object corresponding to a part of an image captured by the first device 100 according to an embodiment of the present invention is displayed on the second device 200.
3 is a flowchart of a method of providing a video including an object corresponding to a partial region of a video being photographed by the first device 100 to the second device 200 according to an embodiment of the present invention.
4 is a flowchart of a method in which the second device 200 according to an embodiment of the present invention generates area information regarding a region of interest.
FIG. 5 shows a video of the changed region of interest and the first device 100 changing the selected region of interest when the communication method between the first device 100 and the second device 200 is changed according to an embodiment of the present invention. Is a flowchart of a method for providing the second device 200.
6 is a first device 100 according to an embodiment of the present invention when the communication method between the second device 200 is changed, the first device 100 changes the video of the resolution of the second device 200 Is a flowchart of how to provide it.
7 is a flowchart of a method of providing a video including an object corresponding to a partial region of a video being photographed by the first device 100 to the second device 200 according to another embodiment of the present invention.
FIG. 8 is a diagram illustrating an example in which the second device 200 determines a partial region of the first video as a region of interest so that an object including a user's touch point is included according to an embodiment of the present invention.
9 is a diagram illustrating an example in which the second device 200 according to an embodiment of the present invention determines a partial region of a preset range as a region of interest from a user's touch point.
FIG. 10 is a diagram illustrating an example in which the second device 200 determines a partial region of the first video as a region of interest by using a user interface for determining a partial region according to an embodiment of the present invention.
FIG. 11 illustrates a case in which the resolution of the second device 200 is changed from the first device 100 when the communication method between the first device 100 and the second device 200 is changed according to an embodiment of the present invention. A diagram showing an example of receiving a video.
FIG. 12 shows that when the communication method between the first device 100 and the second device 200 is changed according to an embodiment of the present invention, the second device 200 changes the image quality from the first device 100. A diagram showing an example of receiving a video.
FIG. 13 shows a partial area in which the second device 200 is changed from the first device 100 when the communication method between the first device 100 and the second device 200 is changed according to an embodiment of the present invention. It is a diagram showing an example of receiving a video for.
14 is a block diagram of a first device 100 according to an embodiment of the present invention.
15 is a block diagram of a second device 200 according to an embodiment of the present invention.
16 is a block diagram of a device according to another embodiment of the present invention.
17 is a diagram illustrating an example of enlarging a partial area of a screen of a first video displayed on the second device 200 according to an embodiment of the present invention.
18 is a diagram illustrating an example in which a screen of a video displayed on the second device 200 according to an embodiment of the present invention is enlarged step by step.
19 is an example in which a second video provided from the first device 100 is displayed after a preview image in which a partial area of the first video is enlarged is displayed on the second device 200 according to an embodiment of the present invention. It is a diagram showing.
20 is a diagram illustrating an example of a Real-time Transfer Protocol (RTP) packet header according to an embodiment of the present invention.
21 is a flowchart of a method for a first device 100 to generate a fourth image according to an embodiment of the present invention.
22 is a diagram of a method for a second device 200 having a video call with the first device 100 to receive a second expanded video with respect to a partial region of the first video according to an embodiment of the present invention. It is a flow chart.
23 is a flowchart of a method in which a second device 200 making a video call with the first device 100 enlarges and displays a partial area of the first video according to an embodiment of the present invention.
24 is a diagram illustrating an example in which an enlarged image of a partial region selected from a first video is displayed on a screen of the second device 200 together with the first video according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art to which the present invention pertains can easily practice. However, the present invention can be implemented in many different forms and is not limited to the embodiments described herein. In addition, in order to clearly describe the present invention in the drawings, parts irrelevant to the description are omitted, and like reference numerals are assigned to similar parts throughout the specification.

Throughout the specification, when a part is "connected" to another part, this includes not only "directly connected" but also "electrically connected" with another element in between. . Also, when a part “includes” a certain component, this means that other components may be further included rather than excluding other components, unless otherwise specified.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is an overall schematic diagram of an image providing system according to an embodiment of the present invention.

Referring to FIG. 1, the first device 100 and the second device 200 can make a video call with each other, and the first device 100 can provide the video being captured to the second device 200. have. In addition, the second device 200 may also provide the first device 100 with the video being captured.

Hereinafter, for convenience of description, a case in which the first device 100 provides a video to the second device 200 in the image providing system according to an embodiment of the present invention will be described. In addition, hereinafter, the content provided by the first device 100 to the second device 200 is described as being a video filmed by the first device 100, but is not limited thereto, and the first device 100 is For example, content such as a video image, a photographic image, a captured image, and an application execution image may be provided to the 2 devices 200.

The first device 100 may provide the first video to the second device 200 based on the video captured by the first device 100, and provide area information about an object included in the first video. It can be received from the second device 200. In addition, the first device 100 tracks a predetermined object included in the first video based on the received area information, and the second device 200 includes a second video including only a partial area including the tracked object. Can provide.

In addition, when the resolution of the video being photographed is greater than the streaming resolution of the video transmitted between the first device 100 and the second device 200, the first device 100 may be displayed in a partial area of the video being photographed. A second video for the second device 200 may be provided.

The first device 100 and the second device 200 are devices that can transmit videos, images, and the like to other devices, for example, a smart phone, a mobile phone, a personal digital assistant (PDA), a laptop, a media player, Global positioning system (GPS) devices, laptops, and other mobile or non-mobile computing devices, but are not limited thereto.

Also, three or more devices may make a video call together. For example, the first device 100, the second device 200, and the third device (not shown) may make a video call together. In this case, the first device 100 may provide the first video to the second device 200 and the third device (not shown). In addition, when the second device 200 selects some areas of the first video, the first device 100 may provide the second device 200 and the third device (not shown) related to the selected area. You can. If the second device 200 and the third device (not shown) respectively select some areas of the first video, the first device 100 removes the video related to the area selected by the second device 200. Provided to the 2 device 200, a video related to the area selected by the third device (not shown) may be provided to the third device (not shown).

Meanwhile, the first device 100 may provide the second device 200 with a third image corresponding to an execution screen of an application executed in the first device 100. In this case, the second device 200 may select a portion of the received third image, and receive a fourth image for the selected portion from the first device 100.

2 is a diagram illustrating an example in which an image including an object corresponding to a part of an image captured by the first device 100 according to an embodiment of the present invention is displayed on the second device 200. 2 (a) is a diagram showing an example of a first video according to an embodiment of the present invention, and FIG. 2 (b) is a diagram showing a second video according to an embodiment of the present invention.

The first device 100 may provide the first video to the second device 200 based on the video being photographed, and may identify the object 22 included in the first video. Also, the first device 100 may provide the second device 200 with a second video including only a partial area including the identified object 22.

The first device 100 may check the object 22 included in the predetermined partial area 20 from the video being photographed, but is not limited thereto. The first device 100 may check the object 22 from the first video provided to the second device 200.

3 is a flowchart of a method of providing a video including an object corresponding to a partial region of a video being photographed by the first device 100 to the second device 200 according to an embodiment of the present invention.

In step S300, the first device 100 provides the first video to the second device 200. The first device 100 may provide the first video to the second device 200 based on the video being photographed. The first device 100 may provide the second device 200 as a first video without processing the video being photographed. Also, the first device 100 may process the video being photographed and provide the processed video as the first video to the second device 200. In this case, the first device 100 may change the resolution of the video being photographed or convert the format of the video being photographed. For example, when the resolution of the video being shot is greater than the streaming resolution between the first device 100 and the second device 200, the first device 100 sets the resolution of the video being captured to be equal to the streaming resolution. The first video of the converted resolution may be provided to the second device 200. In addition, the first video may have a plurality of first frames.

In addition, in step S300, the first device 100 may transmit the first video having the original resolution to the second device 200 in the first resolution. In this case, the first device 100 may obtain and store information regarding the resolution of the display included in the second device 200 in advance, and may store the information based on the resolution of the display included in the second device 200. Resolution can be determined.

Meanwhile, the first device 100 may provide the second device 200 with a third image corresponding to an execution screen of an application executed in the first device 100. The first device 100 may provide a third image to the second device 200 in a third resolution.

In step S310, the second device 200 displays the received first video. The second device 200 may display the received first video on the screen of the second device 200. The second device 200 may display the first video of the first resolution on the screen.

Meanwhile, when the second device 200 receives the third image corresponding to the execution screen of the application executed in the first device 100, the second device 200 may display the received third image. have. The second device 200 may display the third image at the third resolution.

In step S320, the second device 200 generates area information about the object included in the first video. The second device 200 may select a region of the first video as the region of interest based on the user input and generate region information regarding the selected region of interest. Also, the second device 200 may select a predetermined object included in the first video based on the user input and generate area information on the selected object.

The second device 200 may receive a user's touch input for the first video and select a region of interest based on the location of the touch input point. Specifically, when the user touches a predetermined point in the first video, the second device 200 may identify an object including the touched point and select an area including the identified object. For example, when the face portion is touched in the first video, the second device 200 may identify a face that is an object including the touched point. Also, the second device 200 may select an area including the identified face as an area of interest.

In addition, when the user touches a predetermined point in the first video, the second device 200 may select an area of a preset range from the touched point as the region of interest. For example, the second device 200 may select a rectangular area of a predetermined size centered on the touched point as the region of interest.

Also, the second device 200 may display a user interface on the screen for the user to select a region of the first video as a region of interest. For example, the second device 200 may display a rectangle on a portion of the first video, and provide a user interface so that the user can change the size of the rectangle by touching and dragging the border of the rectangle.

In addition, although the second device 200 has been described as selecting a region of interest as a region of interest based on a user input, it is not limited thereto. The second device 200 may recognize a face included in the first video without a separate user input and select a predetermined area including the face as a region of interest.

Also, the size of the region of interest selected from the first video may be determined based on the size of streaming resolution of data transmitted between the first device 100 and the second device 200. Based on the size of the streaming resolution of data transmitted between the first device 100 and the second device 200, an upper limit value of the size of the region of interest selected in the first video may be determined.

Also, the second device 200 may generate coordinate information for the selected region of interest. The second device 200 may generate a coordinate value indicating the location of a region of interest selected from all regions of the first video. Also, the second device 200 may generate a coordinate value indicating the location of the touch input point in the entire area of the first video.

Also, the second device 200 may generate area information regarding a region of interest for a plurality of first frames included in the first video.

On the other hand, when the third device corresponding to the execution screen of the application executed in the first device 100 is provided to the second device 200, the second device 200 selects a part of the third image, Area information for the selected area may be generated.

In step S330, the second device 200 provides the generated area information to the first device 100. The second device 200 may provide coordinate values for the selected region of interest to the first device 100. Also, the second device 200 may provide the first device 100 with the coordinate value of the point touched by the user.

In step S340, the first device 100 identifies an object related to area information. In step S340, the first device 100 may identify the object from the first video based on the received region information. For example, when an object included in an area of the first video corresponding to the received area information is a face, the first device 100 may identify the face from the first video. In addition, for each of the first frames included in the first video, an object related to the received region information may be identified.

Also, the first device 100 may identify an object from a video that is being photographed based on the received region information. Since the received area information is area information for a partial area of the first video, the first device 100 may check an area matching the area of the first video corresponding to the received area information from the video being captured. In addition, an object included in the matched area can be identified from a video being photographed.

Also, the first device 100 may identify an object included in a video being photographed using an object identified from the first video. The first device 100 may identify an object similar to the image of the object identified from the first video from the video being photographed. Whether objects are similar to each other can be determined in various ways. For example, the device 100 may determine whether the objects are similar to each other by comparing at least one of the size, color, and shape of the object. In another example, the device 100 may determine whether the objects are similar to each other by comparing a portion of the objects with each other.

If, when the first device 100 receives the area information for a portion of the third image corresponding to the execution screen of the application executed in the first device 100 from the second device 200, the first device The 100 may identify an object from the third image based on the received area information.

In step S350, the first device 100 tracks the identified object. The first device 100 may track the object identified from the first video. Also, the first device 100 may track the object identified from the video captured by the first device 100. The video captured by the first device 100 includes the video being photographed by the first device 100. The identified object may move or rotate within the video, and the first device 100 may track the object in the video based on the shape and color of the object. Tracking the identified object means to identify the location of the moving object and the area including the object when the object is moving in the video, but is not limited thereto.

In addition, in step S350, the first device 100 may select a frame including the identified object from among the plurality of first frames of the first video. Also, the first device 100 may distinguish a partial region including the identified object from among the plurality of first frames of the first video.

If, when the first device 100 receives the area information for a portion of the third image corresponding to the execution screen of the application executed in the first device 100 from the second device 200, the first device The 100 may track an object from the third image.

In step S360, the first device 100 provides the second device 200 with a second video including only a partial area including the identified object. The first device 100 may cut an area including the identified object from the captured video, and provide the cut video as the second video to the second device 200.

Specifically, the first device 100 may determine an area corresponding to the region of interest on the screen of the first video of the original resolution based on the received region information. Also, the first device 100 may convert the first video to the second video to include only the determined area. Also, the first device 100 may provide the second video to the second device 200 at the first resolution. For example, when the original resolution is 1000 x 1000 and the first resolution is 100 x 100, the first device 100 converts a portion of the first video corresponding to the determined region into a second video, and the second The video may be transmitted to the second device 200 at a resolution of 100 x 100.

Further, the first device 100 distinguishes some regions including the identified object among the plurality of first frames of the first video, and has a plurality of second frames including only a partial region of the plurality of first frames The second video may be provided to an external device.

If the first device 100 selects a frame including the identified object from among the plurality of first frames of the first video, the first device 100 may include a plurality of second frames including only a partial region of the selected frame. 2 Video can be provided to external devices. In this case, some regions may be regions of interest. Also, the first device 100 may generate a second frame by cutting an area corresponding to the region of interest for each selected frame.

Also, according to a communication method between the first device 100 and the second device 200, the first device 100 may determine the size of an area corresponding to the second video.

Also, the second video may have a plurality of second frames including only a partial region of the frame selected in step S350, but is not limited thereto. In addition, the second video may have a plurality of second frames including only a partial region distinguished in step S350, but is not limited thereto.

Also, according to a communication method between the first device 100 and the second device 200, the first device 100 may determine the image quality or resolution of the second video. For example, when the resolution of the area including the identified object is smaller than the streaming resolution of data transmitted between the first device 100 and the second device 200, the resolution of the area including the identified object or You can increase the image quality. Further, for example, when the resolution of the area including the identified object is greater than the streaming resolution of data transmitted between the first device 100 and the second device 200, the area of the area including the identified object Resolution or image quality can be reduced.

The communication method between the first device 100 and the second device 200 may include, for example, 3G (third generation) communication, 4G (fourth generation) communication, and Wifi (wireless fidelity) communication. Does not work.

Also, as the communication method between the first device 100 and the second device 200 changes, the first device 100 may change the size of an area corresponding to the second video or the quality of the second video. The communication method between the first device 100 and the second device 200 may be changed as at least one of the first device 100 and the second device 200 performs handover.

In addition, when the identified object disappears (ie, the identified object is not captured) in the captured video, the first device 100 may change the area corresponding to the second video. For example, when the object identified in the captured video disappears, the first device 100 may provide the entire area of the captured video to the second device 200. In addition, in this case, the first device 100 may provide the entire area of the captured video to the second device 200 based on a separate user input.

In addition, when the disappeared object appears again on the photographed image (ie, when the disappeared object is photographed again), a video of a portion of the entire region of the photographed image including the displayed object is provided to the second device 200. You can.

On the other hand, when the first device 100 receives the area information for a portion of the third image corresponding to the execution screen of the application executed in the first device 100 from the second device 200, the first device The 100 may provide the second device 200 with a fourth image including only a partial area including the identified object. In this case, the first device 100 may provide the fourth image with the third resolution to the second device 200.

In addition, when the first device 100 receives the area information for a portion of the third image corresponding to the execution screen of the application executed in the first device 100 from the second device 200, the third image Objects included in may not be identified or tracked. In this case, the first device 100 may cut a portion of the third image based on the received region information, convert the cut portion to the fourth image, and provide the second device 200.

In step S370, the second device 200 displays the received second video. The user of the second device 200 can check a more detailed video of the object included in the selected partial area. Also, the second device 200 may display a still image of the received second video.

Meanwhile, the second device 200 may display the first video and the second video together on the screen of the second device 200. In this case, in step S360, the second device 200 may receive the first video as well as the second video. Also, the second device 200 may overlap and display the second video on the first video. The second device 200 may overlap the second video in a portion where the user of the first device 100 is not displayed in the first video. Also, the second device 200 may adjust the size of the area where the first video is displayed and the size of the area where the second video is displayed. However, the present invention is not limited thereto, and the second device 200 may display the first video and the second video on the screen of the second device 200 without overlapping.

On the other hand, in the above, it has been described that the second device 200 can receive the second video from the first device 100 and display the first video and the second video together, but is not limited thereto. The second device 200 selects a partial region of the first video, extracts an enlarged image of the selected partial region from the first video, and extracts the enlarged image together with the first video, and the screen of the second device 200 You can also display on.

4 is a flowchart of a method in which the second device 200 according to an embodiment of the present invention generates area information regarding a region of interest.

In step S322, the second device 200 receives a user input for selecting a partial area of the first video. The second device 200 may receive a user's touch input for selecting a partial area of the first video. For example, the second device 200 may receive a user input touching a predetermined point of the first video or a user input touching and dragging a predetermined point of the first video.

Also, the second device 200 may receive a user's touch input through a predetermined user interface displayed on the screen. For example, the second device 200 may display a rectangle on a portion of the first video, and provide a user interface so that the user can change the size of the rectangle by touching and dragging the border of the rectangle. Also, the second device 200 may receive a user's touch input through a provided user interface.

In step S324, the second device 200 confirms a communication method with the first device 100. The second device 200 may check the type and bandwidth of the communication method with the first device 100. Also, the second device 200 may check streaming resolution of data transmitted between the first device 100 and the second device 200. The communication method between the first device 100 and the second device 200 may include, for example, 3G (third generation) communication, 4G (fourth generation) communication, and Wifi (wireless fidelrity) communication, but is not limited thereto. Does not work.

In step S326, the second device 200 selects a partial region of the first video as the region of interest based on the received user input and the identified communication method. The second device 200 may select a partial area based on the location of the touch input point in the first video. Specifically, when the user touches a predetermined point in the first video, the second device 200 may identify an object including the touched point and select an area including the identified object. For example, when the face portion is touched in the first video, the second device 200 may identify a face that is an object including the touched point. Also, the second device 200 may select an area including the identified face as an area of interest.

In addition, when the user touches a predetermined point in the first video, the second device 200 may select an area of a preset range from the touched point as the region of interest. For example, the second device 200 may select a rectangular area of a predetermined size centered on the touched point as the region of interest.

Also, the second device 200 may change the selected region of interest according to a user's touch input through the user interface.

In addition, the size of the region of interest selected from the first video may be determined according to a communication method between the first device 100 and the second device 200. For example, the size of the region of interest selected from the first video may be determined based on at least one of a type of communication method and a bandwidth. Further, for example, the size of the region of interest selected from the first video may be determined based on the size of streaming resolution of data transmitted between the first device 100 and the second device 200. Also, according to a communication method between the first device 100 and the second device 200, an upper limit value of the size of the region of interest selected from the first video may be determined.

In FIG. 4, the second device 200 is described as selecting a partial region of the first video based on the user input, but is not limited thereto. The second device 200 may recognize a face included in the first video without a separate user input and select a predetermined area including the face as a region of interest. Also, in this case, according to a communication method between the first device 100 and the second device 200, the size of the selected region of interest may be determined.

FIG. 5 shows a video of the changed region of interest and the first device 100 changing the selected region of interest when the communication method between the first device 100 and the second device 200 is changed according to an embodiment of the present invention. Is a flowchart of a method for providing the second device 200.

In step S500, the first device 100 determines whether a communication method with the second device has been changed. Specifically, the first device 100 may check whether the type and bandwidth of the communication method with the second device 200 has been changed. Also, the second device 200 may check whether the streaming resolution of data transmitted between the first device 100 and the second device 200 is changed.

The communication method between the first device 100 and the second device 200 may include, for example, 3G (third generation) communication, 4G (fourth generation) communication, and Wifi (wireless fidelrity) communication, but is not limited thereto. Does not work. Also, a communication method between the first device 100 and the second device 200 may be changed as at least one of the first device 100 and the second device 200 performs handover.

As a result of the determination in step S500, when the communication method is changed, in step S510, the first device 100 changes the area corresponding to the second video being transmitted to the second device 200. As the communication method between the first device 100 and the second device 200 changes, the first device 100 may change the size of the area corresponding to the second video.

For example, when the type of communication method between the first device 100 and the second device 200 is changed from 3G communication to WIFI communication, an area larger than the area corresponding to the second video may be selected. When the first device 100 transmits an area of the face part when the type of communication method between the first device 100 and the second device 200 is 3G communication, the first device 100 and the second device If the type of the communication method between 200 is changed to WIFI communication, the first device 100 may select some areas to further include a background portion other than the face. Also, for example, when the type of the communication method between the first device 100 and the second device 200 is changed from 4G communication to WIFI communication, an area smaller than the area corresponding to the second video may be selected.

Also, for example, when a streaming resolution of data transmitted between the first device 100 and the second device 200 is changed, the size of the area corresponding to the second video may be changed.

The size value of the region, which corresponds to at least one of the type of communication method and the bandwidth, may be preset and stored in the first device 100. In addition, the size value of the region, which corresponds to the streaming resolution of data transmitted between the first device 100 and the second device 200, may be preset and stored in the first device 100.

In step S520, the first device 100 provides a video for the changed area to the second device 200. For example, when the type of communication method between the first device 100 and the second device 200 is changed from 3G communication to WIFI communication, the first device 100 is larger than an area corresponding to the second video. The video about the area may be provided to the second device 200. Also, for example, when the size of the streaming resolution of data transmitted between the first device 100 and the second device 200 is reduced, the first device 100 is smaller than an area corresponding to the second video. The video about the area may be provided to the second device 200.

6 is a first device 100 according to an embodiment of the present invention, when the communication method between the second device 200 is changed, the first device 100 changes the video of the changed image quality to the second device 200 Is a flowchart of how to provide it.

In step S600, the first device 100 determines whether a communication method with the second device has been changed. Specifically, the first device 100 may check whether the type and bandwidth of the communication method with the second device 200 has been changed. Also, the second device 200 may check whether the streaming resolution of data transmitted between the first device 100 and the second device 200 is changed.

As a result of the determination in step S600, when the communication method is changed, in step S610, the first device 100 changes the image quality of the second video being transmitted to the second device 200. As the communication method between the first device 100 and the second device 200 changes, the first device 100 may change the quality of the second video to high or low quality.

For example, when the type of communication method between the first device 100 and the second device 200 is changed from 3G communication to WIFI communication, the quality of the second video may be changed to a higher quality than that of 3G communication. . Also, for example, when the type of the communication method between the first device 100 and the second device 200 is changed from 4G communication to WIFI communication, the quality of the second video is changed to a lower quality than that of 4G communication. You can.

Also, for example, when the streaming resolution of data transmitted between the first device 100 and the second device 200 is changed, the quality of the second video may be changed.

An image quality value corresponding to at least one of a type of communication method and a bandwidth may be preset and stored in the first device 100. Also, an image quality corresponding to a streaming resolution of data transmitted between the first device 100 and the second device 200 may be preset and stored in the first device 100.

In step S620, the first device 100 provides the video of the changed image quality to the second device 200. The first device 100 may provide a video of a quality suitable for the changed image quality to the second device 200.

7 is a flowchart of a method of providing a video including an object corresponding to a partial region of a video being photographed by the first device 100 to the second device 200 according to another embodiment of the present invention. In FIG. 7, the first device 100 may receive coordinate information for a predetermined point of the first video from the second device 200 and select some regions as regions of interest using the received coordinate information.

In step S700, the first device 100 provides the first video to the second device 200. The first device 100 may provide the first video to the second device 200 based on the captured video. The first device 100 may provide the captured video as the first video without any additional processing to the second device 200. Also, the first device 100 may process the captured video and provide the processed video as the first video to the second device 200. In this case, the first device 100 may change the resolution of the captured video or convert it into a predetermined format. For example, when the resolution of the captured video is greater than the streaming resolution between the first device 100 and the second device 200, the first device 100 converts the resolution of the captured video to be the same as the streaming resolution. Then, the first video of the converted resolution may be provided to the second device 200.

Also, in step S700, the first device 100 may transmit the first video having the original resolution to the external device in the first resolution. In this case, the first device 100 may obtain and store information regarding the resolution of the display included in the second device 200 in advance, and may store the information based on the resolution of the display included in the second device 200. Resolution can be determined.

In step S710, the second device 200 displays the received first video. The second device 200 may display the received first video on the screen of the second device 200. The second device 200 may display the first video of the first resolution on the screen.

In step S720, the second device 200 receives a user's touch input for the first video displayed on the screen. The second device 200 may receive a user's touch input selecting a point of the first video displayed on the screen. Also, the second device 200 may generate coordinate information about the touched point.

In step S730, the second device 200 provides coordinate information for the touched point to the first device 100.

In operation S740, the first device 100 determines a partial region of the first video to be provided to the second device 200 as the region of interest based on the received coordinate information. The first device 100 may select some regions based on the location of the touch input point. Specifically, the first device 100 may identify an object including the touched point and select an area including the identified object. For example, when the face portion is touched in the first video, the first device 100 may identify a face that is an object including the touched point. Also, the first device 100 may select an area including the identified face as an area of interest.

Also, the first device 100 may select an area of a preset range from the touched point as the region of interest. For example, the first device 100 may select a rectangular area of a predetermined size centered on the touched point as the region of interest.

In step S750, the first device 100 identifies the object included in the determined partial area. For example, when the first device 100 selects an area of a predetermined range from a touched point as an area of interest, the first device 100 may identify an object included in the selected area from the first video. .

Also, the first device 100 may identify an object from the captured video based on the determined region of interest. Since the determined region of interest is a partial region of the first video, the first device 100 may check an area matching the determined region of interest of the first video from the video being photographed. In addition, an object included in the matched area can be identified from a video being photographed.

Also, the first device 100 may identify an object included in a video being photographed using an object identified from the first video. The first device 100 may identify an object similar to an image of the object identified from the first video or more than a preset value from the video being photographed.

In step S760, the first device 100 tracks the identified object. The first device 100 may track the object identified from the first video. Also, the first device may track the object identified from the captured video.

In step S770, the first device 100 provides a second video including the tracked object to the second device 200. The first device 100 may cut an area including the identified object from the captured video, and provide the cut video as the second video to the second device 200.

Specifically, the first device 100 may determine an area corresponding to the region of interest on the screen of the first video of the original resolution based on the received region information. Also, the first device 100 may convert the first video to the second video to include only the determined area. Also, the first device 100 may provide the second video to the second device 200 at the first resolution. For example, when the original resolution is 1000 x 1000 and the first resolution is 100 x 100, the first device 100 converts a portion of the first video corresponding to the determined region into a second video, and the second The video may be transmitted to the second device 200 at a resolution of 100 x 100.

Also, according to a communication method between the first device 100 and the second device 200, the first device 100 may determine the size of an area corresponding to the second video. Also, according to a communication method between the first device 100 and the second device 200, the first device 100 may determine the quality of the second video. The communication method between the first device 100 and the second device 200 may include, for example, 3G (third generation) communication, 4G (fourth generation) communication, and Wifi (wireless fidelrity) communication, but is not limited thereto. Does not work.

Also, as the communication method between the first device 100 and the second device 200 changes, the first device 100 may change the size of an area corresponding to the second video or the quality of the second video. The communication method between the first device 100 and the second device 200 may be changed as at least one of the first device 100 and the second device 200 performs handover.

In step S780, the second device 200 displays the received second video. The user of the second device 200 can check a more detailed video of the object included in the selected partial area.

Meanwhile, the second device 200 may display the first video and the second video together on the screen of the second device 200. In this case, in step S770, the second device 200 may receive the first video as well as the second video. Also, the second device 200 may overlap and display the second video on the first video. The second device 200 may overlap the second video in a portion where the user of the first device 100 is not displayed in the first video. Also, the second device 200 may adjust the size of the area where the first video is displayed and the size of the area where the second video is displayed. However, the present invention is not limited thereto, and the second device 200 may display the first video and the second video on the screen of the second device 200 without overlapping.

FIG. 8 is a diagram illustrating an example in which the second device 200 determines a partial region of the first video as a region of interest so that an object including a user's touch point is included according to an embodiment of the present invention.

Referring to FIG. 8, when a user touches a predetermined point 80 in the first video displayed on the screen of the second device 200, the second device 200 includes the touched point 80 The region 84 including the face 82 may be determined as a partial region of the first video.

Specifically, when the user touches a predetermined point 80 in the first video displayed on the screen of the second device 200, the second device 200 faces the face 82 including the touched point 80 ) Can be identified. Also, the second device 200 may select an area 84 including the identified face 82. In this case, the size of the region 84 may be determined based on a communication method between the first device 100 and the second device 200.

Further, for example, when the user touches a predetermined menu displayed on the screen of the second device 200, a predetermined box is displayed on the screen of the second device 200, and the user moves the displayed box or the box The region of interest may be selected by adjusting the size of. In this case, a separate user interface for moving the box or adjusting the size of the box may be displayed on the screen of the second device 200.

9 is a diagram illustrating an example in which the second device 200 according to an embodiment of the present invention determines a partial region of a preset range as a region of interest from a user's touch point.

Referring to FIG. 9, when a user touches a predetermined point 90 in a first video displayed on the screen of the second device 200, a rectangle of a predetermined size centered on the touched point 90 The region 92 may be determined as a partial region of the first video. In this case, the size of the region 92 may be determined based on a communication method between the first device 100 and the second device 200.

Meanwhile, the second device 200 may determine the region of interest based on the user's unpinching input. When the user enlarges a predetermined portion of the second video displayed on the screen of the second device 200 through unpinching and holds the touched finger for a predetermined time or more, the second device 200 enlarges by the user's unpinching input. The selected area can be determined as an area of interest. Further, the second device 200 may request the second device corresponding to the determined region of interest from the first device 100.

FIG. 10 is a diagram illustrating an example in which the second device 200 determines a partial region of the first video as a region of interest by using a user interface for determining a partial region according to an embodiment of the present invention.

Referring to FIG. 10, the second device 200 may display a guide line 10 for selecting a partial region of the first video on the screen. In addition, when the user touches and drags a part of the guide line 10, the second device 200 may change the size of the guide line 10. Also, the second device 200 may determine an area included in the changed guideline 10 as an area of interest of the first video. In this case, an upper limit value of the size of the area included in the guide line 10 may be determined based on a communication method between the first device 100 and the second device 200.

FIG. 11 illustrates a case in which the resolution of the second device 200 is changed from the first device 100 when the communication method between the first device 100 and the second device 200 is changed according to an embodiment of the present invention. A diagram showing an example of receiving a video. 11 (a) is a diagram illustrating an example of a video displayed on the second device 200 before the communication method is changed, and FIG. 11 (b) is displayed on the second device 200 after the communication method is changed. A diagram showing an example of an old video.

The communication method between the first device 100 and the second device 200 is changed, and the streaming resolution of data transmitted between the first device 100 and the second device 200 may be reduced. In this case, the second device 200 may receive a video having a lower resolution than the video before the communication method is changed from the first device 100. Also, the second device 200 may display a video received after the communication method is changed to a smaller size than the video before the communication method is changed.

FIG. 12 shows that when the communication method between the first device 100 and the second device 200 is changed according to an embodiment of the present invention, the second device 200 changes the image quality from the first device 100. A diagram showing an example of receiving a video. 12 (a) is a diagram showing an example of a video displayed on the second device 200 before the communication method is changed, and FIG. 12 (b) is displayed on the second device 200 after the communication method is changed. A diagram showing an example of an old video.

The communication method between the first device 100 and the second device 200 is changed, and the streaming resolution of data transmitted between the first device 100 and the second device 200 may be reduced. In this case, the second device 200 may receive a video of a lower quality than the video before the communication method is changed from the first device 100. Also, the second device 200 may display a video received after the communication method is changed in the same size as the video before the communication method is changed.

FIG. 13 shows a partial area in which the second device 200 is changed from the first device 100 when the communication method between the first device 100 and the second device 200 is changed according to an embodiment of the present invention. It is a diagram showing an example of receiving a video for. 13 (a) is a diagram showing an example of a video displayed on the second device 200 before the communication method is changed, and FIG. 13 (b) is displayed on the second device 200 after the communication method is changed. A diagram showing an example of an old video.

The communication method between the first device 100 and the second device 200 is changed, and the streaming resolution of data transmitted between the first device 100 and the second device 200 may be increased. In this case, the second device 200 may receive a video on a wider area than the video before the communication method is changed from the first device 100.

14 is a block diagram of a first device 100 according to an embodiment of the present invention.

As shown in FIG. 14, the first device 100 according to an embodiment of the present invention includes an image acquisition unit 110, an image providing unit 120, an object identification unit 130, and an object tracking unit 140 , A network confirmation unit 150, a frame buffer 190, a DB 160, a transmission / reception unit 170, and a control unit 180.

The image acquisition unit 110 acquires an image to be provided to the second device 200. The image acquisition unit 110 may acquire a video captured through a photographing apparatus or an image sensor included in the first device 100, but is not limited thereto. The image acquisition unit 110 may acquire content such as a photo and a captured image.

When a third image corresponding to the execution screen of the application executed in the first device 100 is provided to the second device 200, the image acquisition unit 110 displays the application execution screen from the frame buffer 190. A corresponding third image can be obtained. However, the present invention is not limited thereto, and the image acquisition unit 110 may extract an image corresponding to the execution screen of the application from the frame buffer 190 to be described later, and process the extracted image to generate a third image.

The image providing unit 120 provides the acquired image to the second device 200. The image providing unit 120 may process the acquired image and provide it to the second device 200, but is not limited thereto. The image providing unit 120 may provide the obtained image to the second device 200 without additional processing.

Also, the image providing unit 120 may provide the first video and the second video to the second device 200 based on the video captured by the first device 100. The image providing unit 120 may provide the captured video as the first video without any additional processing to the second device 200. Also, the image providing unit 120 may process the captured video and provide the processed video as the first video to the second device 200. In this case, the image providing unit 120 may change the resolution or image quality of the captured video or convert the video format.

Also, the image providing unit 120 may transmit the first video having the original resolution to the second device 200 at the first resolution. In this case, the first device 100 may obtain and store information regarding the resolution of the display included in the second device 200 in advance, and may store the information based on the resolution of the display included in the second device 200. Resolution can be determined.

Also, the image providing unit 120 may provide the second device 200 with a second video including the object tracked by the object tracking unit 140 to be described later. The second video may include only some areas including the tracked object. In addition, when the communication method between the first device 100 and the second device 200 is changed, the image providing unit 120 changes the resolution or image quality of the second video and changes the video to the second device 200. Can provide. In addition, when the communication method between the first device 100 and the second device 200 is changed, the image providing unit 120 changes the size of some areas corresponding to the second video and videos for the changed some areas Can be provided to the second device 200.

Meanwhile, the image providing unit 120 may provide the second device 200 with a third image corresponding to the execution screen of the application executed in the first device 100. In addition, when the second device 200 selects a portion of the third image received from the first device 100 and requests an image for the selected portion, the first device 100 is a fourth corresponding to the selected portion An image may be provided to the second device 200. In this case, the fourth image may be an image generated by cutting a portion selected from the third image.

The object identification unit 130 identifies an object from the acquired image. The object identification unit 130 may receive area information on a partial region of the first video from the second device 200 and identify an object from the captured video or the first video.

The object tracking unit 140 tracks an object from the acquired image. The object tracking unit 140 may track the identified object in the first video or the captured video.

The network confirmation unit 150 checks a communication method between the first device 100 and the second device 200. The network confirmation unit 150 may check the type and bandwidth of a communication method between the first device 100 and the second device 200. In addition, the network confirmation unit 150 may check streaming resolution of data transmitted between the first device 100 and the second device 200.

The frame buffer 190 stores an image corresponding to an execution screen of an application executed in the first device 100.

The DB 160 stores various information necessary for the first device 100 to provide an image of a predetermined partial region to the second device 200.

The transmitting and receiving unit 170 transmits and receives various information necessary for the first device 100 to provide an image of a predetermined partial region to the second device 200 with the second device 200.

The control unit 180, the image acquisition unit 110, the image providing unit 120, the object identification unit 130, so that the first device 100 provides the second device 200 with an image for a predetermined partial region , The object tracking unit 140, the network confirmation unit 150, the frame buffer 190, the DB 160, and controls the transmission and reception unit 170.

15 is a block diagram of a second device 200 according to an embodiment of the present invention.

15, the second device 200 according to an embodiment of the present invention includes an image receiving unit 210, a user input unit 220, a network confirmation unit 230, an area information generation unit 240, It includes an area information providing unit 250, a display unit 260, a DB 270, a transmitting and receiving unit 280 and a control unit 290.

The image receiving unit 210 receives an image from the first device 100. The image receiving unit 210 may receive the first video and the second video from the first device 100. In addition, when the communication method is changed, the image receiving unit 210 may receive a video with a changed resolution or image quality from the first device 100, but is not limited thereto.

Meanwhile, when a third image corresponding to an execution screen of an application executed by the first device 100 is provided, the image receiving unit 210 may receive a third image from the first device 100.

The user input unit 220 receives a user input for selecting a partial region of the first video. For example, the user input unit 220 may receive a user input that touches a predetermined point of the first video or a user input that touches and drags a predetermined point of the first video. Also, the user input unit 220 may receive a user's touch input through a predetermined user interface displayed on the screen. Meanwhile, when a third image corresponding to an execution screen of an application executed by the first device 100 is provided, the user input unit 220 receives a user input for selecting a partial region of the third image.

The network confirmation unit 230 checks a communication method between the first device 100 and the second device 200. The network checker 230 may check the type and bandwidth of a communication method between the first device 100 and the second device 200. In addition, the network confirmation unit 230 may check streaming resolution of data transmitted between the first device 100 and the second device 200.

The region information generating unit 240 selects a partial region of the first video based on the user input and the identified communication method. The region information generation unit 240 may generate region information related to an object included in the first video. The region information generating unit 240 may select some regions based on the location of the touch input point in the first video. Specifically, when the user touches a predetermined point in the first video, the area information generating unit 240 may identify an object including the touched point and select an area including the identified object.

In addition, when a user touches a predetermined point in the first video, the region information generating unit 240 may select a region of a preset range from the touched point as a partial region. Also, the area information generating unit 240 may change some selected areas according to a user's touch input through a user interface. In addition, the size of some areas selected from the first video may be determined according to a communication method between the first device 100 and the second device 200.

The area information providing unit 250 provides the generated area information to the first device 100.

The area information providing unit 250 may provide coordinate values for selected partial areas to the first device 100. Also, the area information providing unit 250 may provide the first device 100 with the coordinate value of the point touched by the user.

The display unit 260 displays an image received from the first device 100 on the screen. The display unit 260 may display a video received from the first device 100 on the screen.

The DB 270 stores various types of information so that the second device 200 can receive a video for a partial region of the first video received from the first device 100.

The transmitter / receiver 280 transmits and receives various information to and from the first device 100 so that the second device 200 can receive a video on a partial region of the first video received from the first device 100. .

The control unit 290 checks the video receiving unit 210, the user input unit 220, and the network so that the second device 200 can receive a video for a partial region of the first video received from the first device 100. The control unit 230, the area information generating unit 240, the area information providing unit 250, the display unit 260, the DB 270, and the transceiver 280 are controlled.

16 is a block diagram of a device according to another embodiment of the present invention. The device illustrated in FIG. 16 may be the first device 100 or the second device 200 according to an embodiment of the present invention.

The mobile communication unit 1001 performs call establishment and data communication with a base station through a cellular network such as 3G / 4G. The sub communication unit 1002 performs a function for short-range communication such as Bluetooth or NFC. The broadcast unit 1003 receives a DMB broadcast signal.

The camera unit 1004 includes lenses and optical elements for taking pictures or videos.

The sensor unit 1005 may include a gravity sensor that senses the movement of the device, an illuminance sensor that senses the brightness of the light, a proximity sensor that senses the proximity of the person, and a motion sensor that senses the movement of the person.

The GPS receiver 1006 receives a GPS signal from an artificial satellite. Various services may be provided to the user using the GPS signal.

The input / output unit 1010 provides an interface with an external device or person, and includes a button 1011, a microphone 1012, a speaker 1013, a vibration motor 1014, a connector 1015, and a keypad 1016. .

The touch screen 1018 receives a user's touch input. Here, the touch input may be a drag gesture or a tap gesture. The touch screen controller 1017 transmits the touch input input through the touch screen 1018 to the control unit 1050. The power supply 1019 is connected to a battery or an external power source to supply power required for the device.

The controller 1050 displays and changes some areas of the content displayed on the device according to the present invention by executing programs stored in the memory 1060.

The programs stored in the memory 1060 can be classified into a plurality of modules according to their functions. The mobile communication module 1061, the Wi-Fi module 1062, the Bluetooth module 1063, the DMB module 1062, the camera Module 1065, sensor module 1066, GPS module 1067, video playback module 1068, audio playback module 1069, power module 1070, touch screen module 1071, UI module 1072, It may be classified as an application module 1073. The function of each module will be intuitively deduced by those skilled in the art from the name.

Hereinafter, the application module 1073 will be described in more detail.

First, when the device of FIG. 16 is the first device 100, the application module 1073 provides the first video to the second device 200 based on the video being photographed using the modules included in the device , Area information on a partial area of the first video may be received from the second device 200. In addition, the application module 1073 may identify and track an object corresponding to a partial area by using the modules included in the device, and provide a second video 200 including the tracked object to the second device 200 You can. In addition, the application module 1073 may check a communication method with the second device 200 by using modules included in the device, and when the communication method is changed, remove a video with a changed resolution or quality compared to the second video. 2 can be provided to the device 200.

In addition, when the device of FIG. 16 is the second device 200, the application module 1073 receives the first video from the first device 100 using the modules included in the device, and part of the first video You can select an area. In this case, the size of some regions may be determined based on a communication method between the first device 100 and the second device 200. In addition, the application module 1073 may generate area information on a selected partial area using the modules included in the device and provide it to the first device 100. Also, the application module 1073 may display the second video received from the second device 200 using the modules included in the device.

17 is a diagram illustrating an example of enlarging a partial area of a screen of a first video displayed on the second device 200 according to an embodiment of the present invention.

As illustrated in FIG. 17, the second device 200 according to an embodiment of the present invention may enlarge a partial area of the first video based on the user's touch and drag input. In this case, the second device 200 may determine the degree of enlargement of the first video based on the original resolution of the first video. Specifically, as shown in FIG. 17 (a), when the user multi-touches and drags the screen of the first video, a portion of the first video may be enlarged as in FIG. 17 (b). Also, the second device 200 may determine the degree to which the first video can be enlarged by considering the original resolution of the first video and the screen size of the second device 200. Therefore, as in FIG. 17 (c), even if the user continuously drags the touch, the first video may not be enlarged beyond the magnification of FIG. 17 (b).

In addition, the second device 200 may receive and store information on the original resolution of the first video from the first device 100 in advance to determine the enlargement level of the first video, but is not limited thereto.

18 is a diagram illustrating an example in which a screen of a video displayed on the second device 200 according to an embodiment of the present invention is expanded in stages.

As shown in FIG. 18 (a), a partial region of a video may be selected and the selected region may be primarily enlarged, and then a selected region may be secondaryly enlarged by selecting a portion of the enlarged movie. .

19 is an example in which a second video provided from the first device 100 is displayed after a preview image in which a partial area of the first video is enlarged is displayed on the second device 200 according to an embodiment of the present invention. It is a diagram showing. The first device 100 and the second device 200 may make a video call with each other.

19 (a), the user may select a portion 10 of the screen of the first video displayed on the second device 200 as the region of interest. The second device 200 may receive the first video having the first resolution from the first device 100 and display the received first video on the screen of the second device 200.

Thereafter, as shown in FIG. 19 (b), a preview image in which the selected region of interest is enlarged may be displayed on the second device 200, and a portion of the first video may be enlarged on the first device 100. A menu 12 for deciding whether to provide can be displayed. In addition, the first device 100 may display which part of the first video is enlarged in the second device 200 in the predetermined area 14.

In addition, when the user selects 'allow' in the menu 12, as shown in FIG. 19 (c), the first device 100 provides a second video for the selected region of interest to the second device 200 , The second device 200 may display the received second video. In this case, the first device 100 may provide the second video 200 for the selected region of interest to the second device 200 at the first resolution.

Meanwhile, when the first device 100 provides the second video to the second device 200, notification information for notifying that the second video is an enlarged image of the selected region of interest is inserted into the second video. can do. Specifically, the first device 100 may stream the first video and the second video to the second device 200, and when the transmission of the first video is stopped and the second video is transmitted, notification information within the second video Can be inserted. For example, when the first device 100 stops transmitting the first video and transmits the second video, the first device 100 inserts a predetermined parameter value in a predetermined packet at a streaming level By doing so, it is possible to inform the second device 200 that the second video is provided. In addition, the notification information may be inserted into a Real-time Transfer Protocol (RTP) packet header shown in FIG. 20.

20 is a diagram illustrating an example of a Real-time Transfer Protocol (RTP) packet header according to an embodiment of the present invention.

Each frame of the second video provided from the first device 100 to the second device 200 may include a plurality of RTP packets. Also, the first device 100 may insert notification information into a predetermined packet among a plurality of RTP packets corresponding to each frame of the second video. For example, the first device 100 inserts notification information into an RTP packet having a bit value of '1' in the RTP Marker field 90 included in the header of a plurality of RTP packets corresponding to each frame of the second video. can do.

Also, the first device 100 may insert notification information in a predetermined frame among frames of the second video. For example, the first device 100 may insert notification information for a preset number of frames from the first frame of the second video. However, it is not limited thereto.

Also, the first device 100 may insert the notification information into the Header extension field 92 of the RTP packet header. The notification information may have a predetermined bit value. In addition, the notification information may include predetermined area information. The area information may be information indicating which part of the first video clip is cut by the first device 100 to generate the second video clip. For example, the area information may be a value indicating the position of a portion corresponding to the screen of the second video from the entire screen of the first video or the original video. Also, the area information may have a predetermined coordinate value.

21 is a flowchart of a method for a first device 100 to generate a fourth image according to an embodiment of the present invention. The first device 100 may provide a third image corresponding to an execution screen of an application executed in the first device 100 to the second device 200, and the third image from the second device 200. A fourth image may be generated by receiving region information regarding a part, and the generated fourth image may be provided to the second device 200.

In step S2100, the first device 100 acquires a third image from the frame buffer in the first device 100. The frame buffer may store a third image corresponding to an execution screen of an application being executed on the first device 100.

In operation S2200, the first device 100 may cut a part of the third image based on the area information received from the second device 200. Also, the first device 100 may encode a part of the cut third image (step S2300) and packetize (s2400).

22 is a diagram of a method for a second device 200 having a video call with the first device 100 to receive a second expanded video with respect to a partial region of the first video according to an embodiment of the present invention. It is a flow chart.

In step S2205, the first device 100 and the second device 200 are connected for a video call.

In step S2210, the first device 100 provides the first video to the second device 200. The first device 100 may photograph a user of the first device 100 and generate a first video for a video call. Also, the first device 100 may transmit the generated first video to the second device 200.

In step S2220, the second device 200 displays the first video received from the first device 100 on the screen of the second device 200.

In step S2230, the second device 200 selects some areas to be enlarged from the first video. The second device 200 may select, for example, a user's mouth portion of the first device 100 or a cover portion of a book held by the user of the first device 100 in the first video. . In this case, the second device 200 may select some areas based on the user's touch input, but is not limited thereto. Some areas to be enlarged during a video call may be preset, and the second device 200 may select some preset areas from the user's video of the first device 100.

For example, the second device 200 may select the user's mouth as a partial area by tracking the user's mouth from among the face images of the user of the first device 100. Specifically, the second device 200 detects face recognition information from the image of the user of the first device 100, and based on the detected face recognition information, the second device 200 detects the mouth portion of the user's face of the first device 100. You can track and choose. The method of detecting face recognition information may include a feature-based method, an appearance-based method, and a hybrid method. The feature-based method uses the geometric information of the face or uses facial feature components (e.g., eyes, nose, mouth, chin, etc.) in size and shape, correlation, or a mixture of these elements. It is a method of recognizing a face using form information. The feature-based method may include, for example, a pure geometry method, a dynamic link architecture-based method, and a hidden markov model-based method. Also, the appearance-based method is a method of recognizing a face using a model trained by a set of training images. The appearance-based method may include, for example, a method using an intrinsic face generated by principal component analysis, a method using a linear discriminant analysis, a method using a neural network, and a method using a support vector machine. In addition, the hybrid method is a method of mixing a feature-based method and an appearance-based method. The hybrid method may include, for example, a linear feature analysis based method, a shape-normalized based method, and a component based method. However, the method for detecting face recognition information according to an embodiment of the present invention is not limited to the above three methods, and various methods may be used for detecting face recognition information.

In step S2240, the second device 200 transmits coordinate information of the selected partial area to the first device 100.

In step S2250, the first device 100 displays a partial area selected by the second device 200 among the first video. The first device 100 may display a portion selected by the second device among the first videos, for example, with a dotted line. Also, the first device 100 may display only the portion selected by the second device among the first videos on the screen of the first device 100.

In step S2260, the first device 100 determines to provide the second video 200 for the selected partial region to the second device 200. The first device 100 displays a separate UI and, based on a user input through the UI, provides the second device 200 with a second video for a partial region selected by the second device 200 Can decide.

In step S2270, the first device 100 photographs a portion corresponding to the selected partial region. The first device 100 may photograph, for example, a mouth portion of a user of the first device 100 or a cover portion of a book held by a user of the first device 100.

In step S2280, the first device 100 provides the second video to the second device 200. The first device 100 may generate a second video by photographing a portion corresponding to the selected partial area, and provide the generated second video to the second device 200.

In step S2290, the second device 200 displays the second video.

Meanwhile, the second device 200 may display the first video and the second video together on the screen of the second device 200. In this case, in step S2280, the second device 200 may receive the first video as well as the second video. Also, the second device 200 may overlap and display the second video on the first video. The second device 200 may overlap the second video in a portion where the user of the first device 100 is not displayed in the first video. Also, the second device 200 may adjust the size of the area where the first video is displayed and the size of the area where the second video is displayed. However, the present invention is not limited thereto, and the second device 200 may display the first video and the second video on the screen of the second device 200 without overlapping.

23 is a flowchart of a method in which a second device 200 making a video call with the first device 100 enlarges and displays a partial area of the first video according to an embodiment of the present invention.

In step S2305, the first device 100 and the second device 200 are connected for a video call.

In step S2310, the first device 100 provides the first video to the second device 200. The first device 100 may photograph a user of the first device 100 and generate a first video for a video call. Also, the first device 100 may transmit the generated first video to the second device 200.

In step S2320, the second device 200 displays the first video received from the first device 100 on the screen of the second device 200.

In step S2330, the second device 200 selects some areas to be enlarged from the first video. The second device 200 may select, for example, a user's mouth portion of the first device 100 or a cover portion of a book held by the user of the first device 100 in the first video. .

In this case, the second device 200 may select some areas based on the user's touch input, but is not limited thereto. Some areas to be enlarged during a video call may be preset, and the second device 200 may select some preset areas from the video of the user of the first device 100. For example, the second device 200 may select the user's mouth part as a partial area by tracking the user's mouth part among the face images of the user of the first device 100.

In step S2340, the second device 200 displays only a partial area selected in step S2330 among the first videos on the screen of the second device 200. The second device 200 may enlarge a portion selected from the first video through image processing, and display the enlarged image on the screen of the second device 200. For example, when the resolution of the first video is 200 x 200 and the resolution of some selected areas is 50 x 50, the first device 100 may enlarge the resolution of the selected some areas to 200 x 200.

Meanwhile, the second device 200 may overlap the enlarged image on the first video. The second device 200 may overlap the enlarged image in a portion where the user of the first device 100 is not displayed in the first video. Also, the second device 200 may adjust the size of the area where the first video is displayed and the size of the area where the enlarged image is displayed. However, the present invention is not limited thereto, and the second device 200 may display the enlarged image and the second video on the screen of the second device 200 without overlapping.

24 is a diagram illustrating an example in which an enlarged image of a partial region selected from a first video is displayed on a screen of the second device 200 together with the first video according to an embodiment of the present invention.

Referring to FIG. 24 (a), the mouth portion 40 may be selected from the user's face of the first device 100 displayed on the screen of the second device 200. The mouth portion 40 may be selected by the user of the second device 200. However, the present invention is not limited thereto, and according to a preset criterion, the second device 200 may track the mouth portion 40 in the first video and select the mouth portion 40.

In addition, referring to FIG. 24 (b), an enlarged image 44 for the selected mouth portion may overlap on the first video 42. In this case, the second device 200 may determine a portion in which the user of the first device 100 is not displayed in the first video, and may overlap the second movie in the portion in which the user is not displayed.

In addition, referring to FIG. 24 (c), the enlarged image 44 for the selected mouth portion does not overlap on the first video 42, and the enlarged image 44 and the first video 42 for the selected mouth portion ) May be displayed in separate windows.

One embodiment of the invention may also be implemented in the form of a recording medium comprising instructions executable by a computer, such as program modules, being executed by a computer. Computer readable media can be any available media that can be accessed by a computer and includes both volatile and nonvolatile media, removable and non-removable media. In addition, computer readable media may include both computer storage media and communication media. Computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Communication media typically include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave, or other transport mechanism, and includes any information delivery media.

The above description of the present invention is for illustration only, and a person having ordinary knowledge in the technical field to which the present invention pertains can understand that it can be easily modified into other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

The scope of the present invention is indicated by the following claims rather than the above detailed description, and it should be interpreted that all changes or modified forms derived from the meaning and scope of the claims and equivalent concepts thereof are included in the scope of the present invention. do.

Claims (49)

Transmitting the first video having the original resolution to the external device at the first resolution;
Receiving, from the external device, area information on a region of interest of the first video displayed at the first resolution in the external device;
Determining an area corresponding to the region of interest in the first video of the original resolution based on the region information;
Converting a part of the first video of the original resolution into a second video; And
Transmitting the second video to the external device at the first resolution;
It includes,
The determined area is smaller than the first video of the original resolution, and a part of the first video corresponds to the determined area,
The region of interest includes an object included in the first video, and the region information is generated by the external device, and when the screen of the external device is touched, the object including the touched part on the screen The region of interest is selected to be included, and when the screen of the external device is touched, the region of interest including a preset range is selected from a portion touched on the screen, the image providing method.
delete delete delete delete According to claim 1,
When the screen of the external device is touched, a user interface for selecting the region of interest is displayed on the screen of the external device.
According to claim 1,
The first video and the second video are videos for a video call with the external device,
In the step of transmitting the first video to the external device, the first video is transmitted to the external device based on the captured video,
The resolution of the captured video is greater than the streaming resolution of the video call, video providing method.
delete According to claim 1,
The first resolution is determined according to the type of communication method with the external device, the image providing method.
delete delete According to claim 1,
The transmitting of the first video to the external device may include transmitting the first video and the second video to the external device based on the captured video,
If an object included in the region of interest disappears from the captured video, changing the determined region;
Video providing method further comprising.
The method of claim 12,
When the disappeared object appears in the captured video, providing the second video to the external device including only a partial area including the displayed object among all areas of the captured video;
Video providing method further comprising.
According to claim 1,
A method of providing an image, wherein notification information indicating that the second video is transmitted is inserted into the second video.
delete delete delete The method of claim 14,
The notification information includes region information indicating which part of the first video corresponds to the second video.
delete Receiving a first video from an external device;
Selecting a region of interest related to an object included in the first video based on a user input;
Generating region information on the selected region of interest;
Providing the area information on the object included in the received first video to the external device; And
Receiving a second video including only a partial area including the object of the first video from the external device;
It includes,
The resolution of the second video is higher than that of the partial region of the first video,
Some areas including the object are determined by the external device based on the provided area information,
In the step of selecting the region of interest, when the screen of the first video is touched, the region of interest is selected to include an object including the touched portion,
The step of providing the area information is to provide coordinate information for the touched point to the external device when the screen of the first video is touched.
delete delete delete delete The method of claim 20,
When the screen of the first video is touched, displaying a user interface for selecting the area;
It further comprises, a video receiving method.
The method of claim 20,
The first video and the second video are videos for video call with the external device,
The first video and the second video are provided from the external device based on an image captured by the external device,
The resolution of the captured video is greater than the streaming resolution of the video call, video receiving method.
delete The method of claim 20,
According to the type of communication method with the external device, the external device, the resolution of at least one of the first video and the second video is determined, the video receiving method.
delete delete The method of claim 20
The first video and the second video are provided from the external device based on an image captured by the external device,
When the object disappears from the captured image, the partial region of the first video is changed,
Receiving the second video including only the changed partial area from the external device;
Video receiving method further comprising.
The method of claim 31,
When the disappeared object appears on the captured image, receiving the second video including only a partial region including the displayed object from the external device;
Video receiving method further comprising.
A memory in which at least one program is stored; And
And a processor that provides a video to an external device by executing the at least one program.
The at least one program,
Transmitting the first video to an external device;
Receiving area information related to an object included in the first video from the external device; And
Providing a second moving picture including only a partial area including the object identified based on the area information to the external device;
It contains the commands to execute,
The resolution of the second video is higher than that of the partial region in the first video,
The area information is information on a region of interest selected by the external device, and is generated by the external device,
When the screen of the external device displaying the first video is touched, the region of interest including the object including the touched portion is selected,
When the screen of the external device displaying the first video is touched, the region of interest in a preset range is selected from the touched part.
delete delete delete delete The method of claim 33,
When the screen of the external device is touched, a user interface for selecting the region of interest is displayed on the screen of the external device.
The method of claim 33,
The first video and the second video are videos for video calls with the external device,
In the step of transmitting the first video to the external device, the first video is transmitted to the external device based on the captured video,
The resolution of the captured video is greater than the streaming resolution of the video call.
delete The method of claim 33,
A device in which at least one resolution of the first video and the second video is determined according to the type of communication method with the external device.
delete delete The method of claim 33
In the step of transmitting the first video to an external device, the first video and the second video are transmitted to the external device based on the captured video,
If the object disappears from the captured image, changing the partial area corresponding to the second video; And
Providing a second video of the changed partial region to the external device;
The device further comprising instructions for executing the device.
The method of claim 44,
When the disappeared object appears on the photographed image, providing a video including the partial region including the displayed object to the external device among all regions of the photographed image;
The device further comprising instructions for executing the device.
delete delete delete delete

Images