KR20160021687A - Method for displaying image stream side by side and portable electronic divice displaying image stream on a display screen side by side by the method - Google Patents

Abstract

The present invention relates to treatment of an image and, more specifically, to a method for displaying a video stream on a display in a side-by-side form. The method according to the present invention comprises the following steps: storing a video stream received by an electronic device in a memory; reading out a video stream packet appointed to be displayed on a single display by being converted from the video stream stored in the memory, simultaneously through first and second processes; decoding the video packet read out through each of the processes, individually scaling the video frames generated through decoding to be displayed on the appointed first and second display regions of the display, and transmitting the same to a video driving unit; and appointing a video stream packet disposed next to the appointed video stream packet as a novel video stream packet to be displayed on a display, when work with respect to the video stream packet appointed in each of the first and second processes is completed.

Description

≪ Desc / Clms Page number 1 > METHOD FOR DISPLAYING IMAGE STREAM SIDE BY SIDE AND PORTABLE ELECTRONIC DIVISION DISPLAYING IMAGE STREAM ON A DISPLAY SCREEN SIDE BY SIDE < RTI ID = 0.0 > BY THE METHOD}

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to image processing, and more particularly, to a method of displaying a video stream on a display in a side-by-side manner. The invention also relates to a portable electronic device for displaying a video stream on a display in the manner described above.

Various types of head mounted displays (hereinafter referred to as HMDs) have been developed and known. The HMD can provide a visual effect, such as viewing a large screen, while displaying the image on a small display. Among the HMDs, there is known a type of HDM which is provided with a pair of small display devices and can be worn like glasses. The same 2D image or 3D image (stereographic image) can be displayed on each display device of the HMD.

The HMD is a particularly suitable device for viewing stereoscopic images. When a person looks at an object, three-dimensional images can be recognized because the two eyes are spaced a certain distance (about 65mm apart) and placed on the face. When a person views an object, a different two-dimensional image corresponding to the gap between the left eye and the right eye is formed on each retina, and different two-dimensional image information is transmitted to the brain through the retina. The two different image information transmitted from the brain are fused, and the person perceives the perception of depth, presence, and perception of volume of the object corresponding to the image. Therefore, in order to view the stereoscopic image with the HMD, it is necessary to display at least two different images (stereoscopic images) having a binocular disparity on a pair of display devices.

On the other hand, it is known that when a general person watches stereoscopic images for a long time, side effects such as visual fatigue and psychological instability occur. Side effects include, for example, eyestrain, eyes feel heavy, double vision, dry eyes, heavy headaches, headache, stiff shoulders, shoulder pain , Pain such as back pain, nausea, and giddiness. Although the cause of visual and psychological fatigue is not clarified when a person watches a stereoscopic image, it is not clear whether the focal distance and the convergence angle of the two eyes are inconsistent, unnatural image, mismatch of motion parallax, Differences are mentioned. It is understood that these causes are related to erroneous photographing of the stereoscopic image, errors of the stereoscopic image reproducing system, or closely related to the inherent properties of the stereoscopic image itself. Especially, when the moving object is viewed from the stereoscopic image in relation to the property of the stereoscopic image itself, the HMD user feels excessive protrusion feeling and speed feeling. In addition, when the stereoscopic image includes a lot of colorful colors and complicated patterns, the HMD user can feel the afterimage of the pattern and the color body in addition to the stereoscopic effect.

On the other hand, even if the same image is displayed, the viewer feels different depending on the size of the screen on which the image is displayed. When you view the same image on the large screen of a movie theater or on a small screen such as a computer monitor or smart phone, you feel differently or get different impressions. HMD provides the same effect as viewing a small screen on a large screen like a movie theater.

Accordingly, it is possible to provide an impression that the image is viewed using a HMD device as needed on a large screen, and at the same time, when a 3D image or similar image (image when a 2D image is displayed on the HMD) There is a need for a market for display methods and apparatuses that can minimize side effects. There is also a need for technologies and ideas that can provide HMD devices at very low cost. An HMD device has been proposed to satisfy some of these requirements.

US Patent Application Publication No. US 2010/0079356 A1 discloses an HMD using a portable electronic device having a display such as a smart phone. The HMD disclosed in the above patent does not have a separate display device, but instead is configured to mount a smart phone equipped with a display device in the HMD. The HMD of the above patent is advantageous in that the HMD price can be lowered because a part of the functions of the smartphone, which is combined with the smart phone electronically and mechanically, is used as a function of the HMD. In order to utilize a portable electronic device such as a smart phone as an HMD device, a display of a portable electronic device is divided into left and right eye regions, and left and right eye images are displayed in respective regions (hereinafter referred to as' SIDE-BY- SIDE type 'or' SBS type ').

In recent years, with the development of communication technology, portable electronic devices such as Samsung Galaxy ^TM or Apple's iPhone ^TM have been connected to a wireless Internet or a wireless communication network to transmit video information such as TV programs and movies to a video stream Can be displayed. The portable electronic device may be used in combination with an HMD device such as that disclosed in the above-mentioned U.S. Patent Application Publication No. US 2010/0079356 A1 (hereinafter, referred to as 'Combined-HMD'), but a special lens for viewing a SIDE-BY- (Hereinafter referred to as " mounting-HMD "). When a display of a portable electronic device is used as a display of a combined-HMD or a mounted-HMD, a video stream transmitted to a portable electronic device must be converted into a SIDE-BY-SIDE format and displayed.

The image is transferred to the Korean patent application No. 10-2013-0068042 filed by the inventor of the present application filed on the date of filing 2013.8.2, A method of displaying a video stream in a SIDE-BY-SIDE form on one display is disclosed. The method disclosed in the patent application includes decoding a received image data, reducing a screen frame of decoded image data, duplicating a reduced screen frame twice to generate a new screen frame in a SIDE-BY-SIDE format, And transmits the generated screen frame of the SIDE-BY-SIDE format to the video driver and displays the screen frame on one screen. The image stream transmitted at this time is a two-dimensional image, and when displayed in a SIDE-BY-SIDE format on a mounted HMD, provides a stereo-like stereoscopic effect similar to a three-dimensional image. In addition, when the 2D image is mounted in the SIDE-BY-SIDE format and displayed on the HMD, the side effect of the stereoscopic image is mitigated to some extent.

In the method disclosed in the patent, a screen frame is reduced in size, a reduced screen frame is duplicated twice, and a new screen frame in a SIDE-BY-SIDE format is generated and transmitted to the video driver. Accordingly, when the conversion time of the image is longer than the reproduction speed of the image, the image displayed on the display can not be continuously displayed, and some images are skipped and displayed. Therefore, the image displayed on the display is not smooth, and when viewing for a long time, side effects due to stereoscopic viewing can be increased. In addition, the two left and right-eye images included in the SIDE-BY-SIDE format are 2D images and images with a slight drop in stereoscopic effect.

Therefore, when a 2D image is displayed on the display of the HMD in a SIDE-BY-SIDE format, a new image stream display method which can reduce the side effect of the stereoscopic image while increasing the stereoscopic effect is required. On the other hand, among the users of the mounting-HMD, as in the case of strabismus, the focus of the left and right eyes is different from that of the normal person. For these people, it is required to provide a mounting-HMD capable of individually adjusting the position of the left / right eye image displayed in the SIDE-BY-SIDE format on the display of the mounting-HMD so as to coincide with the focus of the right and left pupils.

It is a first object of the present invention to provide a method of quickly displaying a 2D video stream in a SIDE-BY-SIDE format on a display of a portable electronic device used in a mounting-HMD. It is a second object of the present invention to provide a method of rapidly displaying a 2D video stream in a SIDE-BY-SIDE format on a display of a portable electronic device used in a mounting-HMD while increasing a stereoscopic effect. A third object of the present invention is to provide a method of adjusting the position of images displayed in a SIDE-BY-SIDE format on a display of a portable electronic device used in a mounting-HMD. A fourth object of the present invention is to provide a portable electronic device displaying a video stream in a SIDE-BY-SIDE format on a display in the same manner as described above.

According to one aspect of the present invention, a method of displaying a video stream in a side-by-side format on a display of a portable electronic device is provided. The method according to the present invention is preferably carried out in a portable electronic device such as a smart phone, for example, an electronic device including a control part, a display, a video driving part for controlling the display, a wireless communication module and a memory Do.

A method according to the present invention includes the steps of storing a video stream received in an electronic device in a memory, and converting a video stream packet designated to be displayed in a single display into a converted video stream, A first process of decoding a read video packet, scaling video frames generated by decoding to be displayed in a designated first display area of the display, and transmitting the scaled video frames to a video driver A second conversion display step of decoding the read video packet and scaling the video frames generated by the decoding to be displayed in a designated second area of the display and transmitting the scaled video packets to the video driver; , A video specified in each of the first and second processes And converting the video stream packet positioned after the designated video stream packet into a new video stream packet for display on the single display when the operation of the stream packet is completed.

According to the present invention, the first and second processes convert the same video stream packet to display in a pre-designated first area and a second area of a single display. The first area and the second area for dividing the display are displayed in a position corresponding to the viewing angle of the left eye and the right eye of the user when the portable electronic device is used in the mounting type HMD, . According to the present invention, when a 2D video stream is transformed to fit each display area in two processes and directly transmitted to a video driver to display in a side-by-side format, a video stream is displayed on a single display The display speed can be made faster than in the case of displaying in side form. Conventional methods include scaling a video stream to decode the decoded video stream in a given display on a given display and duplicating the scaled image frame twice to create a new image frame in side by side format , And the generated new image frame is transmitted to the video driver and displayed in a side by site format. As described above, the conventional method of displaying an image on a side-by-side basis in a single display requires a lot of time because it has to perform several steps in one process.

Further, when a 2D video stream is displayed side by side on a single display according to the present invention, a stereoscopic effect is displayed more clearly than when a video stream is displayed on a single display by a conventional method. A side-by-side image frame generated by a conventional method is copied on the same image twice and a new image frame arranged in a side-by-side form is displayed on the display. In contrast, the side-by-side images displayed on the display in accordance with the method of the present invention begin to decode the same video stream packets simultaneously on two different processors, transforming the respective image frames and displaying them in a predetermined display area of the display do. Therefore, the two images of the image frame displayed in the side-by-side form slightly differ depending on the conversion speed of each process. In the case where there is a difference between the conversion speed of the process of converting and displaying the video stream in the left eye display area and the conversion speed of the process of converting and displaying the video stream in the right eye display area, , An image frame displayed in the side-by-site form has the same effect as displaying a stereo image. Therefore, according to the present invention, a viewer who views a 2D image displayed in a side-by-side form will feel the same effect as when viewing a stereo image. In the method according to the present invention, it is preferable to control the completion time difference of the image conversion display operation of the first process and the second process so as to be within 5/30 frame, preferably within 3/30 frame of the image stream desirable.

When the consecutive image frames do not largely change the scene as video images, the user is greatly inconvenient when the images displayed adjacent to the side by side are displayed with differences in the range of 3/30 to 5 frames It is possible to feel the same effect as viewing a 3D image. However, even if the image displayed next to the side by side is in the range of 3/5 to 30/30, if the scene of the image changes a lot, the user recognizes the images of scenes completely different from each other as side by side So that the viewer perceives the visual confusion instead of the effect of the 3D image. Therefore, in this case, it is necessary to check a frame in which a scene changes a lot beforehand so that an image of the same scene is displayed in a side-by-side form. For example, the control process reads the specified video stream packet before the first and second processes, decodes the read video packet, and compares the sequentially generated image frames by decoding. The comparison of neighboring image frames may be performed by, for example, performing an exclusive OR operation (XOR operation) on two temporally consecutive image frames, and judging a change of a scene when the value is greater than or equal to a predetermined value. And controlling an image frame in which the image comparison value is equal to or greater than a predetermined value when the comparison value of the image frames is equal to or greater than a predetermined value by simultaneously processing and displaying the image frame in the first process and the second process.

A user with an abnormality in the muscles of the pupils of both eyes, such as strabismus, needs to adjust the side-by-side image displayed on the display to suit his or her vision. That is, it is desirable that the position where the two images are displayed on the display can be changed according to the user so that the position of the two images displayed in the side-by-side form can be viewed at the position of the natural pupil of the user. The method includes the steps of: displaying a target for image display area correction in a first display area and a second display area, respectively; displaying the target for the image display area correction in the first display area or the second display area, Receiving the image display area correction coordinate information, and changing coordinates of the image display area in accordance with the image display area correction coordinate information. The user is moved to a position where the target for the correction of the display area, which is displayed in a manner not corresponding to the position of the pupil of the user, is viewed as one using an input device such as a remote controller dedicated to a smart phone or a touch input device installed on the display Side-by-side image can be changed by changing the position information.

 According to another aspect of the present invention, there is provided a portable electronic device for displaying a video stream in a side-by-side format on a display. A portable electronic device according to the present invention includes a control unit, a display, a video driving unit for controlling the display, a wireless communication module, and a memory for storing a program for converting a video stream to be side- .

The program comprising the steps of: storing a video stream received at the wireless communication module in a memory; and simultaneously reading out a video stream packet designated to be displayed on a single display by converting among the video streams stored in the memory, in a first and a second process A first conversion display step of decoding the read video packet, scaling the video frames generated by the decoding to be displayed in a designated first display area of the display, and transmitting the scaled video frames to the video driver; A second conversion display step of decoding the same video packet read out, scaling the video frames generated by decoding to be displayed in a designated second area of the display, and transmitting the scaled video frames to the video driver, In the first and second processes of When performing operations on the specified video stream packets are all completed, so as to perform the step of specifying the video stream packet which is located next specified video stream packets to the new video stream packet for display by converting the display of the single.

The portable electronic device according to the present invention can be used in a mounting-HMD. In particular, a 2D video stream can be rapidly displayed in a SIDE-BY-SIDE format and a 2D video stream can be simultaneously displayed in a side-by-side form similar to a stereo image, thereby displaying a stereoscopic effect.

The program further includes a step of controlling the image processing speed of the first process and the second process such that the difference in the completion time of the image conversion display operation of the first process and the second process falls within a defined range can do. In addition, the program may be configured to control the completion time difference of the image conversion display operation of the first process and the second process to be within three-thirty frames of the image stream. The program may further include the steps of: displaying a target for image display area correction in the first display area and the second display area, respectively; and displaying the target for image display area correction in the first display area or the second display area, And receiving the coordinate information and changing coordinates of the image display area in accordance with the image display area correction coordinate information.

By displaying the 2D video stream in the side-by-side format on the display of the portable electronic device by the method according to the present invention, the 2D video stream can be displayed quickly and the 2D video stream can be increased in stereoscopic effect. Thus, it is possible to provide the same effect as viewing a 3D video to a mounting-HMD user without converting the 2D video medium to 3D format.

In addition, the method of the present invention can adjust the position of an image displayed on a display of a portable electronic device in a side-by-side format of a 2D video stream. Thus, the visually impaired such as strabismus can display side-by-side images on the display and appreciate them in a form suitable for their own view.

In addition, the portable electronic device according to the present invention can display a 2D video stream in a side-by-side format so as to have a 3D sensation at a high speed, thereby providing the same effect as viewing a 3D image when used as a display device in a mounting- can do.

FIG. 1 is a perspective view of a mounting-and-receiving-type HDD to which a portable electronic apparatus according to the present invention is mounted.
FIG. 2 is a detailed view showing a state in which the portable electronic device is mounted on the mounting-HMD of FIG.
3 is a schematic diagram of a portable electronic device according to the present invention
4 is a diagram illustrating a video stream packet stored in a memory of the portable electronic device according to the present invention
5 is a flow chart illustrating a method for displaying a video stream according to the present invention on a display in a side-
Fig. 6 is an explanatory diagram showing a state in which video stream packets are converted and processed in the first and second processes; Fig.
Fig. 7 is an explanatory view of an example of a target for adjusting an image display area displayed on a display in the portable electronic device according to the present invention and a corrected image display area; Fig.
8 is a flowchart showing an image display area adjustment procedure in the portable electronic device according to the present invention.
9 is a schematic diagram illustrating a method of displaying a conventional image stream on a display in a side-

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

FIG. 1 is a perspective view of a mounting-and-mounting device for mounting and using the portable electronic device according to the present invention, and FIG. 2 is a detailed view showing a state in which a portable electronic device is mounted on the mounting-and-mounting device of FIG. The mounting-HMD 100 includes a housing 10, a holder 20 into which the housing 10 can be slidably inserted, a pair of links 50 connected to both sides of the housing 10, And a head mount (30) connected to the head mount (50). Referring to FIG. 2, the portable electronic device 200 is inserted into the holder 20. Further, the housing 10 is provided with a lens unit 40 provided with a pair of lenses. The lens unit 40 is disposed so as to face the display of the portable electronic device 200. Although not shown, each of the lenses is a lens having a shape for viewing an image displayed in a side by-sight form, It is appropriate that the focal length is approximately 40 to 80 mm. The shape of the lens in the side-by-side form is well known to those skilled in the art and will not be described in detail.

Fig. 3 shows a schematic diagram of a configuration of a portable electronic device according to the present invention. The portable electronic device 200 includes a wireless communication module 210, a memory 220, a CPU 230, a video driver 260, and a display 280. A portable electronic device for use in mounting -HMD a smartphone, for example, the Apple iPhone or the Galaxy ^{TM TM} Although it is preferred to use the same mobile phone, but are not limited to, tablet PC or video which can be connected to the network Players can also be used.

The portable electronic device 200 stores the video stream transmitted through the wireless communication module 210 in the memory 200. Referring to FIG. 4, the video stream is transmitted in a packet form, and is stored in a memory according to the order in which the video stream is transmitted. In each packet, a plurality of encoded image frame information is arranged in accordance with the reproduction order. In order to display the video stream stored in the memory in the side by site format, the video packets are read and decoded according to the stored order, the respective image frames are converted to fit the size of the display, and the video frames are transmitted to the video driver.

In the present invention, the side-by-side form of an image means a form in which an original image or images are scaled and arranged in parallel on a single display to display one image or a plurality of images on a single display. The parallel arrangement includes arrangements not only in the horizontal direction but also in the vertical direction and the oblique direction with respect to the visual field of the user. In addition, in the present invention, scaling is generally used to mean not only reduction of the original image but also image conversion including enlargement, rotation, and movement of a part or all of the original image.

9 is a schematic diagram of a method for displaying a video image in a side-by-side manner according to a conventional method. Referring to FIG. 9, the conventional method includes the steps of generating image frames (A, B, C, ...) included in a video stream from new image frames in a side-by- AA, BB, CC, ...) and displays it on the display. More specifically, the conversion process reads and decodes the video packets stored in the memory, converts each decoded image frame to a size that can be displayed on the display in a side-by-side form, The image is copied to the memory twice to generate a side-by-side image frame, and the generated side-by-side image is transmitted to the video driver. The conventional method has a problem in that it takes a long time to scale and copy each decoded image frame to form a new image frame in a side-by-side form, thereby failing to smoothly reproduce the video stream. Also, since the same images are arranged in parallel in a new image frame generated in a side-by-side form, the viewer can not provide a 3D sensation (depth sense) similar to a 3D image.

3 and 5, a method of converting a video stream into a side-by-side format and displaying it on a display will be described according to the present invention. The CPU 230 of the portable electronic device executes two first and second processes 240 and 250 and a control process 235 for controlling the two processes. A central processing unit (CPU) (central processing unit) can be configured by using one processor or by arranging a plurality of processors in parallel. First, in the first and second processes 240 and 250, among the video stream packets stored in the memory 220, a predetermined conversion target video stream packet is read (S110).

Referring to FIG. 4, a designated video stream packet 222 is encoded with a plurality of image frames 1, 2, 3, 4, .... The video stream packets may be a file compressed according to a predetermined compression standard such as, for example, MP4, avi, wmv, mkv, and the like. The compressed stream packets are decoded using a decoding tool of the standard. For example, it can decode based on 30 frames per second. The first and second processes respectively decode the read video stream packet to extract image frames, and convert the extracted image frames. For example, the first process 240 scales image frames 1, 2, 3, 4, ... to be displayed in the first display area corresponding to the left eye of a single display 280 (S120 ). Also, the second process 250 scales the image frames 1, 2, 3, 4, ... so as to be displayed in a second display area corresponding to the right eye of the single display 280 (S140). For example, when the resolution of the display is 720p (1280x720 pixels), the sizes of the first display area and the second display area can be respectively set to half of the display size. In this case, each image frame is reduced in size by half. As a method of halving the image frame, for example, both the even and odd pixels in the horizontal and vertical directions of the image frame can be removed and the remaining odd-numbered pixels can be used as the converted image frame. The number of pixels of the converted frame is 640x360. If the image is displayed on the display in half-shrinked state without any other conversion, the size of the image displayed in side-by-side form will be half of the display size. That is, the coordinates of the upper left (start position) of the first display area (left eye display area) becomes (0, 179) with respect to the reference coordinates of the display (when the upper left coordinate is set to (0,0) The coordinate of the lower right (ending position) becomes (639, 539). The coordinates of the upper left (start position) of the second display area (right eye display area) is (640, 179) and the coordinates of the lower right end (ending position) are (1279, 539) do. If the resolution of the other video stream is different from the resolution of the display, the image frame can be converted using an appropriate scaling method according to the position and size of the display area of the specified display.

Next, the first process 240 specifies the memory of the video driver corresponding to the first area of the predetermined display to transmit the scaled image frames to the video driver and displays the scaled image frames on the display 280 (S130). The second process 250 also specifies the memory of the video driver corresponding to the second area of the predetermined display to transmit the scaled image frames to the video driver and display on the display 280 S130).

A portable electronic device such as a smart phone provides an operating environment (multitasking) capable of executing a plurality of processes in one processor, or provides multiprocessing to perform a plurality of processes in a plurality of processors. Particularly, in the case where the first process and the second process are performed independently in a plurality of processors, the conversion speed of the video stream packet varies slightly from process to process depending on the environment. Accordingly, although the image frame displayed on the first display area of the display and the image frame displayed on the second display area may be the same at a predetermined time, most different image frames are displayed. For example, referring to FIG. 4, when the stream packet 222 is displayed, the image frame # 3 may be displayed in the first display area and the image frame # 4 may be displayed in the second display area simultaneously. Therefore, there is a slight difference in the image displayed in the side-by-side form on the display. Therefore, when a 2D video stream is displayed in a side-by-side form according to the method of the present invention, an image similar to a stereo image is displayed, though it is not an accurate stereoscopic image. Therefore, the user who uses the mounting-HMD can feel the same effect as viewing a 3D image and a depth image as compared with the case of displaying the 2D image side by side by the conventional method.

Next, it is checked whether the execution of the first process and the second process is completed (whether the format image frame of the designated video stream packet is decoded and converted and displayed on the display) (S160). If only one of the first and second processes is completed, the process waits for completion. If both the first and second processes are completed, the video stream packet of the next sequence is newly designated (S170). If the newly designated video stream packet is larger than the last packet number, the process is terminated. If the newly designated video stream packet is smaller than the last video stream packet number, the process returns to the first step (S110) and the above-described steps are repeatedly performed.

Referring to FIG. 6, when the 221st packet is converted and displayed in the first and second processes, the execution of the first processor is terminated first, and the second process is terminated after a delay of t1 . At this time, the control process 235 stops the first process to read and convert the video stream packet of the next sequence number for t1 time, and checks whether the execution of the second process has ended. When the conversion process for the video stream packet specified in the first and second processes is completed, the control process 235 designates the next video stream packet 222 as a video stream packet to be newly converted. Referring to FIG. 6, the process for the 222nd packet is terminated by the second processor, and the first process is delayed by time t2. At this time, the control process 235 stops the second process to read and convert the video stream packet of the next sequence number by t2, checks whether the execution of the first process is finished, and repeatedly performs the above-described stem. That is, according to the method of the present invention, all the video stream packets start the conversion in the first and second processes at the same time, but the end of the conversion operation is different. Accordingly, the images displayed on the left and right eyes in the side-by-side form are slightly different from each other, and the 2D image is displayed at a high speed in a side-by-side form and simultaneously provided with a stereoscopic effect. The control process 235 preferably controls the difference in time for completing the task of converting and displaying the video stream packet in each process to be within 30/30 frames, preferably within 30/30 frames. If the time difference is too great, the user will feel fatigue rather than stereoscopic effect.

On the other hand, when the consecutive image frames do not largely change the scene as a video, when an image displayed adjacent to the side by side is displayed in a range of 3/30 to 5/30 frames, The user can feel the same effect as viewing the 3D image without inconvenience. However, even if the image displayed next to the side by side is in the range of 3/5 to 30/30, if the scene of the image changes a lot, the user recognizes the images of scenes completely different from each other as side by side Instead of the effect of the 3D image, it gives visual confusion and fatigue.

Therefore, it is necessary to previously examine the image frame in which the scene changes, and to display the same image frame in the side-by-side form with respect to the image frame in which the scene changes. For example, in FIG. 4, it is assumed that image frames 1 to 50 are photographed images indoors, image frames 51 to 100 are images photographed in the sea, and image frames 101 to 150 are images photographed in the room again . In this case, since the scene changes in the case of the image frame 51 and the image frame 101, it is necessary to control so that the same image frame is displayed when side by side is displayed.

In some embodiments, the control process may read a designated video stream packet prior to the first and second processes, decode the read video packet, and compare the sequentially generated image frames by decoding. The sequential comparison of the image frames can be performed, for example, by performing an exclusive-OR operation (XOR operation) on two temporally successive image frames. If the XOR operation for the image frame is equal to or greater than the predetermined value, it is determined that the scene of the image has changed significantly. For example, the value obtained by XORing the 50th image frame and the 51st image frame with respect to the XOR operation value for the 49th image and the 50th image will be increased beyond a predetermined value. In this case, the 51st image frame can be synchronized to be simultaneously displayed in the side-by-side form in the first and second processes. In addition, the 101st image frame can be synchronized to be simultaneously displayed in side by side in the first and second processes. That is, the control process provides the image frame information to be simultaneously transmitted to the video driver, such as the image frames 51 and 101 in the first and second processes, and the first and second processes respectively output the designated image frames to the side- Format, and transmit the next image frame to the video driver.

Referring to FIGS. 7 and 8, a method of adjusting a side-by-side image displayed on a display to a user's own vision by a user who has an abnormality in the muscle of the pupil of both eyes as in strabismus will be described.

First, targets C1 and C2 for image display area correction are displayed on the first display area CS1 and the second display area CS2 of the display 280, respectively (S10). As shown in FIG. 7 (a), the targets C1 and C2 may use circles marked with center lines, but the present invention is not limited thereto, and various shapes such as rectangles and hexagons can be used. 7 (b) shows a state in which the user who is out of focus of the pupil is recognized when viewing the side by side image displayed as horizontally as shown in Fig. 7 (a) using the mounting-HMD. When a user views the side-by-side image as shown in FIG. 7 (b), the second target C2 is moved by -dx and -dy, respectively. That is, a position adjustment signal for instructing to move the coordinates (CS2) of the second display area by (-dx, -dy) through an input device (for example, a remote controller or a joystick connected via Bluetooth) And displays them on the display (S20, S30). When the user views the side-by-side image, when the target is recognized as being matched, the user inputs an adjustment termination signal inputted and records the position of the coordinates of the adjusted first display area and / or the second display area ( S40, S50). FIG. 7 (c) shows a side by side display form for the first display area and the second display area adjusted for the user.

100 HMD device
200 Portable electronic devices
210 wireless communication module
220 memory
230 CPU
240 first process
250 Second Process
235 Control Process
260 video driver
280 display

Claims (10)

A method of displaying a video stream in a side-by-side format on a display of an electronic device,
Storing a received video stream in a memory,
Wherein the first and second processes concurrently read a video stream packet that is transformed among the video streams stored in the memory and designated to display on a single display,
A first conversion display step of decoding the read video packet, scaling the video frames generated by the decoding to be displayed in the designated first display area of the display, and transmitting the scaled video frames to the video driver,
A second conversion display step of decoding the same video packet read out, scaling the video frames generated by the decoding to be displayed in the designated second area of the display, and transmitting the scaled video frames to the video driver,
When a video stream packet specified in each of the first and second processes is completed, a new video stream packet for converting a video stream packet located after the designated video stream packet and displaying the converted video stream packet on the single display And displaying the video stream in a side-by-side form on the display. The method according to claim 1,
Further comprising the step of controlling the image processing speed of the first process and the second process such that the difference in the completion time of the video conversion display job of the first process and the second process falls within the defined range A method for displaying a video stream on a display side by side. 3. The method of claim 2,
Wherein the difference between the completion times of the image conversion display operations of the first process and the second process is controlled to be within thirty-three frames of the video stream. The method according to claim 1,
Reading the designated video stream packet, decoding the read video packet, and comparing the sequentially generated image frames by decoding,
Further comprising the step of controlling the first and second processes to simultaneously transmit the image frame whose image comparison value is greater than or equal to a predetermined value to the video driver when the comparison value of the image frame is equal to or greater than a predetermined value, A method of displaying a stream in a side-by-side form on a display. 5. The method according to any one of claims 1 to 4,
Displaying a target for image display area correction on the first display area and the second display area, respectively;
Receiving the image display area correction coordinate information of the first display area or the second display area as an input device and changing the coordinates of the image display area in accordance with the image display area correction coordinate information To display on a display in a side-by-side form. A portable electronic device for displaying a video stream in a side-by-side format on a display, the portable electronic device comprising: a controller; a display; a video driver for controlling the display; a wireless communication module; And a memory for storing a program for displaying the converted image,
The program includes:
Storing a video stream received at the wireless communication module in a memory,
Simultaneously reading in the first and second processes a video stream packet designated to be converted and displayed on a single display, among the video streams stored in the memory,
A first conversion display step of decoding the read video packet, scaling the video frames generated by the decoding to be displayed in a designated first display area of the display, and transmitting the scaled video frames to the video driver;
A second conversion display step of decoding the same video packet read out and scaling the video frames generated by the decoding to be displayed in a designated second area of the display and transmitting the scaled video frames to the video driver;
When a video stream packet specified in each of the first and second processes is completed, a new video stream packet for converting and displaying the video stream packet located after the designated video stream packet into the single display And displaying the video stream in a side-by-side form on the display. The method according to claim 6,
The program includes:
Further comprising controlling the image processing speed of the first process and the second process such that the difference between the execution completion time of the image conversion display job of the first process and the second process falls within a defined range, A portable electronic device for displaying a stream in a side-by-side form on a display. 8. The method of claim 7,
Wherein the program controls the completion time difference between the first process and the second process conversion display job to be within three-th-th frame of the video stream, and displaying the video stream on the display in a side-by-side manner ≪ / RTI > The method according to claim 6,
The program includes:
Reading the designated video stream packet, decoding the read video packet, and comparing the sequentially generated image frames by decoding,
When the comparison value of the image frame is equal to or larger than the predetermined value, controlling the image frame to be transmitted to the video driver at the same time in the first process and the second process at the same time A portable electronic device for displaying a video stream in a side-by-side format on a display. 10. The method according to any one of claims 7 to 9,
The program includes:
Displaying a target for image display area correction on the first display area and the second display area, respectively;
Receiving the image display area correction coordinate information of the first display area or the second display area as an input device and changing the coordinates of the image display area in accordance with the image display area correction coordinate information For display on a display in side-by-side form.

Images