WO2019004498A1

WO2019004498A1 - Multichannel image generation method, multichannel image playing method, and multichannel image playing program

Info

Publication number: WO2019004498A1
Application number: PCT/KR2017/006869
Authority: WO
Inventors: 정홍수
Original assignee: 포디리플레이 인코포레이티드; 주식회사 이에스엠연구소
Priority date: 2017-06-29
Filing date: 2017-06-29
Publication date: 2019-01-03
Also published as: KR20200001597A

Abstract

A multichannel image generation method, according to one embodiment of the present invention, comprises the steps of: receiving a plurality of images obtained by photographing, at multiple angles, a subject with a plurality of cameras arranged in an array around the subject; correcting each of the plurality of received images; encoding each of the plurality of corrected images through an image compression standard so as to generate a plurality of single channel images; and muxing the plurality of single channel images so as to generate one multichannel image, wherein the plurality of single channel images have a GOP value of 1.

Description

Multi-channel image generation method, multi-channel image reproduction method, and multi-channel image reproduction program

The present invention relates to a multi-channel image generating method, a multi-channel image reproducing method, and a multi-channel image reproducing program, and more particularly, And more particularly to a video playback method and a video playback program capable of switching a channel or a time by an operation of a user.

Currently, a video service is provided that geometrically calibrates and synthesizes a plurality of images obtained by photographing a single subject in various channels with a plurality of cameras, and provides a multi-channel image to a user. These multi-channel images provide realistic images that transcend the concept of high-quality images, thereby allowing users to feel more immersive in the media and to greatly enhance the effect of transmitting video information in advertising, education, medical care, defense and entertainment have.

However, in the conventional multi-channel image, the channel / time switching is simply reproduced by a predetermined merging method when the multi-channel image is produced. That is, conventionally, a plurality of frames are acquired from a plurality of cameras, a part of the acquired frames is selected, and a single channel switching image is produced by merging the frames. Since the channel switching image is a simple combination of a predetermined channel frame at the time of image production, when the corresponding image file is reproduced, the merged frames have a channel switching effect showing a single channel shifting effect. According to such a conventional multi-channel video, the user simply watches a pre-produced channel switching effect. When a viewer manually operates the time switching or channel switching to turn the channel to a desired timing It was impossible to do.

Accordingly, there is a growing demand for multi-channel image generation and multi-channel image reproduction that can produce an effect of manually turning an image according to a user's operation.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned needs, and it is an object of the present invention to provide a method and a method for generating a multi-channel image capable of manually turning an image according to a user's operation.

The objects of the present invention are not limited to those mentioned above, and other objects not mentioned may be clearly understood by those skilled in the art from the following description.

According to a first aspect of the present invention, there is provided a method for generating a multi-channel image, the method comprising: receiving a plurality of images taken from a plurality of cameras arranged in an array around a subject in multiple angles; Correcting each of the plurality of received images; Encoding each of the plurality of corrected images with an image compression standard to generate a plurality of short channel images; And generating a multi-channel image by multiplexing the plurality of short channel images, wherein the plurality of short channel images have a GOP value of 1.

In addition, the image compression standard may be an MJPEG, MPEG4 or H.264 codec.

In addition, the correcting step may correct at least one of a size, a gradient, and a center position of an image.

According to another aspect of the present invention, there is provided a method for reproducing a multi-channel image according to the first aspect of the present invention, A reproducing method, comprising: receiving the multi-channel image from a server; Reproducing the multi-channel image at a predetermined point in time in the received multi-channel image; Receiving at least one of a channel control event, a time control event, a channel and a time control event, and a zoom-in event from a user; And reproducing the multi-channel image while moving a channel or time of the multi-channel image according to the event.

Upon receipt of the user's playback request signal at the end of the control event of the user, the multi-channel image may be played back at the time of the end of the corresponding control event.

Also, the control event may be generated by a user's touch operation on a display unit of a user terminal for reproducing the multi-channel image, wherein the channel control event is generated by a continuous touch operation along a first direction on the display unit, The control event is generated by a continuous touch operation along the second direction on the display unit, and the channel and time control event may be generated by a continuous touch operation along the third direction on the display unit.

In the meantime, the present invention includes a multi-channel image reproducing program stored in a computer-readable recording medium on which a computer program executed by the multi-channel image reproducing method is recorded.

According to another aspect of the present invention, there is provided a method for generating a multi-channel image, the method comprising: receiving a plurality of images of a plurality of cameras arranged in an array around a subject in multiple angles; step; Correcting each of the plurality of received images; Encoding each of the plurality of corrected images with an image compression standard to generate a plurality of short channel images; And extracting frames at a specific time from the plurality of corrected images, and generating a channel switching image composed of the extracted frames.

In addition, the GOP value of the channel switching image may be 1.

According to another aspect of the present invention, there is provided a method for reproducing a multi-channel image according to a second embodiment of the present invention, A playback method comprising: a first step of transmitting a streaming request signal to a server; A second step of selectively receiving one of a plurality of short channel images or a plurality of channel switching images from the server; And a third step of reproducing the received short channel image or channel switching image upon receiving the reproduction request signal from the user.

Also, the second step may receive a predetermined image from the server among the short channel image or channel switching image, or receive the image selected by the user in the step of transmitting the streaming request signal to the server.

In addition, when receiving the short channel image from the server in the second step, after receiving the channel control event from the user after the third step of reproducing the received short channel image, To the server; A fifth step of receiving, from the server, a channel switching image corresponding to a reception time of the channel control event among the plurality of channel switching images; And a sixth step of reproducing the received channel switching image upon receiving the reproduction request signal from the user.

A seventh step of, after the sixth step, transmitting information on generation of the channel control end event to the server upon receiving a channel control end event from the user; An eighth step of receiving, from the server, a short channel image corresponding to a channel at the time of receiving the channel control event end signal among the plurality of short channel images; And a ninth step of receiving a reproduction request signal from the user and reproducing the received short channel image.

Receiving a time control event from a user after a third step of reproducing the received short channel image or after a ninth step of reproducing the received short channel image; And reproducing the time of the received short channel image while moving the time according to the time control event.

In addition, in the case where the channel switching image is received from the server in the second step, after receiving the short channel image playback event from the user after the third step of reproducing the received channel switching image, Transmitting the generated information to the server; A fifth step of receiving, from the server, a short channel image corresponding to a channel at the time of receiving the short channel image reproduction event among the plurality of short channel images; And a sixth step of receiving a reproduction request signal from the user and reproducing the received short channel image.

Receiving a channel control event from a user before or after a third step of reproducing the received channel switching image; And reproducing the channel switching image while moving the viewpoint of the received channel switching image according to the channel control event.

According to another aspect of the present invention, there is provided a multi-channel image reproducing apparatus, comprising: a multi-channel image reproducing program stored in a computer-readable recording medium having a computer program recorded thereon.

According to the multi-channel image generating method, the multi-channel image reproducing method, and the multi-channel image reproducing program according to the embodiment of the present invention, the user directly operates the input means of the terminal to freely switch the channel or time of the reproduced image, Channel image generating method, a multi-channel image reproducing method, and a reproducing program capable of performing multi-channel image reproduction.

In addition, by specifying the GOP size of the multi-channel image to be 1, it provides an advantage that faster decoding can be performed when channel switching or time switching operation is applied to the image.

The effects of the present invention are not limited to those mentioned above, and other effects not mentioned can be clearly understood by those skilled in the art from the following description.

1 is a flowchart illustrating a method of generating a multi-channel image according to a first embodiment of the present invention.

2 is a conceptual view showing a plurality of camera arrangement structures arranged in an array around a subject.

3 is a diagram conceptually showing each image received from a plurality of cameras in a frame.

FIG. 4 conceptually shows a plurality of short channel images generated by encoding a plurality of images.

5 is a diagram conceptually showing one multi-channel image generated by synthesizing a plurality of short channel images.

6 is a flowchart illustrating a method of reproducing a multi-channel image according to the first embodiment of the present invention.

FIG. 7 is a flowchart illustrating a method of generating a multi-channel image according to a second embodiment of the present invention.

8 is a diagram conceptually showing a plurality of short channel images generated by encoding a plurality of images.

9 is a conceptual view illustrating a channel switching image.

10 is a flowchart illustrating a method of reproducing a multi-channel image according to a second embodiment of the present invention.

FIG. 11 is a conceptual diagram illustrating a reproduction scenario of a multi-channel image according to a second embodiment of the present invention.

11 is a conceptual diagram showing another reproduction scenario of a multi-channel image according to the second embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects, features and advantages of the present invention will be more apparent from the following detailed description taken in conjunction with the accompanying drawings, in which: FIG. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unnecessary. The terms described below are defined in consideration of the structure, role and function of the present invention, and may be changed according to the intention of the user, the intention of the operator, or the custom.

The present invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. These embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art to which the present invention pertains, It is only defined by the scope of the claims. Therefore, the definition should be based on the contents throughout this specification.

Throughout the specification, when an element is referred to as " comprising ", it means that it can include other elements as well, without excluding other elements unless specifically stated otherwise.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1st In the embodiment Multi-channel image generation method and multi-channel image reproduction method

[Multichannel image generation method]

1 is a flowchart illustrating a method of generating a multi-channel image according to an exemplary embodiment of the present invention. A multi-channel image generating method according to the present invention includes: a step (S1) of receiving a plurality of images taken from a plurality of cameras arranged in an array around a subject in multiple angles, a step of correcting each of the plurality of received images S2), generating a plurality of short channel images by encoding each of the corrected plurality of images to an image compression standard, and generating a multi-channel image by synthesizing a plurality of short channel images (MUXING) (S4).

First, in a step S1, a plurality of cameras arranged in an array around a subject receives a plurality of images of the subject photographed at multiple angles. 2 is a conceptual view showing a plurality of camera arrangement structures arranged in an array around a subject. The plurality of camera array methods may be, for example, arranging n cameras in a line on substantially the same plane with respect to any camera. In this case, n cameras may be sequentially arranged on a circle spaced a certain distance from the subject, or the distances between the two cameras disposed on both sides of the camera positioned at the center may be matched with the distance between the subject and the camera The distance to the subject may be different. 3 is a diagram conceptually showing each image received from a plurality of cameras in a frame. n cameras capture the same subject at multiple angles, each camera acquires an image for a predetermined time at a specific channel (viewpoint), and the server receives the acquired n images. In the present invention, the subject is not particularly limited. For example, when a batting stadium of a baseball game is set as a subject, n-angle-specific images of the target are received by the server.

Next, in step S2, each of the plurality of received images is corrected. Since the geometric error of the n camera arrays appears as a visual shake in the process of reproducing a multi-channel image, at least one of size, slope or center position of each image is corrected to remove it. Each corrected image may be generated as a container, for example, MP4 or TS.

Generally, a video stream is composed of three types of frames (Frames). The three types of frames are an intra-frame (I frame), a predictive frame (P frame), and a bi-directionally predictive frame (B frame). The I frame does not use the motion vector estimation / compensation but simply DCT-codes the frame. The P frame performs motion estimation / compensation with reference to the I frame or another P frame, and then DCT-codes the remaining difference data , B frame performs motion estimation / compensation from two frames on the time axis using motion compensation like P frame. In a video stream, a sequence is defined by a segment called a group of pictures (GOP), and I, B, B, P, B, B, P ... A frame from an I frame to a next I frame is referred to as a GOP (Group of Picture). Typically, a GOP is composed of a set of images having a duration of a predetermined time (e.g., 0.5 seconds) when displayed at an intended speed. As shown in FIG. 3, each of the corrected images described above is a set of GOPs composed of a single I frame, a single number or a plurality of B frames, and a single or plural P frames. Each corrected image includes one I frame and at least one P frame or B frame.

Next, in step S3, a plurality of (n) short channel images are generated by encoding each of the plurality of corrected images using the software with an image compression standard. In this case, the image compression standard is preferably an MJPEG, MPEG4 or H.264 codec.

FIG. 4 conceptually shows a plurality of short channel images generated by encoding a plurality of images. At this time, the short channel image generated through step S3 as described above is composed of a plurality of I frames.

More specifically, MJPEG is an image compression method for compressing an image frame by frame, and is a kind of video stream in which each frame is compressed by JPEG and time information is inserted between JPEG frames. The corrected image before being encoded by the MJPEG codec is present, for example, as an MP4 or a TS container, and the corresponding image includes one I frame and at least one P frame or B frame as shown in FIG. On the other hand, in the image compression standard, that is, the short channel image encoded by the MJPEG codec, since each image is compressed for each frame, each frame has one complete image. In other words, the above-described short channel image is an image having a GOP size of 1, in which all frames in the image are I frames. In step S3, each of the plurality of images is encoded with the MJPEG codec, thereby substantially converting the GOP size to 1.

If the image is encoded by the MJPEG codec, the capacity becomes larger due to the compression in units of frames. However, unlike image codecs capable of decoding the image in accordance with the GOP unit, It has the advantage of faster decoding in real time access. In the case where each image is encoded with the MPEG4 or H.264 codec, the GOP size of the short channel image is substantially converted to 1, which is the same as the MJPEG described above.

When an operation of turning the reproduction time forward or backward or reproducing a channel (i.e., a time point) during the reproduction of an image is performed, switching to a specific target frame is practically performed. If the GOP of the image is not 1, In the case of an image including other B or P frames, in order to reproduce a perfect image by searching a specific frame designated by the user, the process is not immediate and a time lag occurs. For example, when the target frame is a B frame or a P frame, an I frame immediately before the target frame is searched and the image is connected to the target frame. In this process, a time delay occurs. If a plurality of images are synthesized without being encoded by an MJPEG, MPEG4 or H.264 codec, the synthesized image is not a GOP of 1, so that when a time switching or a channel switching event occurs, Time lag occurs, reproduction is interrupted, and a problem that image reproduction is delayed every time an event occurs continuously occurs. According to the present invention, in step S3, each image having a non-1 GOP size is encoded with an MJPEG, MPEG4, or H.264 codec, thereby substantially converting the GOP size to 1.

Next, in step S4, a plurality of short channel images are synthesized (MUXING) to generate one multi-channel image. 5 is a diagram conceptually showing one multi-channel image generated by synthesizing a plurality of short channel images. A multi-channel image reproduced by combining a plurality of short channel images and regenerated as one media file may be generated as a container, for example, MP4 or TS. In step S4, a plurality of short channel images are synthesized (MUXING) And generates a single multi-channel image.

[Multi-channel video playback method]

6 is a flowchart illustrating a method of reproducing a multi-channel image according to the first embodiment of the present invention. The multi-channel image reproduction method of the present invention is a multi-channel image reproduction method for reproducing a multi-channel image generated by the above-described method in a user terminal.

More specifically, the multi-channel image playback method includes receiving a multi-channel image from the server (S11), receiving a playback request signal from the user, and reproducing the multi-channel image at a predetermined point in time of the received multi- (S31) of receiving a channel control event, a time control event, a channel and a time control event, and a zoom-in event from a user (S31) (S41) of moving a channel or time of the screen or enlarging a screen and reproducing a multi-channel image.

First, in step S11, a multi-channel image is received from the server. A multi-channel image is a combination of images of a subject photographed at various angles for a predetermined time, and has channel information (viewpoint information) together with time information.

Next, in step S21, upon receipt of the user's reproduction request signal, the multi-channel image is reproduced on the user's terminal at a preset specific time point of the received multi-channel image. In the user terminal, for example, a multi-channel image can be reproduced at the time of the fourth camera, and the channel can be preset by the user in the reproduction program and manually selected before reproduction.

Next, in step S31, the user receives at least one of a channel control event, a time control event, a channel and time control event, and a screen zoom event. The above-described user terminal can perform both a display function and an input function, and can be a terminal equipped with a touch screen equipped with various types of touch-based user interfaces. The control event may be generated by a user's touch operation on the display unit of the user terminal, the channel control event is generated by a continuous touch operation along the first direction on the display unit, And the channel and time control event may be generated by a continuous touch operation along the third direction on the display unit. For example, the time can be moved by dragging the liquid crystal screen, which is the display unit, in the left and right direction by using a user's finger or a separate touch input means, and the channel can be moved by vertically dragging, And is set to simultaneously move the channels. In addition, a zoom-in event can be controlled by zooming out two or more points of an area to be enlarged on the screen of the display section or by touching an enlargement icon on the display section. A known touch interface input recognition technique is adopted as a method of recognizing the initial recognition position and the final recognition position when applying the touch operation and the drag input between them.

Next, in step S41, a multi-channel image is reproduced while moving the channel or time of the multi-channel image according to the event. Specifically, when a continuous touch operation is applied from the user along the first direction (that is, when a channel control event is recognized) while a multi-channel image is being reproduced on a specific channel, only the channel is switched while the time is fixed at t ₁ The frame is reproduced to move. For example, assuming that the multi-channel image is reproduced at the viewpoint of the fourth camera and the first direction is the up-and-down direction, if the upward drag input is recognized by the user, it is fixed at time t ₁ , 6, and 7 in the direction of the camera. On the other hand, if the downward drag input is recognized from the user, the channel is switched while being fixed at time t ₁ at the time of inputting the control event while being switched in the direction of the

camera

3, 2 and 1. Further, when the operation direction input from the top switch to the lower drag and drag input, is switched to channel the station only at that time t _1. When a continuous touch operation is applied from the user along the second direction (that is, when a time control event is recognized) while the multi-channel image is being reproduced at a specific time point, The frame is reproduced. For example, assuming that the multi-channel image is reproduced at the viewpoint of the fourth camera and the second direction is the left-right direction, when the dragging input from the user to the left direction is recognized, the channel is fixed and the time is previously switched while being reproduced.

Thereafter, when the control event of the user is terminated, a step S51 of successively reproducing the multi-channel image at the time of the end of the control event upon receiving the user's reproduction request signal is further performed. The control event end signal can recognize a signal that the continuous touch input from the user is no longer recognized by the terminal as a termination signal.

According to such a multi-channel image reproducing method, it is possible to display a multi-channel image while viewing the image by manually turning the reproduction time or the reproduction time according to the user's operation.

[Multi-channel video playback program]

The present invention further includes a multi-channel image playback program stored in a computer-readable recording medium on which the computer program is recorded, the multi-channel image playback method.

Second In the embodiment Multi-channel image generation method and multi-channel image reproduction method

[Multi-channel image generation method]

FIG. 7 is a flowchart illustrating a method of generating a multi-channel image according to a second embodiment of the present invention. A multichannel image generating method of the present invention comprises the steps of: (S1) receiving a plurality of images of a plurality of cameras arranged in an array around a subject at a plurality of angles; A step (S2) of generating a plurality of short channel images by encoding each of the plurality of corrected images with an image compression standard, and a step (S3) of extracting frames at a specific time with respect to the plurality of corrected images , And generating a channel switching image composed of the extracted frames (S4).

Steps S1 to S3 are substantially the same as the method of generating a multi-channel image according to the first embodiment of the present invention, and therefore, the above description is referred to. In step S3, a plurality of short-channel images are generated by encoding each of the plurality of images into an image compression standard. The short-channel image may be produced such that the GOP size is 1 as necessary. That is, the GOP size of the short channel image is substantially converted to 1 by encoding each of the non-1 GOP images into the image compression standard. At this time, the image compression standard is preferably MJPEG, MPEG4 or H.264. The image before being encoded exists, for example, as an MP4 or a TS container, and one I frame and at least one P frame or B frame are included in the image. On the other hand, since a single channel image encoded with an MJPEG, MPEG4 or H.264 codec is subjected to image compression on a frame-by-frame basis, each frame has one complete image. In other words, as shown in FIG. 9, the short channel image can be generated as an image having a GOP size of 1, in which all frames in an image are I frames.

Next, in step S4, the software is used to extract frames at a specific time from the plurality of corrected images, and a channel switching image composed of the extracted frames is generated. 9 is a conceptual view illustrating a channel switching image. More specifically, first, frames at each time are extracted and image-processed to acquire a plurality of images having the same time but moving channels.

By performing this step, the server has a plurality of short channel images fixed at the time taken by each camera and a channel switching image for stopping at a specific time but moving only the channel for each time t.

At this time, the channel switching image can be made to have a GOP value of 1. For example, all frames of an image acquired by each camera may be converted into an I frame image, and a channel switching image may be generated with I frames obtained at each time t. Alternatively, after generating a channel switching image, a plurality of channel switching images may be generated by encoding the corresponding channel switching image into an image compression standard. That is, by performing encoding of a channel switching image with a GOP size of not 1 as an image compression standard, the GOP size of the channel switching image is substantially converted to 1. At this time, the image compression standard is preferably MJPEG, MPEG4 or H.264.

[Multi-channel video playback method]

10 is a flowchart illustrating a method of reproducing a multi-channel image according to a second embodiment of the present invention. The multi-channel image reproduction method of the present invention is a multi-channel image reproduction method for reproducing a multi-channel image generated by the above-described method in a user terminal.

Specifically, a first step (S11) of transmitting a streaming request signal to a server, a second step (S21) of selectively receiving one of a plurality of short channel images or a plurality of channel switching images from the server, And a third step (S31) of reproducing the received short channel image or channel switching image upon receiving the reproduction request signal.

In step S11, a streaming request signal is transmitted to the server. In step S21, one of the plurality of short channel images or the plurality of channel switching images is selectively received from the server. The server may previously specify whether to transmit the short channel image to the user terminal or to transmit the channel switching image to the user terminal. Alternatively, in the step of transmitting the streaming request signal to the server, the user may select one of the short channel image or the channel switching image, transmit the streaming request signal, and receive the designated image. Hereinafter, the case of receiving a short channel image and the case of receiving a channel switching image will be described.

First, when a short channel image is received from the server in the second step, the received short channel image is reproduced upon receiving the reproduction request signal from the user in the third step. At this time, the streaming request signal may include channel information regarding an initial playback start time, and may further include information on an initial playback start time. The user terminal receives and reproduces a short channel image corresponding to the corresponding channel information among a plurality of short channel images from the server. For example, FIG. 11 exemplarily illustrates a video reproduction scenario when a streaming request signal includes a second camera as channel information.

Next, in a fourth step, upon receiving a channel control event from the user, the channel control event generation information is transmitted to the server. In the next step 5, a channel switching image corresponding to the reception time of the channel control event among the plurality of channel switching images is received from the server. In the sixth step, upon receiving the reproduction request signal from the user, the received channel switching image is reproduced.

At this time, the channel control event generation information may include information on the event generation time. Again, 11, the camera 2 when the one-channel image to the point being played, when the user of the channel control event at time t ₁ recognized, the information including information of event occurrence time t ₁ server And the server transmits a channel switching image corresponding to t ₁ of the plurality of channel switching images to the user terminal. The user terminal reproduces the channel switching image received from the server.

The channel control event may be generated by a touch operation of the user with respect to the display unit of the user terminal, and may be generated by a continuous touch operation along the first direction on the display unit of the user terminal. For example, a channel (viewpoint, viewpoint) can be moved by vertically dragging a liquid crystal screen, which is a display unit, using a user's finger or another touch input means. For example, when the first direction is to assume that the upper and lower direction, when a drag input operation is continuous along the lower side by the user at time t ₁ is applied, the user terminal in the channel switching image corresponding to of t _1, a plurality of channel switching image server From the server. When the downward drag input operation is continuously applied from the user, the received channel switching image is reproduced while the channel is being switched in the direction of the

camera

3, 4, 5 while the time is fixed at t ₁ 11). On the contrary, when the upward drag input is recognized by the user, the received channel switching image is reproduced while being switched to the direction of the first camera.

Next, after the sixth step, in the seventh step, upon reception of the channel control end event from the user, information on generation of the channel control end event is transmitted to the server. In the eighth step, a short channel image corresponding to the channel at the time of receiving the channel control event end signal among the plurality of short channel images is received from the server. In the ninth step, the reproduction request signal is received from the user, Plays the channel image.

At this time, the channel control end event generation information may include information on a channel (viewpoint) at the end of the event. 11, when a channel switching image at time t ₁ is being reproduced and a channel control end event of the user is recognized at a time when the channel switching reaches the camera 5, Is transmitted to the server. The channel control end event can recognize a continuous touch input from the user as a signal that is no longer recognized by the terminal as a termination signal. In this case, the server selects a short channel image corresponding to the fifth channel among a plurality of short channel images, and transmits the short channel image to the user terminal. When there is a playback request signal from the user, the user terminal reproduces the short channel image received from the server from time t ₁ onward.

According to another aspect of the present invention, there is provided a method for controlling an image processing apparatus, the method comprising: receiving a time control event from a user; This reproduction process may be performed after the third step of reproducing the short channel image received at the user terminal, or after the ninth step of reproducing the received short channel image. For example, when a specific multi-channel image is being reproduced and a continuous touch operation is applied from the user along the second direction (that is, when a time control event is recognized), the time point (view point) The frame is reproduced so as to be switched. For example, when the multi-channel image is reproduced at the viewpoint of the fifth camera and the second direction is the left-right direction, when the dragging input from the user to the left direction is recognized, the time is previously switched while the viewpoint is fixed.

On the other hand, when receiving the channel switching image from the server in the second step, in the third step, the received channel switching image is reproduced upon receiving the reproduction request signal from the user. At this time, the streaming request signal may include time information regarding an initial playback start time, and may further include information regarding an initial playback start time. The user terminal can reproduce the channel switching video from the predetermined time in the predetermined channel. For example, in FIG. 12, the image reproduction scenario is exemplarily shown when the initial reproduction start time is set to t ₁ in the streaming request signal and the initial reproduction time is set to the second camera.

Next, in a fourth step, a short channel image reproduction event is received from the user, and the generation information of the short channel image reproduction event is transmitted to the server. In the fifth step, a short channel image corresponding to the channel at the time of receiving the short channel image reproduction event among the plurality of short channel images stored in the server is received from the server. In the sixth step, the reproduction request signal is received from the user and the received short channel image is reproduced. At this time, the information on the channel at the time of event generation may be included in the generation information of the short channel image reproduction event. For example, referring to FIG. 12, it is assumed that the channel starts from camera No. 2 while the time is fixed at t ₁ , the image is reproduced while the channel is switched to the camera No. 4, When the user's short channel image reproduction event is recognized, information on generation of the short channel image reproduction event, that is, information on the channel (camera No. 5) is transmitted to the server. At this time, the touch input of the user can be recognized as a short channel image reproduction event. In this case, the server selects a short channel image corresponding to the fourth channel among the plurality of short channel images, and transmits the short channel image to the user terminal. When there is a playback request signal from the user, the user terminal reproduces the fourth channel image received from the server from time t ₁ onward.

Thereafter, receiving the time control event from the user, and reproducing the time-shifted short channel image according to the time control event. This reproduction process may be performed after the third step of reproducing the short channel image received by the user terminal, or after the sixth step of reproducing the received short channel image. For example, referring to FIG. 12, when a continuous touch operation is applied from the user in the second direction (that is, when a time control event is recognized) when the fourth multi-channel image is being played back, The frame is reproduced so as to move only the time while being fixed to the channel. For example, when assuming that the second direction is the horizontal direction, when the leftward drag input recognized by the user, there is the point that a fixed holding time can not be reproduced while switching between t ₁ and t _2.

On the other hand, when the channel control event is received from the user after the sixth step, channel control event generation information is transmitted to the server. Then, a channel switching image corresponding to the reception time of the channel control event among the plurality of channel switching images is received from the server. Upon receiving the reproduction request signal from the user, the received channel switching image is reproduced.

At this time, the channel control event generation information may include information on the event generation time. Again, Referring to Figure 12, fourth camera, when the one-channel image to the point being played, when the user of the channel control event at time t ₂ recognized, the information including information of event occurrence time t ₂ server is sent to, the server transmits a channel switch video corresponding to t ₂ of the plurality of switching video channels in a user terminal. The user terminal reproduces the channel switching image received from the server. For example, when the first direction is to assume that the upper and lower direction, when a drag input operation is continuous along the lower side by the user at time t ₂ is applied, the user terminal in the channel switching image corresponding to of t ₂ a plurality of channel switching image server From the server. Then, when the user is continuing the downward drag input operation, there is reproduced the received channel switching image is reproduced as only the time channels fixed in the holding t ₂ 5, 6, 7 by switching the direction of the camera. On the contrary, when the upward drag input is recognized by the user, the received channel switching image is reproduced while being channel switched to the camera direction # 3.

Next, upon receipt of the channel control end event from the user, the generation information of the channel control end event is transmitted to the server. Next, a short channel image corresponding to the channel at the time of receiving the channel control event end signal among the plurality of short channel images is received from the server, and the received short channel image is reproduced upon receiving the reproduction request signal from the user. At this time, the channel control end event generation information may include information on a channel (viewpoint) at the end of the event. 12, when a channel switching image at time t ₂ is being reproduced and a channel switching event is recognized at the N-1th camera, a channel control end event of the user is recognized, Is transmitted to the server. The channel control end event can recognize a continuous touch input from the user as a signal that is no longer recognized by the terminal as a termination signal. In this case, the server selects a short channel image corresponding to the (N-1) th channel among a plurality of short channel images and transmits the selected short channel image to the user terminal. When there is a reproduction request signal from the user, the user terminal reproduces the N-1th channel image received from the server from time t ₂ onward.

According to another aspect of the present invention, there is provided a method for controlling an image processing apparatus, the method comprising: receiving a time control event from a user; For example, when a continuous touch operation is applied from the user along the second direction (that is, when a time control event is recognized) while the N-1 multi-channel image is being reproduced, the viewpoint is fixed to the N-1 channel The frame is reproduced so that only the time is switched.

On the other hand, the channel switching control is possible before or after the third step, i.e., the step of reproducing the received channel switching image. That is, the user can watch an image while switching the channel even before receiving and reproducing the channel switching video or while reproducing the channel switching video. Specifically, the method may include receiving a channel control event from a user, and reproducing the channel switching image while moving the received channel switching image according to a channel control event. For example, when a downward drag input operation is applied from the user, the channel switching image is reproduced while switching only the channel while the time is fixed. On the other hand, if the user inputs an upward drag input, the received channel switching image is reproduced while the channel is switched in the opposite direction.

[Multi-channel video playback program]

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed embodiments, but, on the contrary, It is to be understood that the invention is not limited thereto and that various changes and modifications may be made therein without departing from the scope of the invention.

Claims

The method comprising: receiving a plurality of images of a plurality of cameras arranged in an array around a subject, the plurality of images being obtained by photographing the subject in multiple angles;

Correcting each of the plurality of received images;

Encoding each of the plurality of corrected images with an image compression standard to generate a plurality of short channel images; And

Generating a multi-channel image by synthesizing the plurality of short channel images;

Lt; / RTI >

Wherein the plurality of short channel images have a GOP value of 1.
The method according to claim 1,

Wherein the image compression standard is an MJPEG, MPEG4 or H.264 codec.
The method according to claim 1,

Wherein the correcting step corrects at least one of a size, a gradient, or a center position of the image.
A multi-channel image reproducing method for reproducing a multi-channel image generated by the method of any one of claims 1 to 3 in a user terminal,

Receiving the multi-channel image from a server;

Reproducing the multi-channel image at a predetermined point in time in the received multi-channel image;

Receiving at least one of a channel control event, a time control event, a channel and a time control event, and a zoom-in event from a user; And

Reproducing the multi-channel image while moving a channel or time of the multi-channel image according to the event;

Wherein the multi-channel image reproduction method comprises the steps of:
5. The method of claim 4,

Upon receipt of a user's playback request signal at the end of the control event of the user, playing back the multi-channel image at a point of time when the control event ends;

Further comprising the steps of:
5. The method of claim 4,

Wherein the control event is generated by a user's touch operation with respect to a display unit of a user terminal through which the multi-

Wherein the channel control event is generated by a continuous touch operation along a first direction on the display unit and the time control event is generated by a continuous touch operation along a second direction on the display unit, Wherein the display is generated by a continuous touch operation along a third direction on the display unit.
A multi-channel image reproducing program stored in a computer-readable recording medium having recorded thereon a computer program executed by the multi-channel image reproducing method of claim 6.
The method comprising: receiving a plurality of images of a plurality of cameras arranged in an array around a subject, the plurality of images being obtained by photographing the subject in multiple angles;

Correcting each of the plurality of received images;

Encoding each of the plurality of corrected images with an image compression standard to generate a plurality of short channel images; And

Extracting frames at a specific time from the plurality of corrected images, and generating a channel switching image including the extracted frames;

And generating a multi-channel image.
9. The method of claim 8,

Wherein the channel switching image has a GOP value of 1.
9. The method of claim 8,

Wherein the image compression standard is an MJPEG, MPEG4 or H.264 codec.
9. The method of claim 8,

Wherein the correcting step corrects at least one of a size, a gradient, or a center position of the image.
A multi-channel image reproducing method for reproducing a multi-channel image generated by the method of any one of claims 8 to 11 in a user terminal,

A first step of transmitting a streaming request signal to a server;

A second step of selectively receiving one of a plurality of short channel images or a plurality of channel switching images from the server; And

A third step of reproducing the received short channel image or channel switching image upon receiving the reproduction request signal from the user;

Wherein the multi-channel image reproduction method comprises the steps of:
13. The method of claim 12,

The second step comprises:

Wherein the server receives a predetermined image from the server of the short channel image or channel switching image or receives a user selected image in the step of transmitting a streaming request signal to the server.
13. The method of claim 12,

When receiving the short channel image from the server in the second step,

After the third step of reproducing the received short channel image,

A fourth step of, when receiving the channel control event from the user, transmitting the channel control event generation information to the server;

A fifth step of receiving, from the server, a channel switching image corresponding to a reception time of the channel control event among the plurality of channel switching images;

A sixth step of reproducing the received channel switching image upon receiving the reproduction request signal from the user;

Wherein the multi-channel image reproduction method comprises the steps of:
15. The method of claim 14,

After the sixth step,

A seventh step of, upon receiving a channel control end event from the user, transmitting the channel control end event generation information to the server;

An eighth step of receiving, from the server, a short channel image corresponding to a channel at the time of receiving the channel control event end signal among the plurality of short channel images; And

A ninth step of receiving a reproduction request signal from a user and reproducing the received short channel image;

Further comprising the steps of:
15. The method of claim 14,

A third step of reproducing the received short channel image, or a ninth step of reproducing the received short channel image,

Receiving a time control event from a user; And

Reproducing the received short channel image while moving the time according to the time control event;

Wherein the multi-channel image reproduction method comprises the steps of:
13. The method of claim 12,

When receiving the channel switching image from the server in the second step,

After the third step of reproducing the received channel switching image,

A fourth step of receiving a short channel image reproduction event from a user and transmitting generation information of the short channel image reproduction event to the server;

A fifth step of receiving, from the server, a short channel image corresponding to a channel at the time of receiving the short channel image reproduction event among the plurality of short channel images;

A sixth step of receiving a reproduction request signal from the user and reproducing the received short channel image;

Wherein the multi-channel image reproduction method comprises the steps of:
18. The method of claim 17,

Before or after the third step of reproducing the received channel switching image,

Receiving a channel control event from a user; And

Moving the viewpoint of the received channel switching image according to the channel control event;

Wherein the multi-channel image reproduction method comprises the steps of:
A multi-channel image reproducing program stored in a computer-readable recording medium on which a computer program recorded by the multi-channel image reproducing method of claim 11 is recorded.