JP2013017071A - Control method of video operation experience - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve usability when video which is cut in accordance with a target region that a user designates is displayed on a display unit.SOLUTION: A video processor includes: video accumulating means for accumulating video; target region designating means for designating a target region in accordance with an operation of a user, and operating a panning function and a tilt function of the target region; video cutting means for cutting the target region in accordance with the operation in the target region designating means from the video accumulated in the video accumulating means; and display means for displaying the video of the target region which is cut out. An operation experience control method of the video includes the steps for: allowing the target region designating means to set initial velocity and final velocity of the panning function and the tilt function in accordance with the operation of the panning function and the tilt function by the user at a rate of respective sides of a display region in the display means; and allowing the video cutting means to cut the video accumulated in a video accumulating section according to the initial velocity and the final velocity of the panning function and the tilt function, which are set by a target region designating step.

Description

  The present invention relates to a video operation experience control method for controlling an operation experience when a video is cut out according to a gaze area interactively designated by a user and displayed on various displays.

  2. Description of the Related Art Conventionally, there has been known a video distribution method in which high-resolution video exceeding HD (high definition) can be cut out according to a gaze area that is interactively designated by a user and displayed on various displays (for example, Non-Patent Document 1). In this video distribution method, when the gaze area size and the display area size on the display device are different, as shown in FIG. 8, the expression of the movement of the pan function on the ROI (gaze area) pixel (in the example shown in FIG. 8). , Pan_v = 1 [ROI pixel / sec]) and the expression of the pan function movement in the display area (pan_v = 2 [display pixel / sec] in the example shown in FIG. 8) are also different. For this reason, when the user interactively designates the gaze area, the correspondence between the pixels on the ROI and the pixels on the display changes, and it is not simple to control the operation experience.

  In the conventional operation sensation control method, for the pan function, the ROI is designated to move horizontally by n pixels (n: integer) for one key event, and for the tilt function, the ROI for one key event. Are vertically moved by n pixels (n: integer). For the zoom-in function, the ROI size is specified to be reduced to (100-m)% (m: integer) for one key event, and for the zoom-out function, the ROI is specified for one key event. The size is specified to be expanded to (100 + m)% (m: integer). That is, in the conventional operation sensation control method, when controlling the user's operational sensation related to the change of the gaze area in a high-resolution video such as a panoramic image, the parameters characterizing the operation are absolute to the gaze area. It was specified as a valid value.

Masayuki Inoue, Hideaki Kizen, Katsuhiko Fukasawa, Nobuhiko Matsuura, “Examination of distribution method in interactive panorama video distribution system,” Information Processing Society of Japan, Audio Visual Complex Information Processing Group, Vol. 2010-AVM-69 No. 9, 2010.

  In the conventional operation experience control method, the parameters that characterize the operation are specified as absolute values for the gaze area, so the speed experience of the pan function and tilt function changes according to the gaze area size specified by the user. Therefore, it is impossible to give the user a speed sensation that does not depend on the size of the gaze area. In addition, regarding the zoom function, it is difficult to control what magnification (quality experience) is obtained after T [s] in the conventional operation experience control method. Therefore, there is a problem that the control method of the operation feeling when the high-resolution video is cut out according to the gaze area that the user specifies interactively and displayed on various displays is very unusable.

  The present invention has been made in view of such circumstances, and it is easy to use the operation experience control when displaying on the various displays the video clipped according to the gaze area that the user specifies interactively. It aims to provide a method.

  The present invention relates to a video storage means for storing video, a gaze area specifying means for designating a gaze area in response to a user's operation and performing a pan function and a tilt function of the gaze area, and storing in the video storage means. Operation experience of a video in a video processing apparatus comprising: a video cutout unit that cuts out the gaze region in response to an operation in the gaze region designation unit from a captured video; and a display unit that displays the video of the cut out gaze region In the control method, the gaze area designating unit determines the initial speed and the final speed of the pan function and the tilt function according to the operation of the pan function and the tilt function by the user. A gaze area designating step set by the gaze area designating step; and the panning and tilt functions set by the gaze area designating step Based on the initial speed and the final speed, and having a video clipping step of cutting out an image stored in the image storage unit.

  The present invention is characterized in that the gaze area designating unit expresses a constant speed pan function and a tilt function by matching the initial speed and the final speed.

  The present invention is characterized in that the gaze area specifying means sets a time until the final speed is reached, and expresses a pan function and a tilt function that accelerate by increasing the final speed from the initial speed. .

  The present invention relates to a video storage means for storing video, a gaze area specifying means for designating a gaze area in response to a user's operation and performing a pan function and a tilt function of the gaze area, and storing in the video storage means. Operation experience of a video in a video processing apparatus comprising: a video cutout unit that cuts out the gaze region in response to an operation in the gaze region designation unit from a captured video; and a display unit that displays the video of the cut out gaze region In the control method, the gaze area designating unit determines the initial speed and the final speed of the zoom function according to an operation of the zoom function of the user's gaze area after the predetermined time has elapsed from the start of zooming. The gaze area designating step set by the gaze area designating means and the video clipping means based on the initial speed and the final speed of the zoom function set by the gaze area designating step. , And having a video clipping step of cutting out an image stored in the image storage unit.

  The present invention is characterized in that the gaze area designation means expresses a constant-speed zoom function by matching the initial speed and the final speed.

  The present invention is characterized in that the gaze area designation means expresses a zoom function that accelerates by making the end speed larger than the initial speed.

  According to the present invention, it is possible to control the operational sensation when displaying the video clipped according to the gaze area that the user specifies interactively on various displays.

It is a block diagram which shows the structure of the 1st Embodiment of this invention. It is a flowchart which shows the processing operation of the video processing apparatus shown in FIG. It is explanatory drawing which shows the processing operation of the video processing apparatus shown in FIG. It is explanatory drawing which shows the processing operation of the video processing apparatus shown in FIG. It is a block diagram which shows the structure of the 2nd Embodiment of this invention. It is a flowchart which shows the processing operation of the video processing system shown in FIG. It is explanatory drawing which shows the definition of a term. It is explanatory drawing which shows the video processing by a prior art.

<First Embodiment>
Hereinafter, an image processing apparatus according to a first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a video processing apparatus in the embodiment. In this figure, reference numeral 1 denotes a video storage unit that inputs and stores video captured by an imaging device such as a video camera from the outside, and includes a storage device or the like. Reference numeral 2 denotes a gaze area designating unit that designates a gaze area by a user operation. Reference numeral 3 denotes a video cutout unit that cuts out and outputs the video stored in the video storage unit 1 based on the designation in the gaze area designating unit 2. Reference numeral 4 denotes a video conversion unit that converts the clipped video into a video having a resolution to be displayed and outputs the video. Reference numeral 5 denotes a display unit that displays the resolution-converted video output from the video conversion unit 4, and is configured from a display device.

  Next, the processing operation of the video processing device 6 shown in FIG. 1 will be described with reference to FIG. FIG. 2 is a flowchart showing the processing operation of the video processing apparatus 6 shown in FIG. First, the video storage unit 1 inputs video data from a video camera or a video file (step S1), and stores the input video data therein (step S2).

  Next, the user operates the gaze area designating unit 2 to designate the gaze area (step S3). In response to this, the video cutout unit 3 cuts out the video stored in the video storage unit 1 and outputs it to the video conversion unit 4 according to the designation of the gaze area (step S4). The video conversion unit 4 converts the resolution of the video clipped to have the resolution of the display unit 5 and outputs the converted video (step S5). The display unit 5 displays the video output from the video conversion unit 4 (step S6). When the user further operates the gaze area designating unit 2 and designates the gaze area, the processing operations in steps S3 to S6 are repeated.

  Here, in order to simplify the following description, terms are defined with reference to FIG. FIG. 7 is an explanatory diagram showing definitions of terms. The original pixel is a pixel of the original image, ORG_width is the number of pixels in the horizontal direction of the original image, and ORG_height is the number of pixels in the vertical direction of the original image. ROI is a gaze area. The ROI pixel is an original pixel in the ROI, ROI_width is the number of pixels in the ROI side, and ROI_height is the number of pixels in the ROI length. Further, the display pixels are pixels in the display area on the display (display unit 5), Window_width is the number of pixels in the display area, and Window_height is the number of pixels in the display area.

  Next, with reference to FIG. 3, the processing operation of the gaze area designation (pan / tilt operation) by the user will be described. FIG. 3 is an explanatory diagram showing the processing operation (step S3) of the gaze area designation (pan / tilt operation) by the user. When panning and tilting the gaze area, the pan setting method specifies the time T [s], x0 [%], x1 [%], y0 [%], and y1 [%] as real numbers. For example, at T [s], the display pixel is set to move from the initial speed (x0 / 100) Window_width [display pixel / s] to the final speed (x1 / 100) Window_width [display pixel / s]. The tilt is set to move on the display pixel from the initial speed (y0 / 100) Window_height [display pixel / s] to the final speed (y1 / 100) Window_height [display pixel / s] at T [s]. To do. Further, the pan speed pan_v (t) [display pixel / s] and the tilt speed tilt_v (t) [display pixel / s] are set independently.

After the time T [s] has elapsed from the start of the pan operation, the display pixel moves on the display pixel from the initial speed (x0 / 100) Window_width [display pixel / s] to the final speed (x1 / 100) Window_width [display pixel / s]. When set, the panning speed pan_v (t) after the elapse of t [s] from the start of panning is calculated by the equation (1).
pan_v (t) = [((x1-x0) / 100) (t / T) + (x0 / 100)] (Window_width) [display pixel / s] (1)

  Here, t, T, x0, and x1 are real numbers, t is an elapsed time from the start of pan operation, T is a set time, and x0 and x1 are moving speeds per 1 [s] when Window_width is 100. It represents a percentage [%]. According to the equation (1), it is possible to set constant speed movement when x0 = x1, and acceleration movement when x0 <x1.

Also, when the gaze area is tilted, after the time T [s] has elapsed from the start of the tilting operation, the initial speed (y0 / 100) Window_height [display pixel / s] to the final speed (y1 / 100) Window_height [display pixel / s]. The tilt speed tilt_v (t) after elapse of t [s] from the start of tilt is calculated by the equation (2).
tilt_v (t) = [((y1-y0) / 100) (t / T) + (y0 / 100)] (Window_height) [display pixel / s] (2)

  Here, t, T, y0, and y1 are real numbers, t is an elapsed time from the start of the tilt operation, T is a set time, and y0 and y1 are moving speed ratios per 1 [s] when Window_height is 100. [%].

  Note that if the pan / tilt operation is continued, the image area protrudes, so that the center of gravity of the ROI does not exceed each side of the original image.

  Next, a specific example of each value in the pan / tilt operation will be described. For example, assume that T = 1, x0 = 10, x1 = 10, y0 = 10, and y1 = 20. In this case, in the pan operation, the display pixel moves at a constant speed from the initial speed (10/100) Window_width to the final speed (10/100) Window_width at T = 1 [s]. In the tilt operation, at T = 1 [s], the display pixel is accelerated and moved from the initial speed (10/100) Window_height to the final speed (20/100) Window_height.

  Since the pan speed pan_v (t) and the tilt speed tilt_v (t) can be set independently, the pan speed pan_v (t) is set to pan_v (t) = [(10−10 / 100) (t / 1) + ( 10/100)] (Window_width) = 0.1 Window_width [display pixel / s]. In addition, the tilt speed tilt_v (t) is tilt_v (t) = [(20−10 / 100) (t / 1) + (10/100)] (Window_height) = (0.1t + 0.1) Window_height [display pixel / S] is controlled.

  Next, with reference to FIG. 4, the processing operation of the gaze area designation (zoom operation) by the user will be described. FIG. 4 is an explanatory diagram showing a processing operation of a gaze area designation (zoom operation) by the user. When zooming the gaze area, T [s], z0 [%], z1 [%], p [%], and q [%] are set to real numbers. When zooming out, that is, when ROI is enlarged, , Z0, z1> 100, constant speed expansion is z0 = z1, and acceleration expansion is z1> z0.

  After the time T [s] has elapsed from the start of zoom-out, when setting to enlarge from the initial ROI enlargement rate z0 (> 100)% to the final ROI enlargement rate z1 (> 100)% with respect to the ROI size at the start of zooming, The ROI size is calculated by the equations (3) and (4).

ROI_width (t) = [[((z1-z0) / 100) (t / T) + (z0 / 100)] ^ (t / T)] ROI_width (t0) [ROI pixel / s] (3 ) ROI_height (t) = [[((z1−z0) / 100) (t / T) + (z0 / 100)])] ^ (t / T)] ROI_height (t0) [ROI pixel / s]・ (4)
Here, ^ is a power, t is an elapsed time [s] from the start of zoom, and t0 is a zoom start time [s].

  When zooming in, that is, when ROI is reduced, z0 and z1 <100 may be set, constant velocity ROI reduction is z0 = z1, and acceleration ROI reduction is z1 <z0. When T [s] has elapsed from the start of zooming in and the ROI size at the start of zooming is set to be reduced from the initial ROI enlargement ratio z0 (<100)% to the final ROI enlargement ratio z1 (<100)%, the ROI size Is calculated by the same formula as formulas (3) and (4).

  In order to limit the zoom operation, when the ROI reduction ratio upper limit is p% (p <100), the ROI enlargement ratio upper limit is q% (q> 100), and the reference ROI size is (Window_width, Window_height), the ROI The reduction rate upper limit ROI_limit and the ROI enlargement rate upper limit ROI_limit are calculated by the following equations (5) and (6).

ROI_limit = (p / 100) (Window_width) or (p / 100) (Window_height) [ROI pixel] (p <100) (stops when the upper limit is reached first) (5)
ROI_limit = (q / 100) (Window_width) or (q / 100) (Window_height) [ROI pixel] (q> 100) (stops when the upper limit is reached first) (6)

  Next, a specific example of each value in the zoom operation will be described. For example, assume that T [s], z0 [%], z1 [%], p [%], and q [%] are set as real numbers. In the case of zooming out, the ROI size at the start of zooming is enlarged at a constant speed from ROI enlargement rate z0 = 105% to ROI enlargement rate z1 = 105%, which is aimed after T = 1 [s]. In the case of zoom-in, the ROI size at the start of zooming is accelerated and reduced from ROI enlargement rate z0 = 95%, which is aimed after T = 1 [s], from ROI enlargement rate z1 = 90%. The upper limit of the ROI reduction ratio is p = 50% (p <100), and the upper limit of the ROI enlargement ratio is q = 200% (p> 100). The reference ROI size is (Window_width, Window_height) = (1920, 1080).

  Since zoom-in and zoom-out can be independently set for speed, ROI_width (t) = [[((105−105) / 100) (t / 1) + (105/100)] ^ (t / 1)] ROI_width Control is performed such that (t0) = [1.05 ^ t] ROI_width (t0) [ROI pixel / s]. ROI_width (t) = [[((90−95) / 100) (t / 1) + (95/100)] ^ (t / 1)] ROI_width (t0) = [(− 0.05t + 0.95 ) ^ T] ROI_width (t0) [ROI pixel / s] The control is performed.

  Also, the ROI reduction rate upper limit ROI_limit = (50/100) (1920) or (50/100) (1080) = width 960 or height 540 [ROI pixel] (p <100), whichever comes first when the upper limit is reached Will stop. Also, ROI enlargement rate upper limit ROI_limit = (200/100) (1920) or (200/100) (1080) = width 3840 or height 2160 [ROI pixel] (q> 100), whichever comes first when the upper limit is reached Will stop.

<Second Embodiment>
Next, a video processing system according to the second embodiment of the present invention will be described. FIG. 5 is a block diagram showing the configuration of the video processing system in the embodiment. In this figure, reference numeral 20 denotes a video distribution server that distributes video data, and is configured by a computer device. Reference numeral 30 denotes a video playback client that plays back video, and is configured by a computer device. The video distribution server 20 and the video playback client are connected by a communication network.

  In FIG. 5, reference numeral 11 denotes a video storage unit that inputs and stores video captured by an imaging device such as a video camera from the outside, and includes a storage device or the like. Reference numeral 12 denotes a gaze area designating unit that designates the gaze area by a user operation. Reference numeral 13 denotes a video cutout unit that cuts out and outputs the video stored in the video storage unit 11 based on the designation in the gaze area designating unit 12. Reference numeral 14 denotes a video conversion unit that converts the clipped video into a video having a resolution to be displayed and outputs the video.

  Reference numeral 15 denotes a display unit that displays the resolution-converted video output from the video conversion unit 14 and is configured of a display device. Reference numeral 16 denotes a video transmission unit that transmits the video data output from the video cutout unit 13 to the video reproduction client 30. Reference numeral 17 denotes a video receiver that receives video data transmitted from the video transmitter 16 and outputs the video data to the video converter 14. Reference numeral 18 denotes a command transmission unit that transmits an operation command for the gaze area operated in the gaze area designating unit 12. Reference numeral 19 denotes a command reception unit that receives the operation command transmitted from the command transmission unit 18 and outputs the operation command to the video cutout unit 13.

  Next, the processing operation of the video processing system shown in FIG. 5 will be described with reference to FIG. FIG. 6 is a flowchart showing the processing operation of the video processing system shown in FIG. First, the video storage unit 11 inputs video data from a video camera or a video file (step S11), and stores the input video data therein (step S12).

  Next, the user operates the gaze area designation unit 2 to designate the gaze area (step S13). In response to this, the command transmission unit 18 transmits a command for designating the gaze area (step S14). This command is received by the command receiving unit 19 and transferred to the video cutout unit 13 (step S15). In response to this, the video cutout unit 3 cuts out the video stored in the video storage unit 11 in accordance with the designation of the gaze area (step S16). The video transmission unit 16 transmits the extracted video data to the video reproduction client 30 (step S17).

  Next, the video receiving unit 17 receives the video data transmitted from the video transmitting unit 16 and delivers it to the video converting unit 14 (step S18). In response to this, the video conversion unit 14 converts the resolution of the video clipped so as to have the resolution of the display unit 15 and outputs the converted video (step S19). The display unit 15 displays the video output from the video conversion unit 14 (step S20). When the user further operates the gaze area designating unit 12 to designate the gaze area, the processing operations in steps S13 to S20 are repeated.

  Note that the processing operation for designating the user's gaze area in step S13 is the same as the processing operation shown in FIGS. 3 and 4, and thus detailed description thereof is omitted here.

  As described above, the initial speed and the final speed at the time of panning and tilting are set by the ratio (%) when the horizontal and vertical widths of the screen resolution are set to 100%, and the time interval T [s] until the final speed is reached. In addition, when zooming, since two types of magnifications are set after T [s] from the ROI size at the start of zooming, high-resolution video is clipped according to the gaze area that the user specifies interactively When displaying on a display having various screen resolutions, the pan / tilt speed and the size of the gaze area (the width of the gaze area, after t [s] (t: real number) after the user starts the gaze area. The vertical width of the gaze area can be controlled. For this reason, it becomes possible to control the operation experience quality.

Specifically, it becomes possible to perform the following five operational feeling controls.
(1) The movement speed experience is made equal regardless of the enlargement rate of the gaze area.
(2) The panning and zooming speed after the elapse of T [s] from the start of operation can be set.
(3) The panning and zooming speed can be set to accelerate as T [s] elapses from the start of the operation.
(4) The zoom magnification after the elapse of T [s] from the start of the operation can be predicted.
(5) As T [s] elapses from the start of the operation, the zoom magnification can be set to accelerate.

  In this way, when controlling the user's operational sensation related to changing the gaze area in a high-resolution video such as a panoramic video, parameters that characterize the operation are designated as absolute values for the gaze area. By specifying the relative value as a relative value, it is possible to obtain a user-friendly experience.

  1 and 5, the program for realizing the functions of the gaze area specifying units 2 and 12 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read by a computer system. By executing this, the operation feeling control process for the video may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer system” includes a WWW system having a homepage providing environment (or display environment).

  The “computer-readable recording medium” refers to a storage device such as a flexible medium, a magneto-optical disk, a portable medium such as a ROM and a CD-ROM, and a hard disk incorporated in a computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.

  The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, what is called a difference file (difference program) may be sufficient.

  The present invention can be applied to an indispensable purpose of controlling an operation experience when a video is cut out according to a gaze area designated interactively by a user and displayed on various displays.

  DESCRIPTION OF SYMBOLS 1,11 ... Video | video accumulation | storage part, 2, 12 ... Gaze area designation | designated part, 3, 13 ... Video cut-out part, 4, 14 ... Video conversion part, 5, 15 ... Display part, 6 ... Video processing device, 16 ... Video transmission unit, 17 ... Video reception unit, 18 ... Command transmission unit, 19 ... Command reception unit, 20 ... Video distribution server, 30. ..Video playback client

Claims (6)

Video accumulation means for accumulating video, gaze area designation means for designating a gaze area in response to a user's operation, and performing a pan function and a tilt function of the gaze area, and video accumulated in the video accumulation means A video operation experience control method in a video processing apparatus comprising: a video cutout unit that cuts out the gaze region in response to an operation in the gaze region designation unit; and a display unit that displays a video of the cut out gaze region. And
The gaze area designation means sets the initial speed and the end speed of the pan function and the tilt function according to the ratio of each side of the display area in the display means according to the operation of the pan function and the tilt function by the user. Steps,
The video cutout means has a video cutout step of cutting out the video stored in the video storage unit based on the initial speed and the final speed of the pan function and tilt function set in the gaze area specifying step. The operation experience control method for the video.   The video manipulation experience control method according to claim 1, wherein the gaze area designation unit expresses a constant-speed pan function and a tilt function by matching the initial speed and the final speed.   2. The gaze area designating unit expresses a pan function and a tilt function for accelerating by setting a time to reach the final speed and increasing the final speed from the initial speed. The operation experience control method of the described image. Video accumulation means for accumulating video, gaze area designation means for designating a gaze area in response to a user's operation, and performing a pan function and a tilt function of the gaze area, and video accumulated in the video accumulation means A video operation experience control method in a video processing apparatus comprising: a video cutout unit that cuts out the gaze region in response to an operation in the gaze region designation unit; and a display unit that displays a video of the cut out gaze region. And
The gaze area designation means sets the initial speed and the final speed of the zoom function according to the size of the gaze area after a predetermined time from the start of zooming according to the operation of the zoom function of the gaze area of the user. Steps,
The video cutout means includes a video cutout step of cutting out the video stored in the video storage unit based on the initial speed and the final speed of the zoom function set in the gaze area specifying step. Operation experience control method.   5. The video operation experience control method according to claim 4, wherein the gaze area designation unit expresses a constant-speed zoom function by matching the initial speed and the final speed.   5. The video operation experience control method according to claim 4, wherein the gaze area designation unit expresses a zoom function that accelerates by making the final speed larger than the initial speed.

Images