WO2018016434A1 - Dispositif et procédé d'acquisition de vidéo condensée, et support d'enregistrement - Google Patents

Dispositif et procédé d'acquisition de vidéo condensée, et support d'enregistrement Download PDF

Info

Publication number
WO2018016434A1
WO2018016434A1 PCT/JP2017/025691 JP2017025691W WO2018016434A1 WO 2018016434 A1 WO2018016434 A1 WO 2018016434A1 JP 2017025691 W JP2017025691 W JP 2017025691W WO 2018016434 A1 WO2018016434 A1 WO 2018016434A1
Authority
WO
WIPO (PCT)
Prior art keywords
frame
video
change information
partial
acquisition unit
Prior art date
Application number
PCT/JP2017/025691
Other languages
English (en)
Japanese (ja)
Inventor
露峰　浩
一巧 黒木
Original Assignee
Ｓｅｖｅｎ Ｄｅｗ株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Ｓｅｖｅｎ Ｄｅｗ株式会社 filed Critical Ｓｅｖｅｎ Ｄｅｗ株式会社
Publication of WO2018016434A1 publication Critical patent/WO2018016434A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
    • H04N21/85—Assembly of content; Generation of multimedia applications
    • H04N21/854—Content authoring
    • H04N21/8549—Creating video summaries, e.g. movie trailer
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor

Definitions

  • the present invention relates to a digest video acquisition device for acquiring a digest video from a video.
  • video search digest generation device that identifies a digest that satisfies a search request and presents the digest to the user (see Patent Document 1).
  • video is input, stored in video content storage means, video is read from video content storage means, analyzed, keywords are extracted, stored in search digest storage means, and video is read from video content storage means Then, it is stored in the search digest storage means as summary information that is information for generating a digest of the section into which the video is divided.
  • search digest storage means acquires a keyword corresponding to the search term designated by the user from the search digest storage unit, acquires summary information corresponding to the keyword from the search digest storage unit, and acquires the summary from the video content storage unit.
  • It is a device that acquires a video of information classification as a digest and outputs it to the display means of the user.
  • the digest video acquisition device includes a change information acquisition unit that acquires inter-frame change information that is information indicating a change between two or more frames of two or more frames constituting a video, A partial video acquisition unit that acquires one or more partial videos that are part of the video using the change information; and a storage unit that stores a digest video having one or more partial videos acquired by the partial video acquisition unit; Is a digest video acquisition device.
  • an appropriate digest video can be acquired because the digest video is acquired using inter-frame change information indicating a change between frames.
  • the digest video acquisition device of the second aspect of the present invention in contrast to the first aspect, when the partial video acquisition unit is information indicating a change that is large enough that the inter-frame change information satisfies a predetermined condition, A digest video acquisition device that acquires one or more partial videos including a frame corresponding to inter-frame change information.
  • the change information acquisition unit acquires two or more feature amounts related to a change between two frames, in comparison with the first or second aspect.
  • the digest video acquisition device further includes a reception unit that receives the number of partial videos, which is the number of partial videos, for any one of the first to third inventions, and the partial video acquisition unit Is a digest video acquisition device that acquires partial videos corresponding to the number of partial videos.
  • the digest video acquisition device is a selection condition storage for storing a first selection condition for selecting a start frame of a partial video for any one of the first to third aspects of the invention.
  • the partial video acquisition unit includes a frame selection unit that selects one or more frames that match the first selection condition, and one or more frames that use one or more frames selected by the frame selection unit as a start frame.
  • a digest video acquisition device comprising partial video acquisition means for acquiring a partial video.
  • the digest video acquisition device of the sixth aspect of the present invention is the fifth aspect of the invention, wherein the partial video acquisition means includes two or more scenes from the start frame and the next start frame or video end frame. It is a digest video acquisition device that acquires a partial video having a length corresponding to the length of a scene.
  • the digest video acquisition device is the fifth aspect of the invention, wherein the partial video acquisition means includes two or more scenes from the start frame and the next start frame or video end frame.
  • This is a digest video acquisition device that acquires a partial video of a certain length.
  • the selection condition storage unit has a second selection condition for selecting an end frame of the partial video, for any one of the fifth to seventh inventions.
  • the storing and partial video acquisition means is a digest video acquisition device that selects one or more frames that match the second selection condition and acquires partial videos from the start frame to the selected end frame.
  • an appropriate digest video can be acquired because the digest video is acquired using the inter-frame change information.
  • Block diagram of digest video acquisition apparatus 1 in the first embodiment A flowchart for explaining the operation of the digest video acquisition device 1 Flow chart for explaining a first example of inter-frame change information acquisition processing The flowchart explaining the 2nd example of the change information acquisition process between the frames Conceptual diagram showing calculation of interframe change information The figure which shows the same frame change information management table Overview of the computer system Block diagram of the computer system
  • a digest video acquisition device that acquires inter-frame change information that is change information between frames of a video and acquires a digest video that is a part of the video using the inter-frame change information will be described. .
  • a part of the video is acquired by using a frame corresponding to information indicating a change large enough that the inter-frame change information satisfies a predetermined condition as a start frame of the scene, and a part of the video is acquired.
  • a digest video acquisition device that acquires a digest video including the above will be described.
  • one frame-to-frame change information is acquired for each frame from two or more feature quantities related to the change between frames, and a digest consisting of a part of the video is used using the inter-frame change information.
  • a digest video acquisition device for acquiring video will be described.
  • the division number N (N is a natural number of 2 or more) of video is received, the number of divisions N is detected, partial video is obtained for every two or more scenes, and the 2 A digest video acquisition device that acquires a digest video obtained by connecting the above partial videos will be described.
  • a digest video acquisition device that acquires a part of a video by using a frame that matches a predetermined condition as a start frame of a scene and acquires a digest video including the part of the video will be described. To do.
  • a digest video acquisition device that acquires a partial video having a length corresponding to the length of the scene from each scene and constitutes the digest video will be described.
  • a digest video acquisition device that acquires a partial video of a certain length from each scene regardless of the length of the scene and constitutes the digest video will be described.
  • a digest video acquisition device that determines an end frame that satisfies a predetermined condition, acquires a partial video from the start frame to the end frame, and constitutes a digest video will be described.
  • FIG. 1 is a block diagram of a digest video acquisition device 1 according to the present embodiment.
  • the digest video acquisition device 1 includes a storage unit 11, a reception unit 12, a processing unit 13, and a transmission unit 14.
  • the storage unit 11 includes a video storage unit 111, a selection condition storage unit 112, and a digest video storage unit 113.
  • the processing unit 13 includes a change information acquisition unit 131, a partial video acquisition unit 132, and a storage unit 133.
  • the partial video acquisition unit 132 includes a frame selection unit 1321 and a partial video acquisition unit 1322.
  • the storage unit 11 stores various types of information.
  • the various information includes, for example, a video from which a digest video is created, a selection condition described later, a digest video, and the like.
  • the video storage unit 111 stores a video having two or more frames.
  • This video is a video from which a digest video is created. It goes without saying that this video may include audio correspondingly.
  • the selection condition storage unit 112 stores a first selection condition for selecting a start frame of a partial video.
  • the first selection condition is a condition using inter-frame change information described later as a parameter.
  • the first selection condition is, for example, that a frame after change between frames whose interframe change information is equal to or greater than a threshold is selected as a start frame, and the value after the change between frames corresponding to the upper N Selecting a frame as a start frame, selecting a frame after X (X is a natural number of 1 or more) frames as a start frame from frames after the change between frames whose inter-frame change information is equal to or greater than a threshold, and changing between frames For example, a frame after X (X is a natural number of 1 or more) frames is selected as a start frame from a frame after a change between frames corresponding to the upper N.
  • the selection condition storage unit 112 also stores a second selection condition for selecting an end frame of the partial video.
  • the second selection condition is, for example, a condition related to the partial video time.
  • the second selection condition is, for example, a frame after the time calculated by the increasing function using the scene length as a parameter from the start frame or after the number of frames calculated by the increasing function using the scene length as a parameter. To select it as the end frame.
  • the second selection condition is to select a frame after a constant time from the start frame as an end frame, and to select a frame after a constant number from the start frame as an end frame.
  • the expression of the first selection condition and the second selection condition does not matter.
  • the first selection condition and the second selection condition are, for example, a logical expression and a program.
  • the digest video storage unit 113 stores the digest video acquired by the processing unit 13.
  • the reception unit 12 receives various instructions and videos.
  • the various instructions and videos are, for example, a start instruction, the number of partial videos, a digest video transmission instruction, and the like.
  • the start instruction is an instruction to start creating a digest video.
  • the number of partial videos is the number of partial videos constituting the digest video.
  • the digest video transmission instruction is an instruction to transmit a digest video and has a video identifier for identifying the video or the digest video.
  • the digest video transmission instruction is normally received from a user terminal device (not shown).
  • reception means reception of information input from an input device such as a keyboard, mouse, touch panel, reception of information transmitted via a wired or wireless communication line, recording on an optical disk, magnetic disk, semiconductor memory, or the like. It is a concept including reception of information read from a medium.
  • the various instructions and video input means may be anything such as a keyboard, mouse or menu screen.
  • the receiving unit 12 can be realized by a device driver for input means such as a keyboard, control software for a menu screen, or the like.
  • the processing unit 13 performs various processes.
  • the various processes are processes performed by the change information acquisition unit 131, the partial video acquisition unit 132, the storage unit 133, the frame selection unit 1321, the partial video acquisition unit 1322, and the like, for example.
  • the change information acquisition unit 131 acquires inter-frame change information from the video in the video storage unit 111.
  • the inter-frame change information is information indicating a change between two or more frames out of two or more frames constituting a video.
  • the change information acquisition unit 131 may acquire, for example, one or more feature amounts related to a change between two frames, and acquire inter-frame change information using the one or more feature amounts.
  • the interframe change information is represented by a natural number from 0 to 9, for example.
  • the inter-frame change information here indicates that the change is greater as the numerical value is larger.
  • the one or more feature amounts are, for example, a DC component, luminance, color value, color difference histogram correlation, and the like.
  • the change information acquisition unit 131 performs the first to third processes for acquiring the above-described interframe change information, and acquires the first interframe change information, the second interframe change information, and the third interframe change information. To do. Then, the change information acquisition unit 131 acquires inter-frame change information using the first inter-frame change information, the second inter-frame change information, and the third inter-frame change information.
  • the first inter-frame change information, the second inter-frame change information, and the third inter-frame change information each have a larger value (larger change), and the larger the inter-frame change information (for example, closer to 9). ).
  • the change information acquisition unit 131 acquires a frame differential value as inter-frame change information.
  • the change information acquisition unit 131 may acquire a set of three or more frames and acquire inter-frame change information from the three or more frame sets.
  • the change information acquisition unit 131 acquires, for example, the frame differential value between the previous two frames in the set of three frames and the frame differential value between the subsequent two frames, and determines the difference between the two frame differential values.
  • Inter-frame change information may be acquired. Note that the change information acquisition unit 131 usually acquires inter-frame change information using an increasing function using a difference between two frame differential values as a parameter.
  • the inter-frame change information may be a differential value of information indicating a change between two or more frames (may be referred to as a degree of change or a change slope). That is, the change information acquisition unit 131, for example, information on two or more changes acquired from one or more of the above-described first interframe change information, second interframe change information, and third interframe change information. May be a differential value.
  • the inter-frame change information may be information regarding changes between two or more different frames.
  • the inter-frame change information is preferably information regarding changes between two or more different frames including adjacent frames.
  • the change information acquisition unit 131 operates as follows, for example. First, the change information acquisition unit 131 sequentially selects a pair of two frames from which inter-frame change information is acquired from the beginning of the video. Next, the change information acquisition unit 131 acquires inter-frame change information between the selected frames.
  • the inter-frame change information is acquired by, for example, a decreasing function (for example, the reciprocal number) using the similarity between two frames as a parameter.
  • the interframe change information is calculated, for example, by one of the following first to fourth processes.
  • the first acquisition process of interframe change information is a pixel-by-pixel comparison process.
  • a color value difference that is a difference between color values (for example, RGB values) of the same pixel in the space of two frames is acquired. If the color values of the pixels at the same position in each of the two frames (frame 1 and frame 2) are (r1, g1, b1) and (r2, g2, b2), respectively, the color value difference (d) is , Expressed by Equation 1.
  • the change information acquisition unit 131 acquires, for example, an average value of color value differences between a plurality of pixels as interframe change information. Further, the change information acquisition unit 131 may acquire, for example, inter-frame change information that has a larger value as the average value of the color value differences is larger. That is, the change information acquisition unit 131 may acquire inter-frame change information by an increasing function using the average value of color value differences as a parameter.
  • the inter-frame change information obtained by the pixel-by-pixel comparison process is referred to as first inter-frame change information.
  • the second acquisition process of interframe change information is a pixel comparison process.
  • Pixel comparison processing calculates the color value difference (d) of pixels at the same position in two frames (frame 1 and frame 2), and if the color value difference (d) is greater than or equal to a threshold or greater than the threshold, an error
  • This is processing for determining a pixel and obtaining the number of error pixels. That is, the change information acquisition unit 131 acquires the number of error pixels as inter-frame change information.
  • the change information acquisition unit 131 may acquire inter-frame change information that has a larger value as the number of error pixels increases. That is, the change information acquisition unit 131 may acquire inter-frame change information using an increasing function using the number of error pixels as a parameter.
  • the inter-frame change information obtained by the pixel comparison process is referred to as second inter-frame change information.
  • the third acquisition process of interframe change information is a color histogram comparison process.
  • the color histogram is a bar graph created by counting how many pixels each color is in the image.
  • the color histogram comparison process is a process for obtaining, for example, the total difference in the number of colors in two frames or the average value or median value of the difference in the number of colors as inter-frame change information.
  • the change information acquisition unit 131 acquires, for example, inter-frame change information that has a large value such as a sum of differences in the number of colors of two frames or an average value or a median of differences in the numbers of colors. Also good.
  • the change information acquisition unit 131 may acquire the inter-frame change information by an increase function using the sum of the differences in the number of colors in two frames or the average value or the median of the differences in the numbers of colors as parameters. good.
  • the inter-frame change information obtained by the color histogram comparison process is referred to as third inter-frame change information.
  • the change information acquisition unit 131 may acquire, for example, normalized inter-frame change information (for example, any one of integers from 0 to 9) in the above processing.
  • the fourth acquisition process of inter-frame change information is a divided area comparison process.
  • the divided area comparison process is the following process. That is, the change information acquisition unit 131 divides each of the two frames (frame 1 and frame 2) into two or more areas to obtain two or more divided areas. Then, the change information acquisition unit 131 acquires, for each of the two or more divided areas, divided area change information that is information indicating the degree of conversion between the divided areas for the same divided area of each of the two frames. . Then, the change information acquisition unit 131 acquires inter-frame change information using two or more divided region change information of each of two or more divided regions.
  • the change information acquisition unit 131 may acquire the divided region change information for each of two or more divided regions by the pixel-by-pixel comparison process described above, or the divided region change information by the pixel comparison process. May be acquired, or divided region change information may be acquired from a difference between one or more feature amounts in each of the two divided regions.
  • the partial video acquisition unit 132 acquires one or more partial videos from the video using the inter-frame change information.
  • the partial video is a part of the video.
  • the process of acquiring the partial video may be a process of specifying the start frame and the end frame of the partial video.
  • the inter-frame change information is information indicating a change that is large enough to satisfy a predetermined condition
  • the partial video acquisition unit 132 includes one or more frames including a frame corresponding to the inter-frame change information. Get partial video.
  • the frames corresponding to the inter-frame change information are usually two or more frames that are subject to calculation of inter-frame change information.
  • the frame corresponding to the inter-frame change information is usually the last frame in time among the two or more frames for which the inter-frame change information is calculated.
  • the frame corresponding to the inter-frame change information is based on any one of two or more frames for which the inter-frame change information is to be calculated, for example, a temporally subsequent frame (for example, a frame after 3 frames, It may be a frame after 10 frames).
  • the partial video acquisition unit 132 may acquire partial videos of the number of partial videos received by the reception unit 12. Note that the length (for example, time and the number of frames) of the partial video acquired by the partial video acquisition unit 132 may be determined in advance or may be dynamically changed.
  • the frame selection unit 1321 selects one or more frames that meet the first selection condition.
  • Each of the one or more frames acquired by the frame selection unit 1321 is a start frame of a partial video.
  • the first selection condition is, for example, that the interframe change information is greater than or equal to a threshold value or greater than the threshold value.
  • the first selection condition is, for example, the top N of interframe change information. Note that N is the number of partial videos or the number of partial videos minus 1, which is a natural number.
  • the partial video acquisition unit 1322 acquires one or more partial videos having one or more frames selected by the frame selection unit 1321 as a start frame.
  • the partial video acquisition means 1322 may acquire a partial video having a length corresponding to the length of the scene from two or more scenes.
  • a scene is a frame group from the start frame to the next start frame, or a frame group from the start frame to the end frame of the video.
  • the partial video acquisition unit 1322 may acquire a partial video having a certain length from two or more scenes. Information for specifying a certain length is stored in the storage unit 11.
  • the partial video acquisition unit 1322 selects one or more frames that match the second selection condition as end frames, and acquires partial videos from the start frame to the selected end frame.
  • the second selection condition is, for example, to select a frame after M frames from the start frame as an end frame.
  • the second selection condition is, for example, determining the length of the partial video according to the length of the scene including the start frame, and selecting the frame corresponding to the length of the partial video as the end frame from the start frame. That is.
  • the accumulation unit 133 accumulates the digest video having one or more partial videos acquired by the partial video acquisition unit 132.
  • the storage unit 133 may link the two or more partial videos acquired by the partial video acquisition unit 132 to generate a digest video, and store the digest video.
  • accumulation is usually storage in the storage unit 11, but it may be accumulated in an external device.
  • the digest video may be stored by adding a flag in association with the start frame and the end frame of the video and configuring a situation in which the digest video can be reproduced using the flag.
  • the flag here is, for example, a start flag or an end flag described later.
  • the transmission unit 14 transmits the digest video to a terminal device (not shown).
  • the transmission unit 14 may transmit one or more partial videos acquired by the partial video acquisition unit 132 to the terminal device.
  • the transmission unit 14 transmits a digest video corresponding to the digest video transmission instruction to the terminal device.
  • This terminal device is a device that has transmitted a digest video transmission instruction.
  • the storage unit 11, the video storage unit 111, the selection condition storage unit 112, and the digest video storage unit 113 are preferably non-volatile recording media, but can also be realized by volatile recording media.
  • information may be stored in the storage unit 11 or the like via a recording medium
  • information transmitted via a communication line or the like may be stored in the storage unit 11 or the like
  • information input via the input device may be stored in the storage unit 11 or the like.
  • the processing unit 13, the change information acquisition unit 131, the partial video acquisition unit 132, the storage unit 133, the frame selection unit 1321, and the partial video acquisition unit 1322 can be usually realized by an MPU, a memory, or the like.
  • the processing procedure of the processing unit 13 or the like is usually realized by software, and the software is recorded on a recording medium such as a ROM.
  • the processing procedure of the processing unit 13 and the like may be realized by hardware (dedicated circuit).
  • the transmission unit 14 is usually realized by a wireless or wired communication means, but may be realized by a broadcasting means.
  • Step S201 The receiving unit 12 determines whether a start instruction has been received. When the start instruction is accepted, the process goes to step S202, and when the start instruction is not accepted, the process goes to step S212.
  • Step S202 The change information acquisition unit 131 assigns 1 to the counter i.
  • Step S203 The change information acquisition unit 131 determines whether or not the i-th set exists in the video in the video storage unit 111. If the i-th set exists, the process goes to step S204. If the i-th set does not exist, the process goes to step S206.
  • the i-th set is two or more frames from which inter-frame change information is acquired. For example, the i-th set is two consecutive frames from which inter-frame change information is acquired.
  • Step S204 The change information acquisition unit 131 acquires inter-frame change information for the i-th group of frames.
  • the inter-frame change information acquisition process will be described with reference to the flowcharts of FIGS.
  • Step S205 The change information acquisition unit 131 increments the counter i by one. The process returns to step S203.
  • the frame selection unit 1321 selects one or more start frames that match the first selection condition.
  • the first selection condition is, for example, a frame in which inter-frame change information is equal to or greater than a threshold value or greater than a threshold value (change is severe).
  • the start frame is usually the last frame in the i-th group of frames.
  • Step S207 The partial video acquisition unit 1322 assigns 1 to the counter j.
  • Step S208 The partial video acquisition unit 1322 determines whether or not the j-th start frame exists. If the jth start frame exists, the process goes to step S209, and if the jth start frame does not exist, the process goes to step S211.
  • the partial video acquisition unit 1322 determines an end frame that corresponds to the jth start frame and that matches the second selection condition.
  • This end frame is a frame temporally before the (j + 1) th start frame.
  • the process in which the partial video acquisition unit 1322 determines the end frame includes, for example, acquiring an end frame identifier (for example, frame number), and corresponding the end frame identifier (for example, frame number) to the start frame identifier. For example, adding a flag indicating that the frame is an end frame, and the like. The content of the process for determining the end frame does not matter.
  • Step S210 The partial video acquisition unit 1322 increments the counter j by 1. The process returns to step S208.
  • the storage unit 133 stores a digest video having one or more partial videos acquired by the partial video acquisition unit 132.
  • the process returns to step S201.
  • the storage unit 133 cuts out a frame group from the start frame to the end frame, acquires partial videos, generates a digest video by connecting two or more acquired partial videos, and stores the digest video. May be.
  • the storage unit 133 may store, in the storage unit 11, a set of the start frame identifier acquired by the frame selection unit 1321 and the end frame identifier acquired by the partial video acquisition unit 1322.
  • the storage unit 133 adds a flag indicating that the frame is a start frame to the start frame corresponding to the identifier of the start frame acquired by the frame selection unit 1321, and adds the flag indicating the end frame acquired by the partial video acquisition unit 1322 to the start frame.
  • a flag indicating the end frame may be added to the corresponding end frame.
  • the process for accumulating the digest video by the accumulating unit 133 may be a process that enables the digest video to be acquired.
  • Step S212 The accepting unit 12 determines whether or not a digest video transmission instruction has been received from a terminal device (not shown). If a digest video transmission instruction is received, the process goes to step S213, and if not received, the process returns to step S201.
  • Step S213 The processing unit 13 acquires a digest video corresponding to the digest video transmission instruction received in step S212.
  • Step S214 The transmission unit 14 transmits the digest video acquired in step S213 to a terminal device (not shown). The process returns to step S201.
  • the set of frames to be subjected to the inter-frame change information acquisition process is two frames and two frames that are temporally continuous. You may do it.
  • the partial video acquisition unit 1322 may use the first frame of the video as the start frame of the partial video and determine an end frame that matches the second selection condition.
  • the processing is ended by powering off or interruption for aborting the processing.
  • step S204 a first example of the inter-frame change information acquisition process in step S204 will be described using the flowchart of FIG.
  • the inter-frame change information acquisition process will be described using the above-described pixel comparison process as an example.
  • two temporally continuous frames that are targets of the inter-frame change information acquisition process are referred to as frame 1 and frame 2.
  • frame 1 and frame 2 two temporally continuous frames that are targets of the inter-frame change information acquisition process.
  • frame 2 exists after frame 1 in the video.
  • Step S301 The change information acquisition unit 131 assigns 1 to the counter i. Also, the change information acquisition unit 131 substitutes 0 for the variable “number of errors”.
  • Step S302 The change information acquisition unit 131 determines whether or not the i-th pixel exists in the frame 1 or the frame 2. When the i-th pixel exists, the process goes to step S303, and when the i-th pixel does not exist, the process goes to step S309. Note that the change information acquisition unit 131 sequentially scans pixels in frame 1 or frame 2, for example, and acquires the i-th pixel.
  • Step S303 The change information acquisition unit 131 acquires the color value of the i-th pixel of the frame 1.
  • Step S304 The change information acquisition unit 131 acquires the color value of the i-th pixel of frame 2.
  • the coordinate position in the frame of the i-th pixel in frame 1 is the same as the coordinate position in the frame of the i-th pixel in frame 2.
  • Step S305 The change information acquisition unit 131 calculates a distance from the two color values acquired in Step S303 and Step S304. This calculation method is Equation 1 described above.
  • Step S306 The change information acquisition unit 131 determines whether or not the distance acquired in step S305 is greater than or equal to a threshold value. If the distance is greater than or equal to the threshold, the process goes to step S307, and if the distance is less than the threshold, the process goes to step S308.
  • Step S307 The change information acquisition unit 131 adds 1 to the variable “number of errors”.
  • Step S308 The change information acquisition unit 131 increments the counter i by one.
  • Step S309 The change information acquisition unit 131 assigns the value of the variable “number of errors” to the interframe change information.
  • the inter-frame change information was acquired using the color value of the pixel.
  • the distance between two pixels (which may be referred to as a difference) is calculated using another feature amount (which may be referred to as an attribute value) of the pixel, and the distance is used.
  • Inter-frame change information may be acquired.
  • the other feature amount of the pixel is, for example, luminance and brightness.
  • step S204 a second example of the inter-frame change information acquisition process in step S204 will be described using the flowchart of FIG.
  • the inter-frame change information acquisition process will be described using the above-described divided region comparison process as an example.
  • two consecutive frames to be subjected to the inter-frame change information acquisition process are referred to as “frame 1” and “frame 2”. Also, assume that frame 2 exists after frame 1 in the video.
  • Step S401 The change information acquisition unit 131 assigns 1 to the counter i.
  • Step S402 The change information acquisition unit 131 determines whether or not the i-th divided region exists in the frame 1 or the frame 2.
  • the process goes to step S403, and when the i-th divided area does not exist, the process goes to step S408.
  • the storage unit 11 stores division area specification (for example, upper left coordinates and lower right coordinates) for specifying two or more division areas.
  • the change information acquisition unit 131 sequentially acquires the divided area specifying information of the storage unit 11 and determines whether or not the i th divided area exists (the i th divided area specifying information exists in the storage unit 11). Whether or not).
  • the change information acquisition unit 131 acquires one or more feature amounts of the i-th divided region of the frame 1.
  • the one or more feature amounts include, for example, an average value of color values of pixels in the i-th divided area, an average value of luminance values of pixels in the i-th divided area, and information on the color histogram of the i-th divided area. Etc.
  • the change information acquisition unit 131 acquires, for example, i-th divided area specifying information from the storage unit 11. Then, for example, the change information acquisition unit 131 acquires one or more feature amounts from the image of the area indicated by the divided area specifying information.
  • Step S404 The change information acquisition unit 131 acquires one or more feature amounts of the i-th divided region of the frame 2.
  • Step S405 The change information acquisition unit 131 acquires a difference between the feature amounts using the one or more feature amounts acquired in step S403 and the one or more feature amounts acquired in step S404. That is, for example, the change information acquisition unit 131 acquires the difference between the average value of the color value of frame 1 acquired in step S403 and the average value of the color value of frame 2 acquired in step S404, and the luminance of the frame 1 And the average value of the luminance of frame 2 are obtained.
  • Step S406 The change information acquisition unit 131 acquires the divided area change information of the i-th divided area from the one or more difference information acquired in step S405.
  • the change information acquisition unit 131 divides the region change information so that the information indicating that the one or more difference information acquired in step S405 indicates a larger difference, the larger the information on the region change information of the i-th region. To get.
  • Step S407 The change information acquisition unit 131 increments the counter i by one. The process returns to step S402.
  • Step S408 The change information acquisition unit 131 acquires inter-frame change information using the divided area change information of each of the two or more divided areas acquired in step S406.
  • the change information acquisition unit 131 may acquire the average value or median value of two or more divided region change information as inter-frame change information, or the maximum value of two or more divided region change information may be changed between frames. It may be acquired as information. That is, the larger the inter-frame change information is acquired, the more the divided region change information indicates a larger value.
  • one video is stored in the video storage unit 111. Then, it is assumed that the user inputs a start instruction having a partial video number “5” to the digest video acquisition device 1. Note that one video is, for example, a video having 18000 frames from frame number 0 to frame number 17999.
  • the receiving unit 12 receives a start instruction. Then, the change information acquisition unit 131 adds inter-frame change information “ ⁇ 1” to the storage unit 11 in association with the frame number 0.
  • the change information acquisition unit 131 acquires frame number 0 and frame number 1 in the video in the video storage unit 111. Then, the change information acquisition unit 131 acquires inter-frame change information indicating the degree of conversion from frame number 0 to frame number 1 by, for example, the divided region comparison process described above.
  • the change information acquisition unit 131 acquires the divided region change information of each divided region of frame number 0 and frame number 1, and uses the divided region change information of each divided region to normalize the inter-frame change information. It is assumed that “1” has been acquired. Then, the change information acquisition unit 131 adds the inter-frame change information “1” to the storage unit 11 in association with the frame number 1. It is assumed that the change information acquisition unit 131 acquires inter-frame change information of any integer from “0” to “9”.
  • the change information acquisition unit 131 has acquired inter-frame change information “2” between frame number 1 and frame number 2. Then, the change information acquisition unit 131 stores the interframe change information “2” in the storage unit 11 in association with the frame number 2.
  • the change information acquisition unit 131 has acquired inter-frame change information “7” between frame number 2 and frame number 3. Then, the change information acquisition unit 131 stores the inter-frame change information “7” in the storage unit 11 in association with the frame number 3.
  • the change information acquisition unit 131 performs the above processing up to the final frame of the video. Then, it is assumed that the change information acquisition unit 131 has calculated inter-frame change information of the concept illustrated in FIG. Further, it is assumed that the change information acquisition unit 131 has accumulated the information of “ID”, “frame number”, and “interframe change information” in the interframe change information management table shown in FIG.
  • the inter-frame change information management table holds records having “ID”, “frame number”, “inter-frame change information”, “start frame flag”, and “end frame flag” for the number of frames.
  • the “start frame flag” is a flag indicating whether it is a start frame
  • the “end frame flag” is a flag indicating whether it is an end frame.
  • the frame selection means 1321 selects the top five start frames having inter-frame change information values.
  • the frame selection unit 1321 has selected the frame numbers “6”, “2517”, “4510”, “8321”, and “12548” whose interframe change information value is “9” as the start frame.
  • the frame selection unit 1321 accumulates the start flag “1” in association with the frame number of the start frame. It is assumed that the frame selection unit 1321 stores the start flag “0” in frame numbers other than the start frame.
  • the start flag “1” indicates a start flag, and the start flag “0” indicates that it is not a start flag.
  • the frame selection means 1321 starts so that the intervals are as even as possible. It is preferable to select a flag.
  • the partial video acquisition unit 1322 determines an end frame for each start frame, for example, as follows.
  • the partial video acquisition unit 1322 determines the end frame so that the partial video has a length corresponding to the length of the scene, for example.
  • the partial video acquisition unit 1322 determines the end frame so that the length of the partial video becomes approximately the ratio of “2512: 1994: 3812: 4228: 5452”, for example, the partial video acquisition unit 1322 Frame number “508” as the end frame corresponding to start frame “6” Then, the partial video acquisition unit 1322 writes the end frame flag “1” in association with the frame number “508.” For example, the partial video acquisition unit 1322 stores the end frame corresponding to the start frame “2517”. The partial video acquisition unit 1322 writes the end frame flag “1” in association with the frame number “2916.” For example, the partial video acquisition unit 1322 includes the start frame “2916”.
  • the frame number “5272” is obtained as the end frame corresponding to “4510.” Then, the partial video acquisition unit 1322 writes the end frame flag “1” in association with the frame number “5272”. The means 1322 uses the end frame corresponding to the start frame “8321”. Then, the frame number “9167” is obtained, and the partial video acquisition unit 1322 writes the end frame flag “1” in association with the frame number “9167.” Also, the partial video acquisition unit 1322 displays the final frame of the video. The end frame flag “1” is written in association with the end frame flag, and the end frame flag “0” is written for a frame that is not the end frame.
  • the partial video acquisition unit 1322 is before or after the frame number, and the change tendency indicated by the inter-frame change information. It is preferable to select a frame whose number is less than a predetermined change as an end frame.
  • the number of frames (or playback time) of the digest video is stored in the storage unit 11 in advance.
  • the number of frames (or playback time) of the digest video may be designated by the user.
  • the accumulating unit 133 acquires videos from the frame corresponding to the start frame flag “1” in FIG. 6 to the frame corresponding to the next end frame “1” as a partial video.
  • the storage unit 133 concatenates the acquired five partial videos, forms a digest video, and stores the digest video in the digest video storage unit 113.
  • the digest video acquisition apparatus 1 has received a digest video transmission instruction from the user terminal device after the above digest video generation processing.
  • the processing unit 13 acquires a digest video corresponding to the received digest video transmission instruction.
  • the processing unit 13 may read the digest video from the digest video storage unit 113, or may construct the digest video from the video in the video storage unit 111 using the management table of FIG.
  • the transmission unit 14 transmits the obtained digest video to the terminal device that has transmitted the digest video transmission instruction.
  • the digest video is acquired using the inter-frame change information, an appropriate digest video can be acquired.
  • two partial videos acquired by the partial video acquisition unit 132 may be continuous in the video. That is, the frame next to the end frame of the partial video may be the start frame of the next partial video.
  • the partial video is obtained using the number of partial videos. That is, in the above specific example, the first selection condition is “select the frame after the change between frames corresponding to the top N as the value of the interframe change information as the start frame”. However, as described above, it goes without saying that the first selection condition may be other conditions such as “select a frame after change between frames whose interframe change information is equal to or greater than a threshold value or greater than the threshold value”. Yes.
  • the processing in the present embodiment may be realized by software. Then, this software may be distributed by software download or the like. Further, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification.
  • video acquisition apparatus 1 in this Embodiment is the following programs. That is, this program causes a computer to change information acquisition unit for acquiring change information between frames, which is information indicating a change between two or more frames of two or more frames constituting a video, and the change information between frames.
  • a partial video acquisition unit that acquires one or more partial videos that are part of the video received by the reception unit, and a digest video that includes the one or more partial videos acquired by the partial video acquisition unit It is a program for functioning as an accumulating unit for accumulating.
  • the partial video acquisition unit may include one or more frames including the frame corresponding to the inter-frame change information when the inter-frame change information is information indicating a change that is large enough to satisfy a predetermined condition.
  • a program that causes a computer to function as the partial video acquisition is preferable.
  • the change information acquisition unit acquires two or more feature amounts related to a change between two frames, and acquires inter-frame change information using the two or more feature amounts. It is preferable that the program is a function that functions.
  • the computer further functions as a reception unit that receives the number of partial videos, which is the number of partial videos, and the partial video acquisition unit functions as a computer that acquires partial videos of the number of partial videos. It is preferable that the program is to be executed.
  • the partial video acquisition unit selects a frame selection unit that selects one or more frames that meet a first selection condition for selecting a start frame of the partial video, and the frame selection unit selects the 1 selected by the frame selection unit.
  • a program that causes a computer to function is preferable as including a partial video acquisition unit that acquires one or more partial videos having each of the above frames as a start frame.
  • the partial video acquisition means acquires a partial video having a length corresponding to the length of the scene from two or more scenes from the start frame and the next start frame or video end frame. It is preferable that the program is a program that causes a computer to function.
  • the partial video acquisition means functions as a computer that acquires a partial video of a certain length from each of two or more scenes from the start frame and the next start frame or video end frame. It is preferable that the program is to be executed.
  • the partial video acquisition means selects one or more frames that match a second selection condition for selecting an end frame of the partial video, and a portion from the start frame to the selected end frame.
  • a program that causes a computer to function as the video acquisition is suitable.
  • FIG. 7 shows the external appearance of a computer that executes the program described in this specification to realize the digest video acquisition apparatus 1 and the like of the various embodiments described above.
  • the above-described embodiments can be realized by computer hardware and a computer program executed thereon.
  • FIG. 7 is an overview diagram of the computer system 300
  • FIG. 8 is a block diagram of the system 300.
  • a computer system 300 includes a computer 301 including a CD-ROM drive, a keyboard 302, a mouse 303, and a monitor 304.
  • the computer 301 in addition to the CD-ROM drive 3012, stores an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012, a ROM 3015 for storing programs such as a bootup program, and the MPU 3013. It includes a RAM 3016 for temporarily storing application program instructions and providing a temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data.
  • the computer 301 may further include a network card that provides connection to a LAN.
  • a program that causes the computer system 300 to execute the functions of the digest video acquisition device 1 and the like of the above-described embodiment is stored in the CD-ROM 3101, inserted into the CD-ROM drive 3012, and further transferred to the hard disk 3017. good.
  • the program may be transmitted to the computer 301 via a network (not shown) and stored in the hard disk 3017.
  • the program is loaded into the RAM 3016 at the time of execution.
  • the program may be loaded directly from the CD-ROM 3101 or the network.
  • the program does not necessarily include an operating system (OS) or a third-party program that causes the computer 301 to execute the functions of the digest video acquisition device according to the above-described embodiment.
  • the program only needs to include an instruction portion that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 300 operates is well known and will not be described in detail.
  • processing performed by hardware for example, processing performed by a modem or an interface card in the transmission step (only performed by hardware) Processing is not included.
  • the computer that executes the program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.
  • two or more communication means existing in one apparatus may be physically realized by one medium.
  • each process may be realized by centralized processing by a single device, or may be realized by distributed processing by a plurality of devices.
  • the digest video acquisition device has an effect that an appropriate digest video obtained by extracting a part from the video can be acquired, and is useful as a digest video acquisition device and the like.

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
  • Television Signal Processing For Recording (AREA)

Abstract

Le problème décrit par la présente invention démontre de manière classique qu'il a été difficile d'acquérir une vidéo condensée appropriée à partir d'une vidéo. La solution selon la présente invention concerne un dispositif d'acquisition de vidéo condensée comprenant : une unité d'acquisition d'informations de changement qui acquiert des informations de changement entre images qui sont des informations qui indiquent un changement parmi au moins deux images parmi au moins deux images qui configurent une vidéo ; une unité d'acquisition de vidéo partielle qui, à l'aide des informations de changement entre images, acquiert une ou plusieurs vidéos partielles qui sont des parties de la vidéo ; et une unité d'accumulation qui accumule une vidéo condensée qui comprend au moins une desdites vidéos partielles que l'unité d'acquisition de vidéo partielle a acquis. Il est ainsi possible d'acquérir une vidéo condensée appropriée à partir d'une vidéo.
PCT/JP2017/025691 2016-07-19 2017-07-14 Dispositif et procédé d'acquisition de vidéo condensée, et support d'enregistrement WO2018016434A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2016141801A JP6878718B2 (ja) 2016-07-19 2016-07-19 ダイジェスト映像取得装置、ダイジェスト映像取得方法、およびプログラム
JP2016-141801 2016-07-19

Publications (1)

Publication Number Publication Date
WO2018016434A1 true WO2018016434A1 (fr) 2018-01-25

Family

ID=60993021

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2017/025691 WO2018016434A1 (fr) 2016-07-19 2017-07-14 Dispositif et procédé d'acquisition de vidéo condensée, et support d'enregistrement

Country Status (2)

Country Link
JP (1) JP6878718B2 (fr)
WO (1) WO2018016434A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7166796B2 (ja) * 2018-06-13 2022-11-08 キヤノン株式会社 情報処理装置、情報処理方法、及びプログラム
US10917487B2 (en) 2019-01-14 2021-02-09 International Business Machines Corporation Generating notification digest based on user presence

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003101937A (ja) * 2001-09-26 2003-04-04 Ricoh Co Ltd 映像記録再生装置および映像配信システム
WO2007039994A1 (fr) * 2005-09-30 2007-04-12 Pioneer Corporation Dispositif de production de condense et programme utilise a cette fin

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP4323601B2 (ja) * 1999-02-15 2009-09-02 キヤノン株式会社 画像処理システム及び画像処理方法
JP2004126811A (ja) * 2002-09-30 2004-04-22 Toshiba Corp コンテンツ情報編集装置とその編集プログラム
JP4572771B2 (ja) * 2005-08-03 2010-11-04 Ａｇｃ硝子建材株式会社 合わせ板状部材の支持構造及び支持方法
JP2007158671A (ja) * 2005-12-05 2007-06-21 Sony Corp 動画記録再生装置、動画記録再生方法、動画記録再生方法のプログラム及び動画記録再生方法のプログラムを記録した記録媒体
CN101427250B (zh) * 2006-04-20 2012-07-04 Nxp股份有限公司 对数据流创建摘要的数据摘要系统和方法
JPWO2011099299A1 (ja) * 2010-02-10 2013-06-13 パナソニック株式会社 映像抽出装置、撮影装置、プログラム及び記録媒体
JP2013239797A (ja) * 2012-05-11 2013-11-28 Canon Inc 画像処理装置
JP2015103217A (ja) * 2013-11-28 2015-06-04 ソニー株式会社 画像処理装置および画像処理方法

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003101937A (ja) * 2001-09-26 2003-04-04 Ricoh Co Ltd 映像記録再生装置および映像配信システム
WO2007039994A1 (fr) * 2005-09-30 2007-04-12 Pioneer Corporation Dispositif de production de condense et programme utilise a cette fin

Also Published As

Publication number Publication date
JP2018013879A (ja) 2018-01-25
JP6878718B2 (ja) 2021-06-02

Similar Documents

Publication Publication Date Title
CN107534796B (zh) 视频处理系统和数字视频分发系统
JP6709986B2 (ja) 輝度特性生成方法
US7251790B1 (en) Media browsing system displaying thumbnails images based on preferences of plurality of users and placing the thumbnails images at a scene change
US10074015B1 (en) Methods, systems, and media for generating a summarized video with video thumbnails
EP2464093B1 (fr) Dispositif de génération de fichier d'image, dispositif de traitement d'image, procédé de génération de fichier d'image et procédé de traitement d'image
US8213730B2 (en) Server, screen transmitting method, and program storage medium
US20150117540A1 (en) Coding apparatus, decoding apparatus, coding data, coding method, decoding method, and program
US10997694B2 (en) Method to generate additional level of detail when zooming in on an image
JP5424930B2 (ja) 画像編集装置およびその制御方法およびプログラム
JP2007124236A (ja) 画像処理装置、画像処理方法及び画像処理プログラム
WO2018016434A1 (fr) Dispositif et procédé d'acquisition de vidéo condensée, et support d'enregistrement
US20020106127A1 (en) Method of and apparatus for retrieving movie image
US9875521B2 (en) Image output apparatus, image output method, and program
JP2006039753A (ja) 画像処理装置、画像処理方法
US7747130B2 (en) Apparatus and method for extracting representative still images from MPEG video
JP4656199B2 (ja) 画像処理装置、サムネイル動画作成方法及びサムネイル動画作成プログラム
US10706891B2 (en) Video image processing apparatus and processing method
CN117176979B (zh) 多源异构视频的内容帧提取方法、装置、设备及存储介质
WO2014113915A1 (fr) Mécanisme permettant de faciliter une détection de phase dynamique avec une forte tolérance à la gigue et destiné à des images de flux multimédias
CN110996173B (zh) 一种图像数据处理方法、装置及存储介质
JP5944205B2 (ja) 信号処理装置、信号処理方法、およびプログラム
CN117714696A (zh) 一种视频码率计算方法及系统
JP2011066483A (ja) 動画サムネイル作成装置、方法及びプログラム
JP6126463B2 (ja) 画像出力装置、画像出力方法、およびプログラム
KR20040035128A (ko) 디지털 비디오의 썸네일 영상 생성 장치

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17830953

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17830953

Country of ref document: EP

Kind code of ref document: A1