WO2012157270A1 - Video editing and playback device - Google Patents

Video editing and playback device Download PDF

Info

Publication number
WO2012157270A1
WO2012157270A1 PCT/JP2012/003206 JP2012003206W WO2012157270A1 WO 2012157270 A1 WO2012157270 A1 WO 2012157270A1 JP 2012003206 W JP2012003206 W JP 2012003206W WO 2012157270 A1 WO2012157270 A1 WO 2012157270A1
Authority
WO
WIPO (PCT)
Prior art keywords
moving image
information
playback apparatus
image editing
handwritten
Prior art date
Application number
PCT/JP2012/003206
Other languages
French (fr)
Japanese (ja)
Inventor
祐介 豊福
Original Assignee
パナソニック株式会社
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by パナソニック株式会社 filed Critical パナソニック株式会社
Publication of WO2012157270A1 publication Critical patent/WO2012157270A1/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/76—Television signal recording
    • H04N5/78—Television signal recording using magnetic recording
    • H04N5/782—Television signal recording using magnetic recording on tape
    • H04N5/783—Adaptations for reproducing at a rate different from the recording rate
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G06T7/248—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving reference images or patches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/30—Scenes; Scene-specific elements in albums, collections or shared content, e.g. social network photos or video
    • G—PHYSICS
    • G11—INFORMATION STORAGE
    • G11B—INFORMATION STORAGE BASED ON RELATIVE MOVEMENT BETWEEN RECORD CARRIER AND TRANSDUCER
    • G11B27/00—Editing; Indexing; Addressing; Timing or synchronising; Monitoring; Measuring tape travel
    • G11B27/10—Indexing; Addressing; Timing or synchronising; Measuring tape travel
    • G11B27/11—Indexing; Addressing; Timing or synchronising; Measuring tape travel by using information not detectable on the record carrier
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N9/00—Details of colour television systems
    • H04N9/79—Processing of colour television signals in connection with recording
    • H04N9/80—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback
    • H04N9/82—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only
    • H04N9/8205—Transformation of the television signal for recording, e.g. modulation, frequency changing; Inverse transformation for playback the individual colour picture signal components being recorded simultaneously only involving the multiplexing of an additional signal and the colour video signal

Definitions

  • the present invention relates to a moving image editing / reproducing apparatus that edits and reproduces a character or a graphic displayed on a moving image together with the moving image.
  • FIG. 10 is a diagram illustrating a menu screen during editing for adding pictographic data to moving image data and a scene of a moving image after editing.
  • a pictograph “Takeshi” is added to the moving image of the athletic meet.
  • a follow-up video designation frame 401 is displayed on the screen.
  • the user operates the recording apparatus so that the follow-up video designation frame 401 surrounds the person corresponding to Takeshi-kun. By doing so, the recording apparatus recognizes the object surrounded by the follow-up video designation frame 401 and follows the object by using the MPEG motion vector. Therefore, as shown in FIG. 10, the pictograph “Takeshi” moves as the object moves.
  • the recording apparatus described in Patent Document 1 described above displays a pictograph following the movement of an object using an MPEG motion vector.
  • the recording apparatus can follow the object until the object is out of frame. For this reason, in the case of a moving image in which the object is out of the frame and then in again, the user must input pictograms and set the follow-up video designation frame 401 each time the object enters the frame.
  • An object of the present invention is to provide a moving image editing / playback apparatus that can continue to follow an object associated with incidental information even if the object is out of frame and then in again.
  • the present invention analyzes moving image data at the time of starting display of supplementary information, which is characters and / or figures displayed superimposed on a moving image, and detects an object in a frame displayed at the start time.
  • Incidental information for registering the incidental state in the memory in association with the object according to the relationship between the first image analysis unit and the position of the incidental information in the frame and the position of the object detected by the first image analysis unit A registration unit; a second image analysis unit that analyzes moving image data in a time period in which the supplementary information is displayed superimposed on the moving image; and detects an object in a frame; and the second image analysis unit
  • An object analysis unit for determining whether an object similar to an object associated with the accompanying information is included in the objects detected by
  • the incidental information display processing unit performs processing to display the character and / or figure indicated by the additional information on the moving image at the position of the similar object.
  • a moving image editing / playback apparatus for registering the incidental state in the memory in association with the object according to
  • the incidental information registration unit A state is associated with the object and registered in the memory.
  • the supplementary information display processing unit performs processing to display characters and / or figures indicated by the supplementary information having a size corresponding to the size of the similar object.
  • the incidental information is registered in the memory as information of a layer different from the moving image.
  • the accompanying information display processing unit reduces the character and / or figure indicated by the accompanying information to a certain size as the similar object changes, the similar object becomes smaller.
  • the size of the character and / or figure indicated by the accompanying information is not changed and the character and / or figure indicated by the accompanying information is enlarged to a certain size in accordance with the change of the similar object, the similar object is obtained. Is processed so as not to change the size of the characters and / or figures indicated by the supplementary information even if becomes larger.
  • the supplementary information display processing unit performs processing to blink and display characters and / or figures indicated by supplementary information associated with a specific object.
  • the first image analyzing unit analyzes moving image data up to an end point of a time zone in which the supplementary information is displayed superimposed on the moving image.
  • the moving image editing / playback apparatus can continue to follow the object even if the object associated with the supplementary information is out of the frame and in again.
  • regeneration apparatus of one Embodiment (A)-(d) is a figure which shows the screen displayed on the display 103 in each step from the preparation of registration of a handwritten character and a figure to completion
  • Diagram showing the relationship between handwritten drawing information recorded in different layers and moving images is a figure which shows the display of the handwritten character and figure at the time of moving image reproduction
  • regeneration The flowchart which shows operation
  • regeneration A figure showing handwritten characters and figures when the object becomes smaller A figure showing handwritten characters / figures when the object becomes very small
  • FIG. 1 is a block diagram illustrating a configuration of a moving image editing / playback apparatus according to an embodiment.
  • a moving image editing / playback apparatus shown in FIG. 1 includes a touch panel 101, a display 103, an editing image analysis unit 105, a handwritten drawing information registration unit 107, a memory 109, a playback image analysis unit 111, and an object analysis unit. 113, a handwritten drawing information display processing unit 115, and a moving image display processing unit 117.
  • the touch panel 101 functions as an input interface of the moving image editing / playback apparatus of the present embodiment, and is bonded to the display surface of the display 103.
  • the user of the moving image editing / playback apparatus draws characters and / or figures to be displayed superimposed on the moving image on the touch panel 101.
  • the characters and / or figures drawn on the touch panel 101 by the user are referred to as “handwritten characters / graphics”.
  • the editing image analysis unit 105 reads from the memory 109 moving image data in a time zone in which handwritten characters / graphics are displayed on the moving image, and detects the object in the frame by analyzing the moving image data. . In particular, the editing image analysis unit 105 determines whether an object exists in an image at the start time for displaying handwritten characters / graphics on a moving image, and if there is an object, information on the shape and position of each object. (Object detection result) is sent to the handwritten drawing information registration unit 107. However, when the object does not exist, the editing image analysis unit 105 sends an object detection result indicating that the object does not exist to the handwritten drawing information registration unit 107.
  • the handwritten drawing information registration unit 107 stores information on the shape and color of handwritten characters / graphics (hereinafter referred to as “handwritten drawing information”) in the memory 109.
  • handwritten drawing information information on the shape and color of handwritten characters / graphics
  • the handwritten drawing information registration unit 107 determines that the distance between the handwritten character / graphic and the object is less than a predetermined value and the handwritten The handwritten drawing information is stored in the memory 109 in association with the characters / graphics.
  • the distance between the object and the handwritten character / graphic is, for example, the distance between the center point of the handwritten character / graphic and the center point of the object.
  • the handwritten drawing information registration unit 107 stores in the memory 109 the time (start time and end time) for displaying the handwritten characters / graphics on the display 103 so as to overlap the moving image. This display time is the time on the playback time of the moving image. If the object detection result obtained from the editing image analysis unit 105 indicates that the object does not exist, or even if the object exists, the distance between the object and the handwritten character / graphic is not less than a predetermined value. For example, the handwritten drawing information registration unit 107 stores only the handwritten drawing information in the memory 109.
  • FIGS. 2 (a) to 2 (d) are diagrams showing screens displayed on the display 103 at each stage from preparation for registration of handwritten characters / graphics to completion.
  • the user pauses the moving image at the display start timing of the handwritten character / graphic.
  • the user draws handwritten characters / graphics on the touch panel 101.
  • the moving image is reproduced until the display end timing (end point frame) of the handwritten character / graphic.
  • the user stops the reproduction of the moving image and confirms the registration of the handwritten character / graphic.
  • the memory 109 stores moving image data, handwritten drawing information obtained from the handwritten drawing information registration unit 107, information related to objects, and metadata including information about time for displaying handwritten characters / graphics.
  • the handwritten drawing information included in the metadata is stored in the memory 109 as layer information different from the moving image, as shown in FIG. Therefore, as shown in FIG. 3, when a plurality of handwritten characters / graphics are registered at the same playback timing of the moving image, the plurality of handwritten drawing information can be stored in the memory 109 as information of different layers. .
  • the reproduction-time image analysis unit 111 reads the moving image data from the memory 109 during the display time of the handwritten characters / graphics indicated by the metadata read from the memory 109, and analyzes the moving image data, thereby Detect objects.
  • the reproduction-time image analysis unit 111 performs the analysis at regular intervals.
  • the object analysis unit 113 is similar to the object associated with the handwritten character / graphic indicated by the metadata among the objects detected by the reproduction image analysis unit 111 during the display time of the handwritten character / graphic indicated by the metadata. It is determined whether the selected object is included. The object analysis unit 113 makes this determination at regular intervals. Whether or not the objects are similar is determined based on parameters such as the color, shape, or feature point of the object. If there is an object similar to the object associated with the handwritten character / graphic, the object analyzing unit 113 instructs the handwritten drawing information display processing unit 115 to display the handwritten character / graphic superimposed on the moving image at the position of the object. To do.
  • the handwritten drawing information display processing unit 115 performs processing to display on the display 103 a handwritten character / graphic having a size corresponding to the size of the object at the position of an object similar to the object associated with the handwritten character / graphic. .
  • the handwritten drawing information display processing unit 115 refers to the handwritten drawing information included in the metadata read from the memory 109 and determines the color and shape of the handwritten character / graphic.
  • the moving image display processing unit 117 performs processing so that a moving image based on the moving image data read from the memory 109 is displayed on the display 103.
  • FIG. 4A is a screen before time for displaying handwritten characters / graphics superimposed on a moving image.
  • the handwritten characters / graphics are superimposed on the moving image as shown in FIG. 4B, and handwritten along with the movement of the object as shown in FIG. 4C.
  • the position and / or size of characters / graphics changes.
  • the object (“Taro” in FIG. 4) corresponding to the handwritten character / graphic is out of frame, and then the object is again framed as shown in FIG. 4E. Even if it is in, the above-described operations of the reproduction-time image analysis unit 111 and the object analysis unit 113 cause the handwritten characters / graphics corresponding to the object to be displayed again.
  • FIG. 5 is a flowchart showing the operation of the moving image editing / playback apparatus of one embodiment when registering handwritten characters / graphics.
  • the editing image analysis unit 105 analyzes moving image data to detect an object (step S101).
  • the editing image analysis unit 105 determines whether or not the object exists as a result of the analysis in step S101 (step S103). If the object does not exist, the process proceeds to step S105. The process proceeds to S107.
  • step S105 the handwritten drawing information registration unit 107 stores the handwritten drawing information in the memory 109.
  • step S107 the handwritten drawing information registration unit 107 determines whether the distance between the object and the handwritten character / graphic is less than a predetermined value. If there is, the process proceeds to step S105.
  • step S ⁇ b> 109 the handwritten drawing information registration unit 107 stores the handwritten drawing information in the memory 109 in association with the object whose distance between the handwritten characters / graphics is less than a predetermined value and the handwritten characters / graphics.
  • FIG. 6 is a flowchart showing the operation of the moving image editing / playback apparatus according to an embodiment when displaying handwritten characters / graphics during moving image playback.
  • the reproduction time image analyzing unit 111 displays the reproduction time of handwritten characters / graphics. It is determined whether it is within the display time (step S203). If the reproduction time is within the display time of the handwritten character / graphic, the process proceeds to step S205.
  • the reproduction-time image analysis unit 111 analyzes the moving image data and detects an object in the frame.
  • step S207 the object analysis unit 113 determines whether or not the detected object includes an object similar to the object associated with the handwritten character / graphic (step S207). If a similar object is included, the process proceeds to step S209. If not, the process proceeds to step S211.
  • step S209 the handwritten drawing information display processing unit 115 performs processing so that a handwritten character / graphic having a size corresponding to the size of the object is superimposed and displayed on the moving image at the position of the similar object.
  • step S211 the handwritten drawing information display processing unit 115 performs processing so as not to display handwritten characters / graphics.
  • step S203 is performed to determine whether the reproduction time image analysis unit 111 has elapsed (step S213). If the predetermined time has elapsed, the process returns to step S203.
  • the moving image editing / playback apparatus As described above, the moving image editing / playback apparatus according to the present embodiment, as shown in FIGS. 4C to 4E, when an object in a moving image is out of frame, a handwritten character / graphic corresponding to the object. When the same object is framed in again without being displayed on the display 103, the displayed handwritten character / graphic corresponding to the object is displayed again on the moving image. Therefore, it is possible to continue displaying handwritten characters / graphics following a specific object in the moving image.
  • the handwritten drawing information display processing unit 115 provided in the moving image editing / playback apparatus of the present embodiment responds to changes in the size of the object accompanying the playback of the moving image. Then, processing is performed so that the size of the handwritten character / graphic corresponding to the object is changed. That is, the size of the handwritten character / graphic is proportional to the size of the object.
  • the size of the handwritten character is limited to a range proportional to the size of the object. That is, as shown in FIG. 7, when the handwritten drawing information registration unit 107 reduces the handwritten character to a certain size in accordance with the change of the object, the size of the handwritten character is not changed even if the object is further reduced. Similarly, when the handwritten character information enlarging unit 107 enlarges the handwritten character to a certain size as the object changes, the size of the handwritten character is not changed even if the object becomes larger. By doing so, handwritten characters can be displayed in a size that is always visible to the user. Furthermore, as shown in FIG. 8, the handwritten drawing information registration unit 107 does not need to display a line or display a handwritten character instead of a handwritten graphic when the size of the object is equal to or smaller than a certain size.
  • the handwritten drawing information display processing unit 115 blinks (blinks) the handwritten character / figure as shown in FIG. 9 when the corresponding object exists in the frame during the display time of the handwritten character / figure. You may process so that it may display. Furthermore, the handwritten drawing information display processing unit 115 may blink and display only handwritten characters / graphics corresponding to an object registered as a main subject among a plurality of objects. The object analysis unit 113 determines whether the object is registered as the main subject.
  • the moving image editing / playback apparatus when reproducing the accompanying information, which is a character or a graphic to be displayed superimposed on the moving image, together with the moving image, the object associated with the accompanying information is framed out and framed in again.
  • the present invention is useful as a moving image editing / playback apparatus that continues to follow the object.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Signal Processing (AREA)
  • Television Signal Processing For Recording (AREA)
  • Studio Circuits (AREA)
  • Image Analysis (AREA)

Abstract

This video editing and playback device is provided with: a means for associating an incidental condition to an object from the relationship between the position of the object and the position of incidental information, which is superimposed on a video, in a frame; a means for detecting the object in a frame by analyzing video data for the time period in which the incidental information is displayed; a means for determining whether a similar object to the object associated with the incidental information is included in the object; and a means for displaying the incidental information in a superimposed manner at the position of the similar object.

Description

動画像編集再生装置Video editing / playback device
　本発明は、動画像に重ねて表示する文字又は図形を編集及び動画像と共に再生する動画像編集再生装置に関する。 The present invention relates to a moving image editing / reproducing apparatus that edits and reproduces a character or a graphic displayed on a moving image together with the moving image.
　特許文献１に記載の録画装置は、記録された動画データに絵文字データを付加する編集機能を有する。図１０は、動画データに絵文字データを付加する編集時のメニュー画面と編集後の動画像の一場面を示す図である。図１０に示した例では、運動会の徒競争の動画像に「たけし」の絵文字が付加される。図１０に示すように、録画装置に絵文字「たけし」が入力されると、画面には追従映像指定枠４０１が表示される。ユーザは、追従映像指定枠４０１がたけし君に該当する人物を囲むよう録画装置を操作する。こうすることで、録画装置は、追従映像指定枠４０１で囲まれたオブジェクトを認識し、ＭＰＥＧの動きベクトルを利用することでオブジェクトに追従する。したがって、図１０に示すように、絵文字「たけし」はオブジェクトの移動に伴い移動する。 The recording device described in Patent Document 1 has an editing function for adding pictographic data to recorded moving image data. FIG. 10 is a diagram illustrating a menu screen during editing for adding pictographic data to moving image data and a scene of a moving image after editing. In the example shown in FIG. 10, a pictograph “Takeshi” is added to the moving image of the athletic meet. As shown in FIG. 10, when the pictogram “Takeshi” is input to the recording apparatus, a follow-up video designation frame 401 is displayed on the screen. The user operates the recording apparatus so that the follow-up video designation frame 401 surrounds the person corresponding to Takeshi-kun. By doing so, the recording apparatus recognizes the object surrounded by the follow-up video designation frame 401 and follows the object by using the MPEG motion vector. Therefore, as shown in FIG. 10, the pictograph “Takeshi” moves as the object moves.
日本国特開２００９－８１５９２号公報Japanese Unexamined Patent Publication No. 2009-81592 日本国特開２００４－１９３７０２号公報Japanese Unexamined Patent Publication No. 2004-193702 日本国特開２００４－１９５２４３号公報Japanese Unexamined Patent Publication No. 2004-195243
　上記説明した特許文献１に記載の録画装置は、ＭＰＥＧの動きベクトルを利用してオブジェクトの移動に追従して絵文字を表示する。但し、この録画装置がオブジェクトを追従できるのは、当該オブジェクトのフレームアウト時までである。このため、オブジェクトがフレームアウトして再びフレームインする動画像の場合、ユーザは、オブジェクトがフレームインする度に絵文字の入力と追従映像指定枠４０１の設定を行わなければならない。 The recording apparatus described in Patent Document 1 described above displays a pictograph following the movement of an object using an MPEG motion vector. However, the recording apparatus can follow the object until the object is out of frame. For this reason, in the case of a moving image in which the object is out of the frame and then in again, the user must input pictograms and set the follow-up video designation frame 401 each time the object enters the frame.
　本発明の目的は、付帯情報と関連付けられたオブジェクトがフレームアウトして再びフレームインしても当該オブジェクトの追従を継続できる動画像編集再生装置を提供することである。 An object of the present invention is to provide a moving image editing / playback apparatus that can continue to follow an object associated with incidental information even if the object is out of frame and then in again.
　本発明は、動画像に重ねて表示される文字及び／又は図形である付帯情報の表示を開始する時点の動画像データを解析して、前記開始時点に表示されるフレーム内のオブジェクトを検出する第１画像解析部と、前記付帯情報のフレーム内の位置と前記第１画像解析部が検出したオブジェクトの位置との関係に応じて、前記付帯状態を前記オブジェクトに関連付けてメモリに登録する付帯情報登録部と、前記付帯情報を動画像に重ねて表示する時間帯の動画像データを一定時間毎に解析して、フレーム内のオブジェクトを検出する第２画像解析部と、前記第２画像解析部によって検出されたオブジェクトの中に、前記付帯情報と関連付けられたオブジェクトに類似したオブジェクトが含まれるかを判断するオブジェクト解析部と、前記オブジェクト解析部によって前記類似したオブジェクトが含まれる判断されると、前記類似したオブジェクトの位置に前記付帯情報が示す文字及び／又は図形を前記動画像に重ねて表示するよう処理する付帯情報表示処理部と、を備えた動画像編集再生装置を提供する。 The present invention analyzes moving image data at the time of starting display of supplementary information, which is characters and / or figures displayed superimposed on a moving image, and detects an object in a frame displayed at the start time. Incidental information for registering the incidental state in the memory in association with the object according to the relationship between the first image analysis unit and the position of the incidental information in the frame and the position of the object detected by the first image analysis unit A registration unit; a second image analysis unit that analyzes moving image data in a time period in which the supplementary information is displayed superimposed on the moving image; and detects an object in a frame; and the second image analysis unit An object analysis unit for determining whether an object similar to an object associated with the accompanying information is included in the objects detected by When the image analysis unit determines that the similar object is included, the incidental information display processing unit performs processing to display the character and / or figure indicated by the additional information on the moving image at the position of the similar object. And a moving image editing / playback apparatus.
　上記動画像編集再生装置では、前記付帯情報登録部は、前記付帯情報のフレーム内の位置と前記第１画像解析部が検出したオブジェクトの位置との間の距離が所定値未満のとき、前記付帯状態を前記オブジェクトに関連付けて前記メモリに登録する。 In the moving image editing / playback apparatus, when the distance between the position in the frame of the incidental information and the position of the object detected by the first image analysis unit is less than a predetermined value, the incidental information registration unit A state is associated with the object and registered in the memory.
　上記動画像編集再生装置では、前記付帯情報表示処理部は、前記類似したオブジェクトの大きさに準じた大きさの前記付帯情報が示す文字及び／又は図形を表示するよう処理する。 In the moving image editing / playback apparatus, the supplementary information display processing unit performs processing to display characters and / or figures indicated by the supplementary information having a size corresponding to the size of the similar object.
　上記動画像編集再生装置では、前記付帯情報は、前記動画像とは異なるレイヤーの情報として前記メモリに登録される。 In the moving image editing / playback apparatus, the incidental information is registered in the memory as information of a layer different from the moving image.
　上記動画像編集再生装置では、前記付帯情報表示処理部は、前記類似したオブジェクトの変化に伴い前記付帯情報が示す文字及び／又は図形を一定の大きさまで縮小すると、前記類似オブジェクトがそれ以上小さくなっても前記付帯情報が示す文字及び／又は図形の大きさを変えず、かつ、前記類似したオブジェクトの変化に伴い前記付帯情報が示す文字及び／又は図形を一定の大きさまで拡大すると、前記類似オブジェクトがそれ以上大きくなっても前記付帯情報が示す文字及び／又は図形の大きさを変えないよう処理する。 In the moving image editing / playback apparatus, when the accompanying information display processing unit reduces the character and / or figure indicated by the accompanying information to a certain size as the similar object changes, the similar object becomes smaller. However, when the size of the character and / or figure indicated by the accompanying information is not changed and the character and / or figure indicated by the accompanying information is enlarged to a certain size in accordance with the change of the similar object, the similar object is obtained. Is processed so as not to change the size of the characters and / or figures indicated by the supplementary information even if becomes larger.
　上記動画像編集再生装置では、前記付帯情報表示処理部は、特定のオブジェクトに関連付けられた付帯情報が示す文字及び／又は図形を点滅して表示するよう処理する。 In the moving image editing / playback apparatus, the supplementary information display processing unit performs processing to blink and display characters and / or figures indicated by supplementary information associated with a specific object.
　上記動画像編集再生装置では、前記第１画像解析部は、前記付帯情報を前記動画像に重ねて表示する時間帯の終点まで動画像データを解析する。 In the moving image editing / playback apparatus, the first image analyzing unit analyzes moving image data up to an end point of a time zone in which the supplementary information is displayed superimposed on the moving image.
　本発明に係る動画像編集再生装置によれば、付帯情報と関連付けられたオブジェクトがフレームアウトして再びフレームインしても当該オブジェクトの追従を継続できる。 The moving image editing / playback apparatus according to the present invention can continue to follow the object even if the object associated with the supplementary information is out of the frame and in again.
一実施形態の動画像編集再生装置の構成を示すブロック図The block diagram which shows the structure of the moving image edit reproduction | regeneration apparatus of one Embodiment （ａ）～（ｄ）は、手書き文字・図形の登録の準備から終了に至るまでの各段階でディスプレイ１０３に表示される画面を示す図(A)-(d) is a figure which shows the screen displayed on the display 103 in each step from the preparation of registration of a handwritten character and a figure to completion | finish. 異なるレイヤーで記録されている手書き描画情報と動画像の関係を示す図Diagram showing the relationship between handwritten drawing information recorded in different layers and moving images （ａ）～（ｅ）は、動画像再生時における手書き文字・図形の表示を示す図(A)-(e) is a figure which shows the display of the handwritten character and figure at the time of moving image reproduction | regeneration 手書き文字・図形を登録する際の一実施形態の動画像編集再生装置の動作を示すフローチャートThe flowchart which shows operation | movement of the moving image edit reproduction | regeneration apparatus of one Embodiment at the time of registering a handwritten character and a figure 動画像再生時における手書き文字・図形を表示する際の一実施形態の動画像編集再生装置の動作を示すフローチャートThe flowchart which shows operation | movement of the moving image edit reproduction | regeneration apparatus of one Embodiment at the time of displaying the handwritten character and figure at the time of moving image reproduction | regeneration オブジェクトが小さくなったときの手書き文字・図形を示す図A figure showing handwritten characters and figures when the object becomes smaller オブジェクトが非常に小さくなったときの手書き文字・図形を示す図A figure showing handwritten characters / figures when the object becomes very small 手書き文字・図形が点滅して表示される画面を示す図The figure which shows the screen where the handwritten character and figure blinks and is displayed 特許文献１に記載の録画装置における動画データに絵文字データを付加する編集時のメニュー画面と編集後の動画像の一場面を示す図The figure which shows the scene of the menu screen at the time of the edit which adds pictographic data to the moving image data in the video recording apparatus of patent document 1, and the moving image after edit
　以下、本発明の実施形態について、図面を参照して説明する。 Hereinafter, embodiments of the present invention will be described with reference to the drawings.
　図１は、一実施形態の動画像編集再生装置の構成を示すブロック図である。図１に示す動画像編集再生装置は、タッチパネル１０１と、ディスプレイ１０３と、編集時画像解析部１０５と、手書き描画情報登録部１０７と、メモリ１０９と、再生時画像解析部１１１と、オブジェクト解析部１１３と、手書き描画情報表示処理部１１５と、動画像表示処理部１１７とを備える。 FIG. 1 is a block diagram illustrating a configuration of a moving image editing / playback apparatus according to an embodiment. A moving image editing / playback apparatus shown in FIG. 1 includes a touch panel 101, a display 103, an editing image analysis unit 105, a handwritten drawing information registration unit 107, a memory 109, a playback image analysis unit 111, and an object analysis unit. 113, a handwritten drawing information display processing unit 115, and a moving image display processing unit 117.
　タッチパネル１０１は、本実施形態の動画像編集再生装置の入力インターフェースとして機能し、ディスプレイ１０３の表示面に貼り合わされている。動画像編集再生装置のユーザは、動画像に重ねて表示したい文字及び／又は図形をタッチパネル１０１上に描画する。以下、ユーザがタッチパネル１０１上に描いた文字及び／又は図形を「手書き文字・図形」という。 The touch panel 101 functions as an input interface of the moving image editing / playback apparatus of the present embodiment, and is bonded to the display surface of the display 103. The user of the moving image editing / playback apparatus draws characters and / or figures to be displayed superimposed on the moving image on the touch panel 101. Hereinafter, the characters and / or figures drawn on the touch panel 101 by the user are referred to as “handwritten characters / graphics”.
　編集時画像解析部１０５は、手書き文字・図形を動画像に重ねて表示する時間帯の動画像データをメモリ１０９から読み出して、当該動画像データを解析することで、フレーム内のオブジェクトを検出する。特に、編集時画像解析部１０５は、手書き文字・図形を動画像に重ねて表示する開始時間の画像にオブジェクトが存在するかを判断し、オブジェクトが存在すれば、各オブジェクトの形状及び位置に関する情報（オブジェクト検出結果）を手書き描画情報登録部１０７に送る。但し、オブジェクトが存在しない場合、編集時画像解析部１０５は、オブジェクトが存在しない旨のオブジェクト検出結果を手書き描画情報登録部１０７に送る。 The editing image analysis unit 105 reads from the memory 109 moving image data in a time zone in which handwritten characters / graphics are displayed on the moving image, and detects the object in the frame by analyzing the moving image data. . In particular, the editing image analysis unit 105 determines whether an object exists in an image at the start time for displaying handwritten characters / graphics on a moving image, and if there is an object, information on the shape and position of each object. (Object detection result) is sent to the handwritten drawing information registration unit 107. However, when the object does not exist, the editing image analysis unit 105 sends an object detection result indicating that the object does not exist to the handwritten drawing information registration unit 107.
　手書き描画情報登録部１０７は、手書き文字・図形の形状や色等に関する情報（以下「手書き描画情報」という）をメモリ１０９に格納する。このとき、編集時画像解析部１０５から得られたオブジェクト検出結果がオブジェクトの存在を示す場合、手書き描画情報登録部１０７は、手書き文字・図形との間の距離が所定値未満のオブジェクトと当該手書き文字・図形を関連付けて手書き描画情報をメモリ１０９に格納する。オブジェクトと手書き文字・図形の間の距離は、例えば、手書き文字・図形の中心点とオブジェクトの中心点の間の距離である。 The handwritten drawing information registration unit 107 stores information on the shape and color of handwritten characters / graphics (hereinafter referred to as “handwritten drawing information”) in the memory 109. At this time, when the object detection result obtained from the editing image analysis unit 105 indicates the presence of the object, the handwritten drawing information registration unit 107 determines that the distance between the handwritten character / graphic and the object is less than a predetermined value and the handwritten The handwritten drawing information is stored in the memory 109 in association with the characters / graphics. The distance between the object and the handwritten character / graphic is, for example, the distance between the center point of the handwritten character / graphic and the center point of the object.
　また、手書き描画情報登録部１０７は、ディスプレイ１０３上に手書き文字・図形を動画像に重ねて表示する時間（開始時間と終了時間）をメモリ１０９に格納する。なお、この表示時間は、動画像の再生時間上の時間である。なお、編集時画像解析部１０５から得られたオブジェクト検出結果がオブジェクトが存在しない旨を示す場合、又は、オブジェクトが存在しても当該オブジェクトと手書き文字・図形の間の距離が所定値以上であれば、手書き描画情報登録部１０７は、手書き描画情報のみをメモリ１０９に格納する。 Also, the handwritten drawing information registration unit 107 stores in the memory 109 the time (start time and end time) for displaying the handwritten characters / graphics on the display 103 so as to overlap the moving image. This display time is the time on the playback time of the moving image. If the object detection result obtained from the editing image analysis unit 105 indicates that the object does not exist, or even if the object exists, the distance between the object and the handwritten character / graphic is not less than a predetermined value. For example, the handwritten drawing information registration unit 107 stores only the handwritten drawing information in the memory 109.
　図２（ａ）～（ｄ）は、手書き文字・図形の登録の準備から終了に至るまでの各段階でディスプレイ１０３に表示される画面を示す図である。図２（ａ）に示すように、ユーザは、手書き文字・図形の表示開始タイミングで動画像を一時停止する。次に、図２（ｂ）に示すように、ユーザが手書き文字・図形をタッチパネル１０１上に描画する。次に、当該手書き文字・図形の表示終了タイミング（終点フレーム）まで動画像を再生する。最後に、図２（ｄ）に示すように、ユーザが動画像の再生を停止して、手書き文字・図形の登録を確定する。 2 (a) to 2 (d) are diagrams showing screens displayed on the display 103 at each stage from preparation for registration of handwritten characters / graphics to completion. As shown in FIG. 2A, the user pauses the moving image at the display start timing of the handwritten character / graphic. Next, as shown in FIG. 2B, the user draws handwritten characters / graphics on the touch panel 101. Next, the moving image is reproduced until the display end timing (end point frame) of the handwritten character / graphic. Finally, as shown in FIG. 2D, the user stops the reproduction of the moving image and confirms the registration of the handwritten character / graphic.
　メモリ１０９は、動画像データと、手書き描画情報登録部１０７から得られた手書き描画情報、オブジェクトとの関連情報、及び手書き文字・図形を表示する時間に関する情報を含むメタデータとを記憶する。なお、メタデータに含まれる手書き描画情報は、図３に示すように、動画像とは異なるレイヤーの情報としてメモリ１０９に格納されている。したがって、図３に示すように、動画像の同じ再生タイミングに対して複数の手書き文字・図形が登録された場合には、複数の手書き描画情報がそれぞれ異なるレイヤーの情報としてメモリ１０９に格納され得る。 The memory 109 stores moving image data, handwritten drawing information obtained from the handwritten drawing information registration unit 107, information related to objects, and metadata including information about time for displaying handwritten characters / graphics. Note that the handwritten drawing information included in the metadata is stored in the memory 109 as layer information different from the moving image, as shown in FIG. Therefore, as shown in FIG. 3, when a plurality of handwritten characters / graphics are registered at the same playback timing of the moving image, the plurality of handwritten drawing information can be stored in the memory 109 as information of different layers. .
　再生時画像解析部１１１は、メモリ１０９から読み出したメタデータが示す手書き文字・図形の表示時間の間、メモリ１０９から動画像データを読み出して、当該動画像データを解析することで、フレーム内のオブジェクトを検出する。再生時画像解析部１１１は、当該解析を一定時間毎に行う。 The reproduction-time image analysis unit 111 reads the moving image data from the memory 109 during the display time of the handwritten characters / graphics indicated by the metadata read from the memory 109, and analyzes the moving image data, thereby Detect objects. The reproduction-time image analysis unit 111 performs the analysis at regular intervals.
　オブジェクト解析部１１３は、メタデータが示す手書き文字・図形の表示時間の間、再生時画像解析部１１１によって検出されたオブジェクトの中に、メタデータが示す手書き文字・図形に関連付けられたオブジェクトに類似したオブジェクトが含まれるかを判断する。オブジェクト解析部１１３は、当該判断を一定時間毎に行う。なお、オブジェクトが類似しているか否かの判断は、オブジェクトの色、形状又は特徴点等のパラメータに基づいて行われる。オブジェクト解析部１１３は、手書き文字・図形に関連付けられたオブジェクトに類似したオブジェクトがあれば、そのオブジェクトの位置に手書き文字・図形を動画像に重ねて表示するよう手書き描画情報表示処理部１１５に指示する。 The object analysis unit 113 is similar to the object associated with the handwritten character / graphic indicated by the metadata among the objects detected by the reproduction image analysis unit 111 during the display time of the handwritten character / graphic indicated by the metadata. It is determined whether the selected object is included. The object analysis unit 113 makes this determination at regular intervals. Whether or not the objects are similar is determined based on parameters such as the color, shape, or feature point of the object. If there is an object similar to the object associated with the handwritten character / graphic, the object analyzing unit 113 instructs the handwritten drawing information display processing unit 115 to display the handwritten character / graphic superimposed on the moving image at the position of the object. To do.
　手書き描画情報表示処理部１１５は、手書き文字・図形に関連付けられたオブジェクトに類似したオブジェクトの位置に、当該オブジェクトの大きさに準じた大きさの手書き文字・図形をディスプレイ１０３に表示するよう処理する。なお、手書き描画情報表示処理部１１５は、メモリ１０９から読み出したメタデータに含まれる手書き描画情報を参照して、手書き文字・図形の色及び形状を判断する。 The handwritten drawing information display processing unit 115 performs processing to display on the display 103 a handwritten character / graphic having a size corresponding to the size of the object at the position of an object similar to the object associated with the handwritten character / graphic. . The handwritten drawing information display processing unit 115 refers to the handwritten drawing information included in the metadata read from the memory 109 and determines the color and shape of the handwritten character / graphic.
　動画像表示処理部１１７は、メモリ１０９から読み出した動画像データに基づく動画像をディスプレイ１０３に表示するよう処理する。 The moving image display processing unit 117 performs processing so that a moving image based on the moving image data read from the memory 109 is displayed on the display 103.
　図４（ａ）～（ｅ）は、動画像再生時における手書き文字・図形の表示を示す図である。図４（ａ）は、手書き文字・図形を動画像に重ねて表示する時間前の画面である。手書き文字・図形の表示開始時間になると、図４（ｂ）に示すように、動画像に手書き文字・図形が重ねて表示され、図４（ｃ）に示すように、オブジェクトの移動に伴い手書き文字・図形の位置及び／又は大きさが変わる。また、図４（ｄ）に示すように、手書き文字・図形に対応するオブジェクト（図４では「たろう」）がフレームアウトし、その後、図４（ｅ）に示すように、当該オブジェクトが再びフレームインしても、上記説明した再生時画像解析部１１１及びオブジェクト解析部１１３の各動作によって、当該オブジェクトに対応する手書き文字・図形が再び表示される。 4 (a) to 4 (e) are diagrams showing display of handwritten characters / graphics during moving image reproduction. FIG. 4A is a screen before time for displaying handwritten characters / graphics superimposed on a moving image. When the display start time of handwritten characters / graphics is reached, the handwritten characters / graphics are superimposed on the moving image as shown in FIG. 4B, and handwritten along with the movement of the object as shown in FIG. 4C. The position and / or size of characters / graphics changes. Further, as shown in FIG. 4D, the object (“Taro” in FIG. 4) corresponding to the handwritten character / graphic is out of frame, and then the object is again framed as shown in FIG. 4E. Even if it is in, the above-described operations of the reproduction-time image analysis unit 111 and the object analysis unit 113 cause the handwritten characters / graphics corresponding to the object to be displayed again.
　図５は、手書き文字・図形を登録する際の一実施形態の動画像編集再生装置の動作を示すフローチャートである。図５に示すように、編集時画像解析部１０５は、動画像データを解析してオブジェクトを検出する（ステップＳ１０１）。次に、編集時画像解析部１０５は、ステップＳ１０１の解析の結果、オブジェクトが存在するか否かを判断し（ステップＳ１０３）、オブジェクトが存在しなければステップＳ１０５に進み、オブジェクトが存在すればステップＳ１０７に進む。 FIG. 5 is a flowchart showing the operation of the moving image editing / playback apparatus of one embodiment when registering handwritten characters / graphics. As shown in FIG. 5, the editing image analysis unit 105 analyzes moving image data to detect an object (step S101). Next, the editing image analysis unit 105 determines whether or not the object exists as a result of the analysis in step S101 (step S103). If the object does not exist, the process proceeds to step S105. The process proceeds to S107.
　ステップＳ１０５では、手書き描画情報登録部１０７は、手書き描画情報をメモリ１０９に格納する。一方、ステップＳ１０７では、手書き描画情報登録部１０７は、オブジェクトと手書き文字・図形の間の距離が所定値未満か否かを判断し、所定値未満であればステップＳ１０９に進み、所定値以上であればステップＳ１０５に進む。ステップＳ１０９では、手書き描画情報登録部１０７は、手書き文字・図形との間の距離が所定値未満のオブジェクトと当該手書き文字・図形を関連付けて手書き描画情報をメモリ１０９に格納に格納する。 In step S105, the handwritten drawing information registration unit 107 stores the handwritten drawing information in the memory 109. On the other hand, in step S107, the handwritten drawing information registration unit 107 determines whether the distance between the object and the handwritten character / graphic is less than a predetermined value. If there is, the process proceeds to step S105. In step S <b> 109, the handwritten drawing information registration unit 107 stores the handwritten drawing information in the memory 109 in association with the object whose distance between the handwritten characters / graphics is less than a predetermined value and the handwritten characters / graphics.
　図６は、動画像再生時における手書き文字・図形を表示する際の一実施形態の動画像編集再生装置の動作を示すフローチャートである。図６に示すように、動画像表示処理部１１７が、動画像データに基づく動画像の再生を開始する（ステップＳ２０１）と、再生時画像解析部１１１は、その再生時間が手書き文字・図形の表示時間内かを判断し（ステップＳ２０３）、再生時間が手書き文字・図形の表示時間内であればステップＳ２０５に進む。ステップＳ２０５では、再生時画像解析部１１１が、動画像データを解析して、フレーム内のオブジェクトを検出する。 FIG. 6 is a flowchart showing the operation of the moving image editing / playback apparatus according to an embodiment when displaying handwritten characters / graphics during moving image playback. As shown in FIG. 6, when the moving image display processing unit 117 starts reproduction of a moving image based on moving image data (step S201), the reproduction time image analyzing unit 111 displays the reproduction time of handwritten characters / graphics. It is determined whether it is within the display time (step S203). If the reproduction time is within the display time of the handwritten character / graphic, the process proceeds to step S205. In step S205, the reproduction-time image analysis unit 111 analyzes the moving image data and detects an object in the frame.
　次に、オブジェクト解析部１１３は、検出されたオブジェクトの中に、手書き文字・図形に関連付けられたオブジェクトに類似したオブジェクトが含まれるか否かを判断する（ステップＳ２０７）。類似したオブジェクトが含まれる場合はステップＳ２０９に進み、含まれない場合はステップＳ２１１に進む。ステップＳ２０９では、手書き描画情報表示処理部１１５が、類似したオブジェクトの位置に、当該オブジェクトの大きさに準じた大きさの手書き文字・図形を動画像に重ねて表示するよう処理する。一方、ステップＳ２１１では、手書き描画情報表示処理部１１５が、手書き文字・図形を表示しないよう処理する。ステップＳ２０９又はＳ２１１の後、ステップＳ２０３を行って一定時間が経過したかを再生時画像解析部１１１が判断し（ステップＳ２１３）、一定時間が経過していればステップＳ２０３に戻る。 Next, the object analysis unit 113 determines whether or not the detected object includes an object similar to the object associated with the handwritten character / graphic (step S207). If a similar object is included, the process proceeds to step S209. If not, the process proceeds to step S211. In step S209, the handwritten drawing information display processing unit 115 performs processing so that a handwritten character / graphic having a size corresponding to the size of the object is superimposed and displayed on the moving image at the position of the similar object. On the other hand, in step S211, the handwritten drawing information display processing unit 115 performs processing so as not to display handwritten characters / graphics. After step S209 or S211, step S203 is performed to determine whether the reproduction time image analysis unit 111 has elapsed (step S213). If the predetermined time has elapsed, the process returns to step S203.
　以上説明したように、本実施形態の動画像編集再生装置は、図４（ｃ）～（ｅ）に示すように、動画像内のオブジェクトがフレームアウトすると、当該オブジェクトに対応する手書き文字・図形をディスプレイ１０３に表示せず、同じオブジェクトが再びフレームインすると、そのオブジェクトに対応する表示されていた手書き文字・図形を再び動画像に重ねて表示する。したがって、動画像内の特定のオブジェクトに追従した手書き文字・図形の表示を継続できる。 As described above, the moving image editing / playback apparatus according to the present embodiment, as shown in FIGS. 4C to 4E, when an object in a moving image is out of frame, a handwritten character / graphic corresponding to the object. When the same object is framed in again without being displayed on the display 103, the displayed handwritten character / graphic corresponding to the object is displayed again on the moving image. Therefore, it is possible to continue displaying handwritten characters / graphics following a specific object in the moving image.
　また、図４（ｂ）及び（ｃ）に示すように、本実施形態の動画像編集再生装置が備える手書き描画情報表示処理部１１５は、動画像の再生に伴うオブジェクトの大きさの変化に応じて、当該オブジェクトに対応する手書き文字・図形の大きさを変えて表示するよう処理する。すなわち、手書き文字・図形の大きさは、オブジェクトの大きさに比例する。 Also, as shown in FIGS. 4B and 4C, the handwritten drawing information display processing unit 115 provided in the moving image editing / playback apparatus of the present embodiment responds to changes in the size of the object accompanying the playback of the moving image. Then, processing is performed so that the size of the handwritten character / graphic corresponding to the object is changed. That is, the size of the handwritten character / graphic is proportional to the size of the object.
　但し、手書き文字の大きさは、オブジェクトの大きさに比例する範囲が限定される。すなわち、図７に示すように、手書き描画情報登録部１０７は、オブジェクトの変化に伴い、手書き文字を一定の大きさまで縮小すると、オブジェクトがそれ以上小さくなっても手書き文字の大きさを変えない。同様に、手書き描画情報登録部１０７は、オブジェクトの変化に伴い、手書き文字を一定の大きさまで拡大すると、オブジェクトがそれ以上大きくなっても手書き文字の大きさを変えない。こうすることで、手書き文字をユーザが常に視認可能な大きさで表示することができる。さらに、手書き描画情報登録部１０７は、図８に示すように、オブジェクトの大きさが一定以下のときは、手書き図形に代えて線を表示するか、手書き文字を表示しなくても良い。 However, the size of the handwritten character is limited to a range proportional to the size of the object. That is, as shown in FIG. 7, when the handwritten drawing information registration unit 107 reduces the handwritten character to a certain size in accordance with the change of the object, the size of the handwritten character is not changed even if the object is further reduced. Similarly, when the handwritten character information enlarging unit 107 enlarges the handwritten character to a certain size as the object changes, the size of the handwritten character is not changed even if the object becomes larger. By doing so, handwritten characters can be displayed in a size that is always visible to the user. Furthermore, as shown in FIG. 8, the handwritten drawing information registration unit 107 does not need to display a line or display a handwritten character instead of a handwritten graphic when the size of the object is equal to or smaller than a certain size.
　また、手書き描画情報表示処理部１１５は、手書き文字・図形の表示時間中に対応するオブジェクトがフレーム内に存在するとき、図９に示すように、当該手書き文字・図形を点滅（ブリンク）して表示するよう処理しても良い。さらに、手書き描画情報表示処理部１１５は、複数のオブジェクトの中でも主要被写体として登録されたオブジェクトに対応する手書き文字・図形のみ点滅して表示しても良い。主要被写体として登録されたオブジェクトか否かはオブジェクト解析部１１３が判断する。 The handwritten drawing information display processing unit 115 blinks (blinks) the handwritten character / figure as shown in FIG. 9 when the corresponding object exists in the frame during the display time of the handwritten character / figure. You may process so that it may display. Furthermore, the handwritten drawing information display processing unit 115 may blink and display only handwritten characters / graphics corresponding to an object registered as a main subject among a plurality of objects. The object analysis unit 113 determines whether the object is registered as the main subject.
　本発明を詳細にまた特定の実施態様を参照して説明したが、本発明の精神と範囲を逸脱することなく様々な変更や修正を加えることができることは当業者にとって明らかである。 Although the present invention has been described in detail and with reference to specific embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made without departing from the spirit and scope of the invention.
　本出願は、2011年5月19日出願の日本特許出願（特願2011-112275）に基づくものであり、その内容はここに参照として取り込まれる。 This application is based on a Japanese patent application filed on May 19, 2011 (Japanese Patent Application No. 2011-112275), the contents of which are incorporated herein by reference.
　本発明に係る動画像編集再生装置は、動画像に重ねて表示する文字又は図形である付帯情報を動画像と共に再生する際に、付帯情報と関連付けられたオブジェクトがフレームアウトして再びフレームインしても当該オブジェクトの追従を継続する動画像編集再生装置等として有用である。 The moving image editing / playback apparatus according to the present invention, when reproducing the accompanying information, which is a character or a graphic to be displayed superimposed on the moving image, together with the moving image, the object associated with the accompanying information is framed out and framed in again. However, the present invention is useful as a moving image editing / playback apparatus that continues to follow the object.
１０１　タッチパネル
１０３　ディスプレイ
１０５　編集時画像解析部
１０７　手書き描画情報登録部
１０９　メモリ
１１１　再生時画像解析部
１１３　オブジェクト解析部
１１５　手書き描画情報表示処理部
１１７　動画像表示処理部 101 Touch Panel 103 Display 105 Editing Image Analysis Unit 107 Handwritten Drawing Information Registration Unit 109 Memory 111 Playback Image Analysis Unit 113 Object Analysis Unit 115 Handwritten Drawing Information Display Processing Unit 117 Moving Image Display Processing Unit

Claims (7)

  1. 　動画像に重ねて表示される文字及び／又は図形である付帯情報の表示を開始する時点の動画像データを解析して、前記開始時点に表示されるフレーム内のオブジェクトを検出する第１画像解析部と、
    　前記付帯情報のフレーム内の位置と前記第１画像解析部が検出したオブジェクトの位置との関係に応じて、前記付帯状態を前記オブジェクトに関連付けてメモリに登録する付帯情報登録部と、
    　前記付帯情報を動画像に重ねて表示する時間帯の動画像データを一定時間毎に解析して、フレーム内のオブジェクトを検出する第２画像解析部と、
    　前記第２画像解析部によって検出されたオブジェクトの中に、前記付帯情報と関連付けられたオブジェクトに類似したオブジェクトが含まれるかを判断するオブジェクト解析部と、
    　前記オブジェクト解析部によって前記類似したオブジェクトが含まれる判断されると、前記類似したオブジェクトの位置に前記付帯情報が示す文字及び／又は図形を前記動画像に重ねて表示するよう処理する付帯情報表示処理部と、
    を備えたことを特徴とする動画像編集再生装置。     First image analysis for detecting an object in a frame displayed at the start time by analyzing moving image data at the time of starting display of supplementary information which is a character and / or figure displayed superimposed on the moving image And
    In accordance with the relationship between the position of the auxiliary information in the frame and the position of the object detected by the first image analysis unit, an auxiliary information registration unit that registers the auxiliary state in the memory in association with the object;
    A second image analysis unit that analyzes moving image data of a time period in which the auxiliary information is displayed superimposed on the moving image at predetermined time intervals, and detects an object in the frame;
    An object analysis unit that determines whether an object similar to the object associated with the supplementary information is included in the objects detected by the second image analysis unit;
    When the object analysis unit determines that the similar object is included, the incidental information display processing is performed to display the character and / or the figure indicated by the incidental information on the moving image at the position of the similar object. And
    A moving image editing / playback apparatus comprising:
  2. 　請求項１に記載の動画像編集再生装置であって、
    　前記付帯情報登録部は、前記付帯情報のフレーム内の位置と前記第１画像解析部が検出したオブジェクトの位置との間の距離が所定値未満のとき、前記付帯状態を前記オブジェクトに関連付けて前記メモリに登録することを特徴とする動画像編集再生装置。     The moving image editing / playback apparatus according to claim 1,
    The auxiliary information registration unit associates the auxiliary state with the object when the distance between the position in the frame of the auxiliary information and the position of the object detected by the first image analysis unit is less than a predetermined value. A moving image editing / playback apparatus characterized by being registered in a memory.
  3. 　請求項１又は２に記載の動画像編集再生装置であって、
    　前記付帯情報表示処理部は、前記類似したオブジェクトの大きさに準じた大きさの前記付帯情報が示す文字及び／又は図形を表示するよう処理することを特徴とする動画像編集再生装置。     The moving image editing / playback apparatus according to claim 1 or 2,
    The moving image editing / playback apparatus, wherein the auxiliary information display processing unit performs processing to display characters and / or figures indicated by the auxiliary information having a size corresponding to the size of the similar object.
  4. 　請求項１～３のいずれか一項に記載の動画像編集再生装置であって、
    　前記付帯情報は、前記動画像とは異なるレイヤーの情報として前記メモリに登録されることを特徴とする動画像編集再生装置。     The moving image editing / playback apparatus according to any one of claims 1 to 3,
    The moving image editing / playback apparatus, wherein the auxiliary information is registered in the memory as information of a layer different from the moving image.
  5. 　請求項１～４のいずれか一項に記載の動画像編集再生装置であって、
    　前記付帯情報表示処理部は、前記類似したオブジェクトの変化に伴い前記付帯情報が示す文字及び／又は図形を一定の大きさまで縮小すると、前記類似オブジェクトがそれ以上小さくなっても前記付帯情報が示す文字及び／又は図形の大きさを変えず、かつ、前記類似したオブジェクトの変化に伴い前記付帯情報が示す文字及び／又は図形を一定の大きさまで拡大すると、前記類似オブジェクトがそれ以上大きくなっても前記付帯情報が示す文字及び／又は図形の大きさを変えないよう処理することを特徴とする動画像編集再生装置。     The moving image editing / playback apparatus according to any one of claims 1 to 4,
    If the accompanying information display processing unit reduces the character and / or figure indicated by the accompanying information to a certain size in accordance with the change of the similar object, the character indicated by the accompanying information even if the similar object is further reduced. If the characters and / or figures indicated by the supplementary information are enlarged to a certain size in accordance with the change of the similar object without changing the size of the figure, even if the similar object becomes larger A moving image editing / playback apparatus, which performs processing so as not to change the size of characters and / or figures indicated by the accompanying information.
  6. 　請求項１～５のいずれか一項に記載の動画像編集再生装置であって、
    　前記付帯情報表示処理部は、特定のオブジェクトに関連付けられた付帯情報が示す文字及び／又は図形を点滅して表示するよう処理することを特徴とする動画像編集再生装置。     A moving image editing / playback apparatus according to any one of claims 1 to 5,
    The moving image editing / playback apparatus, wherein the incidental information display processing unit performs processing to blink and display characters and / or figures indicated by incidental information associated with a specific object.
  7. 　請求項１～６のいずれか一項に記載の動画像編集再生装置であって、
    　前記第１画像解析部は、前記付帯情報を前記動画像に重ねて表示する時間帯の終点まで動画像データを解析することを特徴とする動画像編集再生装置。     The moving image editing / playback apparatus according to any one of claims 1 to 6,
    The moving image editing / playback apparatus, wherein the first image analyzing unit analyzes moving image data up to an end point of a time zone in which the supplementary information is displayed superimposed on the moving image.
PCT/JP2012/003206 2011-05-19 2012-05-16 Video editing and playback device WO2012157270A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2011112275A JP2014143453A (en) 2011-05-19 2011-05-19 Moving image editing reproducer
JP2011-112275 2011-05-19

Publications (1)

Publication Number Publication Date
WO2012157270A1 true WO2012157270A1 (en) 2012-11-22

Family

ID=47176619

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2012/003206 WO2012157270A1 (en) 2011-05-19 2012-05-16 Video editing and playback device

Country Status (2)

Country Link
JP (1) JP2014143453A (en)
WO (1) WO2012157270A1 (en)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP6455193B2 (en) 2015-02-04 2019-01-23 株式会社デンソー Electronic mirror system and image display control program
JP6524831B2 (en) * 2015-07-16 2019-06-05 富士ゼロックス株式会社 Information processing apparatus and program
US11227446B2 (en) 2019-09-27 2022-01-18 Apple Inc. Systems, methods, and graphical user interfaces for modeling, measuring, and drawing using augmented reality

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006115006A (en) * 2004-10-12 2006-04-27 Nippon Telegr & Teleph Corp <Ntt> Individual video image photographing and distributing apparatus, and individual video image photographing and distributing method and program
JP2008131405A (en) * 2006-11-21 2008-06-05 Sony Corp Imaging unit, image processing unit, image processing methods for the units, and program making computer execute the method
JP2009038749A (en) * 2007-08-03 2009-02-19 Canon Inc Image processor, control method of image processor, and program for implementing the same
JP2009081592A (en) * 2007-09-26 2009-04-16 Hitachi Ltd Video image recording apparatus
JP2010034918A (en) * 2008-07-30 2010-02-12 Hitachi Ltd Moving image reproducing apparatus
JP2011023867A (en) * 2009-07-14 2011-02-03 Panasonic Corp Moving image reproducing device and moving image reproducing method

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006115006A (en) * 2004-10-12 2006-04-27 Nippon Telegr & Teleph Corp <Ntt> Individual video image photographing and distributing apparatus, and individual video image photographing and distributing method and program
JP2008131405A (en) * 2006-11-21 2008-06-05 Sony Corp Imaging unit, image processing unit, image processing methods for the units, and program making computer execute the method
JP2009038749A (en) * 2007-08-03 2009-02-19 Canon Inc Image processor, control method of image processor, and program for implementing the same
JP2009081592A (en) * 2007-09-26 2009-04-16 Hitachi Ltd Video image recording apparatus
JP2010034918A (en) * 2008-07-30 2010-02-12 Hitachi Ltd Moving image reproducing apparatus
JP2011023867A (en) * 2009-07-14 2011-02-03 Panasonic Corp Moving image reproducing device and moving image reproducing method

Also Published As

Publication number Publication date
JP2014143453A (en) 2014-08-07

Similar Documents

Publication Publication Date Title
JP6102588B2 (en) Information processing apparatus, information processing method, and program
JP5523090B2 (en) INPUT DEVICE, CONTROL METHOD FOR INPUT DEVICE, PROGRAM, AND STORAGE MEDIUM
EP3120217B1 (en) Display device and method for controlling the same
TW201119368A (en) Image processing device, image processing method, and computer program
JP2007241410A (en) Display device and display control method
US9179090B2 (en) Moving image recording device, control method therefor, and non-transitory computer readable storage medium
AU2013222958A1 (en) Method and apparatus for object size adjustment on a screen
CN101833391A (en) Messaging device, information processing method and program
CN102597941A (en) Portable information terminal
US20190278426A1 (en) Inputting information using a virtual canvas
CN104205032A (en) Information processing apparatus for controlling an image based on duration and distance of a touch input, method, and non - transitory computer - readable medium
JP6255838B2 (en) Display device, display control method, and program
US20150063785A1 (en) Method of overlappingly displaying visual object on video, storage medium, and electronic device
JP5441748B2 (en) Display control apparatus, control method therefor, program, and storage medium
KR20150106330A (en) Image display apparatus and image display method
JP5220157B2 (en) Information processing apparatus, control method therefor, program, and storage medium
KR102255274B1 (en) Method of providing user interface and flexible device for performing the same.
WO2012157270A1 (en) Video editing and playback device
JP2014146127A (en) Information processing device, information processing method, and program
JP2010277551A (en) Data selection device, method for controlling the same, and program
JP2010061343A (en) Voice recording method, voice reproduction method, voice recording program and voice reproduction program
JP2013120421A (en) Information processor and image pickup device
JP2011188315A (en) Image reproduction device and program
US10212382B2 (en) Image processing device, method for controlling image processing device, and computer-readable storage medium storing program
CN103180806A (en) Portable display device, operation control method therefor, and program therefor

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12785273

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12785273

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP