JP2019092025A - Editing system - Google Patents

Abstract

To provide a technology for effectively performing editing such as mosaic processing by making it easy to detect a face image of a person to be detected in a program (in video data).SOLUTION: A similar face image detection device 16 is made by an editor to read a face image file (detection target face image) of a target performer. The similar face image detection device 16 compares the detection target face image with a face image in a face image storage server 127, which results in retrieval of a face image in a scene in which a performer having the same face appears and time code information. The time code information on the detected scene of appearance is transferred to an editing device 14. Without time and labor for searching for a scene in which the target performer appears, the editor is enabled to produce a special feature program on the performer or add a mosaic effect on the performer's image. When a preview on the detected scene of appearance without using the editing device 14 is desired, a video file in a low resolution server 126 is reproduced.SELECTED DRAWING: Figure 16

Description

  The present invention relates to an editing system, for example, in a broadcast station storing video, in order to produce a celebration program or a memorial program, a appearance scene of a specific performer is detected from the video in the past. , And an editing system having a function of assisting editing for program production.

  Conventionally, past video assets have been recorded on VTR tapes, and a huge number of VTR tapes have been stored in a warehouse. A tape number is attached to each VTR tape, and together with the tape number, information of a program name, a cast, and a program content summary recorded on the tape is managed. Therefore, when a video of a specific performer is desired, the person in charge of the broadcast station specified the VTR tape in which the program in which the performer appears is recorded based on the management information of the VTR tape.

  For example, in Patent Document 1, as the same program editing technology, in the case of creating original material information data which is information extracted the relationship between the editing original material and the edited material, and editing again, the edited material and the project Techniques for editing using data and original material information data have been proposed.

JP 2012-34218 A

  By the way, conventionally, in order to specify which scene of VTR tape has appeared, it is necessary to reproduce the VTR tape through the VTR device and visually look for the appearance scene of the performer, and the appearance scene is found In this case, the time code information of the appearance scene is noted and used for editing, and a new technology of measures is required from the viewpoint of work efficiency and accuracy.

  In recent years, a method of dubbing video assets from VTR tapes to magnetic media such as LTO tapes and optical media such as Blu-ray Disc (registered trademark) has been changed to a method of storing as video files in these media. However, there is no change in that it is necessary to play back and view video files in these media in order to find out the appearance scene, and there is a similar problem.

  In addition, if the cast has a problem just before the broadcast after editing of the program is completed and the cast of the cast is not available, the cast will be mosaicted or the cast scene cut Need to re-edit to In order to find out the appearance scene for re-editing, it is necessary to reproduce the video after the editing is completed and the editor visually finds out the appearance scene. This point also has the same problem.

  The present invention has been made in view of such a situation, and an object thereof is to solve the above-mentioned problems.

The present invention is an editing system provided with an editing apparatus for editing a video file used for broadcasting, which acquires a face image of a performer included in the video file, and the face image and the appearance video of each performer The face image storage server for recording in association with time code information, the face image stored in the face image storage server, and the face image to be searched included in the video file of a specific program are compared, Based on the appearance video detection unit for detecting appearance video in a program and the appearance video detected by the appearance video detection unit, other appearance videos in which a person of the face image to be searched for in the specific program is appearance A similar face image detection device that detects time code information of a detected appearance video detected by similar face image search, associates the detected time code information of the appearance video with the performer information that is the search target, and notifies the editing device Wherein the editing device edits the video file of the specific program by using the time code information.
In addition, when editing the video file, if the video file of the performer is stored in the recording device to be processed by the similar face image detection device, the editing device may use the video file of the recording device. It may be possible to display the video confirmation of the appearance video by playing back.
Further, the editing apparatus may reproduce the appearance video detected using the low resolution image.
The face image storage server may store a face image to be detected in association with a face type, and the similar face image detection device may perform a similar face image search according to the face type.

  According to the present invention, it is possible to provide a technique for facilitating detection of a face image of a person to be searched for in a program (in video data) and effectively performing editing such as mosaic processing.

FIG. 1 is a block diagram showing a schematic configuration of a video editing system according to an embodiment. FIG. 1 is a block diagram showing a schematic configuration of a recording apparatus according to an embodiment. FIG. 1 is a block diagram showing a schematic configuration of a similar face image detection device according to an embodiment. It is a block diagram showing a schematic structure of an editing device concerning an embodiment. It is a functional block which shows schematic structure of the automatic edit information creation apparatus based on embodiment. It is a flow chart which shows an example of edit processing concerning an embodiment. It is a flow chart which shows an example of edit processing concerning an embodiment. It is a flow chart which shows an example of edit processing concerning an embodiment. It is a flowchart which shows the operation example of a process target recognition part based on embodiment. It is a figure which shows the example of the display method (display of a touch-panel display) in an editing apparatus based on embodiment. It is a figure which shows the example of the feature-value of the image used as the candidate of the search key image based on embodiment. It is a flowchart which shows the procedure which implements a similar person search (similar face image detection process) based on embodiment. It is a figure which shows the example of the search screen which can be used for a similar face image search system based on embodiment. It is a figure which shows the example of a procedure which accumulate | stores a face image in a face image storage server based on embodiment. It is a figure which shows the example of a procedure which accumulate | stores a face image in a face image storage server based on embodiment. It is a figure which shows the example which similar face image detection was carried out from the face image storage server by making the face image of the target performer into a detection target based on embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
The outline of the present embodiment is as follows.
(1) From the huge past video accumulated in the broadcasting station, only the face images of all the performers are cut out and stored in the face image storage server together with time code information of appearance scenes.
(2) The appearance image of the performer is detected by comparing the accumulated face image and the face image of the target performer.
(3) Using the face image of the appearance scene of the detected result, narrow down the appearance scene similar by the similar face image detection processing.
(4) By passing the time code information of the detected appearance scene to the editing machine, it facilitates the production of a feature program in which the performer appears.
(5) The appearance scene detected using the low resolution video is simply reproduced.
(6) If the video of the performer becomes not available immediately before the broadcast, identify the scene in which the performer is performing and edit (mosaic, cut, etc.).

  FIG. 1 is a block diagram showing a schematic configuration of a video editing system 1 according to the present embodiment. The video editing system 1 includes a camera 10, a recording device 11, a recording device 12 (video server), an automatic editing information creation device 13, an editing device 14, a management terminal 17, a transmission server 18, and a system control unit. 15 are connected by a network 2 such as a LAN line or a predetermined communication line. The system control unit 15 controls the whole of the video editing system 1 in an integrated manner, and may be configured independently or included in the same manner as other devices (such as the recording device 12 and the editing device 14) It may be done.

  The camera 10 performs digital conversion processing on an image captured by a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor) element, or the like, and converts the image data of the conversion result (for example, material video data of the HD-SDI standard) The data is output to the recording device 12 via the network 2 using the recording device 11. The recording device 12 (video server) stores this. The automatic editing information creation device 13 is connected to the recording device 12 via the network 2, and the material video data is input to the automatic editing information creation device 13 and then stored in the recording device 12. However, the material video data may be directly input and stored in the recording device 12 without passing through the automatic editing information creation device 13.

  FIG. 2 is a block diagram showing a schematic configuration of the recording device 12. The recording device 12 includes a recording function and a similar face image detection function (similar face image detection device 16), and a data storage function (121 to 127).

  The data storage function of the recording device 12 will be described. The recording device 12 includes a source video data unit 121 for recording source video data, an edited video data unit 122 for recording edited video data, and an automatically edited video data unit 123 for recording automatically edited video data. A final editing information unit 124 for recording final editing information, an automatic editing information unit 125 for recording automatic editing information, a low resolution server 126 for recording low resolution files, and a face image included in video are recorded and stored. And a face image storage server 127.

  The reason for providing the low resolution server 126 is as follows. That is, in general, the video file in the medium 5 (optical media 5a, magnetic media 5b, VTR tape 5c) needs to be stored with high image quality, so the video file size becomes large. That is, it can not always be stored in accessible HDD storage. If the file is a low resolution file, the file size is small, so it is possible to save the video file on an accessible HDD storage and make a trial copy. Therefore, when dubbing the medium 5, low resolution video (low resolution file) is simultaneously created and recorded in the low resolution server 126.

  Subsequently, the similar face image detection device 16 for realizing the recording function and the similar face image detection function of the recording device 12 will be described with reference to FIG.

  FIG. 3 is a block diagram showing a schematic configuration of the similar face image detection device 16. The similar face image detection device 16 includes an image transmission / reception unit 210, an image recording unit 211, a reproduction control unit 212, a person area detection unit 213, a person feature quantity extraction unit 214, a person feature quantity recording unit 215, and attributes. An information recording unit 216, a request receiving unit 217, a similar person searching unit 218, an appearance event searching unit 219, a search result transmitting unit 220, a keyword recording unit 110, and a keyword searching unit 111 are included.

  The image transmission / reception unit 210 is a processing unit that inputs / outputs an image from the outside of the apparatus, receives input image data from the camera 10 or another apparatus, and transmits output image data to another apparatus (such as the editing apparatus 14). I do.

  The image recording unit 211 writes the input image data to the recording medium and reads the output image data from the recording medium (in the case of the VTR tape 5c, it is connected to the media reproduction device 19). At the time of writing, in addition to the image data, an image ID (image identification information) serving as information when reading out the image data is also recorded. The playback control unit 212 controls video playback to the editing device 14.

  The person area detection unit 213 performs person detection using image recognition technology on input image data, determines presence of a person in an image, and calculates coordinates of the area when a person is present. Also, the human area detection unit 213 specifies an area of the “face” of the human, extracts a face image including the area, and records the face image in the face image storage server 127.

  The person feature quantity extraction unit 214 performs feature quantity calculation on the area detected by the person area detection unit 213 using an image recognition technique. The human feature value calculated here is, for example, the shape and direction of the contour of the person, the color of the skin, the gait (the way of the legs, such as how to move the legs and how to move them), or The shape and direction of the contour of the face, which is a representative part, and the size, shape, arrangement relationship, etc. of main components such as eyes, nose, and mouth can be mentioned. In this embodiment, the type of feature used The number may be any. The person feature quantity extraction unit 214 can distinguish the face type (front, side face, oblique face, back face, laughing face, angry face, etc.) as the type of feature quantity, and the face image to be detected and the like Feature amounts can be associated.

  The person feature quantity recording unit 215 writes and reads the feature quantity calculated by the person feature quantity extraction unit 214 on a recording medium. At this time, the person feature amount is associated when the face image extracted by the person area detection unit 213 is recorded in the face image storage server 127. The face image is associated with the person's name at a predetermined timing (input by the user or automatic assignment by similar face image search).

  The recording medium of the image data in the image recording unit 211 and the recording medium of the person feature amount in the main processing unit may be the same or different.

  The attribute information recording unit 216 writes and reads attribute information related to image data to a recording medium. The attribute information is, for example, a photographing time of an image, an imaging device number, or the like.

  The request receiving unit 217 receives a search request and a keyword assignment request from the editing device 14. The search request includes a similar face image search request and an appearance event search request.

  The similar person search unit 218 performs similar face image search when the request received by the request reception unit 217 is a similar person search request.

  The appearance event search unit 219 performs appearance event search when the request received by the request reception unit is an appearance event search request.

  The search result transmission unit 220 transmits the similar person search result and the appearance event search result obtained from the similar person search unit 218 and the appearance event search unit 219 to the editing device 14.

  The keyword recording unit 110 writes and reads a keyword to the recording medium based on the keyword assignment request received by the request receiving unit 217.

  When the search request data received by the request reception unit 217 includes a keyword, the keyword search unit 111 performs a keyword search.

  Subsequently, the editing device 14 (editing machine) will be described with reference to FIG. FIG. 4 is a block diagram showing a schematic configuration of the editing device 14. The editing device 14 performs editing processing for actually performing rendering processing and the like on the material video data.

  The editing device 14 includes an editing control unit (editing means) 141 having a processor that actually performs this editing operation, and a display unit for displaying a video based on the material video data and the video data after the editing. A similar face image detection / operation unit 103 is provided with an operation panel 142 (operation means) for selecting each part of the image or sound or inputting an instruction. The display unit 142 and the operation panel 143 may be provided as an integrated touch panel display 144.

  The editing control unit 141 reads the material video data and the above-mentioned automatic editing information from the recording device 12 (automatic editing information unit 125), and edits the material video data based on the automatic editing information (new video data (automatically edited video) Data) is created, and the automatically edited video data is stored in the recording device 12 (automatically edited video data unit 123).

  However, in the editing device 14, after the user confirms the image based on the automatically edited video data on the display unit 142, the operation panel 143 is operated to perform processing of the automatically edited video data. It is also possible to issue an instruction to cancel the processing of the portion recognized as not appropriate among them to the editing control unit 141 and cancel this processing. In this case, material video data can also be referenced.

  Similarly, the editing control unit 141 can further perform additional processing on the automatically edited video data. At this time, the part to be newly processed is designated by the user. Also at this time, the user can perform this operation by operating the operation panel 143 after confirming the video based on the automatically edited video data on the display unit 142. By such an operation by the user, final editing information in which the automatic editing information is rewritten is generated. The final editing information is reflected in the editing process on the material video data, and is also used to update the processing target information as described later.

  Similarly, the editing control unit 141 directly reads the material video data from the recording device 12, causes the user to confirm an image based on the material video data on the display unit 142, and then operates the operation panel 143 to perform the automatic editing. It is also possible to specify a portion to be processed and perform an operation to perform rendering processing without using information. In this operation, the user can perform rendering processing on the material video data regardless of the automatic editing information.

  Thus, the editing control unit 141 causes the recording device 12 to edit the automatically edited video data edited based on the automatic editing information, the edited video data edited by the user or the material video data. It can be recorded.

  The similar face image detection operation unit 103 has, as functional components, a search request transmission unit 221, a search result reception unit 222, a search result display unit 223, a reproduction image display unit 224, a screen operation detection unit 225, and a keyword assignment request transmission unit 112. Each processing unit of the multiple search key selection unit 113 is included.

  The search request transmission unit 221 transmits the search request to the recording device 12. In the case of similar person search, the search request data includes the name of a person, a search key image (especially a face image) or its feature amount as a search key for a similar person search. In addition, search request data can also include refinement parameters.

  The search result receiving unit 222 receives the search result from the recording device 12 (similar face image detection device 16). The data received as the search result includes a set of images obtained by performing similar person search or appearance event search in the recording device 12 (similar face image detection device 16). The individual images forming the set are generated from the video recorded in the recording device 12 (similar face image detection device 16) by performing image size reduction processing and the like. Hereinafter, the individual images are also referred to as “search result images”, and data transmitted and received as search results is also referred to as “search result data”.

The search result display unit 223 displays a screen of the search result received by the search result receiving unit 222. An example of the displayed screen will be described later.
The reproduction image display unit 224 performs continuous moving image display on the screen of the image data input from the recording device 12 (similar face image detection device 16).
The screen operation detection unit 225 detects and acquires an operation content by the user.
The keyword assignment request transmission unit 112 transmits a keyword assignment request to the recording device 12 (similar face image detection device 16).
The multiple search key selection unit 113 performs processing to appropriately select a smaller number of search key images when a plurality of search key image candidates are selected.

  FIG. 5 is a functional block of the automatic editing information creation apparatus 13. The automatic editing information creation apparatus 13 includes a processing target recognition unit 131 and an information storage unit 132. The information storage unit 132 includes a final editing information unit 124, an automatic editing information unit 125, and a processing target information unit 128. The final editing information unit 124 and the automatic editing information unit 125 may be the same as those provided in the recording device 12 or may be provided separately.

The automatic editing information creation apparatus 13 reads the material video data, and the processing target recognition unit 131 recognizes a portion to be subjected to rendering processing. At this time, the processor in the processing target recognition unit 131 performs this recognition based on the processing target information stored in the information storage unit 132, and thus information on the portion to be processed and the processing thereof (automatic The editing information is stored in the recording device 12.

  More specifically, the information on the portion to be processed in the contents of the automatic editing information includes the video frame position (time code information) of the portion to be processed, the coordinates on the video, or the case where the processing target is audio. Are the range of the position of the audio sample, the contents of the processing, and the like. The contents of the process include mosaic process, blur process, video cut, increase / decrease process of luminance when the process object is video, mute process, volume adjustment, etc. when the process object is audio. The processing target information also includes the reason for the processing (for example, the name of a specific company that corresponds to a broadcast prohibition, etc.).

  As the processing target information, for example, a plurality of pieces of information can be set according to the delivery destination (purpose) of the video. In response to this, for example, a part not to be processed for one distribution destination is to be processed for another distribution destination, and the contents of the above processing are changed according to the distribution destination. Such operations are possible. In such a case, the processing target information is selected by the user.

  Further, as described later, when the editing is finally performed on the material video data, the portion to be processed and the content of the process are corrected after being confirmed by the user. Information on the difference between the final editing information (final editing information) or the final editing information and the automatic editing information is also stored in the information storage unit 132.

An operation example with the above configuration will be described.
First, an example of editing processing will be described with reference to FIGS. 6 to 10, and then similar person search processing (particularly, similar face detection processing) will be described with reference to FIGS. A processing example in which the similar face detection processing is applied to the editing processing will be described.

  FIG. 6 is an example of a flowchart showing a specific operation performed by the system control unit 15. Here, for the sake of simplicity, it is assumed that the processing specified by the user using the editing device 14 is not performed. Further, in FIG. 1, it is assumed that material video data is input (stored) in the recording device 12 only via the automatic editing information creation device 13.

  First, the recording device 11 obtains material video data (S1). The automatic editing information creation apparatus 13 obtains the material video data, and analyzes whether there is a portion to be processed in the image in the material video data (S2). Here, the processing target recognition unit 131 refers to the information in the information storage unit 132 to recognize whether there is a portion to be processed in the image in the material video data, and when this portion is recognized, The processing for this portion is also determined based on the information in the information storage unit 132 (S3). This creates automatic editing information. If the part to be processed is not recognized (No in S4), the material video data is stored in the recording device 12 as it is (S5).

  When the part to be processed is recognized (Yes in S4), the system control unit 15 inquires the user whether to store the material video data (S6). If not stored (No in S6), as described above, the editing device 14 creates automatically edited video data by editing the material video data based on the automatic editing information (S7). The automatically edited video data and the automatic editing information are stored in the recording device 12 (S8). In this case, the video data stored in the recording device 12 is only the automatically edited video data after editing, or, if the material video data is stored in the recording device 12, the material video data is Replaced with automatically edited video data.

  When the material video data is stored (Yes in S6), the system control unit 15 stores the material video data and the automatic editing information in the recording device 12 (S9), and then performs automatic editing for the user. Check whether or not (S10). When the automatic editing is not performed (No in S10), the process ends. In this case, the recording device 12 stores material video data before editing and automatic editing information. For this reason, although there is no automatically edited video data at this time, the edited device 14 can be used to easily create the automatically edited video data later.

  When the automatic editing is performed (Yes in S10), the system control unit 15 causes the editing device 14 to create automatically edited video data (S11), and stores the video data in the recording device 12 (S12). In this case, the recording device 12 stores all of the original material video data, the automatic editing information, and the automatically edited video data. Therefore, for example, when a plurality of pieces of processing target information are set as described above, it becomes easy to perform later processing using the other pieces of processing target information on the same material video data.

  When the recording device 12 directly receives the material video data from the recording device 11 and stores the data, the steps S6 to S8 become unnecessary. However, material video data may be deleted after the automatically edited video data is stored (S12).

  The flowchart of FIG. 6 shows the operation of the system control unit 15 after the input of material video data. On the other hand, there is a video distribution (output) request to the system control unit 15 in a state where the material video data is stored in advance in the recording device 12, and the video data after the material video data is edited according to this is It may be output.

  FIG. 7 is a flowchart showing an example of the operation of the system control unit 15 in such a case. Here, it is assumed that at least material video data is stored in the recording device 12.

  First, when there is a distribution request (S21), the system control unit 15 checks whether the automatically edited video data is stored in the recording device 12 (S22). If the automatically edited video data is not stored (No in S22), it is checked whether the automatic editing information is stored (S23).

  If the automatic editing information exists (Yes in S23), the system control unit 15 creates the automatically edited video data using the editing device 14 as described above, and stores the video data in the recording device 12 (S24). If the automatic editing information does not exist (No in S23), the system control unit 15 creates the automatic editing information using the automatic editing information creating device 13 (S25), and similarly the automatically edited video using the editing device 14 Data is created and stored in the recording device 12 (S24). As a result, when the automatically edited video data is not stored (No in S22), the automatically edited video data is newly created and stored in the recording device 12.

  When the automatically edited video data is stored (Yes in S22), or when the newly edited video data is newly created and stored as described above (S24), the system control unit 15 performs the automatic editing. An image based on the video data is displayed on the editing device 14 (display unit 142) (S26), and the user is inquired as to whether or not the content may be distributed with this content (S27).

  If the contents may be distributed with this content (Yes in S27), the automatically edited video data is set as the edited video data for which distribution is permitted (S28). On the other hand, when a change from this content is desired (No in S27), the system control unit 15 further edits the automatically edited video data using the editing device 14 (S29), and the edited video data is The edited video data permitted to be distributed is set and stored in the recording device 12 (S30). At this time, final editing information is also created and stored as described above.

  Thereafter, the system control unit 15 distributes the edited video data stored in the recording device 12 as described above (S31).

  Also, even if the automatic editing information is created, it takes time to apply it to create the automatically edited video data, and it also takes time to store various video data in the recording device 12. For this reason, it is preferable not to store or create video data that is clearly unnecessary in distribution. Furthermore, in order to reduce the processing time, the user may check the image simultaneously using another device.

  FIG. 8 is an example of a flowchart showing the operation of the system control unit 15 in consideration of such a point.

  Here, when the recording device 11 obtains material video data (S41), it is determined whether the material video data is to be stored as it is in the recording device 12 (S42). If it is recognized that the storage of the material video data is unnecessary (No at S42), the automatic editing process is performed as described above, and the automatically edited video data is created (S43). It is stored in the recording device 12 as video data for distribution (S44). In this case, the video data recorded in the recording device 12 is only the automatically edited video data.

  If it is recognized that the material video data is stored (Yes in S42), the material video data is stored in the recording device 12 (S45). Thereafter, it is inquired whether or not to analyze the material video data using another device (S46). When analyzing the material video data using another device (Yes in S46), the user analyzes the material video data using the other device (S47), and the process thereafter using the editing device 14 Can be started. This analysis result can be used for the following determination (S50, S56).

  Thereafter, it is inquired whether or not automatic editing is to be performed immediately (S48), and if it is not to be performed immediately (No in S48), automatic editing information is created by the automatic editing information creation device 13 (S49). Inquiring whether or not the contents of the automatic editing information are acceptable (S50).

  When this inquiry is made, the automatically edited video data is not actually created, but as described above, the user can confirm the content after editing based on the automatic editing information. It is possible to do this verification using a still image.

  If it is desired to change this content (No in S50), the editing apparatus 14 causes the user to make the correction (S51). Thereafter, if there is no change in the content (Yes in S50), edited video data in which the actual editing work has been performed on the material video data is created based on the automatic editing information as it is (S52). The edited video data is stored in the recording device 12 as video data for distribution (S53). In this case, edited video data is not created until the content is finally determined.

  When the automatic editing is immediately performed (Yes in S48), the automatic editing information and the automatically edited video data based on the automatic editing information are created (S54), and the automatically edited video data is displayed on the display unit 142 (S55). In this case, the user can check in detail whether or not the edited content is appropriate at all time points of the automatically edited video data (S56).

  After that, when it is desired to correct this editing content (No in S56), the correction work and confirmation are performed in the same manner as described above (S57), and thereafter new video data (based on the corrected editing information) The edited video data is created (S58), and the edited video data is stored in the recording device 12 as video data for distribution (S59). The final editing information created at this time is also stored simultaneously.

  If it is determined that the editing based on the automatic editing information is appropriate (Yes in S56), the automatically edited video data that has already been created is stored in the recording device 12 as video data for distribution (S60) .

  In the above operation, the processing time can be shortened by minimizing the necessity of actually performing the editing process on the material video data, and the user can surely check whether the editing process is appropriate or not. And their corrections will be made.

  Next, processing target information stored in the information storage unit 132 to recognize a portion to be processed in the source video data will be described. The parts to be subjected to such processing include the time display as described above, the registered number of the reflected car, the company name, the face of the reflected person, and the like. The time display and registration number can be recognized by pattern recognition of numbers, company names can be recognized by pattern recognition of characters, and faces can also be recognized by a pattern recognition method.

  As described above, in the editing apparatus 14 described above, when the user operates the operation panel 143, it is possible to set a portion to be a target of such processing, and then the final editing information on which the operation is reflected is It is created. In this case, the processing target recognition unit 131 can recognize the final editing information and update (or create) the processing target information. In this case, the processing target recognition unit 131 functions as processing target information modifying means for updating the processing target information to more preferable contents.

FIG. 9 is a diagram showing the flow of such an operation in the process target recognition unit 131.
First, processing target information in the initial state (initial setting) is created by the user (P1). Here, for example, only those that are minimally necessary for processing and relatively easy to recognize are selected as targets. Also, for example, time display in an image as described above can be such an object. The video editing system 1 is repeatedly used as described above using the processing target information. At this time, as described above, in addition to or in place of the editing by the automatic editing information, the editing work is also performed by the user, and finally the final editing information applied to the material video data is created. The final editing information is also stored in the information storage unit 132.

  Therefore, the processing target recognition unit 131 becomes the basis of the automatic editing information by comparing the automatic editing information that is the basis of the automatically edited video data with the final editing information generated after that. The processing object information stored in can be modified. For example, although a character string in the image was not included in the processing target information in the processing target information, the automatic editing information was not processed in the processing, but is specified later by the user and processed in the final editing information When it is considered as a target, the processing target information can be modified to add this character string as a processing target. Conversely, because a character string in the image is included in the processing of the processing target information, the automatic editing information has been processed, but this designation is later canceled by the user and the final editing information is If the processing target information is not processed, the processing target information can be modified to delete this character string from the processing target. The contents of processing in the processing target information (such as blur processing) can be similarly modified. Such an operation can be performed similarly even in the case of processing a specific person by using face recognition in an image, for example.

  In addition, the processing target recognition unit 131 performs processing target information based on statistical processing in the plurality of final editing information recorded, without using such a simple determination as to whether or not the processing target is selected. It can also be modified. At this time, for example, each difference between the final editing information and the automatic editing information may be evaluated numerically, the total of the numerical values may be calculated as a score, and the processing target information may be modified based on this numerical value. For example, it is possible to extract final editing information in which the score is large (the difference is large), and to incorporate in the processing object information a common processing target among these and not included in the processing target information. it can.

  Therefore, in the flow of FIG. 9, this video editing system 1 is used using the processing object information (P1) in the initial state, and at this time, the final editing information is created by the operation of the user. Are stored (P2). Thereafter, as described above, the difference between the final editing information and the automatic editing information is quantified and evaluated (P3). Based on this numerical value, it is determined whether or not it is preferable to rewrite the current processing target information as a comprehensive analysis (P4), and the processing target information is finally updated (P5). Here, as shown in FIG. 9, in the final determination (P4), not only the difference between the final editing information and the automatic editing information as described above, but also the tendency of the editing work by the user in the editing apparatus 14 (for example, It is also possible to consider conditions for an image added due to circumstances newly generated after initial setting (P1), etc., because there are many editing operations for some users and few editing operations for other users.

  Such modification work of the processing target information may be repeated every time the video editing system 1 is used and the final editing information is created, or may be performed periodically. Moreover, when using said score, you may carry out according to the cumulative value of this score.

  As described above, the operation of modifying the processing target information based on a large number of final editing information can also be performed using a well-known machine learning method (deep learning) or the like. As described above, in the case where a plurality of pieces of processing target information are set according to the delivery destination of the video, etc., these tasks can also be performed for each piece of processing target information.

  The user can use the display unit 142 and the operation panel 143 (touch panel display 144) of the editing device 14 to make an inquiry and a setting for setting the evaluation of the automatically edited video data to be input.

  FIG. 10 is an example of such a display. Here, on the display K, the description of the automatic editing information (the contents of the processing in each of the portions to be processed and the respective portions) and the availability of the application thereof are performed. A rating is entered by the user. Thereafter, by operating the lower display M, editing processing is performed using the final editing information in which the automatic editing information has been modified to reflect the operation of the display K.

  There are various types of material video data, and in some cases, processing may be performed on an uncommon special part. In such a case, even if the difference between the automatic editing information and the final editing information becomes large, it is preferable that the final editing information in this case is not used to modify the generally used processing target information. As shown in FIG. 10, when it is selected that the automatic editing information in this case is not targeted for evaluation, the final editing information in such a special case is not used for modifying the processing object information.

  As described above, various methods other than the above can be applied as a method of feeding back the newly created final editing information and updating the processing target information.

  Also, for example, as a part that can be the target of the above-described processing, there is a face of a person who is reflected, and the processing target recognition unit 131 can recognize the face in the image. Here, for example, when there are a plurality of reflected persons and it is desired to apply the processing only to the face of a specific person, or conversely, the processing may be applied to the faces of all persons other than the specific person. is there. In such a case, if the face of the person is set to the first level in the processing target information, as in the case of the above-mentioned broadcast prohibited term, only a warning is issued, and unless the warning is canceled, automatic editing is performed. What is necessary is just to set it as the structure which does not create finished video data and does not deliver material video data. Thereafter, the user controls the operation panel 143 so that the process is performed only on the face of a specific person among all the reflected faces, or the process is not performed on only the face of a specific person. After the editing information is created and the edited video data is created according to the final editing information, this can be distributed.

  In the above configuration, the automatic editing information includes the processing target recognition unit (processing target recognition unit, processing target information modification unit) 131 and the information storage unit (information storage unit) 132 in the recording device 12 (video server). The editing device 14 including the creation device 13, the editing control unit (editing means) 141, the display unit (display means) 142, and the operation panel (operation means) 143 is connected, and the above operation is performed. However, processing target recognition means, processing target information modification means, information storage means, editing means, display means and the like having the same functions as described above are provided in relation to the material video data, and automatically edited video data, automatic editing information, The specific configuration of the apparatus is arbitrary as long as final editing information and the like can be created. That is, how each means described above is provided in each device to be used is optional, and each means described above may be provided in a single device.

  Next, similar person search processing (in particular, similar face detection processing) will be described with reference to FIGS. The processing is executed by the functions of the similar face image detection device 16 and the editing device 14 (in particular, the similar face image detection operation unit 103), and the technology disclosed in JP 2013-101431 A is a face image recognition process Applied to The following illustrates the main parts disclosed.

  FIGS. 11A to 11G illustrate feature amounts of images that have become candidates for search key images along the procedure of performing similar person search in the present embodiment. FIG. 12 exemplifies a procedure for performing similar person search (similar face detection processing).

  First, in the search processing by the first key image 6001, the first search is performed by the first search key image selected by the user. Here, an image having a feature amount close to the feature amount of the image selected in the first search key image (in this example, the feature amount of a person in the image) is sent to the similar person search unit 218 in the recording device 12 As a result, for example, 10 images are searched.

  In FIG. 11A, the feature quantities of the first search key image are indicated by "o". Here, the feature quantities of the image are expressed in two dimensions for easy understanding of the explanation, but in practice, the feature quantities of the image often have a very large number of dimensions, for example, several hundred dimensions. .

  Here, it is assumed that three out of ten images as search results are the same as the first search key image. In processing 6002 for selecting the same person from the search results, the target 3 images are selected from the 10 search result images. Specifically, for example, the user operates the operation panel 143 or the mouse (not shown) of the editing apparatus 14 to select a target image. A threshold is provided for the feature amount of the image, and if the distance between the feature amount of the first search key image and the feature amount of the search result image is equal to or less than the threshold, it is determined to be the same target (the same person), and the corresponding search result The image may be automatically selected.

  In FIG. 11B, in addition to the contents of FIG. 11A, the feature amount of the image selected by the process 6002 of selecting the same person from the search result is indicated by “Δ”. An image selected by such processing becomes a candidate for a new search key image.

  Here, if it is assumed that the search result image is one of continuous images forming a moving image, the images of the same person are often included before and after the search result image in the moving image. In a process 6003 for selecting the same person before and after the search result, the position, the traveling speed, etc. of the person from a plurality of images included in a predetermined time slot before or after the search result image in the moving image from which the search result image is extracted. And automatically select an image determined to be the same person as the person in the search result image (ie, the same person as the person in the search key image). The user may be able to specify.

  In addition to the content of FIG.11 (b), the feature-value of the image selected by the process 6003 which selects the same person before and behind a search result is shown by FIG.11 (c) by "(square). An image selected by such processing becomes a candidate for a new search key image.

  In the image processing 6004 for adding a mask, an image is generated by adding a mask for covering the nose and the mouth by image processing to the image of a person who is a candidate for a new search key image in the processing up to this point, and a new search key Perform processing to add to image candidates. In contrast to this, when the image of the person before the image processing has a mask covering the nose and the mouth, processing may be performed to remove the mask in the image processing. Moreover, the image of a mask may prepare several types of images.

  The image processing 6005 for adding sunglasses and glasses generates an image in which the sunglasses and glasses are added by image processing to the image of a person who is a candidate of a new search key image by the processing so far, and a new search key image Perform processing to be added to the candidate of. In contrast to this, when the image of the person before image processing wears sunglasses or glasses, processing may be performed to remove the sunglasses or glasses in the image processing. In addition, images of sunglasses and glasses may be prepared with multiple types of images.

  The image processing 6006 for changing the direction of the person generates an image in which the direction of the person is changed by the image processing with respect to the image of the person as a new search key image candidate in the processing up to this point. Perform processing to be added to the candidate of. Although the direction of the person is usually plural, it may be simple left-right inversion.

  In FIG. 11D, in addition to the contents of FIG. 11C, an image processing 6004 for adding a mask, an image processing 6005 for adding sunglasses and glasses, and an image processing 6006 for changing the direction of a person are generated. The feature quantities of a plurality of images are indicated by “x”. An image generated by such processing is added as a candidate for a new search key image.

  Note that the image processing 6004 for adding a mask, the image processing 6005 for adding sunglasses and glasses, and the image processing 6006 for changing the direction of a person are the results of the processing 6002 for selecting the same person from the first search key image and search results. The processing may be performed on any of the image and the image of the result of processing 6003 for selecting the same person before and after the search result. Further, any one image processing may be performed on an image of a target, any two image processing may be performed, or all three image processing may be performed. In addition, image processing other than the above may be performed, such as image processing for changing the contrast of the target image.

  Next, in the clustering processing 6007, a plurality of images which have been made candidates of search key images by the processing 6001 to 6006 thus far are clustered to obtain an image (or a feature value thereof) representative of each cluster. As a clustering method, known techniques such as the k-means method can be used. As an image representing each cluster, for example, an image closest to the average of the feature amounts of the images included in the cluster is used, and the feature amount of the image is used as a new search key. The average of the feature amounts of the images included in the cluster may be used as a new search key as it is.

  FIG. 11E illustrates how new search key image candidates obtained by the processes 6001 to 6006 thus far are divided into clusters by the clustering process 6007 and feature amounts of images representative of the clusters. Yes. In FIG. 11E, three clusters are shown surrounded by a frame line, and feature amounts P11, P12, and P13 of an image closest to the center of gravity of each cluster are respectively selected as feature amounts of an image representative of each cluster. It is done.

  In search processing 6008 using a representative search key, similar image search is performed using the feature amount of the image representative of each cluster obtained by the clustering processing 6007 as a new search key, and the result is output.

  Here, in the example of FIG. 11E, since there are 29 images (images obtained by the processes 6001 to 6006) related to the first search key image, the feature amounts of these images are conventionally used. Although the search with the new search key is repeated 29 times, in the present embodiment, similar face image search is performed using feature amounts of three images representing each cluster obtained by the clustering processing 6007. It is made to be able to do three searches while balancing feature quantities. Here, the number of clusters is three, but this can be changed depending on the setting.

  Next, the screen of the similar face image detection operation unit 103 of the editing device 14 will be described with reference to FIG. FIG. 13 illustrates a search screen that can be used in the similar face image search system of this example.

  The search screen has a reproduction image display area 3001, an image reproduction operation area 3003, a search key image specification area 3004, a search narrowing down parameter specification area 3008, a search execution area 3017, and a search result display area 3020.

  The reproduction image display area 3001 is an area for displaying an image recorded in the recording device 12 (similar face image detection device 16 or the like) as a moving image. Further, the moving image 3002 in the reproduction image display area 3001 is for displaying the image recorded in the recording device 12 as a moving image.

  The image reproduction operation area 3003 is an area for reproducing an image recorded in the recording device 12. Each button constituting the present area 3003 is assigned its own reproduction type. In the figure, an example is shown in which rewind types, reverse playback, playback stop, forward playback, and fast-forward playback types are assigned in order from the left. When the user appropriately presses each button with the mouse 282, the moving image 3002 is switched to the playback type assigned to the button.

  A search key image specification area 3004 is an area for specifying and displaying a search key image. The present area 3004 has a search key image 3005, a video designation button 3006, and a file designation button 3007.

  The search key image 3005 is an image to be a first search key image for similarity search. In the initial state, since the search key image is not specified, the image is not displayed. In addition, when it does not designate, it may be made to indicate that it is not designated, such as displaying an image showing an unspecified state prepared separately.

  The image designation button 3006 is a button for designating, as a search key image 3005, an image displayed in the reproduced image display area 3001 at the time of pressing.

  The file designation button 3007 is a button for designating, as a search key image 3005, an image other than the image recorded in the recording device 12, for example, an image photographed by a digital still camera or an image captured by a scanner. When the button 3007 is pressed, a dialog box for specifying the file of the image is displayed, and the user can specify a desired image there.

  The search refinement parameter designation area 3008 is an area for designating the type of the refinement parameter at the time of the search and the value (range) thereof. This area 3008 has imaging device specification check boxes 3009, 3010, 3011 and 3012, time code specification check boxes 3013 and 3014, and time code specification fields 3015 and 3016.

  The imaging device specification check boxes 3009, 3010, 3011, and 3012 are check boxes for specifying an imaging device (such as the camera 10) to be searched at the time of search. When the check boxes 3009, 3010, 3011, and 3012 are pressed, check marks indicating that they have been selected are displayed. When this mark is pressed again, it becomes non-display, and when it is pressed, display / non-display is repeated.

  The time code specification check boxes 3013 and 3014 are check boxes for specifying a time range to be searched when searching. This check box is the same as the other check boxes as to the display mode. When the time code specification check box 3013 is selected, the start time is given to the time range. In the non-selected state, it means that the head time is not given to the time range, that is, the image of the oldest time recorded in the recording device 12 is set as the search target range.

  Similarly, when the time code specification check box 3014 is selected, the end time is given to the time range. In the non-selected state, it means that the end time is not given to the time range, that is, the image of the newest time recorded in the recording device 12 is set as the search target range.

  The time code specification fields 3015 and 3016 are input fields for specifying the values of the start time and the end time described above. In the initial state, since all time zones are to be searched, the time code specification check boxes 3013 and 3014 are all not selected and the time code specification fields 3015 and 3016 are blank.

  The search execution area 3017 is an area for instructing a search execution. This area 3017 includes a similar person search button 3018 and an appearance event search button 3019, as well as a similar person search button 3300 from the search result, the same scene check box 3201, a mask check box 3202, a sunglasses check box 3203, a different angle check box 3204 Have.

  The similar person search button 3018 is a button for instructing execution of a similar person search (search process 6001 using a first key image) by the search key image 3005. When a parameter is designated in the search narrowing parameter designation area 3008, execution of a similar person search is instructed according to the designated parameter.

  The appearance event search button 3019 is a button for instructing execution of the appearance event search. If a parameter is specified in the search / refinement parameter specification area 3008, execution of the appearance event search is instructed according to the specified parameter.

  The search result display area 3020 is an area for displaying a search result. The display of the search results is carried out by displaying a list of search result images. In the initial state, nothing is displayed in the search result display area 3020.

  Here, the user presses the image designation button 3006, presses the imaging device designation check box 3009, 3010, 3012, and further presses the time code designation check boxes 3013, 3014, and the time code designation field 3015, It is assumed that “15: 30: 20: 17” and “12: 30: 20: 17” are input to 3016 respectively.

  Thereby, as shown in FIG. 13, in the search key image 3005, an image of the person “A” displayed in the moving image 3002 is designated as a search key image, and “camera Three points “1”, “camera 2”, and “camera 4” are designated, and “15: 30: 20: 17 to 12: 30: 20: 17” is designated as a time range to be searched.

  Thereafter, it is assumed that the user presses the similar person search button 3018. Then, in the search result display area 3020, the search result obtained by executing the similar person search using the search key image 3005 is displayed. FIG. 13 shows an example of the search screen in this state. The display of the search results is implemented by displaying a list of search result images (in this example, search result images 3031 to 3141).

  For example, the search result images 3031 to 3141 are displayed in order of similarity with the search key image 3005 from the top left to the right and then from the left to the right in the second row. In this display example, it is shown that the search result image 3031 has the highest similarity to the search key image 3005, and the search result image 3141 has the lowest similarity.

  In the notation of the example shown in this figure, the circle and the alphabet illustrated on the search result images 3031 to 3141 in the search result display area 3020 are a simplified representation of the person's face and person name, for example, the search The result image 3031 shows that the character "Mr. A" appears. Of course, an actual image is displayed in the display in an actual system in the simplified display portion.

  Around the search result image 3031, a cue play button 3032, a search key image designation button 3033, and a search target check box 3301 are provided. The same applies to the other search result images 3041 to 3141.

  The cue reproduction button 3032 is a button for instructing start of continuous moving image reproduction with the search result image 3031 at the top. For example, when the cue reproduction button 3032 is pressed, the moving image 3002 is switched to the search result image 3031, and the user can view the moving image starting with the search result image 3031.

  The search key image designation button 3033 is a button for designating the search result image 3031 as a new search key image. For example, when the search key image designation button 3033 is pressed, the search result image 3031 is displayed on the search key image 3005. Thereby, it is possible to carry out the re-search using the search result image 3031.

  The search target check box 3301 is a check box for specifying the search result image 3031 as a new search key image (or its candidate) when the similar person search button 3300 from the search result is pressed. For example, all images of Mr. "A" appearing in the search result (in this example, search result images 2031 to 3061, 3081, 3091, 3121, 3141) are checked, and the similar person search button 3300 from the search result is displayed. By pressing, it is possible to search for "Mr. A" of various patterns.

  The similar person search button 3300 from the search result is a button for instructing execution of another similar person search (search process with representative search key 6008) based on the result of the similar person search by the search key image 3005. In the similar person search again, the user selects an image (a search target check box is checked) selected by the user from the display of the search result display area 3020 (the result of the search process 6001 by the first key image) as a new search key Re-execute similar person search as an image (or its candidate).

  The same scene check box 3201 executes processing 6003 for selecting the same person before and after the search result for the image selected by the user from the display of the search result display area 3020, and the resulting image (target image This is a check box for designating that the image of the same person as the person in the image before and after) be added to the new search key image candidate.

  Note that the mask check box 3202 executes the image processing 6004 for adding a mask to an image selected by the user from the display of the search result display area 3020, and the resulting image (a person in the target image Is a check box for designating that an image with a mask added or an image with a mask removed from the person is added as a new search key image candidate.

  In addition, the sunglasses check box 3203 executes an image processing 6005 for adding sunglasses or glasses to an image selected by the user from the display of the search result display area 3020, and the resulting image (in the target image It is a check box for designating that an image obtained by adding sunglasses or the like to a person or an image obtained by removing sunglasses or the like from the person is added as a new search key image candidate.

  The different angle check box 3204 executes the image processing 6006 for changing the direction of the person for the image selected by the user from the display of the search result display area 3020, and the resulting image (in the target image This is a check box for designating that an image in which the direction of the person is changed is added to a new search key image candidate.

  When the similar person search button 3300 from the search result is pressed in a state where one or more of these check boxes 3201 to 3204 is checked, each of the display of the search result display area 3020 is selected by the user. Perform image processing corresponding to the checkbox in the checked state for an image, add a new search key image candidate as a result to the generated image, and then add the new search key image candidate On the other hand, clustering processing 6007 is executed to obtain a search key image representing each cluster, and the similar image search is executed using the feature amount of the image representing each cluster as a search key.

  As described above, in the above example, search key determination means for determining the feature amount of the image to be the search key based on the feature amounts of the plurality of images as search key candidates, and the search key by the search key determination means And retrieval means for retrieving an image having a feature amount similar to the feature amount of the image determined in step b), the search key determination means clusters feature amounts of a plurality of images as search key candidates. The feature amount of the image representing the cluster is determined as the search key for each cluster, and the search means is configured to perform the search using the search key for each cluster determined by the search key determination means.

  In this example, the search key determination unit is realized by the function of the multiple search key selection unit 113 of the similar face image detection operation unit 103 of the editing device 14, and the function of the similar person search unit 218 of the similar face image detection device 16. Although the search means is realized, the search key determination means and the search means may be realized according to other aspects.

  Next, a processing example in which the similar person search processing (similar face detection processing) described above is applied to the editing processing will be described with reference to FIGS.

  As described above, in the flow until the cast scene (cast video) of the cast performed conventionally is searched, if the person in charge (editor etc.) searches the information of the cast on the management terminal, the cast A list of appearing programs and VTR tape numbers in which the programs are recorded is displayed. After that, the person in charge takes out the VTR tape of the output tape number from the shelf and reproduces it by the VTR reproducing machine. Then, the reproduction video was visually observed to search for a cast scene, and the time code information of the cast scene was recorded. In such a flow, improvement has been required from the viewpoint of work efficiency and accuracy. Therefore, we introduce the following flow technology.

  FIG. 14 shows a procedure for storing a face image in the face image storage server 127 when the original video is recorded on the medium 5 (optical media 5a, magnetic media 5b and VTR tape 5c). The procedure for finding the media 5 is conventional.

  When the original video is recorded on the optical media 5a and the magnetic media 5b, the video file is taken out from the found media (the optical media 5a and the magnetic media 5b), and the similar face image detection device 16 reproduces the video file. Only the image of the face portion is cut out from the reproduced image using the above-mentioned similar person search processing technology, and the cut-out face image is stored in the face image storage server 127 together with time code information.

  The face images to be stored are not only one type (generally a frontal face) but a plurality of face types (front, side, diagonal, back, laughing, angry, etc.) as face images to be detected. It can be registered and saved, and a face image and its face type are associated and recorded. By preparing a plurality of face images to be detected, in particular a plurality of face images of different types, it becomes possible to more accurately detect a video on which a specific performer has appeared, and It is possible to detect a particularly desired situation (such as wanting a video of a smiling face) in the video of a performer. When the face image is stored, if the names of the performers can be identified, the names may also be registered. When a plurality of face images of the same performer are recorded in the face image storage server 127, a reference face image (reference face image) may be designated. The reference face image is not limited to one, but may be set to one for each face type or one at a predetermined appearance time (for example, five years) from the viewpoint of workability.

  When the original video is recorded on the VTR tape 5 c, the VTR tape 5 c found is reproduced by the VTR reproducing device (media reproducing device 19) and taken into the similar face image detecting device 16. Similar to the case of the optical media 5a and the magnetic media 5b, the similar face image detection device 16 cuts out only the image of the face portion using the technique of similar person search processing as in the case of the optical media 5a and the magnetic media 5b. Are stored in the face image storage server 127 together with time code information.

  FIG. 15 shows a procedure for storing a face image in the face image storage server 127 when the original video is recorded in the low resolution server 126.

  When the original video is recorded on the low resolution server 126, when the person in charge searches for information on the performer on the management terminal 17, the program in which the performer appears and the low on which the program is recorded are recorded. The video file name in the resolution server 126 is output. The information is passed to the similar face image detection device 16 online as it is, that is, via the network 2. As a result, the video file is taken out from the low resolution server 126, and the similar face image detection device 16 reproduces the video file to cut out only the video of the face part from the reproduced video, and the cut out face image together with the time code information It is stored in the accumulation server 127.

  FIG. 16 is a diagram in which a similar face is detected from the face image storage server 127 with the face image of the target performer as the detection target.

  The editor causes the similar face image detection device 16 to read the face image file (the face image to be detected) of the target performer. The face image to be detected may be an image extracted as a representative face image from a video file to be edited, or may be a face image selected from face images included in the face image storage server 127, or an image on the web It may be an image captured from. The similar face image detection device 16 compares the face image to be detected with the face image in the face image storage server 127, and a face image of a scene where a cast of the same face appears and time code information are searched.

  The time code information of the appearance scene detected here is passed to the editing device 14. The editor does not have to search for a scene in which a target performer appears, and can create a feature program of the performer or apply a mosaic to the performer.

  In addition, if it is desired to make a preview of the detected appearance scene without using the editing device 14, the appearance scene preview can be easily performed by reproducing the video file in the low resolution server 126.

  By performing such a similar face detection process, the similar face image detection device 16 automatically appears, for example, when searching for a scene in which a target performer appears from a huge past video of a broadcasting station. It will detect the scene. As a result, the editor need not look at the image of the medium 5 (optical medium 5a, magnetic medium 5b, VTR tape 5c). In the meantime, the editor can do other work, and the efficiency of the editor's work can be greatly improved.

  In addition, since the number of editing devices 14 is limited, when the editing device 14 can not be used, the appearance scene of the target performer is searched in advance, and the appearance scene is previously made using the video file of the low resolution server 126 By previewing in, it is possible to perform pre-editing work.

  In addition, when the cast has a problem just before the broadcast is completed, and the cast of the cast becomes impossible, the above-mentioned technology makes it easy for the cast scene of the desired cast to appear. It becomes possible to perform processing for applying mosaics to the performers, or for processing to cut out the appearance scenes, thereby preventing claims from sponsors and viewers.

  In the above process, a performer is detected from the past video of the broadcast station. However, since the face of the performer changes several decades after the video recording, if the current face image of the target performer is targeted for detection, the accuracy of the detection may be low. In order to solve that, once, it replaces with the face image (the face image of the past which detection accuracy fell) of a result detected as the detection object of the present face image again, and registers it newly as a face image of detection object, The detection accuracy can be improved by detecting similar face images again. That is, improvement in detection accuracy can be expected by two-step detection (new information re-registration of reference face (reference face image) → similar face image search).

  Then, by passing the time code information of the detected appearance video to the editing machine, the editor can mosaic the video on which the cast appears, or cut the appearance video.

  Also, if the appearance time (shooting time) is a face image that is close, similar feature quantities are considered to appear, so it is possible to detect face images with appearance times open while following similar feature quantities. . Similarly, when a side face image is desired, the side face is re-registered as a face image to be detected, and similar face image detection enables detection of a more narrow appearance scene.

  Then, by passing the time code information of the detected appearance scene to the editing machine, the editor can create a special program in which only the scene in which the performer appears.

  At the present broadcasting station, the edited video (edited video) is recorded on the optical media 5a, and it is broadcasted to the reproduction apparatus or broadcasted from the optical media 5a or taken from the optical media 5a to the transmission server 18 and broadcasted. There are many. Therefore, the video file is taken out from the optical medium 5a, the video file is reproduced in the similar face image detection device 16 (similar face detection device), only the video of the face is cut out from the reproduction video, and the cut out face image is a face together with the time code. By storing similar faces in the image storage server 127 and detecting the face image of the target performer as a detection target, the person in charge can search for the appearance scene without looking at the video. Here, by preparing a plurality of face images to be detected, such as a front face, a side face, an oblique face, etc., it becomes possible to more accurately detect a scene in which a performer who wants to search is appearing.

  The present invention has been described above based on the embodiments. This embodiment is an exemplification, and it is understood by those skilled in the art that various modifications can be made to the combination of the respective constituent elements, and such modifications are also within the scope of the present invention.

1 video editing system 2 network 5 media 5a optical media 5b magnetic media 5c VTR tape 10 camera 11 recording device 12 recording device 13 automatic editing information creation device 14 editing device 15 system control section 16 similar face image detection device 17 management terminal 18 transmission server 19 media reproduction apparatus 103 similar face image detection operation unit 110 keyword recording unit 111 keyword search unit 112 keyword assignment request transmission unit 113 multiple search key selection unit 121 material video data unit 122 edited video data unit 123 automatically edited video data unit 124 Final editing information unit 125 Automatic editing information unit 126 Low resolution server 127 Face image storage server 128 Processing object information unit 131 Processing object recognition unit 132 Information storage unit 141 Editing control unit 142 Display unit 143 Operation panel 144 Touch Image display unit 212 image recording unit 212 reproduction control unit 213 person area detection unit 214 person feature amount extraction unit 215 person feature amount recording unit 216 attribute information recording unit 217 request reception unit 218 similar person search unit 219 appearance event search unit 220 search result transmission unit 221 search request transmission unit 222 search result reception unit 223 search result display unit 224 reproduction image display unit 225 screen operation detection unit

Claims (4)

An editing system provided with an editing apparatus for editing a video file used for broadcasting, comprising:
A face image storage server for acquiring the face image of the performer included in the video file and associating the face image with time code information of the appearance video of each performer;
A cast video detection unit that detects a cast video in the specific program by comparing the face image recorded in the face image storage server with the face image to be searched included in the video file of the specific program;
Based on the appearance video detected by the appearance video detection unit, another appearance video in which a person of the face image to be searched for appears in the specific program is detected by similar face image search, and And a similar face image detection device for notifying the editing device of time code information in association with the performer information as the search target,
The editing system is characterized in that the video file of the specific program is edited using the time code information.   The editing device reproduces the video file of the recording device when the video file of the performer is stored in the recording device to be processed by the similar face image detection device when editing the video file. The editing system according to claim 1, wherein the video confirmation of the appearance video is displayed in a possible manner.   The editing system according to claim 1 or 2, wherein the editing apparatus reproduces the appearance video detected using the low resolution video. The face image storage server can store a face image to be detected in association with the type of face, and
The editing system according to any one of claims 1 to 3, wherein the similar face image detection device performs a similar face image search according to the face type.

Images