JP4270703B2 - Automatic person index generator - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an automatic person index generating apparatus that automatically generates an index of persons who participate in a program.
[0002]
[Prior art]
In a television broadcasting station, it is common to produce a new broadcast program (secondary use of a recorded program) by capturing a part of video and audio recorded on a broadcasted VTR tape.
For this reason, if an index corresponding to various features is added to the recorded video, the recorded VTR tape is played back from the beginning to the end for secondary use of the video. Even without it, it is possible to search for a desired video immediately by using the index, and program production can proceed efficiently.
[0003]
It is extremely convenient if one of the features of the video can be selected especially for a participating person and an index of the participating person (hereinafter referred to as a person index) can be generated.
In particular, in a sports video or the like, a person index indicating who is shown in the video is very important. However, in the case of a multiplayer game such as soccer, assigning a person index to the broadcast video currently has to rely on human hands. At present, there is no automatic person index generation or assignment to a program. Therefore, when searching for a video in which a specific person is shown, the video must be viewed from time to time.
[0004]
[Problems to be solved by the invention]
As described above, conventionally, an operation for automatically generating a person index and assigning it to a program has not been performed. However, what can be considered with the conventional technique is to recognize a person shown in a video by image processing and use it as a person index.
[0005]
In this case, a region where a person is shown is cut out from the video, and a featured portion such as a face is extracted from the region and compared with the feature of a human database prepared in advance. A person index is generated. However, with this method, it is difficult to identify an individual if the person in the video is too small to perform analysis for feature extraction or if the face is facing backward (feature extraction is not possible). . This method also has a problem that it takes a lot of time to process when there are a plurality of persons on the screen.
[0006]
The object of the present invention is the above-described problem of the conventional method, i.e., it is too dependent on facial feature extraction, so it is difficult to specify the person if the person on the screen is small or facing backwards. A very practical automatic index generator that eliminates problems such as a lot of processing time if there are multiple people in it, can be processed in real time, and can be realized on a small scale Is to provide.
[0007]
[Means for Solving the Problems]
In order to achieve the above object, the automatic human index generation device of the present invention generates data representing the positions of a plurality of persons from a wide-angle whole image captured by a television camera equipped with a wide-angle lens, and actually programs For the person shown in the video shot by the broadcast TV camera that shoots the video for the broadcast, the data indicating the location of the generated multiple persons and the pan, tilt, zoom and focus of the actual broadcast TV camera It is specified by the shooting range of the TV camera calculated from each camera parameter, and by this specification, an index that the corresponding person is included in the video is generated.
[0008]
That is, the automatic human index generating apparatus of the present invention is an automatic human index generating apparatus that automatically generates an index of persons who participate in a television program, and includes a first television camera that captures a wide range of images, The second television camera that captures a video for a program that is photographed simultaneously with the photographing by the television camera and whose photographing range is included in the photographing range of the first television camera, and the first television camera An overall video analysis unit for analyzing data representing the position of the participating person from each captured video for each participating person, and using the camera data of the second television camera, the second television camera. A shooting range data generating unit that generates shooting range data, and the presence position of a participant who is analyzed and obtained by the entire video analysis unit is the shooting range data generation unit. It is determined whether or not it is included in the shooting range of the second TV camera using the shooting range data generated by the section, and only those included as a result of the determination are determined for each participant. A person index output unit that outputs data representing time existing within the shooting range of the second television camera of the participant as a person index for multiplexing the recording by the second television camera; It is characterized by this.
[0009]
The automatic human index generating apparatus of the present invention includes a plurality of the second TV camera and the shooting range data generating unit for generating shooting range data of the TV camera, and the person index output unit includes the entire person index output unit. The presence positions of the participants who are analyzed and obtained by the video analysis unit are included in the shooting ranges of the respective second TV cameras using the shooting range data generated by each of the plurality of shooting range data generation units. Only for those included as a result of the determination, data representing the time within the shooting range of the second television camera of the participant for each participant, corresponding to each of said plurality of television cameras, configured to output as a person index to multiplexing the recording by the plurality of TV cameras It is characterized in that the.
[0010]
Further, the automatic human index generating apparatus of the present invention is characterized in that the camera data of the second television camera is camera parameters of pan, tilt, zoom and focus of the television camera.
[0011]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, the present invention will be described in detail based on an embodiment of the invention with reference to the accompanying drawings.
FIG. 1 is a block diagram showing an embodiment of the automatic human index generating apparatus according to the present invention.
In FIG. 1, reference numeral 1 denotes a wide-area video shooting TV camera, 2 denotes an overall video analysis unit, 3 denotes a program shooting TV camera, 4 denotes a shooting range data generation unit, and 5 denotes a person index output unit.
[0012]
FIG. 2 shows the operation of the apparatus of the present invention by taking a broadcast broadcast (or recording) of a soccer game as an example.
The operation of the device of the present invention will be described below with reference to FIGS.
As shown in FIG. 2, the television camera 1 for wide-area video photography in FIG. 1 captures a wide-area video over the entire soccer ground from the upper part of the stand of the soccer stadium. Therefore, all the players who have participated (indicated by a to g in FIG. 2) are somewhere in the screen of the video imaged by the camera 1. The captured video signal is supplied to the entire video analysis unit 2 in FIG. 1, and the video is analyzed for each player so as to output data representing the player's position at that time as time passes. Is called. Here, the detection (analysis) of the presence position can be performed using, for example, a well-known pattern matching technique, and the ground is set on a two-dimensional plane so that the player's presence position is displayed in coordinates. Also good.
[0013]
On the other hand, a camera parameter detection sensor such as pan, tilt, zoom, and focus is attached to a television camera for broadcast broadcasting (the TV camera 3 for shooting a program shown in FIGS. 1 and 2). Data (hereinafter referred to as camera data) is obtained.
[0014]
The obtained camera data is supplied to the shooting range data generation unit 4 shown in FIG. 1, where the broadcast broadcast TV camera (also known as the TV camera for program shooting is indicated by reference numeral 3 in FIGS. 1 and 2). In other words, which range is imaged (hereinafter referred to as imaging range data) is generated by calculation as, for example, ground on a two-dimensional plane and region data on the two-dimensional plane. This calculation is well known in the literature (Yamauchi et al., “Electronic Studio System” Television Society Technical Report, Vol. 16, No. 23, pp. 1-6, 1992) and will not be described here.
[0015]
FIG. 3 shows a shooting range of the TV camera 3 for program shooting (see FIGS. 1 and 2) generated as described above.
In FIG. 3, ● indicates the position of each player from a to g, and the hatched portion particularly indicates an area on the ground that is shot by the TV camera 3 for program shooting.
In addition, the area | region image | photographed by the television camera 1 (refer FIG. 1, FIG. 2) for wide-range image | photographing mentioned above covers the ground whole surface (Therefore, the presence position of all the players from a to g is analyzed. Please note that.
[0016]
As shown in FIG. 1, regarding the position of the player on the ground obtained by the entire video analysis unit 2, the data obtained for each player is the television camera 3 for program shooting generated by the shooting range data generation unit 4. Is supplied to the person index output unit 5 together with the shooting range data indicating the shooting range. The person index output unit 5 uses the shooting range data generated by the shooting range data generation unit 4 to indicate the positions of the players a to g obtained by analysis and obtained by the entire video analysis unit 2. It is determined whether or not it is included in the shooting range of the camera 3, and as a result of the determination, the presence of the player is indicated for each player (only in this case, players d and e) included. Output data as person index.
[0017]
Now, in the output of the entire video analysis unit 2, the position of each player from a to g (of course, this position has changed all the time because it is a soccer game) is a two-dimensional coordinate (x _i , y _i ) ( Here, i = a to g), and the shooting range of the TV camera 3 for program shooting indicated by hatching in FIG. 3 is also expressed by an expression f (x, y) that limits the area on the same coordinate system. Suppose that only players (players d, e) whose location is in the region f (x, y) limited by the shooting range are symbols representing players (players a to g in this example) It can be output as a name (person index).
[0018]
FIG. 4 shows an output example of the person index output from the person index output unit 5 as described above.
2 and 3 is taken at time T on the time axis t in FIG. The shooting range of the TV camera 3 for program shooting always changes and the players are always moving. At the shooting time T, only the players d and e are shown on the shooting screen by the TV camera 3. 4 shows. The meanings shown in FIG. 4 are the same at other arbitrary time points. In short, it shows which player is captured at which time of the video imaged by the television camera 3.
[0019]
In the above, for the sake of simplicity, it is assumed that a soccer game is shot with one TV camera for broadcast broadcasting (TV camera 3 for shooting a program), but multiple TV cameras are used for the actual relay. On the other hand, an on-air (broadcast) program is created by switching and using the output videos of the plurality of TV cameras in accordance with the intention of the program producer.
[0020]
FIG. 5 is a block diagram showing another embodiment of the device of the present invention in which the automatic human index generating device of the present invention is adapted in such a case.
5 is different from the configuration of FIG. 1 in that the TV camera for shooting a program and the shooting range data generation unit are respectively 3-1, 3-2, 3-3 and 4-1, 4-2, 4-3. There are a plurality of (three in this example) points from the fact that there are a plurality of points, the person index output unit 5 in FIG. Since a person index is obtained, one camera is selected from the plurality of camera-specific person indexes in accordance with an instruction from an on-air camera instruction unit 7 (so-called switcher table) that instructs an on-air camera. The point is that a final person index selection section 8 is provided as a final person index.
[0021]
Note that, in the above, the camera-specific index output unit 6 is generated by the respective shooting range data generation units from 4-1 to 4-3 where the existence positions of the respective players analyzed and obtained by the entire video analysis unit 2 are obtained. 3-1 to 3-3, it is determined whether or not it is included in the shooting range of each TV camera for program shooting, and data indicating the presence of the player determined to be included (person index) ) Is output for each television camera from 3-1 to 3-3. This is why the number of output lines of the camera-specific person index output unit 6 in FIG. 5 is three.
[0022]
Now that the description of the present invention has been completed, a typical application example of the present invention will be described.
By multiplexing the person index generated by the automatic person index generating apparatus of the present invention shown in FIG. 1 or FIG. 5 with on-air video and audio by various methods, a symbol or name representing a player is used for secondary use of the recorded program. A scene in which a specific player is shown as a key can be automatically searched.
This is a big difference in labor and time that a production person in the past has played a recorded VTR tape from the beginning to the end to search for a specific player for the secondary use of the program.
[0023]
FIGS. 6A, 6B, and 6C show search examples when the present invention is used and a person is specified (for example, a specific player is specified) as described above.
In a certain recorded VTR tape, it is assumed that the indexes of the players P, Q, and R are generated and multiplexed on the program (see FIG. 6A). At this time, in order to search for the video of the players P and R in cooperation play, when searching for the video in which both are shown at the same time, the portion of 3: 03-3: 45 and the portion of 6: 30-7: 51 ( (See FIG. 6B). In this way, the search key can be designated by the logical product or logical sum of the players. As a result, the necessary parts can be narrowed down without viewing the video from the beginning, and the efficiency of program production is greatly improved. FIG. 6C shows an example in which only the portion where the players P and R may play a cooperative play is extracted using the search result.
[0024]
In the above, the automatic person index generating device of the present invention has been described by taking a soccer game relay as an example. However, the present invention is not limited to this, for example, other sports such as American football and basketball, and programs other than sports. It can also be applied to live broadcasts and recordings.
[0025]
【The invention's effect】
When a recorded program such as a recorded VTR is used for the second time, a search for a video showing a specific person often occurs. However, since a person index could not be generated in the past, it is not possible to search for a specific person video. It was possible. The only way to assign a person index in the conventional way is to add a person index by looking at the video from the beginning. If this method shows multiple people on the screen, That work is enormous.
According to the present invention, it is possible to automate these operations, support for the production of a digest program, search for secondary use, and information describing the flow of a game in sports video, which is integrated at a higher level. Information management is expected.
[Brief description of the drawings]
FIG. 1 is a block diagram showing an embodiment of an automatic human index generating apparatus according to the present invention.
FIG. 2 shows the operation of the apparatus of the present invention taking a soccer game broadcast as an example.
FIG. 3 shows a shooting range of a TV camera for shooting a program.
FIG. 4 shows an output example of a person index.
FIG. 5 is a block diagram showing another embodiment of the device of the present invention.
FIG. 6 shows a search example when a person is specified using the present invention.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 Television camera for wide range video photography 2 Whole video analysis part 3,3-1,3-2,3-3 Television camera for program photography 4,4-1,4-2,4-3 Shooting range data generation part 5 Person Index output unit 6 Camera-specific person index output unit 7 On-air camera instruction unit 8 Person index selection unit

Claims (3)

An automatic human index generating device that automatically generates an index of a person participating in a TV program,
A first television camera that captures a wide range of video;
A second television camera that captures an image for a program that is photographed simultaneously with the photographing by the first television camera and whose photographing range is included in the photographing range of the first television camera;
An overall image analysis unit for analyzing data representing the presence position of a participant from a wide range of images taken by the first television camera;
A shooting range data generation unit for generating shooting range data of the second television camera using the camera data of the second television camera;
Whether or not the presence position of the participating person analyzed and obtained by the whole video analysis unit is included in the shooting range of the second television camera using the shooting range data generated by the shooting range data generation unit. For each of the participants only for those included as a result of the determination, data representing the time within the shooting range of the second television camera of the participant is included in the second An automatic human index generating device comprising a human index output unit that outputs a human index for multiplexing in recording by a television camera . The automatic person index generation device according to claim 1,
The apparatus includes a plurality of the second TV camera and the shooting range data generation unit that generates shooting range data of the TV camera, and the person index output unit is analyzed and obtained by the whole video analysis unit. It is determined whether or not the presence position of the entered participant is included in the shooting range of each of the second television cameras using the shooting range data generated by each of the plurality of shooting range data generation units. As a result of the determination, only for those included, data representing the time within the shooting range of the second television camera of the participant for each participant is the data of the plurality of television cameras. corresponding to each automatic person Lee, characterized by being configured so as to output as a person index to multiplexing the recording by the plurality of TV cameras Index generator. In the automatic person index generating device according to claim 1 or 2,
2. The automatic human index generating apparatus according to claim 1, wherein the camera data of the second television camera is pan, tilt, zoom and focus camera parameters of the television camera.

Images