JPH07105400A - Motion picture reproducing device - Google Patents

Abstract

PURPOSE:To allow the motion picture reproducing device to realize the changeover and display of a motion picture on request of a viewer with a simple system available for ordinary viewers by tracing an object selected by the viewer. CONSTITUTION:Plural motion pictures picked up simultaneously are divided again as n-sets of object areas and the result is stored in a time series data storage section 12 as time series data. Furthermore, position information representing a position of each object to be picked up in an entire area is detected and stored in a position information storage section 14. In the case of reproduction, the position information storage section 14 is used to select the time series data representing the picked-up and commanded object and the object is displayed at a desired position.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image reproducing apparatus, and more particularly, a region to be photographed is simultaneously photographed and stored as a plurality of moving images, and a user selects a particular object being photographed. Then, the present invention relates to a moving image reproducing apparatus for selecting and displaying a stored moving image so as to track the selected object.

[0002]

2. Description of the Related Art Moving pictures taken by a video camera are
It is composed of a plurality of objects to be photographed and a background. If the background is also an object, it can be said that the moving image is a plurality of objects that have temporal movements taken at the same time. An animation created by computer graphics can also be treated as a plurality of objects having different movements taken at the same time, like a moving image taken by a video camera.

In the conventional moving image reproducing apparatus, the photographer or an editor who is similar to the photographer decides which one of the moving objects should be focused on for reproduction. For example, when playing a video of a soccer match, in addition to a camera that shoots the ground from the audience side in a wide-angle view, a camera that mainly shoots near the corner of the ground and a camera that shoots the goals of each team from behind are prepared. ， Shoot the subject ball or player from multiple angles simultaneously. Then, a scene in which the movement of the subject at each time is accurately captured is selected from the plurality of captured moving pictures, and one temporally continuous moving picture is edited and reproduced according to the selected scene. As a result, the switching of the shooting angle according to the movement of the ball or the player will be performed in the video being played back, compared to the case of simply playing back the video shooting the game with one camera. I was trying to convey the ball and the movement of the player more accurately.

However, in this method, since the editor is switching the moving images from a plurality of cameras, there is a problem that the reproduced moving image does not always satisfy the viewer. For example, soccer match videos are generally edited to capture the movement of the ball exactly, but even if this satisfies the viewer who wants to enjoy the flow of the match, it is possible to observe the movement of a particular player. I couldn't satisfy the viewers who wanted to. For this reason, there has been a demand for a moving picture reproducing apparatus that allows the reproduced moving pictures to be freely switched depending on the viewer's intention.

As a conventional moving picture reproducing apparatus for solving the above-mentioned problems, for example, it is shown in an article "A high-tech moving picture of this amount can be seen nationwide" in the January 1992 issue of the magazine PIXEL (graphic processing information center publication). As described above, there is a method of simultaneously displaying a plurality of moving images obtained by simultaneously shooting the same subject from a plurality of angles on a large screen or a plurality of displays so that there is no time lag. According to this device, a viewer can freely move his or her visual field on a large screen or a plurality of displays, and watch a plurality of moving images while freely switching according to the taste of each viewer.

[0006] However, in this device, since the switching of the moving image is realized by the viewer moving his or her visual field, the fatigue of the viewer becomes great and, at the same time, the viewer's request is maximally satisfied. There is no guarantee that you have selected a video. Furthermore, when viewing while moving the field of view, it is difficult to reproduce a moving image because it is intuitive and there are no selection criteria. Further, in order to reproduce a plurality of moving images at the same time, it is necessary to prepare a plurality of moving image reproducing devices and a large screen or a plurality of displays. Therefore, a simple video playback device that can be used by more general viewers has been desired.

[0007]

SUMMARY OF THE INVENTION An object of the present invention is to switch and display a moving image so that an object selected by a viewer is displayed at a desired position, for example, in the center of the screen.
It is to realize the switching and display of moving images based on the viewer's request with a simple system that can be used by general viewers.

[0008]

In order to achieve the above object, the moving image reproducing apparatus of the present invention has at least n (n ≧ 2).
Time series data storage means for storing time series data of a book,
In a moving image reproducing apparatus equipped with a moving image display means for reading out m (m ≧ 1) time-series data from the stored n time-series data and synthesizing and displaying a moving image, For each object inside, a position information storage means for detecting and storing position information of the position of the object in the entire area is provided, and when storing a moving image,
A plurality of moving images obtained by simultaneously photographing the entire region are subdivided into n target regions, and the n target regions are respectively stored as time-series data by using the time-series data storage means and are also photographed. The movement history of one or more objects is stored as time-series data of position information by using the position information storage means, and when the user indicates and reproduces the object, the instructed object The time-series data in which an object is photographed and the position in the time-series data are searched using the position information storage means, and m time-series data are displayed so that the object is displayed at a desired position designated in advance. The data is selected and displayed by the moving image display means.

[0009]

According to the present invention, there is provided position information storage means for detecting and storing position information regarding the position of an object in the entire area, which is an object to be photographed. When storing moving images, multiple moving images taken at the same time are subdivided into n target areas and stored as time-series data, and the movement history of each object is stored as time-series data of position information. Since it is stored in the information storage means, it is possible for the user to select a specific object being photographed when playing the video and select and display the video so as to track the selected object. Become.

[0010]

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 shows an example of the configuration of a moving picture reproducing apparatus using the present invention.

In FIG. 1, image input units 10-1 and 10 are provided.
Reference numerals -2 and 10-3 each include a video camera such as a video camera, or a video deck or a laser disk device that stores and reproduces a video captured by the video camera.
When there are a plurality of video input units 10, the video switching unit 11 switches the input video from them and selects one video, sets a spatial processing target area for the selected video, and Digitally encodes the image of the processing area,
This is a mechanism for outputting time-series data by performing processing such as JPEG encoding for reducing the amount of data as necessary. The time-series data storage unit 12 is, for example, a magnetic disk device that stores the time-series data output from the video switching unit 11. The position information creation unit 13 is a mechanism that outputs the position information of the target object for each of the plurality of target objects photographed in the input video selected by the video switching unit 11. The position information storage unit 14 is the position information creation unit 1
3 is a magnetic disk device, for example, which stores time information added to the position information relating to each object output from FIG.

The object pointing unit 15 is a mechanism for interpreting what the user has requested to display. The position retrieving unit 16 is a mechanism for retrieving the position of the object instructed by the object instructing unit 15 at the current time from the position information and time information stored in the position information storage unit 14. The video reproducing unit 17 determines a set of time-series data necessary for displaying the object from the position of the object obtained by the position searching unit 16, and stores the time-series data in the time-series data storage unit 1.
It is a mechanism for reading the data from No. 2 and decoding and synthesizing the images if necessary, and outputting the reproduced video. Video display unit 18
Is a display for displaying the reproduced video output from the video reproduction unit 17.

FIG. 2 is a diagram for explaining an operation using three pieces of time series data by using the apparatus having the configuration shown in FIG.
FIG. 3 is a diagram showing the positional information of the two objects photographed in the image shown in the example of FIG. 2, that is, the two fishes “Karai” and “Katsuo”.

The operation of this embodiment will be described below with reference to FIGS. 2 and 3. First, the operation of creating time-series data and position information from a captured video will be described. FIG. 2 is an image of an aquarium taken simultaneously by using three video cameras. Each image is taken with a continuous region fixed in the X-axis direction. In the video of the aquarium, multiple fish are swimming and moving within the shooting area of the fixed video camera. For this reason, movement of the fish across the video cameras frequently occurs, such as the fish protruding from the shooting area of the first video camera entering the shooting area of the second video camera. Therefore, shooting with a single video camera cannot track the target fish other than moving the video camera to follow a specific fish or changing the angle every moment. Therefore, the target object to be tracked is determined at the time of shooting, and it is not possible to select the target object to track at the stage of reproducing the video.

This problem is solved in the moving picture reproducing apparatus according to the present embodiment by storing a plurality of images simultaneously shot by a plurality of video cameras with fixed angles and switching to a necessary image for reproduction. Therefore, it is possible to perform reproduction such that the target object is tracked with respect to any of the plurality of target objects being photographed.

FIG. 2 shows a state in which three video cameras are lined up in the X-axis direction and photographed, and the time axis is taken in the depth direction. The three time-series data are continuously stored at intervals of d from the time t ₀ when the shooting is started. In a video image taken at a rate of 30 frames per second, d is 1/30 second. Such images are obtained by arranging three video cameras side-by-side and simultaneously shooting them, and digitizing them into time series data, or images shot by a very high-definition video camera or images generated by computer graphics. It can be generated by a simple operation such as dividing the area and cutting out each area as time series data.

Here, when the X coordinate is from 0 to 100,
Image taken by the first video camera, X coordinate is 100
Up to 200, the video imaged by the second video camera is displayed, and when the X coordinate is over 200 and up to 300, the video image is captured by the third video camera.
1 time series data, series # 2 time series data, series # 3
The time series data is stored in the time series data storage unit 12.

In the image of FIG. 2, for the sake of simplicity, only two fish, "Karei" and "Katsuo" are shown as objects. In real aquariums and other captured images, more objects are captured at the same time. However, even if there are multiple objects, the individual objects are tracked by the same operation as described below. be able to.

FIG. 3 shows an example in which the positional information of the two fishes “Karai” and “Katsuo” photographed in the example of FIG. 2 is created and stored in a table format. At time t ₀ , “Karei” is at a position with an X coordinate of 70 and a Y coordinate of 30. This is (70,3
When expressed as 0), the position of "bonito" is (150,5
0). At time t ₀ + d, (80, 4
0), (170, 52). In this way, position information indicating where the object being photographed is located is created in advance for each frame of each video and stored in the position information storage unit 14. Characteristically, at time t ₁ , “Karei” moves from sequence # 1 to sequence # 2, “Katsuo” moves from sequence # 2 to sequence # 3, and at time t ₂ , “Karei” moves to sequence # 2. To move to the series # 3. Therefore, than was playing the data of sequence # 1, but from time t ₀ to time t ₁ it is possible to display the "flatfish", later will not be able to display the "flatfish". Therefore, time t
_By switching the video displayed in 1 from the series # 1 to the series # 2, tracking of "Karei" is realized. Information for that purpose is held as position information.

Such position information does not have to be performed at the same time as the image input or the image is stored as time series data. That is, it is sufficient to sequentially read out the individual time-series data stored in the time-series data storage unit 12 while detecting the read data, that is, the object photographed in the image, and use that position as the data. Is. When the object is cut out mechanically, that is, when the position cannot be detected automatically, it is possible to reproduce the image using time-series data and create the position information semi-automatically by hand. . Since such position information is created only once when the system is provided, even if the position information is created manually, a support tool such as pointing the target object with the mouse while watching the video is used. It can be created sufficiently by using it.

Next, the display operation for tracking the target object using the created position information will be described. In the example of FIG. 2, it is assumed that the user instructs "Karei" to be tracked as a target. It can be seen from the position information shown in FIG. 3 that the X coordinate of “Karei” is 70 at time t ₀ , so the time series data displayed first is series # 1.
Therefore, the video reproduction unit 17 reads the time series data of the series # 1 from the time series data storage unit 12, and the JPE
If G or the like has been encoded, decoding is performed to reproduce the video data, and the video data is sent to the display which is the video display unit 18 to start the display. At the same time as the display starts, the position information storage unit 1
The operation for inquiring about the X coordinate of "Karei" with respect to 4 is repeated while advancing the time. From the results of a query, it can be seen that moved in sequence # 2 from the series # 1 in the "flatfish" the time t _1. Therefore, the data of the series # 1 is read out until the time t ₁ -d, the data of the series # 2 is read out after the time t _1, and the video data is reproduced.
Is sent to the display. Furthermore, "flatfish" at time t ₂ is because moving the series # 3, and continues the reproduction while switching the data read out as before.

When the object to be tracked is "bonito", it can be realized by selecting and switching the time-series data to be read using position information. As shown above,
In this embodiment, an operation of tracking an object moving in a wide area photographed by a plurality of video cameras while switching the cameras can be realized by using the stored time series data. In the embodiment, the case where the time-series data is created by using the video data taken by the video camera as a unit is shown. However, in this method, the switching of the playback video due to the movement of the object is performed by the video image taken by the video camera as a unit. As a result, the switching may be unnatural. For example, Figure 2
In "Karei" tracking, when switching from sequence # 1 to sequence # 2 at time t ₁ , "Karei" swam to the right on the screen, and when it reaches the right end, it suddenly appears from the left end and continues. Movement of fish is not realized.

In such a case, the photographed video is subdivided by area division, the individual subdivided videos are stored as time-series data, and at the time of reproduction, a plurality of time-series data are simultaneously read and combined. It is possible to realize continuous movement of the visual field by displaying them in the same manner.

For example, in the example of FIG. 2, each video image taken by three video cameras is divided into two at the center in the X-axis direction,
By storing as a total of 6 time-series data, it is possible to combine, for example, the right half of the image shot by the first video camera and the left half of the image shot by the second video camera. it can. By combining and displaying the images of the angles that were not originally taken, it is possible to switch to the above-mentioned combined image when "Karei" exceeds 75 in the X coordinate. As described above, by reducing the size of the area handled by one time-series data, the movement of the visual field for tracking the object can be made more continuous.

In this embodiment, the method of creating and storing the position information in frame units, which is the minimum division unit in the time axis direction, has been described, but a method of creating position information in units of a plurality of frames instead of every frame is also possible. is there. In this case, it may not be possible to accurately track an object that is moving at high speed, but even if the object that was originally moving at high speed is displayed so as to be tracked, the human Visual cannot follow. Therefore, the method of creating location information at intervals of multiple frames does not reduce the convenience of the user,
It can be said that this is an effective method for reducing the amount of location information data and the amount of processing required for location retrieval. The interval for creating the position information may be fixed depending on the amount of data that can be stored and the time required for retrieval, or depending on the moving speed of the object, for example, when the object is moving at high speed. Can be set to a short interval, and can be set coarsely when moving at low speed.

Regarding the storage of the position information, the information necessary for selecting the time-series data at the time of reproduction, that is, the time-series data read out in accordance with the movement of the target object is stored only at the time. Even in this case, the operation can be realized in the same manner as shown in the embodiment. Although it has been stated that human vision cannot follow a moving object at high speed, in such a case, an upper limit is set for the moving speed of the displayed image, and the object moves at a speed exceeding the upper limit. When doing so, it is possible to take a method of relatively slowing down the moving speed of the object on the screen by performing so-called panning in which the display range is widened. Further, as another method for coping with such a case, there is a method in which the position of the object is always tracked but the display area for always displaying the object is not tracked. In this method, the movement of the display area does not exactly match the movement of the object. In this method, when the object moves abruptly, there is a moment when the object goes out of the display area, that is, there is a period in which it is not displayed. However, when the movement of the object becomes slow, it is possible to realize control such that the object enters the display area.

In this embodiment, an example in which the position information is stored as X and Y coordinates has been shown, but it may be easier to select the time series data by using polar coordinates depending on the arrangement of the video camera. In short, the position information only needs to have enough information to select the time-series data at the time of video reproduction, and which time-series data is suitable for displaying the target object at a desired position at a certain time. A bitmap that represents is sufficient. Further, although the method of creating and storing the position information in advance has been described in the present embodiment, the position of the target object can be obtained in the process of reproducing the video data. In this case, since the time that can be spent to obtain the position is limited, intermediate data that conforms to the position information is created in advance, and the position is obtained using the intermediate data, so that processing in the reproduction process is performed. It is also possible to take a method of reducing. By adopting such a method, it is possible to reduce the memory / disk capacity required to store the position information created in advance.

In the above description, the size occupied by the object in the image is not mentioned, but by storing information on the size in addition to the position information, a plurality of "karei" are swimming. It is also possible to display "Karei," which is the largest copy at that time, by tracking "Karei" on the image in the aquarium, and to track the result. Further, when the user designates a plurality of objects, it is possible to easily realize a combination application such as selecting and displaying an object which is a large copy at that time.

[0029]

According to the present invention, a moving image reproducing apparatus for reading m (m ≧ 1) time-series data from n (n ≧ 2) time-series data to synthesize and display a moving image is photographed. By providing position information about multiple target objects, the video can be played back to track the movement of the target object specified by the user, and the video that highly satisfies the requirements of each user. The playback device can be easily realized.

[Brief description of drawings]

FIG. 1 is a diagram showing a configuration example of a moving image reproducing apparatus of the present invention.

FIG. 2 is a diagram illustrating an operation using three pieces of time series data in the configuration shown in FIG.

FIG. 3 is a diagram showing position information of two objects photographed in the image shown in the example of FIG.

[Explanation of symbols]

 10-1, 10-2, 10-3 Video input section 11 Video switching section 12 Time-series data storage section 13 Position information creation section 14 Position information storage section 15 Object pointing section 16 Position search section 17 Video playback section 18 Video display Department

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Seiichi Kontani 1-1-6 Uchisaiwaicho, Chiyoda-ku, Tokyo Nihon Telegraph and Telephone Corporation

Claims (1)

[Claims] 1. A time-series data storage means for storing at least n (n ≧ 2) time-series data, and the stored n.
In a moving picture reproducing apparatus provided with a moving picture display means for reading out m (m ≧ 1) time series data from book time series data and synthesizing and displaying a moving picture, each individual moving picture in the entire region to be reproduced is displayed. For a target object, position information storage means for detecting and storing position information in which the target object is located in the entire area, and n pieces of moving images forming the entire area are respectively stored when the moving image is stored. Control for storing as series data using the time-series data storage means and storing movement history of one or more objects included in the entire area as time-series data of position information using the position-information storage means And the means for performing the playback of the designated object,
The time-series data including the instructed object and the position in the time-series data are searched by using the position information storage means, and the instructed object is displayed at a predetermined desired position. Thus, the moving picture reproducing apparatus is provided with a means for controlling to display the moving picture display means by selecting m time-series data.