JPH10187760A - Image display device and moving image retrieval system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To make multi-picture syncronous reproduction handlable as the reproduction of one moving image data, and to reduce the consumption of network resources and system resources by reducing, synthesizing and reproducing moving image data to be reproduced for each same frame of respective moving images. SOLUTION: The reduction scale of image data is calculated from the number of image data to be displayed in a list and the image data are reduced by an image reducing means 101, composited by an image compositing means 102 and displayed by an image display means 103. When the reproducing request of all the correspondent moving image data in the composited image is acquired, the maximum number of frames of moving image data is acquired and just for that number of frames, all the moving images are reduced in the calculated reduction scale by the image reducing means 101, composited by an image composting means 102 and displayed by the image display means 103. In that case, concerning the images less than the number of frames, even after the final frame of these images is reproduced, that final frame is continuously reproduced so that multi-picture synchronous reproduction can be handled as the reproduction of one piece of moving image data.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving picture search on a computer, and more particularly to a moving picture search using multi-screen reproduction.

[0002]

2. Description of the Related Art Conventionally, multi-screen simultaneous reproduction is generally disclosed in, for example, Japanese Patent Application Laid-Open No. 5-19731. The multi-screen simultaneous playback method is determined by the decoding target frame identifying means 3301 for identifying the level of decoding based on the number of data of moving images to be simultaneously displayed and reproduced, as shown in FIG. 33, and the decoding target frame identifying means. An image display device comprising: a target frame decoding means 3302 for decoding video data at a decoding level; and a moving picture reproduction means 3303 for reproducing each decoded video data.

Conventionally, a moving image search method is disclosed in
No. 113790. As shown in FIG. 34, the retrieval method is a representative scene extraction means 3401 for extracting a portion of moving image information representing the feature of the retrieval image as a scene.
Menu image generating means 3402 which edits this scene as a menu image by arranging the scene on the time axis, and menu image reproducing means 3403 which reproduces the menu image for searching. This is a moving image search method characterized by searching for a video.

Further, as a conventional method for selecting data according to the purpose from stored moving image data, there is a moving image search system disclosed in Japanese Patent Application Laid-Open No. Hei 5-81348. FIG. 35 is a block diagram showing a configuration of a moving image search system disclosed in Japanese Patent Application Laid-Open No. 5-81348. In the reduced image creating unit 3502, image data for moving image display is created from all or part of image data of moving image data, and for each of the plurality of moving image data, each moving image data is represented. A reduced still image to be created is created from a part of each moving image data. The still image display 3504 displays the representative images of the moving image data side by side on the screen of the bitmap display 3506. When the representative image on the bitmap display is selected by the screen selecting means 3507, the moving image display image data of the moving image data represented by the selected representative image by the moving image display 3505 is sequentially displayed on the bitmap display 3506. This is a system characterized by displaying continuously.

[0005]

However, in the prior art, in general, when performing multi-screen simultaneous reproduction via a network, a network bandwidth is secured by the amount of reproduced moving image data, and the multi-screen simultaneous When performing playback, system resources for the number of playback moving images are consumed. An object of the present invention is to make effective use of network bandwidth and system resources in such simultaneous playback on multiple screens.

Further, in general, when searching for moving image data, a set of videos characteristic of a moving image is set as a search target. Therefore, similar videos are searched for, and the search efficiency is poor. Processing time is also required. An object of the present invention is to improve search processing speed and prevent omission of detection when moving image data is searched using a still image as a key.

Further, when search conditions are designated from a plurality of terminals and a plurality of different moving images are simultaneously reproduced for each terminal, the load on the server becomes heavy in proportion to the number of terminals in addition to the load on the terminals and the network. In other words, there is a problem that it is difficult to reproduce smooth moving images on a plurality of terminals. An object of the present invention is to provide a moving image search system that reproduces, as smooth moving image data, a list of moving image data that matches search conditions specified by each terminal in a plurality of terminals.

Further, in the conventional method, in order to find a target scene from moving image data to be searched, it is necessary to reproduce and check the entire moving image display image data created in advance, which is very difficult to search. There is also a problem that it takes time and effort. SUMMARY OF THE INVENTION It is an object of the present invention to provide a moving image search system capable of quickly and easily searching for a desired scene from moving image data.

[0009]

In order to solve the above-mentioned problems, the present invention firstly reduces a plurality of pieces of moving image data to be reproduced for each frame and combines them into one moving image. Is intended to be reproduced. As a result, when a plurality of moving image data are simultaneously reproduced on a multi-screen, the number of data streams to be transmitted is reduced, so that the network bandwidth can be effectively used and the consumption of system resources of the reproducing computer terminal is reduced. It has the effect of doing.

Second, when searching for moving image data using a key image, instead of comparing all frames in the key image and moving image data, each scene of the key image and moving image data is not compared. The purpose is to improve the search processing speed by comparing with a representative frame. Thus, when a search is made for moving image data using a still image as a key, the number of frame data to be searched is reduced, so that there is an effect that the search processing speed is improved.

Thirdly, when searching for moving image data using a key image, not only is the key image compared with the representative frame of each scene of the moving image data, but also the representative of the scene matched by the comparison result is obtained. An object of the present invention is to prevent a search from being missed by registering a frame as a new key image and further performing a search using the newly registered key image.
Accordingly, when a search is performed on a moving image data using a still image as a key, the number of still images serving as search keys increases, thereby providing an effect of preventing omission of search.

In a fourth aspect of the present invention, a moving image search server extracts a plurality of moving image data matching a search condition, and an image reducing unit creates a reduced image from moving image data extracted from a moving image storage unit. Then, the reduced image is synthesized by the image synthesizing means, and the synthesized image is stored in the synthetic image storage means for storing the synthesized image. Further, the data stored in the composite image storage means is transmitted to the terminal using the composite image transmission control means, and the received composite image is displayed on the moving image search terminal side. As a result, even when a plurality of moving image data matching the search conditions are displayed on the screen at the same time, the moving image search server receives and reproduces the reduced and combined moving image data via the network. The load on the resources and the network can be reduced to one moving image data.

Fifth, the present invention calculates the time required for generating a composite image, and adjusts the frame interval of image data to be composited according to the load on the server. There is an effect that a moving image having no image can be transmitted to the moving image search terminal.

In the sixth aspect of the present invention, the composite image transmission control means of the moving image retrieval server compares the frame number of the generated composite image with the frame number of the composite image transmitted to the moving image retrieval terminal and transmits the composite image. If it catches up with composition, it repeats sending the composite moving image generated with the sending frame number at the beginning, and then stores the composite moving image data until the transmission catches up with the generation, thereby searching the moving image. The effect that the moving image data on the terminal can be reproduced without delay is obtained.

[0017] Seventh, the present invention calculates the time required to generate a composite image of the scheduled reproduction time, and when the time required to generate the remaining composite image becomes shorter than the scheduled reproduction time, The image transmission is started.
As a result, there is an effect that the reproduction of a plurality of pieces of moving image data that match the search condition can be started with a short waiting time without waiting for the completion of the generation of the combined moving image data of the scheduled reproduction time.

Eighth, the present invention reduces the image data extracted by shifting the reproduction start time of any specified moving image data, generates a synthesized image arranged in time lapse, and searches for a moving image. The composite image is displayed on the terminal. As a result, the selected moving image data can be searched more finely, and an effect that a target scene can be quickly found can be obtained.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below with reference to FIGS. It should be noted that the present invention is not limited to these embodiments and may be implemented in various modes without departing from the scope of the present invention.

(Embodiment 1) First, a first embodiment corresponding to claim 1 will be described. FIG. 1 is an example of a configuration diagram showing an image display device according to the present embodiment. In FIG. 1, reference numeral 101 denotes an image reducing unit that reduces image data, and 102 combines images reduced by the image reducing unit 101. 103 is an image display means for displaying images synthesized by the image synthesis means 102, and 104 is an image synthesis means for synthesizing all the images displayed by the image display means 103 on a frame-by-frame basis. A multi-screen simultaneous reproduction means for reducing by means 101, synthesizing each frame by an image synthesizing means 102, and simultaneously reproducing the synthesized image is shown.

FIGS. 2A and 2B are flowcharts showing the flow of processing of the image display apparatus. The operation will be described below in conjunction with FIG. In FIG. 2A and FIG. 2B, first, the number of image data to be displayed in a list is obtained (201), the reduction ratio of the image data is calculated from the obtained number of image data (202), and the number of start steps (i) is calculated. 0 (203), until the number of start steps (i) becomes equal to the number of image data to be displayed in a list (204), the image data is reduced by the image reduction means 101 at the reduction ratio calculated in step 202 (205), and the process is started. Increase the number of steps by 1 (20
6). When the number of start steps (i) becomes equal to the number of image data to be displayed in a list (204), all the reduced image data are synthesized by the image synthesizing means 102 (207) and synthesized by the image synthesizing means 102 by the image display means 103. The displayed image is displayed (208).

When a reproduction request of all the moving image data corresponding to the composite image is obtained from the user (209, 210),
The frame number (m) of the moving image data having the largest number of frames among all the moving image data to be reproduced is obtained (211), the starting step number (i) is set to 0 (212), and the starting step number (i ) Until the number of frames becomes equal to the number of frames of the maximum moving image data (213), it is checked whether there is moving image data in which the number of frames is less than the start step number (i) in all the moving image data ( 214), the final frame of the moving image data whose number of frames is less than the starting step number (i) is set as the starting step number (i) frame (215), and the starting step number of all moving image data by the image reducing means 101 ( (i) The frame number is reduced at the reduction rate calculated in step 202 (216), the start step number of all moving image data reduced by the image synthesizing means (i) the image of the frame is synthesized (217), and Synchronize the composite image with the composite image of the start step number-1 (i-1) th frame, and And displayed by the display unit 103 (218), the start step number (i) 1
Increase (219).

Returning to step 213, the process is repeated until i = m. In the case of No at step 210, the process is terminated.

As described above, in the present embodiment,
When all the synthesized images are played back at the same time, even if the images have different numbers of frames, even if the images have a small number of frames, the last frame of the image continues to be played even after the last frame of the image is played, so that multi-screen simultaneous playback can be performed. Since it can be handled as one moving image data reproduction, the number of streams can be reduced when reproducing a plurality of moving image data via the network, and system resource consumption can be suppressed in the same terminal, so system resources can be used effectively. It can be used, and its practical effect is great.

(Second Embodiment) Next, a second embodiment corresponding to claim 3 will be described. FIG. 3 is an example of a block diagram showing the image display device according to the present embodiment. In FIG. 3, 301 is an image reducing means for reducing image data, and 302 is a combination of images reduced by the image reducing means 301. Image display means for displaying the image synthesized by the image synthesis means 302, 304 for reproduction image selection means for selecting which image in the image displayed by the image display means 303 is to be reproduced, 305 Is a playback image selection means 1
5 shows a selected image simultaneous reproduction means for simultaneously reproducing the image designated by 5.

FIG. 4 is a flowchart showing the flow of the processing of the selected image reproducing means. The operation will be described below in conjunction with FIGS. 3 and 2-a. In FIG. 2A, the same as in the first embodiment, and in FIG. 2A and FIG. 4, when the user request is a selection screen reproduction request (209, 401), the number of start steps (i) is set to 0.
(402) obtaining information on which moving image data should be reproduced among the moving image data corresponding to the composite image (403), and setting a reproduction range to a rectangular shape based on the information obtained in step 403; (404), and the number of frames of the moving image data having the largest number of frames among the moving image data to be reproduced.
(m) is acquired (405), and the number of start steps is equal to (40) until the number of frames is equal to the number of frames of the maximum moving image data.
6), the number of frames in the playback video is the number of start steps (i)
(407), the last frame of the moving image data whose number of frames is less than the start step number (i) is set as the start step number (i) frame (408), and Check if there is moving image data for which playback is not
09), for the moving image data that is not reproduced, the data of the initially displayed frame is set as the starting step number (i) (410), and the image reducing unit 301 sets the starting step number (i) of the entire moving image data to the (i) th frame. Is reduced by the reduction ratio calculated in step 202 (411), and the images of the start step number (i) frame of all the moving image data reduced by the image combining means 302 are combined.
(412), the synthesized image is synchronized with the synthesized image of the start step number -1 (i-1) frame, displayed by the image display means 303 (413), and the start step number (i) is increased by 1 ( 414), when the number of start steps (i) becomes equal to the number of frames of the moving image data having the maximum number of frames (406), the process ends. In the case of No at step 410, the process is terminated.

As described above, in the present embodiment, the reduction and
When all the images displayed in the composite are displayed, a rectangular image is selected on the screen so as to include all the images to be reproduced at the same time, and when the selected image is reproduced, the first frame is reproduced for the non-selected image. Continuing, even for selected images, for an image having a small number of frames, the last frame of the image is continuously reproduced even after the last frame of the image is reproduced. Can be selected, the amount of useless data can be further suppressed in addition to the effect of the first embodiment, and the practical effect is large.

(Third Embodiment) Next, a third embodiment corresponding to claim 2 will be described. FIG. 5 is an example showing an internal configuration of the image reducing means 101 according to the first embodiment. In FIG. 5, 501 is a macroblocking means for dividing the image data into arbitrary pixel units, and 502 is a macroblocking means. Orthogonal transformation means for performing orthogonal transformation (for example, DCT transformation, etc.) for each macroblock divided by 501; 503, a pixel extraction means for performing the orthogonal transformation means 502 to extract the first low-frequency term; The pixel synthesizing means for synthesizing the pixels extracted by 503 is shown.

FIG. 6 is a flowchart showing the flow of the processing of the image reducing means. The operation will be described below in conjunction with FIG. In FIG. 6, first, the macroblock forming means 501
Divides the image data into macroblocks according to the reduction ratio (601), sets the number of start steps (i) to 0 (602), and if the number of start steps (i) is smaller than the number of macroblocks divided in step 601, (603), orthogonal transformation of the macroblock is performed by the orthogonal transformation means 502 (604), and the pixel of the first low-frequency term after the orthogonal transformation in step 604 is extracted by the pixel extraction means 503 (605), and the number of start steps ( i) is incremented by 1 (606), and when the number of start steps (i) becomes equal to the number of macroblocks divided in step 601 (603), the pixels extracted by the pixel combining means 504 are combined (607) and reduced. Generate an image.

As described above, in the present embodiment, an arbitrary reduced image can be generated by changing the number of divisions of the macroblock, so that any arbitrary reduced image can be easily generated, and The effect is great.

(Embodiment 4) Next, a fourth embodiment of the present invention will be described. FIG. 7 is an example of a configuration diagram showing an image display device according to the present embodiment. In FIG. 7, reference numeral 701 denotes a key image input unit for inputting a key image for performing a search for a moving image, and 702 denotes a moving image. A moving image search means for comparing data with a key image for search; 703, an image reduction means for reducing image data of a search result; 704, an image synthesis means for combining images reduced by the image reduction means 703; An image display means for displaying an image synthesized by the synthesis means 704 is shown.

FIG. 8 is a flowchart showing the flow of processing of the image display apparatus. The operation will be described below in conjunction with FIG. In FIG. 8, first, a key image as a search condition is input by the key image input means 701 (201), and moving image data is searched based on the input key image (802).
If there is a matched moving image by 02 (803), the representative frame of the moving image data of the matched moving image data is reduced by the image reducing means 703 (804), and all the reduced representative frames by the image synthesizing means 704 are combined. Then, the image display means 705 displays the image synthesized by the image synthesis means 704 (806).

As described above, in the present embodiment, the search results are displayed as the list data of the still images, so that the search results can be narrowed down easily, and the practical effect is large.

(Embodiment 5) Next, a fifth embodiment of the present invention will be described. FIG. 9 is an example showing the internal configuration of the moving image search means 702 according to the fourth embodiment. In FIG. 9, reference numeral 901 denotes a cut point detecting means for detecting a cut point serving as a scene change from moving image data; Is a frame extracting means for extracting a representative frame of each scene in the moving image data, 903 is a key image inputting means for inputting a key image as a search condition, 904 is a comparison between the key image and a representative frame of each moving image data. An image comparison / determination means for outputting a representative frame of moving image data that has been matched is shown.

FIG. 10 is a flowchart showing the processing flow of the moving image search means 701 of FIG. 7, and in conjunction with FIG. 8 and FIG.
The operation will be described below. In FIG. 10, first, the cut point detection unit 901 detects the cut point of the moving image data to be searched (1001), the frame extraction unit 902 extracts the representative frame in the scene calculated from the cut point (1002), and the key The key image is input by the image input means 903 (1003), and the representative frame extracted by the frame extracting means is searched based on the key image input in step 1003 (100
4) Thereafter, the same processing as in step 803 and thereafter in FIG. 8 is performed.

As described above, in the present embodiment, by comparing the key image with the representative frame of each scene of the moving image data, it is not necessary to compare the key image with all the frames of the moving image data. The search processing speed is improved, and the practical effect is great.

(Embodiment 6) Next, a sixth embodiment corresponding to claims 6 and 7 will be described. FIG.
FIG. 11 shows another example of the internal structure of the moving image search means in the fifth embodiment. In FIG. 11, reference numeral 1101 denotes a cut point detecting means for detecting a cut point serving as a scene change from the moving image data; Frame extracting means for extracting a representative frame of each scene, 1103 a key image input means for inputting a key image as a search condition, and 1104 comparing a key image with a representative frame of each scene of each moving image data to obtain a match. An image comparison / determination unit that outputs a representative frame of the moving image data, and a key image registration unit 1105 that registers the compared and matched frame as a key image.

FIG. 12 is a flowchart showing the processing flow of the moving image search means 701 in FIG. 7, and the operation thereof will be described below in conjunction with FIGS. In FIG. 12, first, the cut point of the moving image data to be searched is detected by the cut point detecting means 1101 (1201), and the representative frame in the scene calculated from the cut point is extracted by the frame extracting means 1102 (1201).
2) Input a key image by key image input means 1103 (120
3), searching for a representative frame of each scene of the moving image extracted by the frame extracting means 1102 based on the key image input in step 1203 (1204), and if there is a matched representative frame (1205) ), All the representative frames matched by the key image registration means 1105 are registered as key images (1).
206), the representative frame is searched again using the newly registered key image (1207), and the same processing as in step 803 and thereafter in FIG. 8 is performed.

As described above, in the present embodiment, by further providing the key image registration means for registering the coincident frame as the key image, the number of images serving as the search key is increased, so that the omission of the search can be prevented. , Its practical effect is great.

(Seventh Embodiment) Next, a seventh embodiment corresponding to claim 8 will be described. FIG. 13 shows another example of the internal structure of the image search means according to the sixth embodiment. In FIG. 13, reference numeral 1301 denotes a cut point detection means for detecting a cut point serving as a scene change from moving image data;
02 is a frame extracting means for extracting a representative frame of each scene in the moving image data, 1303 is a key image inputting means for inputting a key image which is a search condition, and 1304 compares a key image with a representative frame of each moving image data Image comparison and determination means for outputting matched moving image data, 1305 is an image similarity output means for outputting a frame most similar to the key image from all frames in the scene including the frame matched by the image comparison and determination means 1304 , 1306 is an image similarity output means
The key image registration means for registering the one having the highest similarity output by 1305 as a key image is shown.

FIG. 14 is a flowchart showing the flow of processing of the moving image search means. The operation will be described below with reference to FIGS. In FIG. 14, first, the cut point of the moving image data to be searched is detected by the cut point detecting means 1301.
(1401), extracting a representative frame in the scene calculated from the cut point by the frame extracting means 1302 (1402), inputting a key image by the key image inputting means 1303 (1403), and
The frame extracting means 13 based on the key image input in 1403
Search for the representative frame extracted in 02 (140
4) If there is a matched representative frame (1405), all frames of the scene including the matched representative frame are extracted by the frame extracting means 1302 (1406), and the image similarity output means
The key image and all the extracted frames are compared by 1305, the frame having the highest similarity is registered as the key image by the key image registration unit 1306 (1407), and the representative frame is searched again by the newly registered key image ( 1408), and thereafter, the same processing as in step 803 and thereafter in FIG. 8 is performed.

As described above, in this embodiment, there is further provided image similarity output means for outputting a frame most similar to a key image from all frames in a scene including a frame matched by the image comparison / determination means. Allows the new search key image to be similar to the first key image, improving the accuracy of the search results,
Its practical effect is great.

(Embodiment 8) Next, claims 9 and 10
The eighth embodiment corresponding to is described. Figure 15
Is an example of a configuration diagram showing a moving image search system according to the present embodiment, in which a moving image search server 1500 and a moving image search terminal 1520 are connected via a network 1530.

The moving image search server 1500 searches for moving image data that matches the search conditions.
1. Moving image storage means 15 for storing moving image data to be searched
02, a composite image generation control unit 1503 that controls a series of processes for generating a composite image, an image extraction unit 1504 that extracts an image for each frame from the video data that matches the search conditions in the video search unit 1501, Image reducing means 1505 for arranging reduced images, image combining means 1506 for regularly arranging reduced images and combining them on one screen, combined image storing means 1507 for storing combined images, frame numbers and identifiers of terminals that have made search requests A composite image transmission control unit 1508 for outputting information of a composite image created, and a communication unit 1509 for controlling communication with a terminal.

On the other hand, a moving image search terminal 1520 displays search condition input means 1521 for the user to input search conditions such as keywords, communication means 1522 for controlling communication with the moving image search server 1500, and displays composite image data of search results. And a composite image reproducing unit 1523 for performing the processing.

FIG. 16 is a flowchart showing the flow of the multi-moving image synthesizing process in the moving image search server 1500 of the moving image search system. The operation will be described below with reference to FIG.

First, in the moving image search means 1501,
From the moving image data stored in the moving image storage unit 1502,
Search for video data that matches the search condition (160
1). Further, the number of frames of the data having the longest time among the moving image data matching the search result is obtained and output to the composite image generation control means 1503 (1602). Further, the read frame number (i) is initialized to 0, and the identifier of the moving image data that matches the search condition and the start of generation of the composite image are output to the image extracting means (1603).

Next, the image extracting unit 1504 extracts the i-th frame image of the read frame number from each of the moving image data that matches the search condition obtained by the moving image searching unit 1501 (1604).

Next, the image reducing means 1505 reduces the i-th frame image data of the moving image data extracted by the image extracting means 1504 at an arbitrary predetermined reduction ratio (1605). This reduction rate is, for example, 4 ×
In the case of combining four sides, each side is reduced to に. In addition,
Here, for the sake of explanation, the case where the image is reduced to 1/4 has been described. However, even if the image is reduced vertically and horizontally at an arbitrary reduction ratio according to the number of moving image data to be displayed at the same time, other operations are not affected. It is clear.

Further, in the image synthesizing means 1506,
The reduced images of the i-th frame reduced by the image reducing unit 1505 are arranged so as not to overlap, and are combined into one image (1606). FIG. 17 is a diagram illustrating an example of screen composition in the image composition unit 1506. Here, the description has been made using the case where the reduced images are arranged in the order shown in FIG. 17. However, if the reduced images are arranged so that they do not overlap, they can be arranged vertically or regularly using another method. I do not care.

Further, the image synthesizing means 1506 stores the generated synthesized image of the i-th frame in the synthesized image storage means 1507.
To the composite image storage means 15 (1607).
The storage destination such as the file name and address of the composite moving image written in 07, the frame number (i), and the identifier of the terminal that has made the search request are output to the composite image transmission control means 1508 (1608).

Further, the read frame number (i) is set to 1
And the process proceeds to the next frame (1
609). Steps 1604 to 1609 are repeatedly performed while the read frame number (i) is smaller than the number of frames of the longest moving image data among the moving image data matching the search condition obtained by the moving image search unit 1501.
Synthesized image data is generated by reducing and synthesizing the moving image data that matches the search condition (1610-Y).

When the read frame number (i) becomes larger than the number of frames of the longest moving image data, the creation of the composite image data is terminated (1610-N).

As described above, a single moving image is generated by combining a plurality of moving image data that match the search conditions for each terminal, and an identifier such as an address or a file name to be stored for each terminal that has made a search request is added. Then, it is stored in the composite image storage unit 1507 of the moving image search server 1500.

Next, the flow of processing relating to the transmission control of the composite image to the terminal that has input the search request in the composite image transmission control means 1508 will be explained using FIG.

The composite image transmission control means 1508 is a composite image storage means 1 output from the image composition means 1506.
The identifier representing the storage destination such as the address or file name of the composite moving image written in 507, the frame number (i) of the created composite image, and the identifier of the terminal that made the search request are input (1801). When the frame number (i) of the created composite image becomes larger than an arbitrary value (1802)
-Y), the transmission frame number (j) is initialized to 0 (18)
03), transmission of the composite image to the terminal that has made the search request is started.

Next, the composite image transmission control means 1508
The image corresponding to the transmission frame number (j) is extracted from the composite image stored in the composite image storage unit 1507 (180).
4) The extracted composite image data and the identifier of the terminal that has issued the search request are output to the communication unit 1509 (180).
5). Further, the transmission frame number (j) is incremented by one, and the process proceeds to the next frame (1806).
Steps 1804 to 1801 until the transmission frame number (j) exceeds the frame number (i) of the created composite image.
6 is repeatedly executed (1807-Y).

The communication unit 1509 transmits the combined moving image data to the moving image search terminal 1520 corresponding to the terminal identifier output from the combined image transmission control unit 1508.

As described above, under the control of the composite image transmission control means 1508, the composite moving image data is transmitted to the moving image search terminal 1520 to which the search condition distinguished by the identifier is inputted.

On the other hand, in the moving image search terminal 1520, the communication means 1522 receives the synthesized moving image data transmitted from the moving image search server 1500, and the synthesized image reproducing means 1523 displays the received synthesized moving image data.

Incidentally, as in the tenth aspect of the present invention,
In addition to the above-described configuration, a configuration in which the combined image as a search result is further compressed and transmitted, and then expanded and displayed on the receiving side can be naturally considered. At that time, the method of compression / decompression is not particularly limited.

As described above, in the present embodiment, when a plurality of moving images to be searched are simultaneously displayed, the moving image search server 1500 creates one moving image by reducing and combining a plurality of moving image data. Since the moving image search terminal 1510 reproduces only one synthesized moving image, the amount of data flowing through the network and the amount of data to be processed on the terminal side need only be one synthesized moving image. Therefore, the load on the network and the terminals can be significantly reduced. Further, in order to simultaneously communicate a plurality of moving image data, the moving image search server 15 via a network having a low communication speed.
00 is connected to the video search terminal 1520,
Even when the processing capability of the moving image search terminal 1520 is insufficient to simultaneously reproduce a plurality of moving image data, it is possible to simultaneously reproduce a plurality of moving image data as a search result on one terminal, If transmission and reception are performed after compressing the synthesized image, the communication load can be greatly reduced, and the practical effect is large.

(Embodiment 9) Next, a ninth embodiment corresponding to claim 11 will be described. FIG. 19 shows an example of a configuration diagram of a moving image search system according to the present embodiment. In addition to the configuration of the eighth embodiment, the moving image search server 1900 includes a frame interval setting unit 1909 for setting an interval between frames to be extracted from moving image data stored in the moving image storage unit 1902.

FIG. 20 is a flowchart for explaining the flow of the moving image synthesis processing in the moving image search server 1900 in the present embodiment.

First, in the moving image search means 1901,
From the video data stored in the video storage unit 1902,
Search for video data that matches the search conditions (200
1). Further, the number of frames of the data having the longest time among the moving image data matching the search result is obtained and output to the composite image generation control means 1903 (2002). Furthermore, the read frame number (i) is initialized to 0, and the identifier of the moving image data that matches the search condition and the start of generation of the composite image are output to the image extracting unit 1904 (2003). next,
In the image extraction means 1904, the moving image search means 190
The image of the i-th frame of the read frame number is extracted for each of the moving image data that matches the search condition obtained in 1 (2004).

The image reduction means 1905 reduces the image data of the i-th frame of the moving image data extracted by the image extraction means 1904 at an arbitrary reduction ratio set in advance (2005).

Next, the image combining means 1906 arranges the reduced images of the i-th frame reduced by the image reducing means 1905 so that they do not overlap, and combines them into one image (2006). Further, the image combining means 1906 stores the generated combined image of the i-th frame in the combined image storing means 190.
7 (2007), and further, the combined image storage means 19
The storage destination such as the file name and address of the composite moving image written in 07, the frame number (i), and the identifier of the terminal that has made the search request are output to the composite image transmission control means 1908 (2008). Also, the frame number (i) is output to the frame interval control means 1909 (2009).

The frame interval control means 1909 stores the frame number output from the image synthesizing means 1906 and the time at which the frame number was received in pairs, and stores the frame number (i) and From the time of the frame number (i-n), the time required to create a composite image of an arbitrary number of frames (n) is calculated (2
010). Further, the time required to create one composite image is obtained, and the frame skipping interval (r) is obtained from the number of frames per second of the original moving image data (20).
11).

In this embodiment, every time one composite image is created, the time when the synthesis of the frame with the frame number and the previous frame number is completed is recorded to calculate the time required for the synthesis image creation. Steps 2010 and 20 are performed at intervals at which generation of two or more arbitrary number of composite images is completed.
11 may be performed to calculate and change the frame interval.

Next, the composite image generation control means 1903 sets the read frame number (i) to the frame interval (r).
And the process proceeds to the next frame (2
012). If the read frame number (i) is equal to or less than the number of frames of the longest moving image, steps 2004 to 201
2 is repeated to generate composite image data in which the moving image data that matches the search condition is reduced and synthesized (2013-
Y).

If the read frame number (i) is larger than the longest moving image data frame number, the creation of the composite image data is terminated (2013-N).

As described above, in the present embodiment, the frame interval is adjusted appropriately when the load on the server is increased due to the search requests from a plurality of terminals or when the image synthesizing process in the server is delayed. While composing a moving image,
By transmitting the created moving image, the moving image search server side can play the moving image in real time frequently on the moving image search terminal side without interruption, and its practical effect is great.

(Embodiment 10) Next, a tenth embodiment corresponding to claim 12 will be described. FIG.
3 is a diagram showing an example of a configuration of a synthetic image transmission control unit in the moving image search server of the present embodiment. FIG. 39 shows an example of a block diagram for explaining in more detail the configuration of the synthesized image transmission control means 1508 in the moving image search server 1500 of the moving image search system described in the eighth embodiment. 21, reference numeral 2101 denotes an image synthesizing unit 15.
A write frame storage means 210 for storing a write frame number (i) output from 06, a transmission start control means 2104 for controlling start and end of transmission of the created composite image to the terminal, and 2103 a transmission frame number (j) The transmission frame storage means 2104 compares the writing frame number (i) stored in the writing frame storage means 2101 with the transmission frame number (j) stored in the transmission frame storage means 2103, and sets the next transmission frame number. The frame comparison unit 2105 reads the composite image of the transmission frame number (j) set by the frame comparison unit 2104 from the composite image storage unit 1507 and stores the read composite image of the transmission frame number (j) in the communication unit 15.
09 is a composite image transmission means for transmitting the composite image to the image data 09.

FIG. 22 is a flowchart showing the flow of the composite image transmission control processing by the composite image transmission control means 1508 in the present embodiment. The operation will be described below in conjunction with FIG.

First, the write frame storage means 2101
Stores the writing frame number (i) of the synthesized image output from the image synthesizing unit 1506. The transmission start control means 2102, when the previous writing frame number (i) becomes larger than an arbitrary frame number set in advance,
The first frame is set as the transmission frame number (j), and the transmission of the composite image is instructed to the frame comparison means 2104 (2201).

Next, the frame comparing means compares the sending frame number (j) with the writing frame number (i). If the sending frame number (j) is smaller than the writing frame number (i) (2002-Y), Then, j + 1 is set as the next transmission frame number, and the transmission frame number is output to the composite image transmission means 2105 and the transmission frame storage means 2103 (2003).

Conversely, when the transmission frame number (j) is larger than the write frame number (i) (2002-
N) sets the first frame as the next transmission frame number, and outputs the transmission frame number to the composite image transmission means 2105 and the transmission frame storage means 2103 (2004).

Here, the case where all frames are transmitted has been described as an example. However, when transmitting a composite image every n frames, j + n + 1 is set as the next transmission frame number. Also, the next transmission frame number (j + n + 1)
Is larger than the write frame number (i), the leading frame number 0 is set as the next transmission frame.

Next, the composite image transmitting means 2105 acquires the composite image of the transmission frame number set by the frame comparing means 2104 from the composite image storage means 1507, and outputs the acquired composite image to the communication means 1509 (220).
5). The processing of the above steps 2202 to 2205
This is performed until the transmission start control unit 2102 notifies the frame comparison unit 2104 of the request to stop transmitting the composite image (2106-Y).

The method of stopping the transmission of the synthesized image includes a method of completing the generation of the synthesized image having the longest data length among the original data of the synthesized moving image data and the writing frame number (i). A transmission start control unit 2102 determines whether the transmission frame numbers (j) have become equal, or a method of receiving a request to stop transmission of a composite image from the moving image search terminal via the communication unit 1509.
It is apparent that the processing of steps 2202 to 2205 in the present embodiment is not affected even if the transmission of the composite image is stopped by any method.

FIG. 23 is a diagram showing how the combined moving image data is reproduced in the present embodiment. As shown in FIG. 23, when the reproduction up to the frame number for which the combined moving image data has been completed has been completed, the reproduction is performed again from the first frame to make maximum use of the combined image being created. Then, the combined moving image data is repeatedly reproduced.

As described above, in the present embodiment, the composite image sending control means compares the write frame with the send frame, and when the send frame number becomes equal to or larger than the write frame number, Is set to 0 and the reproduction of the moving image from the beginning is repeated. In this way, by making maximum use of the moving image being composited, by repeatedly reproducing the composite image, the same scene is repeatedly reproduced, and the newly generated composite image is added little by little. This has the effect of making it easier to find the desired scene. Furthermore, since the transmission of the composite image data and the reproduction on the retrieval terminal can be started before all the generation of the composite moving image data is completed, the waiting until the reproduction of a plurality of moving image data that matches the retrieval condition is started. The time is short and the effect is great.

(Eleventh Embodiment) Next, an eleventh embodiment corresponding to claim 13 will be described. FIG.
4 is a diagram showing an example of a configuration of a composite image transmission control unit 1508 in the moving image search server 1500 according to the present embodiment. Composite image image transmission control means 15 of the eighth embodiment
In 08, 2401 is a writing frame storage means for storing the writing frame number (i) output from the image synthesizing means 1506, 2402 is a synthetic image generation speed calculating means 2403 for calculating the synthetic image generation speed by the image synthesizing means 104. Is a scheduled playback time storage means for storing a scheduled playback time of the synthesized moving image data for search, 2404 is a playback schedule stored in the synthesized image generation speed calculation means 2402 and a scheduled playback time storage means 2403. Transmission start control means for controlling the start of transmission of the composite image being generated from time, 2405 is a transmission frame storage means for storing the transmission frame number (j), 2406 is a write frame number (i) stored in the write frame storage means 2401. ) And the transmission frame number stored in the transmission frame storage means 2405 j) comparing, the frame comparing means for setting the frame number to be sent, 2407 sends frame number set in the frame comparing means 2406 (j)
Reading the composite image of from the composite image storage means 1507,
It is a composite image transmitting means for transmitting the read composite image of the transmission frame number (j) to the communication means 1509.

FIG. 25 is a flowchart showing the flow of the image synthesizing process by the moving image search system according to the present embodiment. The operation will be described with reference to FIG.

In the image synthesizing means 1506, the scheduled playback time is stored as the playback time of the search synthesized moving image data, which is the longest moving image data time among the moving image data matching the search conditions obtained by the moving image search means 1501. Means 2403
(2501).

Next, while the write frame number (i) is shorter than the scheduled reproduction time, the following processing is repeated. (25
02). The image synthesizing unit 1506 stores the writing frame number (i) and the time required to write the i-th frame from frame number 0 to the writing frame storage unit 2401 in a pair (2503).
The composite image of the write frame number (i) is stored in the composite image storage unit 1507 (2504).

Next, if the transmission of the composite image is not yet started, the transmission start check processing is performed (2505-
Y).

The above steps 2503 to 2506 are repeated until the write frame number becomes larger than the number of frames of the scheduled playback time, and synthetic image data having the scheduled playback time is generated.

FIG. 26 is a flow chart showing the flow of the transmission start check process, and the combined image transmission start check process will be described with reference to FIG.

In the composite image generation speed calculation means 2402, the time required for generating the composite image is calculated based on the write frame number (i) stored in the write frame storage means 2401 and the time required for writing the i-th frame. Ask (2601).

The transmission start control means 2404 synthesizes the scheduled playback time from the time required to generate the synthesized image calculated by the synthetic image generation speed calculation means 2402 and the scheduled playback time stored in the scheduled playback time storage means 2403. The time required to generate an image is determined (2602).

Further, in the transmission start control means 2405,
From the writing frame number (i) stored in the writing frame storage means 2401 and the time required to generate the composite image for the calculated expected reproduction time, the time required to generate the remaining composite image of the expected reproduction time (260)
3).

When the time required to generate a moving image corresponding to the remaining scheduled playback time is shorter than the scheduled playback time (2603-Y), a transmission start instruction of the moving image data is output to the frame comparison means 2406 (step 603). 2604).

FIG. 27 is a flow chart showing the flow of the moving picture sending process in the frame comparing means 2406, and the moving picture sending process will be described with reference to FIG.

The frame comparison means 2406 receives the transmission start instruction output from the transmission start control means 2404,
The sending frame number (j) is initialized (2701).

Further, while the frame number stored in the frame storing unit 2405 is smaller than the frame number stored in the frame storing unit 2401 (2702-Y), the frame comparing unit 2406 calculates the frame number (j). The image is output to the composite image sending means 2407 (2703). Further, the transmission frame number is incremented by one (2704), and the processing from steps 2702 to 2704 is repeated while the transmission frame number (j) is smaller than the writing frame number (i) (2702-).
Y).

The composite image transmitting means 2407 determines the number of transmission frames from the transmission frame number output from the frame comparing means 2406.
The composite image data of the transmission frame number stored in the composite image storage unit 1507 is transmitted to the moving image search terminal 1520.

As described above, in the present embodiment, the time required to generate the synthetic moving image data for search having the length of the expected reproduction time is calculated from the scheduled reproduction time of the search moving image and the synthetic image generation speed. The transmission of the composite image after a certain time has elapsed so that the generation of all the composite video data for search is completed before the reproduction of the composite video data for search at the scheduled playback time of the search video is completed. This is what we started. Thus, reproduction of a plurality of pieces of moving image data that match the search condition can be started with a short waiting time without waiting for the completion of generation of the combined moving image data of the scheduled reproduction time, and the practical effect is large.

(Embodiment 12) Next, a twelfth embodiment corresponding to claims 14 and 15 will be described.
FIG. 28 shows an example of a configuration diagram of the moving image search system according to the present embodiment. In addition to the configuration of the eighth embodiment, the moving image search terminal 2820 is provided with an image selection unit 2824 for selecting desired moving image data from the combined images that match the search conditions, and the moving image search server 280.
0, a screen division number storage unit 2811 that stores the number of screen divisions of the composite image to be reproduced in the composite image reproduction unit 2823 of the video search terminal 2820;
A time-division extracting unit 2810 for extracting moving image data whose moving start time of moving image data selected by using the 20 image selecting units 2814 is shifted from the moving image storage unit 2802 is provided.

First, a method of selecting a moving image by the image selecting means 2824 in the moving image search terminal 2820 will be described. In the moving image search terminal 2820, the user uses the image selection unit 2824 to select one desired data from the composite image being reproduced by the composite image reproduction unit 2823. As the selection method, a method of selecting and selecting an area on the screen of the reproduced composite image using a pointing device such as a mouse, or displaying individual information of moving image data constituting the composite image is displayed. Although a method of selecting from a list or the like is conceivable, any method may be used for the selection method.

Image selection means 28 of moving image search terminal 2820
The identifier of the moving image data selected by 24 is transmitted to the moving image search server 2800 via the network 2830 and passed to the time division unit 2808 of the moving image search server 2800.

FIG. 29 shows a flow of an image synthesizing process for synthesizing an image obtained by extracting an image from the selected moving image data with a slight delay by the image reducing unit 2805, the image synthesizing unit 2806, and the time division unit 2810. It is a flowchart explaining.

In the time division means 2810, the moving image search terminal 2
The length of the moving image data selected by the image selection unit 2824 of 820 is obtained, and the time interval corresponding to the number of screen divisions stored in the screen division number storage unit 2811 is calculated (2
901). The subsequent processing is repeated until the generation of the composite image data having the calculated time interval length is completed.

In the time division means 2810, the moving picture storage means 2
Image data of frames at every calculated time interval is extracted from the selected moving image data stored in 802 according to the number of divisions of the screen, and the image reducing means synthesizing means 28 is arranged in time order.
05 (2902).

[0103] The image reducing means 2805
The image data output from 10 at every time interval corresponding to the number of screen divisions is reduced and output to the image synthesizing unit 2806 in chronological order (2903).

The image composition means 2806 is a time division means 28.
Images that have been extracted and reduced by shifting the reproduction start time in order of time divided in 10 are arranged as shown in FIG. 30 to synthesize the images and stored in the synthesized image storage means 2807 (290).
4).

Next, the synthetic image generation control means 2803
It is determined whether the length of the created composite image reaches the calculated time interval. If the created composite image does not reach the calculated time interval (2905-Y), the image of the next frame is extracted by the image extraction means. Instruct to 2803 (290
6). As described above, a composite image obtained by reducing and synthesizing image data extracted by shifting the reproduction start time in the order of the time divided by the time division unit 2810 is created in the number of frames corresponding to the time interval, and the composite image of the moving image search server 2800 is created. It is stored in the storage unit 2807.

The synthesized moving image data stored in the synthesized image storage means 2807 is controlled by the synthesized image transmission control means 2808 of the moving image search server 2800 to control the moving image search terminal 2808.
820 and displayed on the composite image display unit 2823 of the moving image search terminal 2820.

As described above, in this embodiment, in addition to the eighth embodiment, a time-division extracting unit 2810 extracts a moving image from desired moving image data designated by the image selecting unit 2824 of the moving image search terminal 2800. A synthetic moving image is generated by reducing and synthesizing the moving image data whose reproduction start time is shifted according to the length of the data, and the generated synthetic moving image is displayed on the synthetic image display means 2823.
As a result, the contents of the selected moving image data can be viewed more precisely and simultaneously, and a target scene can be quickly searched, and the practical effect is large.

Incidentally, it corresponds to claim 15, that is, FIG.
The moving image data whose playback start time is shifted from the moving image data that matches the search condition obtained by the moving image search unit 3101 is extracted from the moving image storage unit 3102 by the time division extraction unit 3104. As shown in FIG. 32, the image data extracted and reduced from the moving image data that matches the search condition are arranged in the vertical direction, and the reproduction start time of each moving image data is shifted and the reduced image data is arranged in the horizontal direction. It is also possible to generate the synthesized image and reproduce the generated synthesized image by the synthesized image reproducing means 3123 of the moving image search terminal 3120. This makes it possible to divide the video data that matches the search conditions into smaller pieces and view them at the same time, making it possible to see all of the video data that matches the search conditions in a short time, and quickly find the target scene The practical effect is great.

[0109]

As described above, according to the first aspect of the present invention, when simultaneously reproducing all the moving image data displayed on the multi-screen, the moving image data to be reproduced is reduced for each same frame of each moving image. By synthesizing and reproducing, it is possible to handle as one moving image data, and an advantageous effect that consumption of network resources and system resources can be reduced can be obtained.

When a part of moving image data displayed on the multi-screen is reproduced simultaneously, each moving image to be reproduced is reduced for each frame, and the moving image data not to be reproduced is reproduced. By reducing the data of the currently displayed frame, and combining and reproducing the reduced image to be reproduced and the reduced image not to be reproduced,
Further, an advantageous effect that unnecessary data can be reduced can be obtained.

Further, in the second invention, when the moving image data is searched using the still image as a key, the search processing speed is improved by comparing the key image with the image of the representative frame of each scene of the moving image data. Has the advantageous effect of improving

In the third invention, when moving image data is searched using a still image as a key, a frame of a moving image that matches the key image is newly registered as a key image, and the newly registered key image is registered. By performing the search again, the advantageous effect of preventing omission of detection can be obtained.

When searching for moving image data using a still image as a key, the key image is compared with the representative frame image of each scene of the moving image data, and a scene including a matched moving image frame is searched for. By registering a frame most similar to the key image in the scene as a new key image and performing a search again using the newly registered key image, an advantageous effect that the accuracy of the search is further improved can be obtained.

In the fourth aspect of the present invention, the moving image search server extracts and reduces a plurality of moving image data matching the search conditions, and combines and transmits the plurality of moving image data. Display an image. Accordingly, even when a plurality of moving image data that match the search conditions are displayed on the screen at the same time, the moving image search server receives and reproduces the reduced and combined moving image data via a network. The load on the resources and the network can be reduced to one moving image data.

Further, in the fifth invention, in the moving image search server, the frame interval setting for setting the frame interval for taking out the moving image data by the moving image search means from the time required to create a fixed number of composite images Means for changing a frame interval of a combined image obtained by reducing and combining a plurality of pieces of moving image data that match a search condition according to a frame interval set by the frame interval setting unit, so that the frame interval is not interrupted according to a load on a server. The effect that the moving image can be transmitted to the moving image search terminal is obtained.

In the sixth aspect of the present invention, the composite image transmission control means of the moving image retrieval server compares the frame number of the generated composite image with the frame number of the composite image transmitted to the moving image retrieval terminal, and If the frame number of the composite image sent to the search terminal is equal to or greater than the frame number of the generated composite image, the next frame is transmitted from the first frame, and the generated composite image is repeatedly reproduced. In addition, it is possible to reproduce the moving image data that matches the search condition without waiting for the generation of all the combined moving image data to be completed. In addition, the same scene is repeatedly played back by repeatedly using the moving image being combined to the maximum, and the newly generated combined image is added little by little, so that the target scene can be easily found. This has the effect.

Further, in the seventh invention, in the composite image transmission control means of the moving image search server, the writing frame storage means and the composite image generation speed calculation means calculate the time required for generating the composite image of the scheduled reproduction time. When the time required to generate the remaining composite image becomes shorter than the scheduled reproduction time, the transmission of the composite image is started, and the reproduction of a plurality of pieces of moving image data that match the search condition is performed. The effect is obtained that the processing can be started in a short waiting time without waiting for the completion of the generation of the moving image data.

In the eighth invention, there is provided a time-division extracting means for extracting, from the moving image storage means, an image of which the reproduction start time of the specified moving image data is shifted, and the arbitrary moving image data designated by the image selecting means is provided. Furthermore, it is possible to search the selected moving image data more finely by dividing the extracted image data by shifting the playback start time and reducing the extracted image data, arranging them in chronological order and synthesizing them. , And an advantageous effect that a desired scene can be quickly searched can be obtained.

[Brief description of the drawings]

FIG. 1 is a configuration diagram illustrating an image display device according to a first embodiment of the present invention.

FIG. 2A is a flowchart showing processing up to image data display in the present invention; FIG. 2B is a flowchart showing multi-screen simultaneous reproduction processing in the first embodiment;

FIG. 3 is a configuration diagram illustrating an image display device according to a second embodiment of the present invention.

FIG. 4 is a flowchart showing a process of reproducing a selected image according to the second embodiment;

FIG. 5 is a configuration diagram of an image reducing unit according to the first embodiment.

FIG. 6 is a flowchart showing a flow of processing of an image reducing unit.

FIG. 7 is a configuration diagram showing an image display device according to a fourth embodiment of the present invention.

FIG. 8 is a flowchart showing the flow of processing in the fourth embodiment.

FIG. 9 is a configuration diagram of a moving image search unit according to a fourth embodiment.

FIG. 10 is a flowchart showing the flow of processing of the moving image search means of FIG. 9;

FIG. 11 is a configuration diagram showing a moving image search unit according to a sixth embodiment of the present invention.

FIG. 12 is a flowchart showing the processing flow of the moving image search means of FIG.

FIG. 13 is a block diagram showing a moving image search unit according to a seventh embodiment of the present invention.

FIG. 14 is a flowchart showing the flow of processing of the moving image search means in FIG. 13;

FIG. 15 is a configuration diagram showing a moving image search system according to an eighth embodiment of the present invention.

FIG. 16 is a flowchart showing a flow of a multi-moving image synthesis process of the moving image search server according to the eighth embodiment of the present invention.

FIG. 17 is a diagram illustrating an example of image composition according to the eighth embodiment of the present invention.

FIG. 18 is a flowchart showing the flow of processing of a composite image transmission control unit according to the eighth embodiment of the present invention.

FIG. 19 is a configuration diagram showing a moving image search system according to a ninth embodiment of the present invention.

FIG. 20 is a flow chart showing the flow of multi-moving image compositing processing of the moving image search server in the ninth embodiment of the present invention.

FIG. 21 is a configuration diagram showing a configuration of a synthetic image transmission control unit of the moving image search server according to the tenth embodiment of the present invention.

FIG. 22 is a flowchart showing a flow of synthetic image transmission control processing of the moving image search server according to the tenth embodiment of the present invention.

FIG. 23 is a diagram showing a state of reproducing a composite image according to the tenth embodiment of the present invention.

FIG. 24 is a configuration diagram showing a configuration of a composite image transmission control unit of a moving image search server according to an eleventh embodiment of the present invention.

FIG. 25 is a flowchart showing the flow of image compositing processing in the eleventh embodiment of the present invention.

FIG. 26 is a flowchart showing the flow of a transmission start check process according to the eleventh embodiment of the present invention.

FIG. 27 is a flowchart showing the flow of a moving image sending process in the eleventh embodiment of the present invention.

FIG. 28 is a configuration diagram of a moving image search system according to a twelfth embodiment of the present invention.

FIG. 29 is a flowchart showing the flow of synthetic image generation processing in the twelfth embodiment of the present invention.

FIG. 30 is a diagram illustrating an example of an arrangement of reduced images according to a twelfth embodiment of the present invention.

FIG. 31 is a configuration diagram of a moving image search system according to a twelfth embodiment of the present invention.

FIG. 32 is a diagram illustrating an example of an arrangement of reduced images according to a twelfth embodiment of the present invention.

FIG. 33 is a configuration diagram showing an image display device in the related art.

[Fig. 34] Fig. 34 is a configuration diagram showing a moving image search means in a conventional technique.

FIG. 35 is a configuration diagram showing a moving image search system in the related art.

[Explanation of symbols]

 101 image reducing means 102 image synthesizing means 103 image displaying means 104 multi-screen simultaneous reproducing means 304 reproducing image selecting means 305 selected image simultaneous reproducing means 501 macroblocking means 502 orthogonal transforming means 503 pixel extracting means 504 pixel synthesizing means 701 key image input Means 702 Moving image search means 703 Image reduction means 704 Image synthesis means 705 Image display means 901 Cut point detection means 902 Frame extraction means 903 Key image input means 904 Image comparison determination means 1105 Key image registration means 1305 Image similarity output means 1500 Video Image search server 1501 Moving image search means 1502 Video storage means 1503 Synthetic image generation control means 1504 Synthetic image extraction means 1505 Image reduction means 1506 Image synthesis means 1507 Synthetic image storage means 150 8 synthetic image sending control means 1509 communication means 1520 moving image search terminal 1521 search condition input means 1522 communication means 1523 synthetic image reproducing means 1530 network 1900 moving image search server 1901 moving image searching means 1902 moving image storing means 1903 synthetic image generation controlling means 1904 Synthetic image extracting means 1905 Image reducing means 1906 Image synthesizing means 1907 Synthetic image storing means 1908 Synthetic image sending control means 1909 Frame interval setting means 1910 Communication means 1920 Moving image search terminal 1921 Search condition input means 1922 Communication means 1923 Synthetic image reproducing means 1930 Network 2101 Write frame storage means 2102 Transmission start control means 2103 Transmission frame control means 2104 Frame comparison means 2105 Synthetic image transmission Means 2401 Writing frame storage means 2402 Synthetic image generation speed calculation means 2403 Scheduled reproduction time storage means 2404 Transmission start control means 2405 Transmission frame control means 2406 Frame comparison means 2407 Synthetic image transmission means 2800 Moving image search server 2801 Moving image search means 2802 Animation Storage means 2803 Synthetic image generation control means 2804 Synthetic image extraction means 2805 Image reduction means 2806 Image synthesis means 2807 Synthetic image storage means 2808 Synthetic image transmission control means 2809 Communication means 2810 Time division extraction means 2811 Screen division number storage means 2820 Moving image search Terminal 2821 Search condition input means 2822 Communication means 2823 Synthetic image reproduction means 2824 Image selection means 2830 Network 3100 Moving image search server 3101 Video Retrieval means 3102 Video storage means 3103 Synthetic image generation control means 3104 Time division extraction means 3105 Image reduction means 3106 Image synthesis means 3107 Synthetic image storage means 3108 Synthetic image transmission control means 3109 Communication means 3110 Screen division number storage means 3120 Moving image search terminal 3121 Search condition input means 3122 Communication means 3123 Synthetic image reproduction means 3130 Network

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor: Manabu Nakamura 1006 Kadoma Kadoma, Osaka Prefecture Matsushita Electric Industrial Co., Ltd.

Claims (15)

[Claims] 1. An image reducing unit for reducing an image, an image combining unit for combining the reduced images, an image display unit for displaying the combined image, and a multi-screen simultaneous playback unit for playing back the image. The multi-screen simultaneous reproduction means reduces the image for each frame to be reproduced by the image reduction means, synthesizes the image reduced for each frame by the image synthesis means for each same frame of each moving image data, and An image display device, wherein the reproduced image is an object to be reproduced. 2. An image reduction unit comprising: a macroblock dividing unit that divides an image into arbitrary pixel units; an orthogonal transformation unit that performs orthogonal transformation for each macroblock; and a pixel extraction unit that extracts a first low-frequency term. 2. The image display device according to claim 1, further comprising a pixel synthesizing unit for synthesizing the extracted pixels. 3. An image reducing means for reducing an image, an image synthesizing means for synthesizing the reduced images, an image display means for displaying the synthesized image, and which image of the synthesized image is reproduced. A selected image reproducing means for selecting and reproducing the image, the selected image reproducing means reducing the image not reproduced by the image reducing means and the image to be reproduced frame by frame, and the image combining means for reproducing the image not reproduced and the reproduced image. An image display device, comprising: synthesizing an image for each frame of an image to be processed; and selecting and synthesizing the synthesized image. 4. A key image input means for inputting an image serving as a search key, a moving image search means for searching a moving image using the image as a key, and an image reducing means for reducing the image of the searched moving image. An image display device comprising image combining means for combining the reduced images and image display means for displaying the combined image. 5. A moving image search means comprising: a cut point detection means for detecting a cut point of a video to be searched; a frame extraction means for extracting an arbitrary frame in a scene; and a comparison between the extracted frame and a key image. The image display device according to claim 4, further comprising: an image comparison / determination unit for performing. 6. The moving image search means further includes key image registration means for adding a frame which is compared by the image comparison and determination means as a new key image, and the image comparison and determination means is newly registered. The image display device according to claim 5, wherein the key images are compared again. 7. A key image registration in which the moving image search means newly adds, as a key image, a frame closest to the key image among all the frames in the scene including the frames matched by the image comparison and determination means. 6. The image display device according to claim 5, further comprising means, wherein the image comparison / determination means compares the newly registered key image again. 8. The image search means further comprises image similarity output means for outputting the similarity between the frame and the key image compared by the image comparison / determination means, and the one having the highest similarity among the similarities. The image display device according to claim 6, wherein the key is registered in a key image registration unit. 9. A moving image search server for searching for a moving image, and a moving image search terminal for requesting the moving image search server to search for a moving image, which are communicable via a network, The moving image search server stores moving image data to be searched, a moving image search unit that searches for moving image data stored in the moving image storage unit, and a moving image that matches a search condition in the moving image search unit. An image extracting unit that extracts an image for each frame from image data, an image reducing unit that creates a reduced image from the moving image data extracted by the moving image extracting unit, and an image combining unit that creates an image that combines the reduced images. Synthetic image generation control means for controlling generation processing of a synthetic image, synthetic image storage means for storing a synthesized image, and synthetic image stored in the synthetic image storage means Data transmission control means for controlling transmission of data to the moving image search terminal, and communication means for controlling communication with the moving image search terminal. A search condition input means for inputting, a communication means for controlling communication with the moving image search server, and a synthesized image reproducing means for displaying synthesized image data synthesized by the moving image search server; The server reduces a plurality of pieces of moving image data that match the search condition, generates composite image data by combining them, transmits the generated composite image data via a network, and the moving image search terminal receives the A moving image search system for reproducing synthesized image data as a moving image. 10. The moving image search server includes a combined image compression unit for compressing the image data combined by the image combining unit and storing the compressed image data in the combined image storage unit, and the moving image search terminal includes the communication unit. The composite image expansion means for expanding the composite image data received by the device and outputting the expanded composite image data to the composite image reproducing means is provided, storing the composite image data and communicating between the moving image search server and the moving image search system. The moving image search system according to claim 9, wherein compressed data is used as the data. 11. The moving image search server comprises frame interval setting means for setting a frame interval for extracting moving image data by the moving image search means from a time required to create a fixed number of composite images. 10. The frame interval of a synthesized image to be generated is changed by changing the extraction frame interval of a plurality of pieces of moving image data that match a search condition in the image extraction unit according to the frame interval set by the interval setting unit. The moving image search system according to 1. 12. A composite image transmission control means for storing a frame number of the composite image generated by the image composition means, and transmitting the composite image based on the write frame number stored in the write frame storage means. Transmission start control means for controlling the start; transmission frame storage means for storing a frame number of the composite image transmitted from the video search server to the video search terminal; and transmission of the generated composite image stored in the writing frame storage means. A frame comparison unit that compares the frame number with the frame number of the composite image sent from the moving image search server to the moving image search terminal stored in the transmission frame storage unit, and sets the frame number to be sent next, and the frame comparison unit. The composite image of the transmission frame number set in step 2 is read from the composite image storage means, and the communication means Means for transmitting the synthesized image to the moving image search terminal, and comparing the frame number of the generated synthesized image with the frame number of the synthesized image transmitted to the moving image search terminal. 10. When the frame number of the generated composite image is greater than or equal to the frame number of the generated composite image, the reproduction of the generated composite image is repeated with the frame number of the frame to be transmitted next as the head. Video search system. 13. A writing frame storing means for storing a frame number of a synthesized image generated by the image synthesizing means, and a synthesized image based on a frame number stored in the writing frame means and a writing time. Composite image generation speed calculating means for calculating the generation speed of the composite image, transmission start control means for controlling the start of transmission from the synthetic image generation speed and the scheduled playback time obtained by the composite image generation speed calculation means, A scheduled playback time storage means for storing a scheduled playback time, a sending frame storage means for storing a frame number of a composite image sent from the video image search server to the video image search terminal, and a write frame storage means The frame number of the composite image and the moving image search server stored in the transmission frame storage means are transmitted to the moving image search terminal. A frame comparison unit that compares the frame numbers of the output composite images and sets a frame number to be transmitted next, and reads the composite image of the transmission frame number set by the frame comparison unit from the composite image storage unit and outputs the composite image to the communication unit. A writing frame storage means and a synthetic image generation speed calculating means for calculating a time required for generating a synthesized image of a scheduled reproduction time, and a time required for generating a remaining synthesized image. 10. The moving image search system according to claim 9, wherein the transmission of the synthesized image is started at a time point when the time becomes shorter than the scheduled reproduction time. 14. A moving image search terminal comprising image selecting means for selecting a desired moving image data from a synthesized moving image generated by reducing a plurality of moving image data matching a search condition and synthesizing them. The moving image search server includes a screen division number storage unit that stores the number of divisions of the composite image, and a time division extraction unit that extracts the image by shifting the reproduction start time of the specified moving image data from the moving image storage unit, Image data extracted by dividing the arbitrary moving image data designated by the moving image search server by the image selecting means into units of time to be stored in the screen division number storing means is reduced, arranged and arranged in the order of elapse of time, and synthesized. The moving image search system according to claim 9, wherein a synthetic image is generated. 15. The time-division extracting means, wherein the moving-image search server comprises, in place of the image extracting means, a time-division extracting means for extracting, from the moving-image storage means, an image whose reproduction start time of a plurality of designated moving image data is shifted, The extracting means extracts the moving image data stored in the moving image storage means, and the image synthesizing means shifts the reproduction start time between the reduced moving image data meeting the search condition and the moving image data meeting the search condition. 10. A composite image in which reduced image data extracted and arranged in order of time elapse is generated.
The moving image search system described in.