JP2009055147A - Motion picture data transmitting apparatus and operation control method thereof - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To transmit a motion picture file of the receivable capacity to a portable terminal. <P>SOLUTION: When a motion picture file of data quantity not more than the capacity of a reception memory of a portable terminal is not formed (NO in step 14), it is confirmed whether or not a summary motion picture should be formed (step 17). If the summary motion picture should be formed (YES in step 17), a summary motion picture file having the data quantity not more than the capacity of the reception memory of the portable terminal as a transmission target and representing the summary of a motion picture is formed on the basis of motion picture conversion information unique to the motion picture file, terminal information indicating the memory capacity of the portable terminal and feature amount of the motion picture file (step 18). In response to a request of the portable terminal, the summary motion picture file is transmitted to the portable terminal (step 23). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a moving image data transmission apparatus and an operation control method thereof.

Mobile terminal devices such as mobile phones often have different video display capabilities for each mobile terminal device. For this reason, the moving image display capability may be stored in correspondence with the portable terminal device and converted into moving image data corresponding to the moving image display capability of the portable terminal device to be transmitted (Patent Document 1).
JP 2006-121614 A

Moreover, although there exists a thing which shortens a long-time moving image (patent document 2), the thing which produces | generates the moving image data with the data amount according to the capacity | capacitance of the receiving memory of a portable terminal device is not considered.
JP 2006-148397

  An object of the present invention is to transmit moving image data within the capacity of a reception memory of a portable terminal device to the portable terminal device.

  The moving image data transmitting apparatus according to the present invention includes a data amount determining means for determining whether or not the data amount of moving image data is larger than the capacity of the reception memory of the portable terminal device that transmits the moving image data, and the data amount determining means. When it is determined that the data amount of the moving image data is larger than the capacity of the reception memory of the portable terminal device, the feature amount and the reproduction time of the moving image in each frame constituting the moving image represented by the moving image data, Based on the relationship between the feature quantity detected by the feature quantity-reproduction time detecting means, the feature quantity detected by the feature quantity-reproduction time detection means, and the playback time of the moving picture, one or more moving image portions having a large feature quantity are detected. Summary video data representing summary video having a data amount equal to or less than the capacity of the reception memory of the portable terminal device is generated from the video data so as to be included. Summary video data generation means for, and is characterized in that it comprises a summary video data transmission means for transmitting the summary video data generated by the summary video data generating means to said portable terminal device.

  The present invention also provides an operation control method suitable for the moving image data transmitting apparatus. That is, in this method, the data amount determination means determines whether the data amount of the moving image data is larger than the capacity of the reception memory of the portable terminal device that transmits the moving image data, and the feature amount-reproduction time detection means. However, in response to determining that the data amount of the moving image data is larger than the capacity of the reception memory of the portable terminal device by the data amount determining means, in each frame constituting the moving image represented by the moving image data The relationship between the feature quantity and the playback time of the video is detected, and the summary video data generation means detects the feature quantity based on the relationship between the feature quantity detected by the feature quantity-playback time detection means and the playback time of the video. Summary video data representing summary video having a data amount equal to or less than the capacity of the reception memory of the portable terminal device so as to include one or more large video parts, Generated from image data, summarized moving picture data transmission means, a summary video data generated by the summary video data generating means is to transmit to the mobile terminal.

  According to the present invention, when the data amount of the moving image data is larger than the capacity of the reception memory of the portable terminal device that transmits the moving image data, the feature amount and the moving image in each frame constituting the moving image represented by the moving image data The relationship with the playback time is detected. Based on the relationship between the detected feature quantity and the playback time, summary video data including a video part with a large feature quantity and having a data amount less than the capacity of the reception memory of the portable terminal device is generated from the video data. The The generated summary moving image data is transmitted to the portable terminal device.

  Since the amount of summary video data transmitted to the portable terminal device is less than the capacity of the reception memory of the portable terminal device, the summary movie data can be received by the portable terminal device, and the summary movie data represented by the summary movie data can be received. It can be played. Since the summary moving image includes a moving image portion having a large feature amount, a characteristic portion of the moving image represented by the moving image data is represented. The user of the portable terminal device can browse important parts of the moving image.

  You may further provide the switching location detection means which detects the switching location of the scene of the moving image represented by the said moving image data. In this case, the summary moving image data generating means will generate summary moving image data so as not to include, for example, the scene change location detected by the change location detecting means.

  The moving image data may represent moving images including a plurality of groups in which a plurality of difference images representing a difference between a complete image of one frame and a previous frame are included. In this case, the summary moving image data generating means can determine, for example, one or more moving image portions having a large feature amount based on the relationship between the feature amount detected by the feature amount-reproduction time detecting means and the reproduction time of the moving image. Summary video data representing a summary video having a data amount equal to or less than the capacity of the reception memory of the portable terminal device is generated from the video data, and the first frame image is a complete image. This will extract the summary video data.

  FIG. 1 shows an embodiment of the present invention and is a block diagram showing an electrical configuration of a moving image file transmission apparatus.

  The moving image file transmitting apparatus according to this embodiment transmits a moving image file to a requested portable terminal in response to a request from a portable terminal (portable terminal apparatus) such as a mobile phone. The mobile terminal is provided with a receiving memory, and can receive a moving image file having a data amount less than the capacity of the receiving memory, but cannot receive a moving image file having a data amount larger than the capacity of the receiving memory. A moving image represented by the received moving image file is displayed on a display screen of a display device provided in the mobile terminal. In this embodiment, a moving image file having a data amount less than the capacity of the reception memory of the mobile terminal is transmitted to the mobile terminal.

  The overall operation of the moving image file transmission device is controlled by the control device 3.

  A moving image file to be transmitted to the mobile terminal is input to the moving image file transmitting apparatus via an input interface (not shown). The input moving image file is given to the moving image database 6 and stored. In addition, data representing moving image conversion information when the input moving image file is transmitted to the mobile terminal is input to the moving image file transmitting device via an input interface (not shown). Data representing the input moving image conversion information is stored in association with the moving image file given to the conversion information management device 4 and stored in the moving image database 6. The moving image conversion information is stored in the conversion information management device 4 corresponding to the file name of the moving image file.

  FIG. 2 is an example of moving image conversion information.

  The video conversion information items include priority setting, stereo setting, and capacity excess setting.

  The priority setting is used to set the quality of the moving image represented by the moving image file when the moving image file is transmitted to the mobile terminal. The priority settings include standard image quality, image quality priority, motion priority, and audio priority. Image quality priority is set to reduce the frame rate of moving images and increase the video bit rate. Motion priority is set to increase the video bit rate and decrease the audio bit rate. Audio priority is set to reduce the video bit rate and increase the audio bit rate.

  The stereo setting is used to set whether to output automatically stereo audio or to monaural audio.

  The capacity excess setting is for setting processing when the data amount of the moving image file to be transmitted exceeds the capacity of the reception memory of the portable terminal to be transmitted. You can set whether to send the summary video or the first part of the video. A summary movie is a movie in which a characteristic part is extracted from the whole movie.

  Such moving image conversion information is stored in the conversion information management device 4 corresponding to a large number of moving image files stored in the moving image database 6.

  Returning to FIG. 1, the conversion information management device 4 stores a terminal information table.

  FIG. 3 is an example of a terminal information table.

  The terminal information includes the video file format of the mobile terminal, the display size of the image that can be displayed on the display screen provided in the mobile terminal (display screen size), the upper limit video bit rate, and the lower limit video bit rate. , Frame rate, audio codec type, upper limit audio bit rate, lower limit audio bit rate, sampling frequency, number of channels, reception memory capacity (KB), and the like.

  The video bit rate represents the amount of data per second of image data. In portable terminals, since the upper limit and lower limit of the video bit rate are determined, the upper limit video bit rate and the lower limit video bit rate are defined in the terminal information. The frame rate is the number of frames per second of images constituting a moving image. The audio bit rate represents the amount of data per second of audio data. As for the audio bit rate, since the upper limit and the lower limit are determined in the mobile phone, the upper limit audio bit rate and the lower limit audio bit rate are defined in the terminal information.

  The display size is the size of the display screen provided in the mobile phone. Sub-QCIF (Quarter Common Intermediate Format) is 128 pixels by 96 pixels. QCIF is 176 × 144 pixels. QVGA (Quarter Video Graphics Array) is 320 pixels by 240 pixels.

  Returning again to FIG. 1, a moving image file to be converted is read from a large number of moving image files stored in the moving image database 6 and input to the feature quantity extraction device 7. In the feature quantity extraction device 7, a feature quantity is extracted from each of the moving image represented by the moving image file. In the feature amount extraction device 7, the relationship between the feature amount and the playback time of the moving image (images of each frame constituting the moving image) is obtained. The feature amount is an amount indicating that the scene of the moving image has been switched (an amount indicating a scene change), an amount indicating the similarity of a face included in the image, an amount indicating a change in volume, and an image constituting the moving image. This is the amount when expressed by a motion vector. By using the amount indicating the switching of the moving image scene as the feature amount, a summary moving image including the switching of the moving image scene can be generated. By using the amount representing the degree of similarity of the face included in the image as the feature amount, a summary moving image including the face image can be generated. By using the amount indicating a change in volume and the amount represented by a motion vector as a feature amount, a summary moving image in which a portion where the moving image is changed is extracted is generated. In any case, the feature quantity extraction device 7 extracts the relationship between the feature quantity and the playback time of the moving image. Data representing the relationship between the feature amount and the playback time of the moving image is given to the feature amount management device 8 and stored for each moving image file.

  When a moving image file to be transmitted to the portable terminal is generated, the moving image file is read from the moving image database 6. The read moving image file is input to the moving image conversion / generation device 2. Also, conversion information corresponding to the read moving image file is read from the conversion information management device 4 and input to the moving image conversion / generation device 2. Further, terminal information about the portable terminal to be transmitted is read from the terminal information management device 5 and input to the moving image conversion / generation device 2. In the moving image conversion / generation device 2, a moving image file having a data amount suitable for the portable terminal to be transmitted is generated. The conversion or generation processing of the moving image file in the moving image conversion / generation device 2 will be described in detail later. The moving image file converted or generated by the moving image conversion / generation device 2 is given to the moving image database 6 and stored for each corresponding portable terminal.

  When there is a request for a moving image file from the mobile terminal to the moving image file transmission device, the moving image file corresponding to the mobile terminal that requested the moving image file is read from the moving image database 6. The read moving image file is input to the moving image transmission interface 1. The moving image transmission interface 1 causes the moving image file to be transmitted to the mobile terminal that requested the moving image file.

  FIG. 4 is a flowchart showing a processing procedure of the moving image file transmission apparatus.

  As described above, a moving image file to be converted or generated for a mobile terminal to be transmitted is read from a large number of moving image files (original moving images) stored in the moving image database 6 (step 11). The read moving image file is input to the feature quantity extraction device 7. The feature amount extraction device 7 detects the feature amount for each reproduction time of the read moving image file (step 12). Thereby, data representing the relationship between the feature amount and the reproduction time for the moving image file is obtained. Data representing the relationship between the feature quantity and the reproduction time is given to the feature quantity management device 8 and stored therein. Also, the moving image conversion information for the moving image file is read from the conversion information management device 4, and the terminal information about the portable terminal to be transmitted is read from the terminal information management device 5 (step 13). Data representing the relationship between the feature quantity and the reproduction time, moving image conversion information and terminal information, and a moving image file to be converted or generated for the portable terminal to be transmitted are input to the moving image conversion / generation device 2.

  The data amount of the moving image file obtained when the moving image file read from the moving image database 6 is converted in accordance with the conversion information about the moving image file read from the moving image database 6 and the terminal information of the mobile terminal to be transmitted is the transmission It is confirmed whether or not the reception memory capacity of the target portable terminal is not exceeded (step 14). For example, if the priority setting is set to standard image quality in the conversion information, the video portion (excluding audio) in the video file is set so that the ratio between the video bit rate and the audio bit rate is 4: 1. Image portion) and sound portion are assigned.

  If a video file having a data amount less than the capacity of the receiving memory of the mobile terminal to be transmitted can be generated (YES in step 14), the video file is transmitted from the video file transmission device without shortening the playback time of the entire video. Since the mobile terminal that is the transmission target can receive the moving image file, the moving image file is converted according to the moving image conversion information and the terminal information (step 15). The converted moving image file is stored in the moving image database 6 for each portable terminal (step 16).

  If it is not possible to generate a video file with a data amount less than the capacity of the receiving memory of the mobile terminal that is the transmission target (NO in step 14), the over-capacity setting of the video conversion information corresponding to the video file It is confirmed whether it is set to transmit or set to transmit the moving image of the head part. If it is set to transmit the summary movie (YES in step 17), a summary movie file is generated based on the movie conversion information, the terminal information, and the feature amount (step 18). The method for generating the summary moving image file will be described later in detail. The generated summary moving image file is stored in the moving image database 6 corresponding to the mobile terminal (step 19).

  If the excess capacity setting is set to transmit the first part of the moving picture (NO in step 17), the terminal has a data amount that can be received by the receiving memory based on the moving picture conversion information and the terminal information. A moving image file representing the first moving image is generated (step 20). The generated moving image file is stored in the moving image database 6 corresponding to the terminal device (step 21).

  As described above, the moving image file to be transmitted to the portable terminal is converted or generated in advance for each portable terminal and stored in the moving image database 6.

  When there is a request for a moving image from the portable terminal (YES in step 22), the requested moving image file for the portable terminal that is a moving image file corresponding to the request is read from the moving image database 6. The read moving image file is transmitted from the moving image interface 1 to the requested portable terminal (step 23).

  FIG. 5 is a flowchart showing a summary moving image generation processing procedure (processing procedure of step 18 in FIG. 4).

  First, the time of the summary video to be generated is calculated (step 31). The time of the summary video can be calculated from Equation 1 with reference to the capacity and bit rate of the reception memory of the mobile terminal. The bit rate is an addition value of the determined video bit rate and audio bit rate.

Summary video time (s) = {Receiving memory capacity (KB) x 8} / bit rate (kbps)
... Formula 1
For example, if the capacity of the receiving memory is 240 KB and the bit rate is 160 kbps, the summary animation time is 12 seconds.

  Subsequently, the range of the moving image to be extracted is defined based on the relationship between the feature amount and the reproduction time of the moving image and the calculated time of the summary moving image. First, a local maximum value of a feature amount equal to or greater than a predetermined threshold value is detected (step 31). It is defined so that the range of the moving image near the detected maximum value becomes the time of the summary moving image calculated when all are combined (step 32). A summary moving image is generated by combining moving images within a specified range (step 34).

  6 and 7 show the relationship between the feature amount and the playback time of the moving image.

  Referring to FIG. 6, the horizontal axis represents the moving image reproduction time, and the vertical axis represents the feature amount of each image corresponding to the moving image reproduction time. Maximum values D1 and D2 that are equal to or greater than a predetermined threshold are detected. Reproduction times t1 and t2 giving the detected maximum values D1 and D2 are calculated.

  Referring to FIG. 7, moving image ranges of predetermined times Δt1 and Δt2 are defined around reproduction times t1 and t2 at reproduction times t1 and t2 at which local maximum values D1 and D2 are given. These predetermined times Δt1 and Δt2 are defined so as to be the time of the summary video when these predetermined times Δt1 and Δt2 are added (Δt1 + Δt2 = time of the summary video). The range of the moving images of these predetermined times Δt1 and Δt2 is combined to form a summary moving image.

  FIG. 8 shows the relationship between the feature amount and the moving image playback time.

  As described above, when there are local maximum values D3, D4, and D5 that are equal to or greater than the threshold value, the ranges Δt3, Δt4, which are defined from the reproduction times t3, t4, and t5 that give these local maximum values D3, D4, and D5. A summary moving image can be obtained by using a moving image portion in which Δt5 is combined. It goes without saying that Δt3 + Δt4 + Δt5 = summary movie time.

  FIG. 9 also shows the relationship between the feature amount and the moving image playback time.

  It is assumed that there are local maximum values D6, D7, and D8 that are equal to or greater than a predetermined threshold value. In the example shown in FIG. 8, a summary video is generated by defining a range from a playback time that gives a maximum value greater than or equal to a predetermined threshold. However, from the playback time that always gives a maximum value greater than or equal to a predetermined threshold. It is not necessary to generate a summary video using all the specified ranges.

  When local maximum values D6, D7, D8 exceeding a predetermined threshold value exist, local maximum values D7, D8 are selected from among the local maximum values D6, D7, D8. For the local maximum values D7 and D8, it is preferable to select a local maximum value having a large feature amount, but it is not always necessary to select a local maximum value having a large feature amount. Reproduction times t7 and t8 giving the selected maximum values D7 and D8 are detected. Ranges Δt7 and Δt8 are defined from the detected reproduction times t7 and t8. A summary moving image is generated by combining the defined ranges Δt7 and Δt8. Also in this case, Δt7 + Δt8 = summary movie time.

  FIG. 10 shows a modification and shows the relationship between the feature amount and the playback time of the moving image.

  In this modified example, all the images that make up the movie are not complete images like a movie by Motion JPEG (Joint Photographic Experts Group), but they are made up as a movie by MPEG (Moving Picture Experts Group). The image to be created is a complete frame image at a rate of one frame to multiple frames, and a summary movie file is generated from a movie file in which data representing the difference in motion from the previous frame image is inserted between the complete frames. It is. Video files are based on MPEG, but it goes without saying that they are not necessarily MPEG.

  As described above, the maximum values D1 and D2 are detected, and the ranges of the summary moving images Δt1 and Δt2 are defined. As shown in the lower part of FIG. 10 (the same applies to the range Δt2), these moving images in the range Δt1 are different from an I frame image (indicated by the symbol I) that can form a complete single frame image alone. A B-frame image (B-frame image) and a P-frame image (denoted by reference symbol P) (GOP: group of picture) that cannot constitute a complete single-frame image by the representation alone appear periodically at a predetermined ratio.

  In this embodiment, as described above, the first image in the range Δt1 defined for generating the summary moving image is an I frame image that can constitute a complete single frame image. The range Δt1 (also Δt2) may be defined so that the first image is an I-frame image, and when the first image is a B-frame image or a P-frame image, the B-frame image Alternatively, an I frame image is generated from the P frame image, and the first image in the range Δt1 (also Δt2) is set as the I frame image. If the first image is not a complete single frame image, the complete single frame image cannot be played back until the complete single frame image appears. Since the first image in the range is a complete single frame image, the complete single frame image can be reproduced from the beginning of the moving image portion of each range constituting the summary video.

  FIG. 11 and FIG. 12 show another embodiment and show the relationship between the feature amount and the moving image playback time.

  In this embodiment, switching of moving image scenes is not included in each range constituting the summary moving image. Since the summary video does not include a scene switching portion, it is possible to prevent the scene from switching suddenly within each range constituting the summary video. If the scene switching is not included in each range constituting the summary video, an amount indicating that the video scene has been switched will not be used as the feature amount.

  Referring to FIG. 11, it is assumed that the moving image scene has been switched at locations indicated by reference numerals S1, S2, S3 and S4. The detection of the switching of the moving image scene can be detected by the feature amount extraction device 7 based on the amount of change between the previous and next images. For example, when comparing the previous and next images within a certain range constituting the images, if almost all the pixels in the range are completely different, the scene is switched.

  The maximum values D1 and D2 that are equal to or greater than the threshold value are detected, and the ranges Δt1 and Δt2 in which the summary video is to be constructed are provisionally defined as described above. If there is no scene switching within the defined ranges Δt1 and Δt2, a summary video is generated using these ranges Δt1 and Δt2. As shown in FIG. 11, when the scene change S2 exists in the range Δt1, the range Δt1 is not used for the summary moving image.

  Referring to FIG. 12, a new maximum value D0 is detected. A new range Δt0 is defined from the detected maximum value D0. A summary video is generated using the defined ranges Δt2 and Δt0.

  The switching of scenes included in the summary video can be reduced.

It is a block diagram which shows the electric constitution of a moving image file transmission apparatus. It is an example of moving image conversion information. It is an example of a terminal information table. It is a flowchart which shows the process sequence of a moving image file transmission apparatus. It is a flowchart which shows the summary moving image production | generation process procedure. The relationship between the feature amount and the moving image playback time is shown. The relationship between the feature amount and the moving image playback time is shown. The relationship between the feature amount and the moving image playback time is shown. The relationship between the feature amount and the moving image playback time is shown. The relationship between the feature amount and the moving image playback time is shown. The relationship between the feature amount and the moving image playback time is shown. The relationship between the feature amount and the moving image playback time is shown.

Explanation of symbols

1 Video Transmission Interface 2 Video Conversion / Generation Device 3 Control Device 6 Video Database 7 Feature Extraction Device

Claims (4)

Data amount determination means for determining whether the data amount of the moving image data is larger than the capacity of the reception memory of the portable terminal device that transmits the moving image data;
The feature amount in each frame constituting the moving image represented by the moving image data when the data amount determining means determines that the data amount of the moving image data is larger than the capacity of the reception memory of the portable terminal device Feature amount-reproduction time detection means for detecting the relationship between the image and the reproduction time of the video,
Based on the relationship between the feature quantity detected by the feature quantity-playback time detecting means and the playback time of the moving picture, the reception memory of the portable terminal device includes the one or more moving picture parts having a large feature quantity. Summary video data generating means for generating summary video data representing a summary video having a data amount equal to or less than the capacity from the video data, and sending the summary video data generated by the summary video data generation means to the portable terminal device Summary video data transmission means,
A moving image data transmission device. A switching position detecting means for detecting a switching position of a moving image scene represented by the moving image data;
The summary video data generating means generates summary video data so as not to include the scene switching part detected by the switching part detection means.
The moving image data transmission device according to claim 1. The above-mentioned moving image data represents a moving image composed of a plurality of groups in which a difference image representing a difference between a complete image of one frame and a previous frame is composed of a plurality of images.
The above summary video data generation means
Based on the relationship between the feature value detected by the feature value-playback time detection means and the playback time of the moving image, the reception memory of the portable terminal device includes one or more moving image portions having a large feature value. Summary video data representing a summary video having a data amount less than or equal to the capacity is generated from the above video data, and the summary video data in which the first frame image is a complete image is extracted.
The moving image data transmission device according to claim 1 or 2. A data amount judging means judges whether the data amount of the moving image data is larger than the capacity of the reception memory of the portable terminal device that transmits the moving image data;
The feature amount-reproduction time detecting means is represented by the moving image data in response to the data amount determining means determining that the data amount of the moving image data is larger than the capacity of the reception memory of the portable terminal device. Detect the relationship between the feature value of each frame that makes up the movie and the playback time of the movie,
Based on the relationship between the feature amount detected by the feature amount-reproduction time detection unit and the playback time of the movie, the summary moving image data generation unit includes the one or more moving image portions having a large feature amount. A summary video data representing a summary video having a data amount equal to or less than the capacity of the receiving memory of the mobile terminal device is generated from the video data,
Summary video data transmission means transmits the summary video data generated by the summary video data generation means to the portable terminal device;
An operation control method for a moving picture data transmitting apparatus.

Images