JP4239496B2 - Information signal processing apparatus and information signal processing method - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a signal processing technique for performing digest reproduction of the contents of a recording medium as efficiently as possible when a recording medium on which an image / audio signal is recorded, such as DVD software, is mounted on a predetermined apparatus. In a recording device that records information signals such as video signals and audio signals on a recording medium such as a magneto-optical disk, hard disk, semiconductor memory, etc., for example, when a broadcast program is recorded and played back, it is similar to any specified scene in the program Information signal processing device that can selectively reproduce a scene and perform a skimming operation to efficiently grasp a desired content in a short time, Information signal processing method Is.
[0002]
[Prior art]
When playing back software for the first time in the prior art, even if you want to get an overview of the contents of the software, it is normal if you come back to a fun or desired playback scene by digest playback at several times speed playback. There is only a technique such as reproduction, and there is no effective confirmation means of effective contents.
[0003]
In addition, there is no signal processing apparatus and signal processing method for displaying that a digest reproduction ID for video and audio to be reproduced has already been generated.
[0004]
Further, for example, as a known technique for analyzing a video structure from an audio signal and a video signal and performing special reproduction such as digest reproduction, Japanese Patent Laid-Open No. 10-507555 “Method for running and reading digital audio and video data” And its system ", Japanese Patent Laid-Open No. 11-239322," Video Browsing / Viewing System ", Japanese Patent Application Laid-Open No. 2000-50233" Broadcast-type distribution method, computer-readable recording medium having recorded thereon a program for causing the computer to execute the method, and And a computer-readable recording medium on which video content is recorded.
[0005]
[Problems to be solved by the invention]
However, the above-described conventional technique has a problem that there is no effective technique for classifying image scenes in real time at the time of recording, for example, and assigning a predetermined scene ID to the classified images.
[0006]
In addition, there is a problem that the digest reproduction operation cannot be performed by using the scene ID for recording from the recording medium on which the scene ID is recorded.
[0007]
Therefore, when soft reproduction is performed from the above viewpoint, there is a problem that must be solved by a signal processing technique that efficiently performs digest reproduction.
[0008]
[Means for Solving the Problems]
In order to solve the above problems, an information signal processing apparatus according to the present invention, Information signal processing method The configuration is as follows.
[0009]
(1) The information signal processing device extracts a feature of the image and sound from the reproduction unit that reproduces the first recording medium on which the image and audio signal is recorded, the image and audio signal reproduced by the reproduction unit, and the feature Recording means for creating a digest reproduction ID based on the extracted image sound and recording it on the second recording medium, and extracting the feature of the image sound, A feature is to be extracted within a normal playback time of a section a including a smaller intro portion than a normal playback section H. .
(2) The information signal processing apparatus reproduces the first recording medium on which the image / sound signal is recorded, and extracts the feature of the image / sound from the image / sound signal reproduced by the reproducing unit, Recording means for creating a digest reproduction ID based on the extracted image sound and recording it on the second recording medium, and extracting the feature of the image sound, When the playback sections H1, H2, and H3 are long, features are extracted within the normal playback time of the playback section b1 that is much shorter than the playback section H1, and for the next playback section H2, The feature is extracted within the normal playback time of the playback section b2 that is much shorter than that, and the feature is extracted within the normal playback time of the playback section b3 that is much shorter than the next playback section H3. .
[0010]
(3) In the information signal processing method, the first recording medium on which the image / audio signal is recorded is reproduced, the image / audio feature is extracted from the image / audio signal reproduced by the reproducing means, and the feature is extracted. Based on the image and sound, a digest playback ID is created and recorded on the second recording medium. A feature is extracted within a normal playback time of a section a including a smaller intro portion than a normal playback section H. That is.
(4) In the information signal processing method, the first recording medium on which the image / audio signal is recorded is reproduced, the image / audio feature is extracted from the image / audio signal reproduced by the reproducing means, and the feature is extracted. Based on the image and sound, a digest playback ID is created and recorded on the second recording medium. When the playback sections H1, H2, and H3 are long, features are extracted within the normal playback time of the playback section b1 that is much shorter than the playback section H1, and for the next playback section H2, The feature is extracted within the normal playback time of the playback section b2 that is much shorter than that, and the feature is extracted within the normal playback time of the playback section b3 that is much shorter than the next playback section H3. .
[0015]
DETAILED DESCRIPTION OF THE INVENTION
Next, an information signal processing apparatus according to the present invention, Embodiment of information signal processing method Will be described with reference to the drawings.
[0016]
First, the outline of the information signal processing apparatus according to the first embodiment of the present invention will be described. Recording such as rental software (DVD or the like) such as a recording medium on which an image / audio signal is recorded in advance or a hard disk or the like installed in the apparatus. When the predetermined image / audio information data recorded on the medium is reproduced for the first time by the device, this is automatically detected, and the summary reproduction ID is generated while performing the feature extraction of the image / audio at the same time as the reproduction. Store in a predetermined recording medium.
That is, the feature is not extracted at the normal playback time of all playback sections, but, for example, as shown in FIG. 1A, a section including a smaller intro portion than the normal playback section H Features are extracted within the normal playback time of a. FIG. 1B shows a technique for extracting features when the playback sections H1, H2, and H3 are long. For the playback section H1, normal playback of a playback section b1 that is much shorter than that is shown. The feature is extracted within the time, the feature is extracted within the normal playback time of the playback section b2 that is extremely shorter than the next playback section H2, and the next playback section H3 is more than that. The feature is extracted within the normal playback time of the extremely short playback section b3. As described above, when the playback section is long, the playback section is divided, and the features are extracted in a part of the playback section.
FIGS. 2A, 2B, and 2C show the processing steps of H1, H2, and H3 in FIG. 1A (Embodiment 1) or FIG. 1B (Embodiment 2). In the initial state of processing, as shown in FIG. 2 (A), the normal playback position and the feature extraction position have the same start position, and the normal playback time elapses as shown in FIG. 2 (B). As the playback proceeds, the position of the feature extraction processing advances toward the end of the predetermined feature extraction section, and when a predetermined normal playback section in the hatched section is normally played back as shown in FIG. The feature extraction process for the feature extraction section ends.
In this way, when performing digest playback using feature extraction data, normal playback is performed until the feature extraction processing for a predetermined section is completed.
When the digest playback mode using the feature extraction data is started by the signal processing as described above, even if the feature extraction data is not processed in advance, the predetermined digest playback operation cannot be performed until the feature extraction processing is completed. It is possible to prevent the sound from being output, and furthermore, since the time of the feature extraction process according to the present invention is completed in a short time, the user can immediately perform the normal reproduction operation during the feature extraction process. Thus, it can be felt that a predetermined digest reproduction operation using the feature extraction data has started, and an efficient and effective operation can be performed.
[0017]
In another method described with reference to FIG. 2, when a playback section is long, a feature is extracted as time elapses, and the feature is extracted within a few seconds from the start of processing (see FIG. 2). 2 (A)), repeating the feature extraction again t seconds after the start of processing (FIG. 2B), and finally extracting the feature near the end of the predetermined section (FIG. 2C). Like that. In this way, for those whose reproduction time is not known, features may be extracted every time a predetermined reproduction time elapses.
In the description of FIG. 1 and FIG. 2 described above, the normal reproduction is performed simultaneously during the feature extraction processing period. However, the present invention is not limited to this, and the feature extraction processed when the feature extraction processing for a predetermined section is completed. A predetermined digest reproduction operation using data may be started. In such a process, since the predetermined digest operation does not start until the predetermined feature extraction process is completed, a display indicating that the feature is being extracted may be displayed to the user.
[0018]
When the software is played again, it is determined that the software is selected. When the summary playback mode is set, the ID is read to quickly perform special playback such as effective summary playback. Can be migrated.
[0019]
If ID processing has already been performed, the processing is not performed and the presence of ID is displayed on a predetermined display means.
[0020]
Further, instead of storing the ID in a predetermined recording medium in the apparatus, the ID is stored in a predetermined storage unit (a case, a predetermined recording area of the recording medium, etc.) of the recording medium itself, and the special reproduction mode operation at the time of reproduction is The same processing as described above is performed.
[0021]
If the ID process has already been performed, the above feature extraction process is not performed, and the fact that there is an ID is displayed on a predetermined display means.
[0022]
When a recording medium on which a predetermined image and sound is recorded is mounted on the apparatus, if the digest ID processing is not performed, the ID processing is automatically performed by high-speed playback.
[0023]
Hereinafter, a first embodiment of the present invention will be described in the following order with reference to the drawings.
[0024]
[1] Operating principle
[2] Whole block configuration example
[3] Operation flowchart
[0025]
[1] Operating principle
(1.1) Characteristic signal detection method
Here, for the sake of simplicity, it is assumed that the characteristic signal is a video / audio signal compressed by MPEG and a predetermined characteristic signal is detected from the MPEG stream.
[0026]
For example, as shown in FIG. 3, an I picture is detected from an image sequence, a characteristic vector is generated by dividing the DC coefficient of a DCT (Discrete Cosine Transform) for each I picture, and the characteristic vector of the characteristic vector is generated. By calculating the distance, it is also possible to detect a motion vector in a P picture and perform similar image detection and cut point detection.
That is, as shown in FIG. 3, the frame is divided into, for example, 25 areas, the average value of the DC coefficient of the luminance signal y and the color difference signals cb and cr is calculated for each area, and the luminance signal y using the calculated value as a component. Characteristic vector Vy and color difference signals cb, cr characteristic vectors Vcb, Vcr are calculated, and a vector distance is calculated using a characteristic vector at a predetermined time as a reference vector. Here, even if the number of divided areas of the frame is not 25, for example, it may be slightly increased as 36 areas or slightly decreased as 16 areas. The number of divisions may be set according to processing time, processing accuracy, and the like.
[0027]
(1.2) Method for detecting a predetermined recording medium
If there is a label, the characteristic signal (characteristic vector) detected as described above is stored in a recording medium in the apparatus together with the label so that it can be determined that the characteristic signal belongs to a predetermined recording medium.
[0028]
If there is no label on the recording medium, or if the label is duplicated, it can be assumed that if the data of the intro part of the recording medium is stored in the recording medium in the device, the recording medium is attached to the device. In this case, the recording medium can be determined by matching the data of the intro portion.
[0029]
In some cases, a list of recording media already processed by the apparatus may be displayed on the display system 13 of a block diagram shown later together with a label or title, if there is a label or title.
[0030]
[2] Whole block configuration example
As shown in FIG. 4, the information signal processing apparatus includes a recording medium system 1, a reproduction processing system 2 that reproduces information recorded in the recording medium system 1, and a reproduction data separation processing system that separates reproduced data. 3, a video decoding processing system 4 that decodes video data out of the separated data, a video D / A processing system 5 that converts the decoded video signal into an analog signal and outputs the video signal, and a recording medium system 1 A recording medium detection system 6 for detecting the recording on the recording medium, a reproduction control system 7 for controlling the reproduction of the recording medium system 1, and an identification signal for detecting an identification signal among the data separated by the reproduction data separation processing system 3 The detection system 8, the audio decoding processing system 9 that decodes the audio signal among the data separated by the reproduction data separation processing system 3, and the decoded audio signal are converted into analog values and output as an audio signal. Feature extraction system 11 for extracting features from voice D / A processing system 10, video decoding processing system 4 and audio decoding processing system 9, recording medium detection system 6, reproduction control system 7, identification signal detection system 8, feature extraction The system 11 includes a system controller 12 that controls the system 11 and the display system 13, a display system 13 that is displayed under the control of the system controller 12, a user input I / F system 14 through which a user inputs a command and the like, and a memory system 15.
[0031]
In such a configuration, the recording medium detection system 6 detects that a predetermined recording medium is installed, starts reproduction of the intro part, and whether the predetermined program of the recording medium is subjected to ID processing, or the memory system 15 It is determined by comparing with the data stored in
In the case where the recording medium system 1 is a recording medium such as a hard disk installed in the apparatus, a predetermined program recorded by the user is set in a reproduction operation state, and an intro part or predetermined information is recorded. The reproduction of the TOC (Table of contents) portion is started, and it is determined whether the predetermined program has been subjected to ID processing by comparing it with data stored in the memory system 15.
[0032]
When the ID process is not performed, the ID process is performed simultaneously with the normal reproduction or the high-speed reproduction.
[0033]
The audio / video data reproduced from the recording medium is extracted by the feature extraction system 11 after being decoded and stored in the memory system 15 via the system controller 12.
In the case where the recording medium system 1 is a recording medium such as a DVD or CD (CD-RW) (CD Rewritable) that can be recorded by the user, or a recording medium such as a hard disk installed in the apparatus. Instead of the predetermined recording area in which the image / audio information is recorded, a predetermined recording area in which data recording is permitted can be provided, and the predetermined data can be recorded instead of recording in the memory system 15. .
[0034]
In the special reproduction mode, the characteristic data is appropriately called from the memory system 15 by the user command input, and the reproduction control system 7 is controlled by the system controller 12 to perform a predetermined digest reproduction.
[0035]
[3] Operation flowchart
The operation of the information signal processing apparatus having the above configuration will be described below with reference to the flowcharts shown in FIGS.
[0036]
First, when a recording medium is detected, data is read from the memory system 15 (steps ST11 and ST12).
[0037]
Then, the reproduction of the intro part starts, and the comparison process reproduction count is determined (steps ST13 and ST14).
[0038]
If it is the first time, the fact that it is the first time is displayed, mode 0 (Mode = 0) is set and the number of times is written to the memory (steps ST15, ST16, ST17, ST18).
[0039]
If it is not the first time in step ST15, the number of times of being the second or more is displayed (step ST19).
[0040]
Then, mode 1 (Mode = 1) is set and the number of times is written in the memory (steps ST20 and ST21).
[0041]
Then, the reproduction mode input is waited for, and when there is no input and the operation is stopped, the reproduction mode is stopped (steps ST22, ST23, ST24, ST25).
[0042]
In step ST23, if there is an input, it is determined whether or not digest playback is to be performed next. If the mode is digest playback and mode 1 (Mode = 1), ID reading of the memory system 15 is read. Performed (steps ST26, ST27, ST33).
[0043]
In step ST27, in the case of mode 0 (Mode = 0), next, high-speed playback processing is performed, characteristic detection processing is performed, ID processing is performed, and data is written in the memory system 15 (steps ST28, ST29, ST30, ST31). .
[0044]
Next, when the processes of steps ST28 to ST31 are not completed, the process goes to step ST28, and when the processes are completed, the ID is read into the memory system (steps ST32 and ST33).
[0045]
Next, referring to FIG. 7, a frame number detection process is performed, and a memory read process is performed (steps ST41 and ST42).
[0046]
Next, digest ID frame comparison processing is performed (step ST43). If it is not a digest frame, playback processing is performed. If it is a digest frame, skip processing is performed (steps ST44, ST45, ST46).
[0047]
Next, the process goes to step ST37 shown in FIG. 6 to perform normal playback mode processing.
[0048]
In step ST26 shown in FIG. 5, if the digest reproduction is not performed, the process goes to step ST34 shown in FIG. 6, and after the ID is read from the memory system 15, the digest reproduction process is performed. If not, it is determined whether there is a mode change (steps ST34, ST35, ST36).
[0049]
If there is no mode change in step ST36, the process goes to step ST34 to perform digest playback again, and if there is a mode change, the normal playback mode is processed (step ST37).
[0050]
After the normal playback mode processing, the process stops in the case of stop, and in the case of no stop, the normal playback mode process of step ST37 is performed (steps ST38 and ST39).
[0051]
In step ST39, if there is a mode change, if mode 0 (Mode = 0), go to step ST28 shown in FIG. 5 to perform high-speed playback processing, and if mode 1 (Mode = 1). Then, the process goes to step ST33 shown in FIG. 5 to read the ID into the memory system 15 (steps ST40 and ST33).
[0052]
Next, referring to FIG. 7, a process for detecting the frame number is performed (step ST41).
A reading process is performed from the memory system 15, and a digest ID frame comparison process is performed (steps ST42 and ST43).
[0053]
If there is no digest frame, playback processing is performed. If there is a digest frame, skip processing is performed, and normal playback mode processing in step ST37 shown in FIG. 6 is performed (steps ST44, ST45, ST46).
[0054]
[4] Device display example
FIG. 8 shows a display example of the apparatus, which displays the number of reproductions of the recording medium, whether digest ID processing has been performed, and the like.
[0055]
Such a display may be a display for displaying a reproduced image or a display system such as a liquid crystal display provided in the apparatus main body.
[0056]
FIG. 8A shows a case where a plurality of discs such as a changer can be mounted and reproduced in the apparatus, and it can be displayed whether ID processing is performed for each disc. In FIG. The number of reproductions of 1 is 1, the title is “movie”, and a display in which digest ID processing is performed is made. Disc No. The number of playbacks of 2 is 0, the title is “music”, and the display without digest ID processing is made. Disc No. The number of reproductions of 3 is 2, the title is “sports”, and the display without digest ID processing is made.
[0057]
In this case, a disk may be designated to display whether ID processing is performed for each program of the disk, as in the single disk playback device shown in FIG.
[0058]
FIG. 8 (2) shows the case of the single disk playback apparatus, and displays whether ID processing is performed for each program. For example, the disc No. The number of reproductions of 1 is 1, the title is “movie”, and a display in which digest ID processing is performed is made. Disc No. The number of playbacks of 2 is 0, the title is “music”, and the display without digest ID processing is made. Disc No. The number of reproductions of 3 is 2, the title is “sports”, and the display without digest ID processing is made.
[0059]
Next, an information signal processing device and an information signal processing method according to a second embodiment will be described with reference to the drawings.
[0060]
First, an outline of the information signal processing apparatus and the information signal processing method of the second embodiment will be described.
Characteristic data in an image scene is input in time series, and predetermined calculation processing is performed between the already stored characteristic data and this input characteristic data, and a predetermined scene ID is obtained from the result and motion vector information. Assign to the image scene. Each time one process is completed, the input data area is sequentially shifted.
[0061]
Hereinafter, a second embodiment of the present invention will be described in the following order with reference to the drawings.
[1] Operating principle
[2] Block configuration example
[3] Operation flowchart
[0062]
[1] Operating principle
FIG. 9 (1) shows a data memory system for storing characteristic data, in which Da is stored in all recording areas as initial data. In this case, the first input data D0 is also stored. good.
[0063]
FIG. 9B shows a state in which Aa is stored in all recording areas as initial data in the ID memory system. In this case, 0 may be used as the initial data.
The address of the data memory system shown in FIG. 9 (1) and the address of the ID memory system shown in FIG. 9 (2) have a one-to-one correspondence. The ID data stored in the address of the ID memory system corresponding to the system address is read out, and this ID data is assigned to the image frame on which the characteristic data is input and the arithmetic processing is performed.
[0064]
Here, for the sake of simplicity, it is assumed that characteristic data is generated from MPEG image data and is input to the data memory system for each I picture.
[0065]
As shown in FIGS. 9A and 9A, when the first I picture is detected, the characteristic data D0 is stored in the first memory area.
[0066]
Using D0 as reference data, a predetermined vector distance calculation is sequentially performed with data in other memory areas as shown in the figure.
[0067]
After a predetermined calculation, the value Vd that minimizes the calculation result is compared with a predetermined threshold value Th,
Vd <Th Formula (1.1)
In this case, ID data in the ID memory area corresponding to the smallest area is assigned to D0, and the ID data is stored in the memory area of the ID memory system corresponding to the memory area of D0.
[0068]
In this case, since D0 is still the first area, when 0 is set as the initial ID, 0 is assigned as ID data.
[0069]
Vd> = Th Formula (1.2)
In this case, data obtained by adding 1 to the maximum value IDmax of data values stored in the ID memory system is used as the ID.
[0070]
However, in the above case, the input data is not yet input until the entire area of the ID memory system is entered, and therefore, the expression (1.2) does not occur.
[0071]
When the next I picture is detected, the memory area is shifted by one as shown in FIGS. 9 (1), (a2), and (a3), and processing similar to the above is performed.
[0072]
If the data is stored in the entire memory area as shown in FIGS. 9 (1) and (a4), the next I picture detection returns to the first memory area as shown in FIGS. 9 (1) and (a5).
[0073]
The same applies to the ID memory system, and calculation processing and IDs are assigned in this way.
[0074]
FIG. 10 shows a recorded image series, where there is a camera pan in the sections a to b, the sections b to c, the sections c to d, and the sections d to, and the time points a, b, c, and d are cut points. Suppose a scene change occurs.
[0075]
When considering image classification based on a person or object, for example, when a person or object is moving or there is a camera pan, the image is determined to be a similar image, or another image There is no absolute criteria for whether or not, and it depends on the user's own preference.
[0076]
Here, when the ID processing described above is performed in such an image series, a different ID may be assigned because the image scene moves in the camera pan sections a and b.
[0077]
In other words, it may be better to assign the same ID for classification only by moving the image.
[0078]
Therefore, if the average scalar value Md of the predetermined motion vector is smaller than the predetermined threshold Mth even if the ID is different from the previous time at a certain time using the motion vector, the ID of the previous I picture The same ID is assigned.
[0079]
By processing in this way, the same predetermined ID is assigned to similar image frames even if the image scene moves slightly across the section of the cut point, and the ID is used according to the user's preference. It can also be applied to retrieval of similar images, image organization, digest reproduction of similar images, and the like.
[0080]
FIG. 11 is a block configuration example of the classification processing system and the ID generation system, and the arithmetic processing is performed by the system controller.
The configuration includes a characteristic data detection system 30 for inputting image data and detecting characteristic data, a characteristic data memory system 31 for storing characteristic data, and characteristic data and characteristic data detection system 30 read from the characteristic data memory system 31. An arithmetic processing system 32 that performs a comparison operation with the characteristic data detected in step 1, an identification information data generation system 33 that generates identification information data, an identification information data memory system 34 that stores identification information data, and these systems. And a system controller 35.
[0081]
In the characteristic data detection system 30, the characteristic data is detected by dividing the data as shown in FIG.
Here, the characteristic data is obtained by dividing the frame into a plurality of areas as described in the first embodiment [1] principle of operation (1.1) of the present invention, and the DC coefficient of the DCT for each divided area. Considering a vector having an average value as a component, the component of the vector can be used as characteristic data. However, as another method, the image frame is wavelet transformed, and the luminance signal and the color difference signal are subjected to multi-resolution analysis. As shown in the figure, the HH region subjected to the multi-resolution analysis is divided into, for example, 36, an average value of wavelet coefficients is calculated for each region, and a vector having the calculated value as a component can be assumed. Thinking in the same way as the case, vector components can be used as characteristic data. That is, the average value of DC coefficients of DCT and the average value of wavelet coefficients for each divided region are used as characteristic data.
Here, the number of area divisions shown in FIG. 12 is not limited to 36, and as described in the first embodiment [1] operating principle (1.1) of the present invention, depending on the case, Since the number of divided areas and the order of characteristic data are the same, the order of characteristic data may be 25th and 16th according to the number of divided areas.
[0082]
The generation of the identification information data in the identification information data generation system 33 is performed as shown in FIG. 13 when the ID is returned to a predetermined value in the ID data return processing.
That is, if the initial value of the ID data is set to min (0000H) and a similar image is not subsequently detected by a predetermined calculation process as described above, the ID increases by 1 for each predetermined calculation, When the set maximum value max (FFFFH) is reached, the initial value is reset to min.
Here, when the ID is reset as described above and the ID processing is continued, it is considered that the same ID as a certain frame image subjected to the ID processing before is assigned. If max is set to FFFFH, the time interval for assigning the same ID is long, so it is not a problem even if the same ID is assigned even though it is not a similar image to the previous image.
Assuming that the frame rate of the image is, for example, 30 frames / second and MPEG 1 GOP (Group of Pictures) is 15 frames, an I picture of ID processing unit is detected and processed every 0.5 seconds every 15 frames. Even if a different ID is assigned to each I picture, FFFFH = 65535,
65535 × 0.5 (seconds) = 32767.5 (seconds), that is, about 9 hours.
As described above, since the time interval in which the same ID is assigned even though it is not a similar image is long, even if the user notices that it is not a similar image but the same ID, the recording / reproducing apparatus as in this embodiment hardly is not a problem.
[0083]
[2] Block configuration example
As shown in FIG. 14, the configuration of the information signal processing apparatus to which the present invention is applied includes an audio A / D conversion processing system 101 that inputs an audio signal, performs AD conversion, and outputs an audio signal converted into a digital value. An audio encoder processing system 102 that encodes the audio signal, a video signal, a feature extraction signal from the audio signal, and a feature extraction signal from the image signal, and a multiplexed signal for recording the multiplexed signal. A recording processing system 104 that performs the above-described processing, a recording medium system 105 that records the multiplexed signal, a video A / D conversion processing system 106 that inputs a video signal, performs AD conversion, and outputs it, and a digital value. A video encoder processing system 107 that encodes the recorded video signal, a playback control system 108 that performs playback processing on the data recorded in the recording medium system 105, and an audio encoder An audio signal feature extraction system 109 for extracting a characteristic signal from the audio signal encoded by the science system 102; and an image signal feature extraction system 110 for extracting a characteristic signal from the video signal encoded by the video encoder processing system 7. A reproduction processing system 111 that performs reproduction processing of data recorded in the recording medium system 105, a reproduction data separation processing system 112 that separates the reproduced signal into an audio signal, a video signal, and an identification signal, and reproduction data separation processing. Audio decoding processing system 113 for decoding the audio signal separated by the system 112, audio D / A processing system 114 for converting the signal in the audio decoding processing system 113 to an analog value, and video separated by the reproduction data separation processing system 112 Video decoding processing system 115 that decodes the signal, and video that converts the signal in the video decoding processing system 115 into an analog value / A processing system 116, audio signal feature extraction system 109, image signal feature extraction system 110, multiplexing processing system 103, system controller 118 for controlling reproduction control system 108, and user input I / O for inputting a signal from a user An F system 119 and a reproduction mode display system 120 are included.
[0084]
In the information signal processing apparatus having such a configuration, the audio signal is first subjected to predetermined AD conversion by the audio A / D conversion processing system 101, and then subjected to predetermined encoder processing by the audio encoder processing system 102. The processed signal or the encoded signal is input to the audio signal feature extraction system 109 to perform predetermined feature extraction.
[0085]
The same applies to the video signal. After the predetermined AD conversion is performed by the video AD conversion processing system 106, the predetermined encoder processing is performed by the video encoder processing system 107, and this AD converted signal or the encoded signal is Then, it is input to the image signal feature extraction system 110 to perform predetermined feature extraction.
[0086]
Via the user input I / F system 119, the characteristic data is preliminarily obtained by the audio signal feature extraction system 109 and the image signal feature extraction system 110 in a predetermined section as described in the above [1] operation principle. The data is stored in a predetermined memory in 118.
[0087]
After that, feature data is sequentially detected, and a predetermined vector distance calculation is performed in the system controller 118 when a predetermined correlation calculation or a vector having each feature extraction data described above as a component is used. .
[0088]
Image / audio data such as a predetermined broadcast program is recorded in the recording medium system 105 through a predetermined multiplexing processing system 103, a recording processing system 104, and the like.
[0089]
In the special reproduction mode, the system controller 118 controls the reproduction data from the recording medium system 105 as described in the above [1] operation principle via the reproduction control system 108.
[0090]
[3] Operation flowchart
A calculation method using a motion vector in the system controller 118 will be described below with reference to flowcharts shown in FIGS.
[0091]
First, as shown in FIG. 15A, the counter n = 0 is set and initial setting processing is performed (steps ST101 and ST102).
In this initial setting process, as shown in FIG. 15B, the motion vector is processed, and the frame number m = 0 and the calculated value V = 0 are set (step ST113).
[0092]
Next, in step ST103 in FIG. 15A, a stream is detected. If it is not an I picture, intra-motion vector frame detection processing is performed (steps ST103, ST104, ST105).
In this motion vector frame detection process, as shown in FIG. 15C, first, it is determined whether or not it is a P picture, and if it is not a P picture, it is confirmed that it is not a stop and the process proceeds to the next step (step ST114). , ST115).
In step ST114, if it is a P picture, a motion vector is detected, and an intraframe average (Vf) is calculated (steps ST114, ST117, ST118).
Then, the average value is calculated by calculating the calculated value V = V + Vf (step ST119).
Next, the frame number m is incremented (m = m + 1) (step ST120).
Then, the process goes to step ST103 in FIG. 15A to detect a stream (step ST103).
[0093]
In step ST104, in the case of an I picture, the average (Vav) of motion vectors is calculated by Vav = V / m (step ST106).
[0094]
Next, ID determination processing is performed (step ST107).
In this ID determination process, as shown in FIG. 16, first, characteristic data (Dt) is generated (step ST121).
When the counter n = 0, initial value setting processing is performed (steps ST122 and ST123).
In the initial value setting process, data buffer initial setting Dt (0) is input, ID buffer initial setting id = 0 is input, and data input position inplace = 0 is input.
[0095]
In step ST122, when the counter n is not 0, the vector distance array is reset and the vector distance calculation is initialized (steps ST124 and ST125).
[0096]
Then, the vector distance minimum value calculation at the distance Vp and the position p is performed (step ST126).
[0097]
Then, the threshold value Vp is determined, and if it is not the threshold value, ID overflow determination processing is performed (step ST129).
In this ID overflow determination process, as shown in FIG. 17A, first, it is determined whether or not the value obtained by adding 1 to the ID buffer id is larger than the ID buffer maximum value idmax (step ST140). If not, the ID buffer id is incremented by 1, and the ID buffer id is set to 0 for a larger multiple, and ID buffer id = IDn is set (steps ST142, ST141, ST143).
[0098]
Returning to FIG. 16, if it is within the threshold value in step ST127, the ID buffer data ID (p) is read out and input to IDn, and the data input position "inplace" is incremented by one (steps ST128 and ST130).
[0099]
If it is over in the position determination, the data input position inplace is set to 0, and if the position is within the memory position, it is left as it is, and the data buffer Dt of the data input position inplace is stored (steps ST131, ST132, ST133).
[0100]
Further, ID (p) is stored in the data input position inplace in the ID buffer (step ST134).
[0101]
Then, the motion vector (Vav) is determined and the threshold value Vp is determined again (step ST135).
[0102]
If it is other than the threshold Vp, pre-process ID assignment, IDnv = IDpre is set, and if it is within the threshold, scene ID (motion vector correspondence) is assigned, IDnv = ID (p), and IDpre = IDn is set in the IDn memory (step ST136). , ST137, ST138, ST139).
[0103]
After performing the ID determination processing in this way, in step ST108 shown in FIG. 15, post-motion vector frame detection post-processing is performed (step ST108).
This post-detection process within the motion vector frame is a motion vector process as shown in FIG. 17B, in which the number of frames m = 0 and the calculated value V = 0 are set (step ST144).
[0104]
After the post-detection process in the motion vector frame of step ST108 in FIG. 15, the scene ID recording process corresponding to the motion vector IDnv is performed (step ST109).
[0105]
Then, increment the counter n and confirm that it is not stopped. Going to detection of the next stream of step ST103 (steps ST110, ST111, ST112).
[0106]
【The invention's effect】
As described above, in the present invention, when the first playback medium is viewed, the outline can be grasped in a short time, so that it can be used as a reference when viewing in the normal playback time. effective.
In addition, since normal reproduction is performed while performing feature extraction processing for digest reproduction, the image and sound during the digest reproduction operation are not interrupted even during the feature extraction processing period, and an effective operation can be performed.
[0107]
Furthermore, since the outline of the content of the reproduction medium is automatically recorded on another recording medium, when searching for various reproduction media and trying to view the reproduction medium again, the outline is understood. There is also an effect that the re-operation can be omitted.
[0108]
Also, when the recording medium is inserted, the digest reproduction ID processing is automatically performed at a high speed, so that the normal operation is saved and it becomes a reference for normal reproduction.
[0109]
In addition, when recording broadcast programs, etc., classification processing can be performed in real time, and timer recording can be used to know the classification result immediately after the recording is completed, thereby eliminating the waiting time for processing. There is an effect that can be.
[0110]
Furthermore, there is also an effect that image classification and rearrangement, editing, digest reproduction, and the like can be performed using the scene ID subjected to classification processing.
[Brief description of the drawings]
FIG. 1 is an explanatory diagram showing an operation principle of a first embodiment according to the present invention.
FIG. 2 is an explanatory diagram showing an operation principle of the first embodiment according to the present invention.
FIG. 3 is an explanatory diagram showing an operation principle of the first embodiment according to the present invention.
FIG. 4 is a block diagram showing the configuration of the information signal processing apparatus according to the first embodiment of the present invention.
FIG. 5 is a flowchart showing the operation.
FIG. 6 is a flowchart showing the operation.
FIG. 7 is a flowchart showing the operation.
FIG. 8 is an explanatory diagram showing a display example of the same device.
FIG. 9 is an explanatory diagram showing an operation principle of a second embodiment according to the present invention.
FIG. 10 is an explanatory diagram showing an operation principle of a second embodiment according to the present invention.
FIG. 11 is a block diagram of an image separation processing system according to a second embodiment of the present invention.
FIG. 12 is an explanatory diagram when selecting a divided region on the same multi-resolution analysis plane;
FIG. 13 is an explanatory diagram of the identification information data return processing;
FIG. 14 is a block diagram showing a configuration of an information signal processing apparatus according to a second embodiment of the present invention.
FIG. 15 is a flowchart showing the operation.
FIG. 16 is a flowchart showing the operation.
FIG. 17 is a flowchart showing the operation.
Device display example
[Explanation of symbols]
1; recording medium system, 2; reproduction processing system, 3; reproduction data separation processing system, 4; video decoding processing system, 5; video D / A processing system, 6; recording medium detection system, 7; Identification signal detection system, 9; audio decoding processing system, 10; audio D / A processing system, 11; feature extraction system, 12; system controller, 13; reproduction mode display system, 14; user input I / F system, 15 Memory system, 30; characteristic data detection system, 31; characteristic data memory system, 32; arithmetic processing system, 33; identification information data generation system, 34; identification information data memory system, 35; system controller, 101; D conversion processing system 102: Audio encoder processing system 103: Multiplexing processing system 104: Recording processing system 105: Recording medium system 106: Video A / D conversion processing system 107: Video encoder Science system 108; Playback control system 109; Audio signal feature extraction system 110; Image signal feature extraction system 111; Playback processing system 112; Playback data separation processing system 113; Audio decoding processing system 114; Audio D / A processing system, 115; video decoding processing system, 116; video D / A processing system, 117; identification signal detection system, 118; system controller, 119; user input I / F system, 120;

Claims (4)

Reproducing means for reproducing the first recording medium on which the image / audio signal is recorded;
Recording means for extracting features of the image and sound from the image and sound signal reproduced by the reproducing means, creating a digest reproduction ID based on the extracted image and sound, and recording the digest reproduction ID on a second recording medium; With
In the extraction of the feature of the image and sound, the information signal is characterized in that the feature is extracted within the normal reproduction time of the section a including the intro part smaller than the section H to be reproduced normally. Processing equipment. Reproducing means for reproducing the first recording medium on which the image / audio signal is recorded;
Recording means for extracting features of the image and sound from the image and sound signal reproduced by the reproducing means, creating a digest reproduction ID based on the extracted image and sound, and recording the digest reproduction ID on a second recording medium; With
When the playback sections H1, H2, and H3 are long , the feature extraction of the video and audio is performed for the playback section H1 within the normal playback time of the playback section b1, which is much shorter than the playback section H1, For the next playback section H2, features are extracted within the normal playback time of the playback section b2 that is much shorter than that, and for the next playback section H3, the normal playback time of the playback section b3 that is much shorter than that is extracted. An information signal processing device characterized by extracting features within the information signal processing device. Playing back the first recording medium on which the image / audio signal is recorded,
Extracting the feature of the image and sound from the image and sound signal reproduced by the reproduction means, creating a digest reproduction ID based on the image and sound from which the feature is extracted, and recording it on the second recording medium,
The information audio processing method is characterized in that the feature extraction of the image and sound is performed within a normal playback time of a section a including a smaller intro portion than a section H for normal playback . Playing back the first recording medium on which the image / audio signal is recorded,
Extracting the feature of the image and sound from the image and sound signal reproduced by the reproduction means, creating a digest reproduction ID based on the image and sound from which the feature is extracted, and recording it on the second recording medium,
When the playback sections H1, H2, and H3 are long , the feature extraction of the video and audio is performed for the playback section H1 within the normal playback time of the playback section b1, which is much shorter than the playback section H1, For the next playback section H2, features are extracted within the normal playback time of the playback section b2 that is much shorter than that, and for the next playback section H3, the normal playback time of the playback section b3 that is much shorter than that is extracted. An information signal processing method characterized in that features are extracted from within .

Images