JPH09247617A - Video signal recording/reproducing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate discrete video signals by thinning video signals and to increase the density of the video signal of a part at which a characteristic point or a singular point is extracted. SOLUTION: When a voice recognition circuit 1 extracts a singular point of a voice signal separated from an input composite video signal and a motion vector detection circuit 2 extracts a characteristic point from the video signal, an interrupt signal d1 is fed to a gate signal generating circuit 6. A time lapse command signal generating circuit 7 gives a command signal d2 to set a prescribed interval to the gate signal generating circuit 6 based on a synchronizing signal from a synchronizing signal separator circuit 3 and a synchronizing signal generating circuit 4. A gate circuit 5 provides a discrete video signal generated by the gate signal from the gate signal generating circuit 6. The voice recognition circuit 1, the motion vector detection circuit 2, the time lapse command signal generating circuit 7 and a time lapse recording/reproducing medium 8 are controlled by a microprocessor 9.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention is to confirm the contents of a large amount of various image information in a short time, which is mainly represented by television broadcasting, digital television broadcasting, and video server. The present invention relates to a video signal recording / reproducing device capable of performing the above.

[0002]

2. Description of the Related Art Today, information collectively called television programs and multimedia is inundated, and this tendency tends to increase day by day. If there are people's original desire to concentrate on watching programs that are useful, entertaining, and entertaining by deleting programs that the viewers generally deem to be of low value, but sometimes they miss those programs. There is. Even in the case of image information, which is generally referred to as package media, there is a reality that many information that is flooded due to lack of time cannot be fully accessed and utilized. As means for avoiding these, there are means such as a program guide, a newspaper, or a preview. There is also a means to record it on video tape and use it at another free time.

In addition, a fast playback function is known for saving time. On the recording side, there is a digest version in which the original source is subjected to some kind of secondary processing, and such a digest version is often used in sports programs on television. However, this digest version requires a long time for editing work, and a general household user does not have such a facility, and there is a limit in utilizing it at home.

As one of means for solving these problems, there is a means usually called a ring buffer. For example, a continuous image can be recorded for 60 seconds and the image can be viewed retroactively when necessary. As an example of this ring buffer, a hard disk, a semiconductor memory (flash memory, IC memory, DRAM, ferroelectric memory), or the like is used. There is a system used as a recording / reproducing medium.

Although this system is a means for interpolating an oversight, it limits the recording / reproducing time depending on the capacity of the memory used and cannot always secure a necessary and sufficient recording / reproducing time.

[0006] At present, there is a tape streamer that can be used in the market and is viewed from the viewpoint of obtaining more recording capacity. This tape streamer is based on a magnetic recording tape, 2.5 gigabytes, and based on optical recording (a type that uses a medium in which an optical recording medium is coated on a tape-shaped base film). The capacity of is realized. By using such a means, it is possible to record for a longer time. However, in the case of the search, access is time-consuming and complicated even if there is a means such as using a data code for the search. For this reason, it is not generally used except for special cases, and is used only for computer data backup.

Further, in a system using a hard disk, a system is constructed which secures a large capacity and emphasizes easy accessibility of the disk. In order to realize this large capacity, there is a system in which a plurality of small capacity disks are configured in a stack and used as a video server. This system is based on RAID (Redandant Array of In
Currently used as expensive disks).

Further, in order to secure desired information,
It is necessary to reduce the redundancy of images, and for this purpose, image processing techniques such as motion vector detection are used to detect scene changes in images and to use them for cut-based editing and browsing (display). There is.

As a well-known method for detecting this motion vector, there is a block matching method. As shown in FIG. 7, this block matching method uses a reference frame 51 and a search frame 53 to subtract all the pixels of the image (m × n) between the inspection block 55 in the search area 54 and the reference block 52. Done,
The absolute value is obtained, the product sum operation is executed, the minimum value of the residual is obtained within the search range, and the relative position is used as the motion vector.

Second, there is the integral shadow method. This integral shading method simplifies the arithmetic circuit by shading two-dimensional data in one dimension. Take the full addition of horizontal pixels and vertical pixels of all pixels of the inspection block size,
As described above, the minimum residual value is obtained and used as the motion vector.

[0011]

It is impossible to access all the information transmitted at one time, such as television broadcasting and digital television broadcasting, but it is impossible to store all the information once. If there is a recording / reproducing medium to be used, the utilization of such information can be expanded. From this point of view, if a means for selectively recording desired information in the process of recording a video signal and deleting a redundant portion and not recording it is installed in a recording / reproducing device, it will be a short time in a flooded information society. You can access more information at and take advantage of them.

Conventionally, there has been a so-called short-time video in which the time for viewing is shortened, but this video originally records all too much information. Therefore, the ability to access necessary information selectively and generally in a short time has been limited. For this reason, it is hard to say that this short-time video is suitable for the original way of utilizing information. Therefore, the present invention realizes an information collecting means that has an information selecting function at the time of recording and a thinning function of a video signal by the information selecting function and overviews the whole information in a short time.

Further, it is difficult to realize only by the image processing technique in order to execute the scene change detection in which the redundancy of the image is reduced and the desired and appropriate information is secured intensively. The algorithm for ensuring the necessary and sufficient functions becomes complicated and the device becomes huge. Therefore, at present, there are no cases of commercial operation. Therefore, the present invention intends to realize a device for practical use by thinning out spatial images in the time axis direction as a redundancy reducing means and interpolating insufficient elements by image processing technology and voice recognition technology. It is a thing.

An algorithm for detecting or extracting a scene of an image used at this time is a system including flexibility and flexibility in consideration of convenience of viewing or utilizing information recorded later. It should be and should have the user option function. In other words, in the video of a certain genre like video on demand,
Areas of particular interest to the user should be focused on, even if they become redundant. Therefore, the present invention realizes a function in which an algorithm for feature extraction can be adaptively changed in association with a keyword for scene change extraction that can be requested in advance.

Accordingly, an object of the present invention is to thin out a large amount of image information from a wide variety of images in a short period of time, and to decimate the images at regular intervals. One scene is characterized by thinning out at variable intervals. Furthermore, by selecting less mediocre images from redundant information, the function to extract highlight scenes with priority, and by recording / playing back those It is an object of the present invention to provide a video signal recording / reproducing apparatus capable of giving an overview of all information.

[0016]

According to a first aspect of the present invention, a video signal recording / reproducing for limiting an image to be recorded on a recording / reproducing medium by temporally and spatially extracting a part of the video signal. In the device, a plurality of time series fixed time intervals adapted to the genre of the video signal or a keyword, or a specific sequence unit combining a plurality of time series is extracted, and the extracted video signal is recorded on a medium. And a video signal recording / reproducing device.

A part of the image is automatically thinned from the input video signal at a constant time interval. For example,
Decimate a part of a field / frame from a binary arranged interval. On the other hand, there are highlight scenes in sports competitions that compete for performance within a certain period of time, and some highlight scenes can be predicted in advance. The intervals are combined to increase the recording density in a specific area due to such elapsed time. Further, in an unpredictable highlight scene, since there is a peculiar spectrum and volume as represented by the sound of the pistol at the start, a voice recognition means is also used to make the interval thinning variable. Further, the specific scene in the input image is extracted by the keyword selected by the user on the menu screen, and the thinning time of the image is made variable in the extracted specific scene. These specific restricted appropriate scenes are recorded on the record / playback medium.

[0018]

BEST MODE FOR CARRYING OUT THE INVENTION An embodiment of the present invention will now be described in detail with reference to the drawings. An embodiment of the video signal recording / reproducing apparatus of the present invention is shown in FIG. A signal selected from a plurality of television broadcasts is input. The input signal is a composite video signal in which the audio signal and the video signal are composited. The input composite video signal is supplied to the voice recognition circuit 1, the motion vector detection circuit 2, the synchronization signal separation circuit 3 and the gate circuit 5. In the voice recognition circuit 1,
As will be described later, the singular point is extracted and its volume is also analyzed. The analysis result is supplied to the microprocessor 9. The motion vector detection circuit 2 is controlled by the microprocessor 9 to detect the motion vector of the video signal separated from the input composite video signal as described later. A feature point is extracted from the detected motion vector, and the interrupt signal d1 is sent to the motion vector detection circuit 2 according to the feature point.
Is supplied to the gate signal generation circuit 6.

The sync signal separation circuit 3 separates the sync signal from the supplied input video signal, and the separated sync signal is supplied to the sync signal generation circuit 4. The sync signal generation circuit 4 generates a sync signal in response to the supplied sync signal. The generated sync signal is the sync signal generation circuit 4
Is supplied to the time-lapse command signal generation circuit 7. The time-lapse command signal generation circuit 7 is controlled by the microprocessor 9 and a command signal d2 after a predetermined time has elapsed from the supplied synchronization signal is supplied to the gate signal generation circuit 6.

The gate signal generating circuit 6 generates a gate signal based on the interrupt signal d1 and the command signal d2. For example, in the gate signal generation circuit 6, a gate signal is generated by the logical sum of the interrupt signal d1 and the command signal d2. The generated gate signal is supplied to the gate circuit 5. In the gate circuit 5, the input composite video signal is gated in a unit of an integral multiple of the field or frame corresponding to the supplied gate signal. The output signal from the gate circuit 5 is supplied to the time-lapse recording / reproducing medium 8 as a discrete video signal. That is, the gate circuit 5 performs sampling of the video signal at a controlled interval, thereby thinning out in the time direction. The time-lapse recording / reproducing medium 8 is controlled by the microprocessor 9 and records / records the supplied input video signal.
Playback is performed.

As described above, the microprocessor 9 controls the motion vector detection circuit 2 and the time lapse command signal generation circuit 7, and the feature points extracted from the video signal and the singular points of the audio signal are constantly checked. Specifically, if there is a desired feature point or singular point, its output (interruption signal d1 and command signal d2) is output from the OR gate circuit 6.
To either of the signals d1 and / or d2
The output of the gate circuit 5 is gated by a unit of an integral multiple of a field or a frame. The gated output becomes a discrete video signal, which is supplied to the time-lapse recording / reproducing medium 8 and recorded. Further, the video signal reproduced from the time-lapse recording / reproducing medium 8 is displayed on the television monitor 10. A menu screen 11 is also displayed on the television monitor 10 as described later.

Next, a detailed block diagram of the motion vector detection circuit 2 according to the present invention is shown in FIG. The motion vector detection circuit 2 includes a horizontal / vertical pixel thinning circuit 21, a horizontal / vertical spatial filter 22, a projection circuit 23,
It is composed of a correlation detection calculation circuit 24, a control / discrimination circuit 25, a field memory 26 and a line memory 27.
First, the video signal separated from the input composite video signal is supplied to the horizontal / vertical pixel thinning circuit 21. In the horizontal / vertical pixel thinning circuit 21, the pixels of this video signal are thinned out in half, for example, both horizontally and vertically. Pixels are decimated in half horizontally and vertically, that is, the number of pixels is 1 /
The video signal of 4 is subjected to the image processing technique in the horizontal / vertical spatial filter 22, that is, subjected to the two-dimensional spatial filter, and supplied to the projection circuit 23.

The projection circuit 23 uses the data in the field memory 26 and the line memory 27 to detect a search area or the like for detecting a motion vector in the image. The line memory 27 is, for example, a memory for 1H. In the correlation detection calculation circuit 24, as will be described later, for example, the correlation detection of two types of images is used together. The calculation result is supplied from the correlation detection calculation circuit 24 to the control / determination circuit 25. The control / discrimination circuit 25 controls the motion vector detection circuit 2 according to an instruction from the microprocessor 9. further,
The control / determination circuit 25 determines whether or not the supplied calculation result is a motion vector, and when it is determined that it is a motion vector, the interrupt signal d1 is supplied to the gate signal generation circuit 6.

A block diagram of the speech recognition circuit 1 of the present invention is shown in FIG. The input input composite video signal is supplied to the voice recognition circuit 1. The voice recognition circuit 1 includes a voice separation circuit 31 and a DSP (FFT) spectrum /
It is composed of a volume analysis circuit 32. The input composite video signal supplied to the voice recognition circuit 1 is supplied to the voice separation circuit 31. The audio separation circuit 31 separates the audio signal from the supplied input composite video signal, and the separated audio signal is supplied to the spectrum / volume analysis circuit 32. In the spectrum / volume analysis circuit 32, the DSP (Digital
FFT (Fast Fourier T
ransformatiom) conversion, and a distinctive and representative spectrum is detected from the supplied audio signal. A singular point is extracted from the detected spectrum and its volume is also analyzed. The analysis result is supplied from the voice recognition circuit 1 to the microprocessor 9. The microprocessor 9 controls the time-lapse command signal generation circuit 7 based on the analysis result. As an example, the thinning time interval of an image and the time interval (interval) between fields / frames are controlled.

An example of the menu screen 11 is shown in FIG. First, in the genre 41, as shown in the figure, each genre of sports is displayed. For example, when golf is selected from this sports genre, keywords related to golf are displayed in the keyword 42 as shown in the figure. Then, a desired keyword is selected from the keywords 42, and an image related to the selected keyword is displayed on the television monitor 10. At this time, it is also possible to select a plurality of keywords. As an example of the displayed video signal, a standard time series of the input video signal is shown in FIG. 5A.

Then, the desired interval is set in consideration of the keyword selected using the time lapse time 43. An interval is set by the time lapse time 43, and an example of a time series of discrete video signals is shown in FIG. 5B. In this time-lapse time 43,
The interval is considered and displayed according to the genre of the video signal, for example, the type of sport. The user selects a substantially optimum initial value of the plurality of displayed intervals in a menu format.

Further, the microprocessor 9 judges the analysis result from the voice recognition circuit 1 and the interrupt signal d1 from the motion vector detection circuit 2 shown in FIG. 3, the characteristic points are extracted, and the result is set to a desired value. When it arrives, the time series of the discrete video signal shown in FIG. 5B changes into a coarse and fine time series as shown in FIG. 5C.

Further, a time zone selection 45 and a number of frames 46 in which a pre-arranged time interval different from a certain fixed interval is included in the time keeper 44.
Is set using. The pre-arranged time interval is a combination of intervals that increases the recording density in a specific area by the set interval. A time zone is set at a predetermined time using the time zone selection 45, and the required number of fields / frames can be set using the number of frames 46 for each of the set time zones. As a result, an example of the discrete video signal obtained is shown in FIG. 5D. This FIG. 5D shows three time zones tα1 to tα2, tα3 to tα4, tα5 to tα6.
Are set respectively, and the time, the interval, and the required number of frames in the set time zone are set independently.

Further, tβ1 and tβ2-shown in FIG. 5D.
At tβ3, the microprocessor 9 judges the analysis result from the voice recognition circuit 1 and the interrupt signal d1 from the motion vector detection circuit 2 shown in FIG. 3, the feature point is extracted, and the extracted feature point has a desired value. Is an example in which a characteristic scene, that is, a field / frame is added to the video signal set by using the pre-arranged time interval.

As described above, the video signal input as shown in FIG. 5A is set according to the setting of the time lapse time 43.
It becomes a discrete video signal as shown in FIG. 5B. A feature point is extracted according to the analysis result from the voice recognition circuit 1 and the interrupt signal d1 from the motion vector detection circuit 2, and when the extracted feature point reaches a desired value, the The series changes into a coarse and fine time series as shown in FIG. 5C or FIG. 5D. Therefore, a certain time domain from the input video signal is time-lapse recorded in a time-lapse manner, that is, as shown in FIGS. 5B, 5C, and 5D, or at a standard video signal cycle in a relatively short time. / It is recorded on the reproduction medium 8, for example, a hard disk, a time-lapse VTR, or the like.

More specifically, the time-lapse time 4
Although the video signal is controlled to be recorded / reproduced at the interval of the initial value set in 3, the interrupt function operates according to the interrupt signal d1 from the motion vector detection circuit 2. Even if the extracted feature points reach a desired value even if they are operating at the interval (time-lapse mode) set as the initial value by this interrupt function, a scene with the feature is added (variable timing lapse). Mode) operation is automatically switched. And
When the interrupt signal d1 disappears, the time-lapse mode of the original set value is automatically returned.

Here, the above-mentioned motion vector detection circuit 2
The motion vector detection algorithm used for is briefly described. In this motion vector detection, the correlation of two types of images is used together. With ordinary unnormalized correlation,
These are two normalized correlation functions. The former is a product-sum operation of N pixels of the same type as convolution as shown in Expression (1).

[0033]

[Equation 1]

This equation (1) has a high calculation processing speed, but has the drawback that, for example, when the image is uniformly white, the maximum value is detected even if it is not similar at all. On the other hand, the normalized correlation function represented by the equation (2) has an advantage that even if there is a linear conversion (constant gain or offset), it is not affected by the gain or offset.

[0035]

[Equation 2]

However, this equation (2) has a drawback that it takes a long processing time. Therefore, it is controlled by the microprocessor 9 and used properly depending on the case. The microprocessor 9 controls the motion vector detection circuit 2 by an algorithm that takes into consideration the average value, the minimum value of the correlation calculation values and their temporal changes. Here, N used in the equations (1) and (2) is N pixels, R is an original pixel serving as a reference, and M is a pixel in an area where motion should be detected.

From the viewpoint of improving the processing time of the correlation calculation value of the motion vector detection circuit 2, as an example, a search area specialized for each keyword of the sports genre is set. For this purpose, as pre-processing, pixels are thinned out, spatial filtering is applied, high-frequency components of the background are removed, and dilation processing (geometric operation, processing to enlarge the area around the object) is added if necessary. Ferret diameter at
Find X and Y. The ferret diameter X is a horizontal diameter obtained by projecting an image on the coordinate axis X, and the ferret diameter Y is a vertical diameter projected at the coordinate axis Y. Then, here, the value of the Feret diameter and the search area are matched.

Therefore, as shown in FIG. 6, the lengths of the horizontal coordinates X1, X2 and the vertical coordinates Y1, Y2 projected on the coordinate axes X and Y can be known. The coordinates and the detection area and search area of the pattern matching method used for motion vector detection are made to match.

The correlation detection method used in this motion vector detection is expanded to the pattern matching function in this embodiment. For example, the menu screen 1 shown in FIG.
1 is applied to the case of recording on the time-lapse recording / reproducing medium 8 including a specific scene such as a golf putter scene, and a scene similar thereto, from the genre of sports.

As described above, in this embodiment, the algorithm is constructed so that similar highlight scenes related to a specific keyword are approximately extracted. In this case, the geometrical shape in the image of the putter operation, for example, a few typical examples from the address to the follow-through is registered as a reference template, and compared with the input video signal, and similar. The scene is controlled to extract the entire scene even if it is somewhat redundant.

In the case of sports, the camera angle is subject to physical restrictions according to its genre. For example, up and down in ski competition, diagonal in marathon,
As you can see, there are many phenomena and vectors that can be predicted. Therefore, a function is provided that allows the user to specify the detection area while looking at the grid-shaped blocks appearing in the menu.

In particular, in sports competitions and the like, there are highlight scenes, and it is not just information on movement changes. The viewer or the information they want to use should be left to their individual judgment. In other words, how to use information, how to use it, and how to enjoy it are different for each person. Therefore, these programs over a long time are digested in a short time,
When it is desired to have an overview (overview), the necessary decisive scene needs a function of selecting the image of the outline almost automatically based on the individual selection. Therefore, the area of motion area, direction (vector), speed of motion,
Further, attributes such as information about colors are taken into consideration, and a template and its extraction algorithm are installed in advance in the system. Therefore, an appropriate extraction function is realized and the practicality is improved.

As an example of extraction of this highlight scene, in golf, a sharp sound is heard at the moment of hitting a golf ball with a driver, in baseball, a sound of hitting a ball with a bat, and in soccer, when kicking a ball with a corner kick. There is a sharp plosive, or a shrill or loud voice when the announcer yells at a goal, home run, intercept, etc. Using such information of voice and impact sound as a medium,
It is characterized in that it is used as additional information for extracting an accurate feature point / singular point, and is added with a weighted genre, keyword, and the like, a highlight scene, and a phenomenon that occurs in association with movement to the video signal.

In this embodiment, the motion vector detection circuit 2
The gate signal is generated by using the interrupt signal d1 from the CPU and the command signal d2 from the time lapse command signal generation circuit 7, but the motion vector detection circuit 2 and the time lapse command signal generation circuit 7 are controlled by the microprocessor 9. Therefore, the command signal d2 of the time-lapse command signal generation circuit 7 can be directly supplied to the gate circuit 5 to generate a discrete video signal.

[0045]

According to the present invention, it is possible to avoid the inconvenience of the work of looking at a keyword of a certain genre for a long time, and
Multi-faceted and wide-ranging information can be accessed in a short time, and the range of information utilization can be dramatically expanded.
Furthermore, it is possible to selectively extract substantially desired information selected by the user from the input video signal, improve the quality of usable information, and record / reproduce the information.

[Brief description of drawings]

FIG. 1 is an embodiment of a video signal recording / reproducing apparatus to which the present invention is applied.

FIG. 2 is an example of a motion vector detection circuit to which the present invention is applied.

FIG. 3 is an example of a voice recognition circuit to which the present invention is applied.

FIG. 4 is an example of a menu screen to which the present invention is applied.

FIG. 5 is a schematic diagram showing a time series of video signals used for explaining the present invention.

FIG. 6 is a schematic diagram used for explaining motion vector detection to which the present invention is applied.

FIG. 7 is a schematic diagram used to describe a block matching method.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 ... Input video signal selection circuit, 2 ... Motion vector detection circuit, 3 ... Sync signal separation circuit, 4 ... Sync signal generation circuit, 5 ... Gate circuit, 6 ... OR gate , 7 ... Time-lapse command signal generation circuit, 8 ...
Time-lapse recording / playback medium, 9 ... Microprocessor, 10 ... TV monitor, 11 ... Menu screen

Claims (4)

[Claims] 1. A video signal recording / reproducing apparatus for limiting an image to be recorded on a recording / reproducing medium by temporally and spatially extracting a part of the video signal, and adapted to a genre and a keyword of the video signal. A video signal recording / reproducing apparatus, characterized in that a plurality of time series are extracted at fixed time intervals or a specific sequence unit in which the plurality of time series are combined, and the extracted video signal is recorded on a medium. 2. The video signal recording / reproducing apparatus according to claim 1, wherein the means for sampling a part of the video signal that is continuous in the specific sequence unit, and the motion detection information in the video signal, A video signal recording / reproducing apparatus comprising: a means for variably controlling a sampling rate of a specific sequence unit; and a means for recording only a video signal extracted by the sampling means on a recording medium. 3. The video signal recording / reproducing apparatus according to claim 2, further comprising means for extracting characteristic changes such as audio information, motion detection information in the video signal, and the extracted features. The video signal recording / reproducing apparatus characterized in that the sampling rate in the specific sequence unit is changed according to the dynamic change. 4. The video signal recording / reproducing apparatus according to claim 1, wherein the video signal to be extracted can be selected by a menu format, and a related scene associated with the selected video signal is extracted and recorded. A video signal recording / reproducing device characterized in that