JP3158291B2 - Display device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device suitable for use in, for example, editing a video tape.

[0002]

2. Description of the Related Art An outline of a material recorded as a visible image such as a so-called cine film and selection of a desired cut can be performed simply by visually observing the material.

[0003] In the case of a material such as a video tape or a video disk on which moving image data is recorded in an invisible state, to know the outline of the material, a. A method of performing a search or the like (b) A method of reducing a plurality of frames of moving images on a monitor and displaying them in a scrolling manner by multi-screen display (see Japanese Patent Publication No. 61-44437).

In the method (a), it takes more time to confirm the outline of a video tape of, for example, a one-hour TV program, and there is a problem that the editing efficiency is deteriorated.
Further, in the method (b), a short cut such as a TV commercial may be overlooked, and there is a problem that the confirmation is not reproducible and the operator varies.

Therefore, the applicant of the present invention has proposed a system in which an outline of a series of moving image data is displayed as a compressed still image corresponding to the passage of time, and the entire flow of the image can be accurately recognized ( JP-A-2-260075).

[0006]

By the way, for example, in the editing of a video tape, it becomes more convenient if not only the overall flow of an image but also the overall flow of a sound can be recognized.

In view of the above, the present invention provides a display device capable of easily grasping the overall flow of images and sounds.

[0008]

A display device according to the present invention moves a series of two-dimensionally displayed moving image data sequentially to a first display position of a display element at each sampling position in one direction. A first display unit for displaying a still image composed of a slit-shaped image based on image data obtained by sampling one-dimensionally in another direction, and a series of audio data corresponding to the series of moving image data. more, each by images data that is obtained by the sampling
The outline detection means for detecting the outline of the audio data respectively corresponding to the slit-shaped image; and the above-mentioned display at the second display position of the display element at the first display position based on the detection output of the outline detection means. Each slit-shaped image corresponding to each slit-shaped image constituting the still image is corresponded to each slit-shaped image.
The summary of the speech data to respond in which and a second display means for visually displaying.

[0009]

According to the present invention, a still image composed of a plurality of slit-like images showing the overall flow of an image is displayed on the display element, and each of the slit-like images constituting the still image corresponds to each of the slit-like images. Let each slit-shaped
An outline of audio data corresponding to the image , for example, a level, a type, and the like are visually displayed. Therefore, it is possible to accurately recognize the entire flow of images and sounds.

[0010]

An embodiment of the present invention will be described below with reference to the drawings.

FIG. 2 shows processing of moving image data according to the present embodiment. In FIG. 1, reference numeral 1 denotes a video image group as a whole. The video image group 1 can be considered as a sequence of moving images corresponding to a series of moving image data recorded on a video tape, a video disk, or the like, arranged in the unit of frame 2 in the time (t-axis) direction. .

The frame frequency of the video signal is 30 Hz
(NTSC system), 1
30 frames 2 are arranged per second. 2A to 2E
2 shows a series of frames of a video image group 1.

Reference numeral 4 denotes a vertical slit (input slit) for inputting image data set on the frame 2, and an image of the frame 2 is sampled by the vertical slit 4. The vertical slit 4 scans in the horizontal direction (H direction) at a predetermined speed, and when it reaches the right end of the frame 2, it repeatedly scans from the left end in the H direction again. Therefore, the vertical slit 4 is inclined Ht in a three-dimensional space including the time axis.
Scan in the direction.

When the vertical slit 4 scans the video image group 1 from the left end to the right end in the Ht direction, the vertical slit 4 crosses f frames 2, and the vertical slit 4 causes n (normally n = 1) frames. Assume that one slit-like image is sampled for.

When f is selected to be a multiple of n, f = nX is satisfied using a predetermined integer X, and a frame group (3) composed of f frames 2
A, 3B, etc.), each of which samples X slit images.

In this example, the time required for the vertical slit 4 to scan the video image group 1 from the left end to the right end in the Ht direction is set to 12 seconds, n is set to 1, and f is set to 1 X = 12 × 30 = 360.

Then, after the X slit images obtained from the frame group 3A are connected in the horizontal direction, they are compressed in the horizontal and vertical directions respectively to reduce the reduced screen 6.
A is formed. The reduced screen 6A is fitted in the display screen 5 corresponding to one frame memory. Similarly, after the X slit images obtained from the frame group 3B are connected in the horizontal direction, they are compressed and the reduced screen 6
B is formed, and this reduced screen 6B is fitted next to the reduced screen 6A in the display screen 5.

A reduced screen is formed in the same manner even for the following frame groups, and is sequentially fitted in the display screen 5.

Actually, since the slit-like images sampled by the vertical slits 4 are generated one by one in time series, the images are compressed in the order of generation and fitted into the display screen 5 one by one.

For example, the vertical slits 4A to 4E are assigned so as to sequentially scan in the H direction corresponding to the frames 2A to 2E. The images on the slits sampled by the vertical slits 4A to 4E are respectively compressed and the vertical slits (output slits) 7A to 7A to
This is an image of the portion 7E.

As a method of compressing images sampled by the vertical slits 4A to 4E, there are a method of simply thinning out image data and a method of obtaining a weighted average of a predetermined area. In the case of simply thinning out image data, the slits 4A to 4E are H
It may have a width of one pixel in the direction.

An audio display unit 60 is provided below each reduced screen of the display screen 5. In each audio display unit 60,
As shown in FIG. 3 in an enlarged manner, an audio level Ea corresponding to each of the slit-shaped images constituting each reduced screen is displayed. The display of the audio display unit 60 corresponding to each reduced screen is performed in a color corresponding to the type of voice corresponding to each reduced screen, for example, human voice, music, and the like.

FIG. 1 shows a display device of this embodiment. In the figure, reference numeral 8 denotes a host computer. This host computer 8 functions as control means for the entire apparatus.

Reference numeral 11 denotes a system bus, and 12 denotes a keyboard.
And host computer 8 via system bus 11
To give various commands.

Reference numeral 14 denotes a VTR as a moving image data source,
5V, 15A: A / D converter, 16V: Video RAM
(VRAM1) and 16A are memories (ARAM) for audio signals. The memory 16V stores one frame of video signal, and the memory 16A stores one frame of audio signal.

A video signal (for example, Y, RY, BY or R, G, B component signal) output from the VTR 14 is converted into digital data by an A / D converter 15V and then converted to a video RAM 16V. Is written to.
The audio signal output from the VTR 14 is A
After being converted into digital data by the / D converter 15A, it is written into the memory 16A.

The storage area of the video RAM 16V (VRAM1) corresponds to the actual display screen, as shown in FIG. 4, HL dots in the horizontal direction (VX1 direction) and HL dots in the vertical direction (VY1).
(One direction). This video RAM 16
The address of each pixel data read from V is represented by coordinates (VX1, VY1) (0 ≦ VX1 ≦ HL-1, 0 ≦ VY).
1 ≦ VL−1).

Reference numeral 17 denotes an image processor.
When creating a video index, the image processor 1
7, the data of the portion surrounded by the vertical slit 4 (see FIG. 4) is read from the image data for one frame of the video RAM 16V, and the data is compressed and the vertical portion of the video RAM (VRAM2) 23 composed of a frame memory is read. The data is written in a portion surrounded by a slit (output slit) 7 (see FIG. 5). In addition, image processor 1
Reference numeral 7 has a function of displaying an instruction cursor on a screen corresponding to the video RAM 23.

When the image processor 17 is expressed as a set of means corresponding to functions, the image processor 17 includes an input slit moving means 18, a slit data reading means 19, an output slit moving means 20, a slit data writing means 21, and an instruction cursor display. Means 22.

As shown in FIG. 5, the storage area of the video RAM 23 (VRAM2) also corresponds to the actual screen as in the case of the video RAM 16V, and is HL dots in the horizontal direction (VX2 direction) and VL dots in the vertical direction (VY2 direction). Dots.
The address of each pixel data to be written into the video RAM 23 is specified by coordinates (VX2, VY2).

Reference numeral 30 denotes an audio processor. At the time of creating an audio index, the audio processor 30 reads audio data for one frame sequentially written into the memory 16A, performs integration processing, and detects an audio level Ea. And video RA
In the slit area of the audio display unit 60 below the vertical slit 7 of M23, color data is written by a length corresponding to the audio level Ea.

In a period corresponding to one reduced screen, audio data is read from the memory 16A.
For example, feature points are extracted by a neural network. As a result, the types of human voice, music, and other voices are determined, and a color is selected from a color map according to the type, and the data of this color is stored in the voice display unit 60 at the bottom of each reduced screen as described above. This is the color data to be written.

When the audio processor 30 is expressed as a set of means corresponding to functions, the audio processor 30 includes an audio data reading means 31, a feature point extracting means 32, a color determining means 33, a level detecting means 34, and a color data writing means. It consists of 35.

Reference numeral 24 denotes a cursor RAM for storing cursor data. The pixel data read from the video RAM 23 and the cursor data read from the cursor RAM 24 are supplied to a combining circuit 25 to form combined image data.

The synthesized image data output from the synthesizing circuit 25 is converted into an analog signal by a D / A converter 26 and supplied to a monitor 27 and a video printer (not shown), and also to an external storage device (VTR, Floppy disk, etc.)
28.

The video signal reproduced from the external storage device 28 is supplied to the A / D converter 29 and the system bus 11
Can be written to the video RAM 23 via the.

At the time of index creation, the VTR 14
The output image data of the video image group 1 is written as a series of slit data to the video RAM 23 (shown in FIG. 5) via the video RAM 16V (shown in FIG. 4), and further, based on the audio data, the video RAM 2
A series of operations performed when the color data indicating the audio level Ea and the type are written in No. 3 will be described step by step along the flowchart of FIG.

In this case, X slit data extracted one by one from each frame of the video RAM 16 V and compressed together are stored in the video RAM 23 (X × Y).
.. Are written as reduced images 6A, 6B,.

[Step 101] According to the following equation, △
Calculate X and △ Y.

ΔX = HL / X ΔY = VL / Y ΔX and ΔY need not be integers, and the video RAM 16V
The pixel data of the block 33 composed of (△ X × △ Y) pixels is compressed to the value of one pixel 34 of the video RAM 23.

In this example, in order to easily perform compression, the coordinates (VX2, VY2) of the video RAM 23 are directly used as the coordinates of the pixels (VX2, VY2) of the pixel having the upper left corner (VX1, VY1) as the address. Is the data of the pixel 54 to be set. If ΔX and ΔY are non-integers, the coordinates (VX1, VY1) are not a pair of integers, and the value of the pixel indicated by the coordinates (VX1, VY1) is calculated by interpolation from the values of surrounding pixels. .

In the video RAM 23, the number h of the horizontal arrangement of reduced screens (6A, 6B, etc.) composed of X vertical slits 7 is calculated according to the following equation. Xs is the number of pixels in the horizontal margin.

H = (HL-Xs) / X Further, the numbers FR of the reduced screens 6A, 6B... Are respectively set to 0, 1,..., FR0, and FR = 0 is initialized.

The image data for one frame output from the VTR 14 is written into the video RAM 16V,
The audio data correspondingly output from the TR 14 is written into the memory 16A.

[Step 102] The audio data is read from the memory 16A,
Then, the extraction of the feature points and the color determination processing are started.

[Step 103] With the coordinates of the upper left corner of the reduced screen (6A, 6B, etc.) of the number FR of the video RAM 23 as (BX, BY), BX, BY are calculated according to the following equations. Xs1 is the number of pixels in the left margin, and Ys1 is the number of pixels in the top margin in the vertical direction. YA = Y + Y '
+ Y ″, Y ′ is the number of pixels in the vertical direction of the audio display unit 60, and Y ″ is the number of pixels in the vertical direction between the audio display unit 60 and the reduced screen (see FIG. 5).

BX = (FRmod h) X + Xs1 BY = [FR / h] YA + Ys1 In these equations, (FRmod h) represents the remainder of FR / h, and [FR / h] represents the maximum not exceeding FR / h. Are shown as integers.

[Step 104] The initial values of the coordinates VX1 of the vertical slit 4 of the video RAM 16V and the coordinates VX2 of the vertical slit 7 of the video RAM 23 are set to 0 and BX, respectively.
Set to.

[Step 105] The initial values of the coordinates VY1 of the vertical slit 4 of the video RAM 16V and the coordinates VY2 of the vertical slit 7 of the video RAM 23 are set to 0 and BY, respectively.
Set to. Initially, N = 0 is set.

[Step 106] The value of N is increased by one.

[Steps 107 and 108] The image processor 17 sets the coordinates (VX1, VX) of the video RAM 16V.
After reading the data of the pixel of Y1) and writing it as the data of the pixel of the coordinates (VX2, VY2) in the video RAM 23, the value of the coordinate VY1 is increased by ΔY, and the coordinate VY2
Is incremented by one.

[Step 109] When the data of the vertical slit 4 of the video RAM 16V is read in the direction D1, the data of the vertical slit 7 of the video RAM 23 becomes D2.
Written in the direction. When the coordinate VY1 of the vertical slit 4 of the video RAM 16V is equal to or smaller than VL, the process returns to step 106, and when the coordinate VY1 exceeds VL, the process proceeds to step 110.

[Step 110] The audio processor 30 reads and integrates all or part of the audio data for one frame period written in the memory 16A.

[Step 111] When the integrated output corresponding to the number of pixels in the vertical direction of the audio display unit 60 of the video RAM 23 is DM and the integrated output is DI, A = [DI / DM × Y '] is calculated. . [DI / DM × Y ′] is DI / DM × Y ′
Is the largest integer that does not exceed The data of A indicates the audio level Ea.

[Step 112] DXN = VX2 DYN = VY2 + Y'-A DYN '= VY2 + Y'-1

[Steps 113 and 114] The value of VX1 is increased by ΔX, and the value of VX2 is increased by 1. This means
This means that the position of the vertical slit 4 of the video RAM 16V is shifted to the right by ΔX, and the position of the vertical slit 7 of the video RAM 23 is shifted to the right by 1.

The host computer 8 is connected to the VTR 1
4, image data and audio data of the n-th frame from the current frame are input and written to the video RAM 16V and the memory 16A, respectively.

[Step 115] If the coordinate VX1 of the vertical slit 4 of the video RAM 16V is less than or equal to HL, the flow returns to step 105. If the coordinate VX1 exceeds HL, this means that one horizontal scan of the vertical slit 4 of the video RAM 16V has been completed, and the process proceeds to step 116.

[Step 116] The color data of the color determined by the audio processor 30 is written to the coordinates (DXN, DYN) to (DXN, DYN ') of the video RAM 23. Here, N = 1 to X. Thereby, the video R
The sound level corresponding to the image data of each slit is indicated and the color data indicating the type is written in the sound display section 60 of the AM.

[Steps 117 and 118] The number FR of the reduced screen is increased by one, and the number FR is stored in the video RAM2.
If the number is smaller than the allowable number of reduced screens FR0, the process returns to step 102. When the number FR exceeds the number FR0, it means that the creation of the video index for one screen has been completed. Therefore, the process proceeds to step 119 to perform post-processing.

As post-processing, the image data in the video RAM 23 is stored in the external storage device 28 via the D / A converter 26, and the image data is supplied to the monitor 27 via the D / A converter 26. It is conceivable that.

The video image group 1 which is a set of moving image data is stored in the external storage device 28 in a compressed form by a method which can be expressed as a kind of video slice (shown in FIG. 7). The monitor 27 displays a plurality of reduced screens, an audio level corresponding to each reduced screen, and an index indicating the type.

Thereafter, the operation returns to step 101 again by the operation of the operator, and an index for the subsequent video signal and audio signal from the VTR 14 is created.

The index created as described above is a reduced screen obtained by compressing and connecting image data sampled by changing the position of the vertical slit 4 for every n frames of the video image group 1, and The outline of the moving image data of the group 1 can be confirmed in correspondence with the passage of time.

Here, in order to change the position of the vertical slit 4 so as to scan from the left end to the right end of one screen in 12 seconds, for example, N reduced images 6A and 6
In the case of forming B,..., 12 × N = 12 N [sec], 30 × 12 × N = 360 N [frame], and 12N seconds worth (360 N frames) in one display screen 5 (frame) Is compressed and displayed, and a series of moving image data is compressed at an extremely high compression ratio.

When the moving image of the video image group 1 changes slowly on the basis of 12 seconds, the vertical slit 4
Scans from the left end to the right end, it is possible to restore a substantially original state of a moving image. On the other hand, when the moving image of the video image group 1 changes abruptly like a commercial, for example, a discontinuity that changes discontinuously on any of the reduced screens is formed. Therefore, although the moving image data is compressed and displayed at a compression ratio of about 1 / 360N, the outline of the slowly changing portion can be checked, and the rapidly changing portion can be disconnected. Can be confirmed as

The audio display section 60 displays an audio level corresponding to a slit image constituting each reduced screen. That is, the audio level is displayed as time elapses. In addition, the display of the audio level is given a color corresponding to the type of audio for each reduced screen. Therefore, the entire flow of the sound together with the image can be easily grasped, and the editing operation and the like can be performed more efficiently.

The individual frames 2 of the video image group 1
Instead of sampling the image data using the vertical slits 4, the image data may be sampled using the horizontal slits. In this case, the horizontal slit is periodically scanned at a predetermined speed from the upper end of the frame 2 to the lower end in the vertical direction (V direction). The audio display unit 60 is provided on the left or right side of the reduced screen.

In the above embodiment, the type of sound is displayed by changing the color, but it may be displayed by changing the size and brightness of the display area.

[0070]

According to the present invention, a still image composed of a plurality of slit-like images showing the overall flow of an image is displayed on the display element, and each slit-like image constituting the still image is displayed. Corresponding to each slit
Summary of audio data corresponding to the shape of the image , for example, level,
The type and the like are visually displayed, the overall flow of the image and the sound can be recognized with high accuracy, and the editing operation and the like can be performed more efficiently.

[Brief description of the drawings]

FIG. 1 is a configuration diagram of an embodiment.

FIG. 2 is a diagram illustrating the concept of an index.

FIG. 3 is an enlarged view of a voice display unit.

FIG. 4 is a diagram showing a data structure of a video RAM.

FIG. 5 shows a data structure of a video RAM.

FIG. 6 is a flowchart showing an operation at the time of creating an index.

FIG. 7 is a diagram illustrating a relationship between an input video signal and a video index.

[Explanation of symbols]

 Reference Signs List 1 video image group 2 frame 3A, 3B frame group 4 vertical slit (input slit) 5 display screen 6A, 6B reduced screen 7 vertical slit (output slit) 8 host computer 14 VTR 16A memory for audio data 16V, 23 video RAM 17 Image processor 27 Monitor 30 Audio processor 60 Audio display unit

──────────────────────────────────────────────────の Continued on the front page (51) Int.Cl. ⁷ identification code FI G11B 27/00 E (58) Investigated field (Int.Cl. ⁷ , DB name) G11B 27/00-27/038 H04N 5 / 445 H04N 5/7826 H04N 5/93 G06T 13/00

Claims (1)

(57) [Claims] 1. A series of moving image data two-dimensionally displayed at a first display position of a display element is sampled one-dimensionally in another direction while sequentially moving each sampling position in one direction. first display means for displaying a still image composed of the slit-shaped image by the obtained image data, from a series of audio data corresponding to the series of moving picture data, images data that is obtained by the sampling each according to
An outline detection means for detecting an outline of audio data respectively corresponding to the slit-shaped image; and a display at the second display position of the display element at the first display position based on a detection output of the outline detection means. And second display means for visually displaying an overview of the audio data corresponding to each slit-shaped image corresponding to each slit-shaped image constituting the still image. Display device.