JP2000278641A - Device and method for processing moving picture - Google Patents

Abstract

PROBLEM TO BE SOLVED: To generate a free index picture by selection by a user by providing a means designating a picture whose information quantity is highest among pictures which are the nearest in terms of time sequence from inputted moving pictures when an index designation signal is generated. SOLUTION: A generation means generating an index designation signal and a designation means designating a picture whose information quantity is highest among pictures which are the nearest in terms of time sequence from inputted moving pictures when the index designation signal is generated are installed. An electronic still camera 10 is provided with key switches 12-21 containing a shutter key 12 in a camera main body 11, for example. An index picture designation program is executed by CPU at the recording mode or the reproduction mode of the moving picture. When a trigger generation key (shutter key 12) for designating the index picture is depressed, the index picture can be generated by a past I picture nearest to the time of a depression operation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a moving picture processing apparatus and a moving picture processing method, and more particularly to a moving picture processing apparatus and a moving picture processing method for extracting an image in a moving picture as an index image.

[0002]

2. Description of the Related Art FIG. 12 is a block diagram of an electronic still camera for electronically recording a still image of a subject.
Is a color CCD that periodically captures a two-dimensional image of the subject
(Charge Coupled Device), 101 is a sample hold circuit for sampling the CCD output, and 102 is S / S
A / D for converting the H output into a digital signal;
A color processing circuit for generating an image signal by combining the luminance information Y of the / D output and the color difference information Cb and Cr; 104, an image buffer for holding the image signal; 105, a reduction processing of the image signal in the image buffer; And a video encoder 106 for converting the signal into a signal suitable for display.
A liquid crystal display 107 for displaying the output of the image buffer 105 is a JPEG (joint photographic) image signal in the image buffer.
experts group) JPEG compression unit for compressing, 108 is JPEG for JPEG decompressing an arbitrary image in the storage medium 109
An expansion unit 109 is a storage medium such as a flash memory. A control unit for controlling each unit, a key input unit including a shutter key, an optical system, and the like are omitted for the sake of illustration.

In such a configuration, when recording a still image, first, an image signal periodically picked up by the CCD 100 is fetched into the image buffer 104, and the image signal is displayed on the liquid crystal display 106. The display image on the liquid crystal display 106 is synchronized with the imaging cycle of the CCD 100, and if the subject moves, the display image also moves,
Also, if the composition is changed, the display image is similarly changed. Therefore, the liquid crystal display 106 can be used as a finder, and an optical finder can be assisted.

When a desired composition is obtained, a shutter key (not shown) is pressed. The shutter key is generally a two-step operation. The exposure and the shutter speed are determined by the first-stage half-press operation, and a still image is recorded by the second-stage full-press operation.

To record a still image, first, an exposure and a shutter speed are set, then the output of the CCD 100 is taken into an image buffer 104 via a sample hold circuit 101, an A / D 102 and a color process circuit 103, and the image signal Is compressed by the JPEG compression unit 107, the bit rate is reduced, and the data is stored in the storage medium 109.

[0006] The compressed image stored in the storage medium 109 is
By reading the image into the image buffer 104 via the JPEG decompression unit 108, the image is reproduced as an image equivalent to an original image (a complete original image cannot be obtained because JPEG is a type of lossy encoding). Is displayed on the liquid crystal display 106.

The recording and reproduction of a still image in an electronic still camera are as described above. Recently, however, an electronic device in which a simple moving image is recorded and reproduced using a periodic imaging function of a CCD has been described. Still cameras have been put to practical use.

FIG. 13 is a block diagram of the system, and FIG.
The difference from this is that a plurality of image buffers 200 ₁ to 200 _n are provided in the image buffer 200.

According to this configuration, a periodic image signal picked up by the CCD 100 is converted into n image buffers 200 ₁ to 200 ₁ .
200 _n sequentially, and each held image is JPEG
By reducing the bit rate by performing JPEG compression in the compression unit 107, a plurality of JPEG compressed images that are continuous in time series can be stored in the storage medium 109.
Displaying a moving image on the liquid crystal display 106 by JPEG decompressing the stored image in the JPEG decompression unit 108 and sequentially storing the stored images in n image buffers 200 ₁ to 200 _n and sequentially reading them out through the video encoder 105 Can be.

JPEG compression / expansion is a standard compression / expansion technology for still images, and is an effective technology that can reduce the bit rate of a still image to reduce the file size. There is a disadvantage that it is not efficient in terms of reduction effect. This is because each image constituting a moving image is encoded as a still image, for example,
To record a moving image of 30 frames per second for 4 seconds, 30
X4 = 120 still images are recorded, and the capacity of the storage medium 109 must be increased accordingly. Therefore, there is known an electronic still camera to which a standard coding method of moving image compression (MPEG: Moving Picture Experts Group) is applied.

FIG. 14 is a block diagram thereof, and FIG.
The difference between this is that the MPEG compression unit 201 and the MPEG decompression unit 2
02.

FIG. 15 shows the syntax (syntax) of MPEG.
Order and contents of the bit stream).
This syntax includes a sequence layer having several GOPs (Group Of Pictures) between a sequence header and a sequence end, and a GOP layer below the sequence layer.
The layer has a structure having a plurality of picture frames (encoded screen data) after the GOP header.

One GOP is one unit of random access, and reproduction from the middle (hereinafter, referred to as a trick mode) is enabled in this unit. The type (picture type) of a picture frame is one of an I picture (abbreviation: I), a P picture (abbreviation: P), and a B picture (abbreviation: B). The contents of each picture type are as follows. is there. -I picture Abbreviation of Intra-Coded Picture.
This is an image in which all of the screen is intra-coded. It differs from other picture types in that it has independence within the GOP (does not require data of other frames).・ P picture Inter-frame forward prediction coded image (Predictive-Coded P
icture). This is an image predicted in the forward direction from the previous I picture or P picture. -B picture Frame interpolation bidirectionally coded image (Bidirectionally
Predictive-Coded Picture). This is an image that is bidirectionally predicted from preceding and succeeding I or P pictures.

FIG. 16 is a diagram showing an example of the GOP structure. The number of pictures (N parameter) of the GOP is set to "15", and the period of the I picture and P picture (M parameter) is set to "3". This is an example of the case. That is, 1
A GOP is composed of 15 frames, and IOP
An example in which the number of frames from a picture (or P picture) to the next P picture is three is shown.

In FIG. 16, an I picture is an intra-coded picture that does not require data (reference picture) of another frame, while a P picture and a B picture are predictive coded pictures in a forward direction and a bidirectional direction, respectively. , As shown in the figure,
The P picture is an image that has been forward-predicted and bi-directionally predicted using an already coded I picture or P picture as a reference image, and a B picture using the preceding or succeeding I picture or P picture as a reference image.

FIG. 17 shows the screen order of the original images (B0, B1,.
.,... Are partially replaced at the stage of the encoding process, and the original sequence is restored at the stage of the reproduced image. Insertion of a B picture at the encoding processing stage is performed after encoding of the preceding and succeeding I pictures (or P pictures) and P pictures. For example, focusing on B3 and B4 of the original image, after encoding I2 and P5,
B3 and B4 are encoded using these I2 and P5 as reference images, and inserted after I2 and P5. As a result, the order of “I2, B3, B4, P5” at the stage of the original image is
.., P5, B3, B4.

Here, important parameters of the GOP structure are the above-mentioned "N parameter" and "M parameter", that is, the number of pictures (N) in the GOP and the period (M) in which I or P pictures appear. . There are no restrictions on the use of these parameters. The value can be freely selected as long as the first GOP on the bit stream satisfies the condition that the first picture is an I picture and the last picture in the order of the original images is an I or P picture. Is set to a value that is considered to be optimal in terms of image quality, motion of a moving image, and the like.

For example, M is often selected to be a value of about 2 to 3, and N is often selected to be a value such that a random access unit is about 0.4 to 1 second. Incidentally, the optimal value of M is determined by the motion of the moving image (small motion is small M and gentle motion is large M), but the optimal value of N is determined by compromise between image quality and random access unit. This is because if N is reduced, random access units become finer, and the convenience of the trick mode can be improved, but on the other hand, the coding efficiency is reduced and the image quality is deteriorated.

The MPEG image is composed of at least N frame images in order to predictively encode the first and subsequent images of N frame images that are continuous in time series (encoding that requires data of another frame). The compression efficiency is higher than that of JPEG, in which all are compressed as still images, and the bit stream can be greatly reduced to reduce the file size.

On the other hand, not only the electronic still camera but also an image processing apparatus that handles moving images requires a stop motion of a moving image and a cueing function of a scene change. For example, an image in a moving image can be displayed as an index image. There is known an image processing apparatus that can perform cueing and grasp of image contents.

FIG. 18 is a conceptual diagram of this, in which the head picture (ie, I picture) of the GOP is taken out and the taken out I picture is set at a predetermined magnification (P / N; 1/4 in the figure).
To display a multi-screen (16 in the figure). By watching the multi-screen display, the contents of the moving image can be easily grasped, and an arbitrary divided image can be selected and a moving image can be reproduced from the divided image.

[0022]

However, FIG.
In the conventional example, since each head picture of a continuous GOP on the time axis is fixedly displayed as an index image, not only a user-selectable free creation of an index image but also a change in a scene can be performed. There is a problem that an appropriate index image cannot be obtained.

Therefore, an object of the present invention is to enable creation of an index image freely by a user selection, or creation of an appropriate index image corresponding to a scene change or the like.

[0024]

SUMMARY OF THE INVENTION The present invention provides a method and apparatus for responding to an intentional operation of an operator or detecting a specific change in an image contained in a bit stream.
An image subjected to intra-frame encoding is extracted from the encoded bit stream, and an index image is created using the extracted image.

More specifically, the invention as set forth in claim 1 is characterized in that: input means for inputting a chronologically continuous moving image; generating means for generating an index designation signal; Designating means for designating, as an index image, an image having the largest amount of information among the most recent images in chronological order from the moving image input to the input means. Or, the invention of claim 2
Input means for inputting a bit stream of a moving image generated by performing intra-frame encoding on the first of N time-series continuous frame images and performing predictive encoding on other images, and index designation A signal generating unit, and when the index designation signal is generated, designates, as an index image, an intra-frame coded image located immediately before or immediately in time series from a bit stream input to the input unit. And designation means. Alternatively, the invention described in claim 3 is based on claim 2
In the invention described above, the generation means generates an index designation signal in response to an intentional operation of an operator. Alternatively, the invention according to a fourth aspect is the invention according to the second aspect, further comprising a detection unit that detects a change tendency on the time axis of the image, wherein the generation unit is configured to detect a specific change tendency by the detection unit. Is detected, the index designation signal is generated. Alternatively, the invention according to claim 5 is the invention according to claim 4, wherein the change tendency is a correlation between images adjacent on a time axis. Or, the invention according to claim 6 is:
According to a fourth aspect of the present invention, the change tendency is a change tendency of luminance information or color information of images adjacent on a time axis. Alternatively, according to a seventh aspect of the present invention, in the second aspect of the present invention, there is provided a detecting means for detecting a specific change tendency of audio information recorded or reproduced simultaneously with the image, and the generating means comprises the detecting means. When the specific change tendency of the voice information is detected by the means, the index designation signal is generated. Or
According to an eighth aspect of the present invention, in the second, third, fourth, fifth, sixth, or seventh aspect of the invention, the intra-frame coded image designated by the designation means is extracted. And generating means for generating an index image from the image. Alternatively, according to the ninth aspect of the present invention, one of N frame images continuous in time series is provided.
A first step of inputting a bit stream of a moving image generated by performing intra-frame encoding on a sheet and performing prediction encoding on another image, a second step of generating an index designation signal, and the index designation When a signal is generated, a fourth intra-frame coded image that is positioned immediately before or immediately after in the bit stream input in the first step is designated as an index image.
And step.

[0026]

DESCRIPTION OF THE PREFERRED EMBODIMENTS Embodiments of the present invention will be described below with reference to the drawings, taking an electronic still camera as an example.

FIG. 1 is an external view of an electronic still camera. The illustrated electronic still camera 10 includes various key switches 12 to 21 (details will be described later) including a shutter key 12 (generation means) on a camera body 11, and a strobe 22, a photographic lens 23, and a finder 2 on its front surface.
4 and an auto-focus unit 25, and a viewfinder window 24a and a liquid crystal display 26 on the back.

One of the key switches 12 to 21 is the shutter key 12 as described above. Other than that,
For example, a plus key 13, a minus key 14, a power switch 15, a menu key 16, a display key 17,
There are a recording mode key 18, a self-timer key 19, a strobe mode key 20, a REC / PLAY key 21, and the like, and the functions (roles) of these keys are as follows.

Shutter key 12: In the recording mode, as the name implies, it is a key that functions as a "shutter key" (exposure and focus are fixed by half-pressing and an image is captured by full-pressing). When the menu key 16 is pressed in the mode (the mode in which a captured image is reproduced or output to another device), the YES key for confirming various selection items displayed on the liquid crystal display 26 may be used. Working multi-function key. In the present embodiment, it is also used as a “trigger generation key” for specifying an index image when recording or reproducing a moving image.

The plus key 13 is a key used to select a reproduced image and various system settings. “Plus” means the selection direction. For example, in the case of image selection, the direction of the latest image,
If the system setting is selected, the liquid crystal display 26
Scanning direction.

Minus key 14: Same function as the plus key except that the direction is reversed. Power switch 15: a switch for turning on and off the power of the camera.

Menu key 16: A key for performing various system settings. In the playback mode, various items such as a delete mode (image erasing mode) and a moving image display mode are displayed on the liquid crystal display 26, and in the recording mode, necessary items for recording a still image or a moving image are displayed.
For example, the liquid crystal display 26 displays selection items such as the definition of a recorded image, on / off of auto focus, and shooting time of moving image shooting.

Display key 17: A key for overlappingly displaying various information on an image displayed on the liquid crystal display 26. For example, in the recording mode, the remaining number of recordable images and the form of photographing (normal photographing, panoramic photographing) ,
Information such as movie shooting) is displayed in an overlapped manner. In playback mode, attribute information of the playback image (page number, definition, etc.)
Are overlapped.

Recording mode key 18: A key which can be used only in the recording mode. In addition to selecting normal shooting, panoramic shooting, and the like, in the present embodiment, simple moving image shooting in the MPEG system is selected. -Self-timer key 19: A key for turning on and off a self-timer function. A strobe mode key 20: a key for various settings related to a strobe, for example, forcibly emitting light, prohibiting light emission, and preventing red-eye.

REC / PLAY key 21 A key for switching between a recording mode and a reproduction mode. In this example, the switch is a slide switch. When the switch is slid up, the recording mode is set, and when the switch is slid down, the playback mode is set.

FIG. 2 is a block diagram of the electronic still camera according to the present embodiment. In FIG. 2, 28 is an actuator for a mechanical shutter, and 29 is an actuator 2
A mechanical shutter driven by 8; a CCD 30;
1 is a driver of the CCD 30, 32 is a timing generator,
33 is a sample and hold circuit, 34 is an analog-to-digital converter, 35 is a color process circuit, 36 is a video transfer circuit, 37 is a buffer memory, 38 is a compression / expansion circuit (input means), 39 is a flash memory,
0 is a CPU (generating means), 41 is a key input unit (generating means), 42 is a digital video encoder, and 43 is a bus. In addition, 23 is a photographic lens, and 26 is a liquid crystal display.

The functions of these units are generally as follows. The photographic lens 23 is for forming an image of a subject on the light receiving surface of the CCD 30, and has a focusing mechanism for an automatic focusing function. Note that a zoom function may be provided, or a retractable type may be provided.

Actuator 28 and mechanical shutter 2
9: Mechanically cuts off light incident on the light receiving surface of the CCD 30. The actuator 28 opens and closes the mechanical shutter 29 according to a control signal from the CPU 40. The mechanical shutter 29 opens and closes the optical path to the light receiving surface of the CCD 30 when the mechanical shutter 29 is open. Open and close the optical path when closed.

CCD 30: An interlaced CCD, although not particularly limited, which can switch between field reading and frame reading by external control. The exposure time at the time of field reading is controlled by the opening and closing operation of the electronic shutter (charge accumulation time), and the exposure time at the time of frame reading is controlled by the opening operation of the electronic shutter and the closing operation of the mechanical shutter 29.

The CCD 30 of the present embodiment is a color CCD. Generally, since the pixel information of a CCD itself does not have color information, a color filter array (a primary color filter using three primary colors of light or a complementary color filter using three primary colors of colors) is mounted on the front surface of a color CCD, and furthermore, the color filter array is mounted on the front surface An optical low-pass filter for removing a false color signal having a frequency component corresponding to the pitch of the color filter array is mounted, but is omitted in the drawing.

Driver 31 and timing generator 32:
This is a part for generating a drive signal necessary for reading the CCD 30. The CCD 30 outputs a periodic image signal in synchronization with the driving signal.

Sample hold circuit 33: CCD 30
A time-series signal (which is an analog signal at this stage) read out from the CPU 30 is sampled (generally, correlated double sampling) at a frequency suitable for the resolution of the CCD 30. Note that automatic gain adjustment may be performed after sampling.

An analog-to-digital converter 34 converts a sampled signal into a digital signal.

Color process circuit 35: A portion for generating a luminance / color difference multiplex signal (hereinafter, referred to as a YUV signal) from the output of the analog-to-digital converter 34. The reason for generating the YUV signal is as follows.

The output of the analog-to-digital converter 34 is
Except for the difference between analog and digital, and errors in sampling and digital conversion, the data substantially correspond to the output of the CCD 30 on a one-to-one basis, and are the light primary color data (RGB data) itself. This causes inconvenience in the use of the memory resources and processing time.
Therefore, it is necessary to reduce the amount of data to some extent by some method. In general, each component data (R data, G data, and B data) of RGB data can be represented by three color difference signals GY, RY, and BY with respect to a luminance signal Y. By removing the redundancy of these three color difference signals, GY need not be transferred, and GY = α.
It can be said that it is a kind of data amount reduction signal based on the principle that it can be reproduced by (RY) -β (BY). Here, α and β are synthesis coefficients.

The YUV signal is converted to a YCbCr signal (Cb
And Cr are sometimes referred to as BY and RY, respectively, but in this specification, they are unified to YUV signals. The signal format of the YUV signal is composed of three fixed-length blocks called “components” each independently including a luminance signal and two color difference signals, and has a length (number of bits) of each component. The ratio is called the component ratio. Although the component ratio of the YUV signal immediately after the conversion is 1: 1: 1, the two components of the color difference signal are shortened, that is, 1: x: x (where x
<1) can also reduce the data amount. This utilizes the fact that human visual characteristics are less sensitive to color difference signals than to luminance signals.

Video transfer circuit 36: The video transfer circuit 36 includes a color process circuit 35 (constituting an exit of the imaging system), a buffer memory 37,
It controls the flow of data flowing between the digital video encoder 42 (constituting the entrance of the display system) and the compression / expansion circuit 38 (constituting the main part of the compression / expansion system). In the photographing preparation stage of adjusting the composition while watching the display on the liquid crystal display 26, the first flow and the second flow shown in the drawing are permitted, and the shutter key 12 is pressed to display the displayed image in the flash memory 39.
The third flow shown in the drawing is permitted in the recording stage of capturing the image, and the second flow and the fourth flow shown in the reproduction stage of reading a desired image from the flash memory 39 and displaying the desired image on the liquid crystal display 26.

The "flow" is a convenient expression conceptually capturing the movement of data flowing between the color process circuit 35, the buffer memory 37, the digital video encoder 42, and the compression / expansion circuit 38. Although the words themselves have no special meaning, generally for digital systems, the fast movement of data directly affects its performance, especially for high-definition electronic still cameras that handle a large amount of pixel information, the fast movement of data. Is one of the design conditions that must be taken into consideration, so that all or a part of the above flow is a data flow that makes full use of high-speed data transfer techniques.

That is, the first to fourth flows are as follows:
For example, the flow is based on DMA (direct memory access) transfer. The video transfer circuit 36 includes a control unit (DMA controller) and other peripheral parts (for example, a FIFO memory and an interface circuit for adjusting a transfer speed) necessary for the transfer. The color processing circuit 35 and the buffer memory 3
7. Arbitrate "quick data transfer" (for example, DMA transfer) between the digital video encoder 42 and the compression / expansion circuit 38.

The buffer memory 37 is composed of a DRAM which is a kind of rewritable semiconductor memory. Generally, DRAMs are different from static RAMs (SRAMs) in that data is dynamically rewritten (refreshed) in order to retain stored contents. However, although the writing and reading speeds are inferior to SRAMs, the unit cost per bit is low. This is particularly suitable for an electronic still camera because a large-capacity temporary storage can be configured at low cost. However, the present invention is not limited to a DRAM. Any rewritable semiconductor memory may be used.

A compression / expansion circuit 38: performs JPEG encoding (compression) and decoding (expansion) when recording and reproducing a still image, and performs MPEG encoding and decoding when recording and reproducing a simple moving image. Part. These encoding methods are currently the de facto standard for still images and moving images (substantial international standard accompanying widespread natural spread), and in particular in terms of compression efficiency, the combination of still images and JPEG encoding The combination of moving picture and MPEG coding is best.

It should be noted that JPEG stands for joint photographic.
An abbreviation of experts group, an international coding standard for color still images (full-color or grayscale still images that do not include binary or moving images). In JPEG, lossless encoding that can completely restore compressed data,
Two methods of irreversible coding that cannot be undone are defined, but in most cases, the latter irreversible coding having a high compression ratio is used. The ease of use of JPEG lies in the fact that the degree of image quality degradation accompanying encoding can be freely changed by adjusting parameters used for compression (compression parameters).

That is, on the encoding side, an appropriate compression parameter can be selected from the trade-off between the image quality and the file size, or on the decoding side, the decoding speed is increased at the expense of some quality, It is easy to use because you can choose to play it at the highest quality even if it is applied. The practical compression ratio of JPEG is about 10: 1 to 50: 1 for lossy codes. Generally, if the ratio is from 10: 1 to 20: 1, visual deterioration does not occur, but if some deterioration is allowed, even from 30: 1 to 50: 1 can be used sufficiently. Incidentally, the compression ratio of other still image coding systems is, for example, GIF (graphics interchangeable).
format), the ratio is about 5: 1, so JPEG
The advantage of is obvious. However, its superiority is only in the case of still images. In the case of moving images, the MPEG encoding described at the beginning is superior to JPEG. This is because MPEG performs inter-frame prediction except for the first image of a GOP, and therefore has only a small amount of data corresponding to a prediction error.
If the number of images (N parameters) constituting a GOP is 2 or more, a higher compression ratio than JPEG can be obtained.

Flash memory 39: rewritable read-only memory (PROM: programmable read on)
ly memory) that can be electrically rewritten by erasing the contents of all bits (or blocks). Flash EEPROM (flash electrically erasable
PROM). The flash memory 39 in the present embodiment may be a fixed type that cannot be removed from the camera body, or may be a removable type such as a card type or a package type. Alternatively, instead of the flash memory 39, a storage medium (magnetic disk, optical disk, magnetic tape, or the like) that stores information by magnetism or light may be used.

Note that the storage medium such as the flash memory 39 must be initialized (formatted) in a predetermined format, whether it is a built-in type or a removable type. The number of images according to the storage capacity can be recorded on the initialized storage medium. For example, if the image size after compression is 100K
Assuming B, 40 cards with a capacity of 4 MB and 8 with a capacity of 8 MB
0 sheets can be recorded.

CPU 40: centrally controls the operation of the camera by executing a predetermined program. The program is written in an instruction ROM (not shown) in the CPU 40. In the recording mode, a program for the mode is stored. In a reproduction mode, a program for the mode is stored in the instruction ROM from the CPU.
The internal working RAM (not shown) is loaded and executed.

Key input section 41: A section for generating operation signals for various key switches provided on the camera body 11.

Digital video encoder 42: buffer memory 37 via video transfer circuit 36
The digital value display image (the image reduced according to the number of pixels of the liquid crystal display 26) read from the image buffer is converted into an analog voltage, and is sequentially converted at a timing according to the scanning method of the liquid crystal display 26. Output.

The bus 43 is a data (and address) transfer path shared between the above components. Although not shown in the figure, necessary control lines (control lines) are also provided between the units.

Next, the operation will be described. First, an outline of recording and reproduction of an image will be described. First, in the still image recording mode, the CCD 30 disposed behind the photographic lens 23 is driven by a signal from the driver 31. The drive mode at this time is a field readout mode, in which the video collected by the photographic lens 23 is photoelectrically converted at regular intervals and a video signal for one image is output. Then, this video signal is sampled by the sampling and holding circuit 34 and converted into a digital signal by the analog-to-digital converter 34, and then a YUV signal is generated by the color process circuit 35. The YUV signal is transferred to an image buffer of a buffer memory 37 via a video transfer circuit 36, and after the transfer to the buffer is completed, read out by the video transfer circuit 36 and sent to the liquid crystal display 26 via the digital video encoder 42. Sent and displayed as a through image.

When the direction of the camera is changed in this state, the composition of the through image displayed on the liquid crystal display 26 changes. When the shutter key 12 is "half-pressed" at an appropriate time (when a desired composition is obtained), exposure and focus are set, and then "full-press", the driving mode of the CCD 30 is switched to the frame read mode, and the exposure is performed. After the mechanical shutter 29 is closed at an appropriate timing according to the above, the YUV signal for one screen is taken into the image buffer of the buffer memory 37. And the buffer memory 3
The YUV signal captured in the image buffer 7 is fixed at the current YUV signal, and the through image displayed on the liquid crystal display 26 is also fixed at the same point.

Then, the YUV signal stored in the image buffer of the buffer memory 37 at that time is sent to the compression / expansion circuit 38 via the video transfer circuit 36, and the YUV signal is sent to each of the Y, Cb, and Cr components. After being JPEG encoded in units called basic blocks of 8 pixels, it is written to the flash memory 39 and recorded as a still image for one image.

Next, in the still image reproduction mode, the CCD 3
After the path from 0 to the buffer memory 37 is stopped, the latest still image is read out from the flash memory 39 and subjected to JPEG decompression processing by the compression / decompression circuit 38. Sent to the image buffer. Then, the data in the image buffer is sent to the liquid crystal display 26 via the video transfer circuit 36 and the digital video encoder 42, and is displayed as a reproduced image.

The plus key 13 and the minus key 14
By pressing, this operation can be repeated while moving the still image read from the flash memory 39 forward or backward, and the desired image can be reproduced.

Next, the recording and reproducing modes of the moving image will be described. First, in the recording mode, the CCD 30 is driven in the field read mode even when the image is captured, and the moving image periodically output from the CCD 30 is stored in the buffer memory. 37, the images are displayed on the liquid crystal display 26, and simultaneously recorded on the flash memory 39 while being compressed by the compression / expansion circuit 38 by MPEG. One M is stored in the flash memory 39 in one movie recording.
A PEG file is created. And in playback mode,
The MPEG file generated in the flash memory 39 is read by the compression / decompression circuit 38 and subjected to MPEG decompression, and the decompressed images are displayed on the liquid crystal display 26 while being sequentially written to the buffer memory 37.

Next, a method of generating an index image of a moving image will be described. An index image is an arbitrary image extracted from a moving image composed of a plurality of images that are continuous in chronological order. Depending on the application, an index image may be used as a mere heading image, or an image itself may be used. It may be treated as a still image. The former is used, for example, for a multi-screen display for grasping the contents of a moving image, and the latter is sometimes used, for example, as a high-definition captured image similar to a normal still image shooting.

In any case, it must be possible to select "arbitrarily" from a plurality of images constituting a moving image. This is because an image representing a scene change, for example, must be able to be selected not only when used as a still image but also when used as a headline image.

In the prior art described at the beginning, the leading image of each GOP is designated as an index image. For example, M
In the case of a GOP in which the parameter is “15”, an index image is specified for every 15th image. This specification is a fixed specification depending on the M parameter, and it is apparent that the index image is arbitrarily (or freely) specified. Sex).

FIG. 3 is a schematic flowchart of the index image designation program according to the present embodiment. This program is a video recording mode (during shooting)
Alternatively, it is executed by the CPU 40 in the reproduction mode (during reproduction).

In this flowchart, first, the i-th (i = 1) picture constituting the GOP is read (S
1) It is determined whether the picture type is an I picture (S2). If the picture type is an I picture, the picture is temporarily stored (S3).

Next, it is determined whether a "trigger generation key" (the shutter key 12 in the present embodiment) for designating an index image is to be depressed (S4). While an index image is created with the current picture (S5), if no depressing operation is performed, i is updated to i = i + 1 and step S1 is performed.
And the i-th picture is read again, and the steps after step S2 are repeated.

FIG. 4 is a time chart of the flowchart of FIG. 3, where I, B and P represent pictures of the same code type. In MPEG coding, the first picture of a GOP is always an I picture. That is, when i = 1, since the picture is always an I picture, the picture is temporarily held, and thereafter, the index is specified until the start of the next GOP (see the white arrow in FIG. 4). Is performed, an index is created in the held I picture.

Therefore, according to the flowchart of FIG. 3, when the "trigger generation key" is depressed, an index image can be created with the "past" I picture closest to the time of the depressing operation.
Since the operation is an intentional operation by the operator (user) of the electronic still camera, a free index image can be created by the user's own selection.

FIG. 5 is a schematic flowchart of another index image designation program according to the present embodiment. This program is executed by the CPU 40 in the recording mode or the reproduction mode of the moving image similarly to the program of FIG.

In this flowchart, first, it is determined whether the "trigger generation key" (the shutter key 12 in the present embodiment) for depressing the index image has been depressed (S11). GOP
Of the i-th picture (i = 1) constituting
12), it is determined whether or not the picture type is an I picture (S13). If it is not an I picture, i = i + 1 is updated and step S12 is repeated.
If the picture is an I picture, an index image is created with the picture (S14).

FIG. 6 is a time chart of the flowchart of FIG. 5, where I, B and P represent pictures of the same code type. It is now assumed that an operation of depressing a “trigger generation key” (designating an index) is performed at an appropriate timing in the GOP. The white arrow in the figure indicates the timing.

When the index is specified, the subsequent picture types are sequentially determined, and when an I picture is determined, an index image is created from the picture.

Therefore, in this program,
When the “trigger generation key” is pressed down, an index image can be created with the “future” I picture closest to the time of the pressing down operation. Like the above-mentioned program, the pressing down operation is performed by operating the electronic still camera. Since this is an intentional operation by the user (user), the user can freely create an index image by his / her own selection. In addition, since an index image is not created using the B and P pictures at the time of index image designation but is created using the latest I picture, an index image with high image quality can be obtained by simple image processing without special processing. Can be created. The created index image can be stored in, for example, a storage medium such as the flash memory 39. However, in order to prevent the storage capacity from being overwhelmed, the index image is designated without storing the actual index image. May be stored. In this case, when displaying the index image later, the corresponding I picture is specified from the address information, and the index image is created and displayed using this I picture.

In the above embodiment, the index image is created in response to the user's own intentional operation. This method is excellent in that a user can freely create an index image by his / her own selection. However, when this method is used, an index image representing a switching portion of a screen (such as a scene change) in a moving image is created. There is a drawback in that it must be manually specified, which is troublesome and troublesome.

FIG. 7 is an example of the improvement, in which a switching portion of an image in a moving image is detected and an index image is automatically created.

In FIG. 7, the detection level is a detection level of, for example, the amount of motion of each image constituting the moving image. The motion amount is represented by, for example, the correlation between images adjacent on the time axis. An image having a large correlation has little motion, and an image having a small correlation has large motion. Since the correlation between the images before and after the change of the screen in the moving image is naturally small, the signal corresponding to the operation of pressing down the "trigger generation key" at the timing when the amount of motion is detected and the level reaches the maximum value Is generated, an index image representing a switching portion (scene change or the like) of a screen in a moving image can be automatically generated using the program in FIG. 3 or FIG.

The detection level is not limited to the motion amount of the image. Any amount indicating a specific change tendency representing a switching portion (scene change or the like) of a screen in a moving image may be used. For example, it may be luminance or color difference information of an image, or an electronic device having a recording function. In the case of a still camera, the information may be audio (not limited to human voice, including all sounds that can be perceived by hearing; the same applies hereinafter).

As shown in FIG. 8, the detection level (the level of the above-mentioned motion amount, luminance, color difference information, audio information, or the like) is compared with a predetermined threshold value. Alternatively, it may be determined that a peculiar change tendency has appeared, and a signal corresponding to the pressing operation of the above-described “trigger generation key” may be generated.

The index generated as described above
The image is, for example, a multi-screen image as shown in FIG.
Still image (I₁~ I₉From the nine divided images up to
May be used as a single still image) or B) in FIG.
The movie shown in the figure (I ₁~ I₉Up to 9 images
(A single moving image). Still image shown
Is displayed on the liquid crystal display 26, nine images are displayed at a time.
You can check the image, and if you display the movie shown in the same way,
You can display a simple video like a drop-down
The whole picture can be easily grasped.

In the case of an electronic still camera with a recording function, as shown in FIG. 10, at the same time as the creation of an index image, audio information for a predetermined time may be recorded in association with the index image.

FIG. 11 is a diagram showing the structure of an MPEG bit stream. V1, V2, V3,... Represent video packets, and A1, A2, A3,. The padding is a bit adjusting unit for keeping the data length of the packet constant.

In this structural diagram, for example, the packet number (or the content of the audio packet) of one or several consecutive audio packets corresponding to the video packet of the I picture used for the index image
May be recorded in association with the index image.

[0088]

According to the first aspect of the present invention, input means for inputting a moving image which is continuous in time series, generating means for generating an index specifying signal, and when the index specifying signal is generated, Since there is provided a designating means for designating, as an index image, an image having the largest amount of information in the most recent image in chronological order from the moving image input to the input means, if an intentional operation of the operator is made to respond, In addition, it is possible to freely create an index image by selecting a user (operator), and when an image is changed in response to detection of a specific change, an appropriate index image corresponding to a scene change or the like is created. Can be made possible.
Alternatively, according to the second aspect of the present invention, a moving image generated by performing intra-frame encoding on the first of N time-series continuous frame images and performing predictive encoding on other images Input means for inputting a bit stream of, and generating means for generating an index designating signal, and when the index designating signal is generated, positions are located immediately before or after the bit stream input to the input means in chronological order. Since there is provided a designating means for designating an intra-frame encoded image to be used as an index image, an index image can be generated using an intra-frame encoded image having the same information as a still image. According to a third aspect of the present invention, in the second aspect of the present invention, the generating means generates an index designation signal in response to an intentional operation of an operator. Can be used to generate an index image. Alternatively, according to a fourth aspect of the present invention, in the second aspect of the present invention, the image processing apparatus further comprises a detecting means for detecting a tendency of the image to change on a time axis, wherein the generating means has a specific Since the index designation signal is generated when a change tendency is detected, an index image can be generated by using a changed intra-frame encoded image. Or, according to the invention described in claim 5,
In the invention described in claim 4, since the change tendency is a correlation between images adjacent on the time axis, for example, an index image can be generated by detecting a change such as a scene change. Alternatively, according to the invention described in claim 6, according to claim 4,
In the invention described, since the change tendency is a change tendency of luminance information or color information of images adjacent on a time axis,
For example, an index image can be generated by detecting a unique image in a moving image. According to a seventh aspect of the present invention, in the second aspect of the present invention, there is provided a detecting unit for detecting a specific change tendency of audio information recorded or reproduced simultaneously with the image, and the generating unit comprises: When the detecting means detects a specific change tendency of the audio information, the index designation signal is generated, so that not only a change in an image but also a change in a sound or a background sound can be detected to generate an index image. According to an eighth aspect of the present invention, in the second, third, fourth, fifth, sixth, or seventh aspect of the invention, the intra-frame code designated by the designation means. Since the image processing apparatus includes a generation unit that extracts a coded image and generates an index image using the image, an index image can be generated using an intra-frame coded image having the same information as a still image. According to the ninth aspect of the present invention, a moving image generated by performing intra-frame coding on the first of N time-series continuous frame images and performing predictive coding on other images. A first step of inputting the bit stream of the above, a second step of generating an index specifying signal, and when the index specifying signal is generated, immediately before or in time series from the bit stream input in the first step. And a fourth step of designating an intra-frame coded image located immediately after the index image as an index image. Therefore, when an intentional operation of an operator is made to respond, an index image can be freely created by user (operator) selection. It can be made possible, and if it responds to the detection of specific changes in the image, create an appropriate index image corresponding to scene changes etc. Able to allow.

[Brief description of the drawings]

FIG. 1 is an external view of an electronic still camera.

FIG. 2 is a block diagram of an electronic still camera.

FIG. 3 is a schematic flowchart of an index image designation program according to the embodiment.

FIG. 4 is a time chart for specifying an index image in the embodiment.

FIG. 5 is a schematic flowchart of another index image designation program in the embodiment.

FIG. 6 is a time chart for specifying another index image in the embodiment.

FIG. 7 is a time chart for specifying an index image based on a specific change of an image.

FIG. 8 is a time chart for specifying another index image based on a specific change of an image.

FIG. 9 is a multi-display example using an index image and a time chart of a simple moving image file.

FIG. 10 is a time chart of an index image using audio sampling together.

FIG. 11 is a structural diagram of an MPEG system stream including audio packets.

FIG. 12 is a simplified block diagram of an electronic still camera for still images.

FIG. 13 is a simplified block diagram of an electronic still camera for JPEG moving images.

FIG. 14 is a simplified block diagram of an electronic still camera for MPEG moving images.

FIG. 15 is an MPEG syntax structure diagram.

FIG. 16 is a GOP structure diagram.

FIG. 17 is a frame sequence diagram of an MPEG image.

FIG. 18 is a conceptual diagram of a conventional index image generation.

[Description of Signs] 12 Key switch (generation means) 38 Compression / expansion circuit (input means) 41 Key input section (generation means) 40 CPU (generation means)

Claims (9)

[Claims] 1. An input means for inputting a moving image which is continuous in time series, a generating means for generating an index specifying signal, and when the index specifying signal is generated, a time from a moving image input to the input means A moving image processing apparatus comprising: a designating means for designating, as an index image, an image having the largest amount of information among images that are the most recent in sequence. 2. An input for inputting a bit stream of a moving image generated by performing intra-frame encoding on the first of N frame images consecutive in time series and performing predictive encoding on other images. Means for generating an index designating signal, and when the index designating signal is generated, an intra-frame coded image located immediately before or immediately in time series from a bit stream input to the input means. A moving image processing apparatus, comprising: a designation unit that designates an index image. 3. The moving picture processing apparatus according to claim 2, wherein said generating means generates an index designation signal in response to an intentional operation of an operator. 4. A detecting means for detecting a changing tendency of the image on a time axis, wherein the generating means generates the index designation signal when a specific changing tendency is detected by the detecting means. 3. The moving picture processing device according to claim 2, wherein: 5. The moving image processing apparatus according to claim 4, wherein the change tendency is a correlation between adjacent images on a time axis. 6. The moving picture processing apparatus according to claim 4, wherein the change tendency is a change tendency of luminance information or color information of images adjacent on a time axis. 7. A detecting means for detecting a specific change tendency of audio information recorded or reproduced simultaneously with the image, wherein the generating means detects a specific change tendency of the audio information by the detection means. 3. The moving image processing device according to claim 2, wherein the index designation signal is generated when the moving image is generated. 8. The image processing apparatus according to claim 2, further comprising a generation unit that extracts an intra-frame coded image specified by the specification unit and generates an index image using the extracted image.
Claim 3, Claim 4, Claim 5, Claim 6, or Claim 7
The moving image processing device according to the above. 9. A method of inputting a bit stream of a moving image generated by performing intra-frame encoding on the first of N frame images consecutive in time series and performing predictive encoding on other images. 1 step, a second step of generating an index specifying signal, and, when the index specifying signal is generated, an intra-frame code located immediately before or after the bit stream input in the first step in time series. A fourth step of designating the converted image as an index image.