JP2009055618A - Electronic camera and control program of electronic camera - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To easily execute various editings in good operability without being limited by performance of hardware. <P>SOLUTION: A mobile camera body 2 photographs a moving picture by a zoom imaging portion 4, for example encodes moving picture data by a compression encoding/decoding portion 24, and records them in a still-picture/moving-picture/voice-data memory 28. An editing/reproduction controller 26 inputs editing operation to the moving picture input from an operation input portion 15, generates a play list (reproduction control information) which expresses an editing content applied to the moving picture according to the input editing operation, and records it in a play list memory 30. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an electronic still camera, a video camera, a camera-equipped mobile phone, an information communication terminal with camera, and the like, and a control program for the electronic camera.

  In recent years, video cameras and the like have become widespread, and it has become possible to easily shoot and record moving images even in ordinary households. Small digital cameras are also capable of video recording and recording functions. Many digital camcorders allow you to create titles at the time of shooting or to shoot including various special effects, but you can use a dedicated video editing device or personal computer to edit and process the video once recorded. A video editing program to be executed is required.

  The personal computer has sufficient hardware such as a large memory capacity and high processing capability (CPU performance), a large display screen size, and a pointing device such as a mouse can be used, and the operating environment is excellent. Therefore, it is now possible to freely edit moving images using a moving image editing program.

  For example, in a recent video editing program, a timeline (time axis scale) is displayed on the screen of a personal computer, and a zone indicating the time of an index image, video or audio file is dragged with the mouse or the like as an object on this timeline. By arranging by & drop operation, docking and cut editing such as insertion, deletion, and combination can be performed on moving image data. Thereby, it is easy to understand visually, and detailed editing can be freely performed.

  However, small devices such as portable electronic cameras have limited processing capability and memory capacity, and the display screen is small and can only be operated with limited buttons and keys. It has been difficult to freely edit the images in the device.

  Conventionally, the following methods have been considered for editing and editing operations of moving images and sounds.

For example, representative frame images of moving images stored in units of cuts are taken out according to the cut numbers in the cut order table, displayed as a list of still images, the cut numbers are arbitrarily modified, and the reproduced moving images are recorded according to the cut numbers. Edit a moving image (for example, Patent Document 1)
In addition, the beginning image of each divided video is displayed as a list by scene index, a clip index obtained by clipping a plurality of scene indexes is displayed, editing information is generated based on the video division information and clip information, and the editing information is referred to Digital video is generated and stored (for example, Patent Document 2).
Japanese Patent No. 2698695 Japanese Patent No. 3276596

  Thus, conventionally, as described in Patent Documents 1 and 2, various methods have been considered regarding editing methods and editing operations of moving images and sounds, but all have sufficient hardware environments such as personal computers. Since it is premised to be executed by an apparatus equipped with an operating environment, it cannot be applied to a portable electronic camera having a small display screen size and inferior operability. That is, with a conventional portable electronic camera, it has been difficult to visually edit a moving image and to perform detailed editing due to hardware limitations and operating environment limitations.

  An object of the present invention is to provide an electronic camera and an electronic camera control program capable of easily executing various types of editing with good operability without being limited by hardware performance.

  The invention described in claim 1 is a moving image photographing means for photographing a moving image, a moving image recording means for recording a moving image photographed by the moving image photographing means, and a moving image recorded by the moving image recording means. An editing operation input means for inputting an editing operation on the video, and a reproduction control information generating means for generating reproduction control information representing the editing content to be applied to the moving image in response to the editing operation input by the editing operation input means. And reproduction control information recording means for recording the reproduction control information generated by the reproduction control information generation means in association with the moving image recorded by the moving image recording means.

  According to a second aspect of the present invention, in the first aspect of the present invention, the moving image recorded by the moving image recording unit represents the moving image recorded by the reproduction control information recording unit with respect to the moving image. It further comprises reproduction output means for reproducing and outputting while performing editing in accordance with the editing content.

  According to a third aspect of the present invention, there is provided a moving image photographing means for photographing a moving image, a moving image recording means for recording a moving image photographed by the moving image photographing means, and a moving image recorded by the moving image recording means. An editing operation input means for inputting an editing operation for adding a special effect for transitioning the image, and a transition source image and a transition destination image according to the editing operation input by the editing operation input means. The image processing apparatus includes an editing unit that generates a continuous image representing the transition of the image and edits the moving image recorded in the moving image recording unit.

  According to a fourth aspect of the present invention, there is provided a moving image photographing unit for photographing a moving image, a moving image recording unit for recording data of a moving image photographed by the moving image photographing unit, and a moving image recorded in the image recording unit. Display means for generating and displaying a list of a plurality of images based on image data; an editing operation for specifying a reproduction order for the plurality of images displayed by the display means; or an arbitrary one of the plurality of images An editing operation input means for inputting at least one of editing operations for designating addition of an editing icon representing a special effect to the image of the image, and the moving image recording means based on the editing operation input by the editing operation input means. An editing means for editing the recorded moving image data is provided.

  According to a fifth aspect of the present invention, in the fourth aspect of the present invention, the degree of correlation of information included in the image included in the moving image is calculated, and images between which the degree of correlation is lower than a predetermined value are identified as scene switching positions. A scene identifying unit for displaying the representative image for each scene identified by the scene identifying unit.

  According to a sixth aspect of the present invention, in the fourth aspect of the invention, when the moving image photographing unit captures a moving image, the separation position input unit inputs an operation for designating a separation position, and the separation position input unit inputs the operation. A position recording means for recording the data indicating the delimited position in association with the moving image data, and a scene for identifying a scene switching position based on the data indicating the delimiter position recorded by the position recording means And an identification unit, wherein the display unit displays a representative image for each scene identified by the scene identification unit.

  The invention according to claim 7 is recorded in the moving image photographing means for photographing the moving image, the moving image recording means for recording the moving image data photographed by the moving image photographing means, and the moving image recording means. A dividing unit that divides moving image data into predetermined units; an editing operation that specifies a reproduction order for moving images for each predetermined unit divided by the dividing unit; or an arbitrary one of the moving images for each predetermined unit An editing operation input means for inputting at least one of editing operations for designating addition of an editing icon representing a special effect to the moving image, and the moving image recording means based on the editing operation input by the editing operation input means. An editing means for editing the recorded moving image data is provided.

  According to an eighth aspect of the present invention, in the invention according to any one of the third to seventh aspects, the reproduction control representing the editing content to be applied to the moving image in accordance with the editing operation input by the editing operation input means. Reproduction control information generation means for generating information, and reproduction control information recording means for recording the reproduction control information generated by the reproduction control information generation means in association with the moving image recorded by the moving image recording means. The editing unit edits the moving image data recorded by the moving image recording unit based on the editing content represented by the reproduction control information recorded by the reproduction control information recording unit with respect to the moving image. It is characterized by that.

  The invention according to claim 9 is the invention according to any one of claims 3 to 7, further comprising post-edit moving image recording means for recording moving image data edited by the editing means. .

  A tenth aspect of the present invention is the first aspect of the invention according to the ninth aspect, wherein the moving image data recorded in the edited moving image recording means is reconfigured by changing the order in predetermined units, Based on the moving image data recorded in the edited moving image recording means, an image representing a special effect is generated, and the generated image is combined with moving image data in a predetermined unit to be reconstructed. And a configuration means.

  According to an eleventh aspect of the present invention, in the invention according to any one of the third to tenth aspects, the image processing apparatus further comprises reproduction output means for reproducing and outputting the moving image edited by the editing means.

  The invention described in claim 12 is a control program executed by a computer incorporated in an electronic camera, wherein the computer captures a moving image capturing means for capturing a moving image and a moving image captured by the moving image capturing means. A moving image recording means for recording, an editing operation input means for inputting an editing operation on the moving image recorded by the moving image recording means, and the moving image according to the editing operation input by the editing operation input means. Reproduction control information generating means for generating reproduction control information representing the editing content to be applied to the reproduction control information generated by the reproduction control information generating means is recorded in association with the moving image recorded by the moving image recording means. It functions as reproduction control information recording means.

  According to a thirteenth aspect of the present invention, there is provided a control program executed by a computer incorporated in an electronic camera, wherein the computer captures a moving image capturing unit that captures a moving image, and a moving image captured by the moving image capturing unit. A moving image recording means for recording data, a display means for generating and displaying a plurality of images based on moving image data recorded in the image recording means, and a plurality of images displayed by the display means An editing operation input means for inputting at least one of an editing operation for designating a reproduction order for images, or an editing operation for designating addition of an editing icon representing a special effect on an arbitrary image among the plurality of images, It functions as an editing means for editing the moving image data recorded in the moving image recording means based on the editing operation input by the editing operation input means. That.

  According to the first aspect of the present invention, in an electronic camera that captures and records a moving image, an editing operation on the captured moving image is input, and editing is performed on the moving image in accordance with the input editing operation. By generating the reproduction control information (playlist) representing the contents, the actual editing process for the moving image data is not executed. Therefore, when the performance of the hardware mounted on the electronic camera is not sufficient, for example, the CPU performance is low. Even when the memory capacity is small, the contents of various edits to the moving image instructed by the user can be recorded. Thus, by recording the playback control information together with the moving image, it is possible to separately output the moving image based on the playback control information in the own electronic camera or in another device.

  According to the second aspect of the invention, in addition to the effect of the first aspect of the invention, the recorded moving image is reproduced and output while being edited in accordance with the editing content represented by the reproduction control information. It is possible to reproduce and output a moving image by repeatedly performing various edits simply by changing. Since editing processing such as alteration or modification is not actually performed on the moving image data, the image quality is not deteriorated even if various editing operations are repeatedly executed.

  According to the third aspect of the present invention, when the reproduction control information represents a special effect for transitioning an image, a continuous image representing the transition of the image is generated based on the transition source image and the transition destination image. Thus, since it is synthesized and output to the moving image, it is possible to reproduce and output the moving image that has been edited using various special effects without changing the original moving image data.

  According to the fourth aspect of the present invention, in an electronic camera that captures a moving image and encodes and records the moving image data, a plurality of images are generated based on the recorded moving image data and displayed in a list. When the editing operation to specify the playback order for the displayed images and the editing operation to specify the addition of editing icons representing special effects are input, the hardware installed in the electronic camera is not sufficient Even if, for example, the display screen size is small or the devices (buttons and switches) used for input operations are limited, for example, images are selected in the order of playback from the list display, and special effect editing icons are applied to the images. Since the editing operation by a simple selection operation such as selecting and adding can be performed, the editing operation can be facilitated.

  According to the fifth aspect of the invention, in addition to the effect of the fourth aspect of the invention, when displaying a list of images, the representative image for each scene of the moving image is displayed, and the image included in the moving image has Since the scene switching position can be identified based on the degree of correlation of information, editing operations such as changing the playback order and adding special effects can be facilitated for each scene.

  According to the sixth aspect of the invention, in addition to the effect of the fourth aspect of the invention, when displaying a list of images, a representative image for each scene of the moving image is displayed. By inputting the operation to specify and recording the input delimiter position data in association with moving image data, the scene switching position can be identified based on the delimiter position data. Therefore, editing operations such as changing the playback order and adding special effects can be facilitated for each scene.

  According to the seventh aspect of the present invention, since an editing operation for moving picture data for each predetermined unit, for example, an image for each GOP (Group Of Pictures) is input when encoding is performed by the MPEG system, encoding is performed. Editing in a predetermined unit (GOP) can be easily performed with the moving image data as it is.

  According to the eighth aspect of the present invention, in addition to the effect of any of the third to seventh aspects, the editing content to be applied to the moving image is represented in accordance with the editing operation input by the editing operation input means. If the performance of hardware mounted on the electronic camera is not sufficient because the generation of playback control information (playlist) does not execute actual editing processing on moving image data, for example, the CPU performance is low, the memory capacity Even if there are few cases, it is possible to record the contents of various edits to the moving image instructed by the user. Thus, by recording the playback control information together with the moving image, it is possible to separately output the moving image based on the playback control information in the own electronic camera or in another device.

  According to the ninth aspect of the invention, in addition to the effect of the invention according to any one of the third to seventh aspects, the moving image data is recorded after being edited. By reproducing the data as it is, it is possible to output a moving image that has been edited according to various editing operations.

  According to the tenth aspect of the invention, in addition to the effect of the ninth aspect of the invention, editing for reconfiguring moving image data by changing the order in predetermined units, and special effects based on moving image data Is generated, and editing is performed by combining and reconstructing the generated image with a predetermined unit of moving image data. For example, when encoding of moving image data is performed by the MPEG method, Editing can be performed for each GOP (Group Of Pictures) as a predetermined unit, and editing can be easily performed with the encoded moving image data.

  According to the eleventh aspect of the present invention, in addition to the effect of the third aspect, the edited moving image can be reproduced and output.

  According to the twelfth aspect of the present invention, the computer inputs an editing operation on the captured moving image, and reproduction control information (playlist) representing editing contents to be applied to the moving image in accordance with the input editing operation. ), The actual editing process for the moving image data is not executed, so that the computer performance is insufficient, for example, even when the CPU performance is low or the memory capacity is small. The contents of various edits to the moving image instructed by the user can be recorded. Thus, by recording the playback control information together with the moving image, it is possible to separately output the moving image based on the playback control information in the electronic camera or in another device.

  According to a thirteenth aspect of the present invention, in an electronic camera that captures a moving image and encodes and records the moving image data, the computer generates a plurality of images based on the recorded moving image data. It was installed in the electronic camera by displaying it in a list and operating it to input an editing operation that specifies the playback order for the displayed images and an editing operation that specifies the addition of editing icons representing special effects. Even if the performance of the hardware is not sufficient, for example, when the display screen size is small or the devices (buttons and switches) used for input operations are limited, images are selected in the order of playback from the list display, for example, The editing operation is easy because it can be done by a simple selection operation, such as selecting and adding an edit icon for the special effect to be applied. Door can be.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a circuit system configuration of an electronic camera according to the first embodiment of the present invention. The present invention can be applied to portable electronic cameras such as an electronic still camera, a video camera, a camera-equipped mobile phone, and a camera-equipped information communication terminal equipped with a function capable of recording and reproducing moving images.

  The electronic camera according to the first embodiment can capture a moving image in addition to a still image, and a moving image that has been subjected to various edits such as trimming, cut editing, and addition of special effects to the moving image data after shooting and recording. A moving image editing / playback function is provided that enables playback to be performed using a built-in low-speed CPU or small-capacity memory.

  The electronic camera includes a portable camera body 2 and a zoom imaging unit 4. The zoom imaging unit 4 is coupled to the portable camera body unit 2 by a connector unit 6 so as to be rotatable, for example. By enabling the zoom imaging unit 4 to rotate, it is possible to arbitrarily set the shooting direction by the zoom imaging unit 4 while maintaining the state of the portable camera body 2.

  The portable camera body 2 can be attached to the charging / PC transfer cradle 7. The portable camera body 2 can be charged by receiving power supply via the charging / PC transfer cradle unit 7. The portable camera body 2 can be connected to other information devices such as a personal computer via the charging / PC transfer cradle unit 7, and still images, moving images, etc. can be connected to other information devices. It is possible to send and receive various data or to receive programs.

  The charging / PC transfer cradle unit 7 supplies power to the portable camera body 2 and mediates transmission and reception of various data between the portable camera body 2 and other information devices. The charging / PC transfer cradle unit 7 is connected to the AC adapter unit 8 and inputs DC (direct current).

  The AC adapter unit 8 is connected to an AC power source, converts the AC power source into a DC output, and supplies the DC output to the charging / PC transfer cradle unit 7.

First, details of the configuration of the portable camera body 2 will be described.
The portable camera body 2 is provided with a control circuit 10 that controls the entire camera. The control circuit 10 is composed of a CPU, a ROM for fixedly storing programs for realizing various processes described later, image data to be displayed on the screen in each operation mode, and a RAM used as a work memory. The The electronic camera according to the first embodiment encodes and records moving image data captured by the zoom imaging unit 4, receives editing operations on the moving image, creates reproduction control information (a playlist to be described later), It is possible to reproduce and output based on the encoded data while referring to the reproduction control information when reproducing the moving image.

  The control circuit 10 executes control and setting of various functions in accordance with instructions input from various input devices through the input circuit 12. The input circuit 12 notifies the control circuit 10 of inputs from, for example, a still image shooting half-press / shutter button 13, a moving image shooting start / stop button 14, and other operation input units 15. The operation input unit 15 includes, for example, a power ON / OFF switch for switching on / off the power, a mode switch for switching an operation mode (for example, a shooting mode (moving image, still image), a playback mode, an editing mode, etc.), Zoom operation lever for zoom operation, menu key for displaying menu, cursor key for inputting up / down / left / right direction, set key for instructing selection execution, display screen switching and selection target A display button or the like for instructing a change is included.

  The control circuit 10 captures an image (a still image or a moving image) by controlling a control unit 70 (described later) provided in the zoom imaging unit 4 via the connector unit 6.

  The control circuit 10 includes a display memory 17, a display drive unit 18, an image buffer memory 20, an image signal processing unit 22, a compression encoding / decoding unit 24, an editing / playback control unit 26, a still image / moving image / audio data memory. 28, a playlist memory 30, a speech decoding unit 31, an external memory interface 36, a speech encoding unit 40, a power control unit 44, a power control unit 44, an external I / O interface 48, and the like are connected.

  The display memory 17 stores image data to be displayed on the display unit 19 such as image data of an image photographed by the zoom imaging unit 4 or image data of various menus.

  The display drive unit 18 causes the display unit 19 to display an image based on the image data recorded in the display memory 17. For example, in the shooting mode, an image shot by the zoom imaging unit 4 is displayed through in the display memory 17 and functions as a viewfinder of the electronic camera. The display unit 19 is configured by, for example, a liquid crystal display. When the electronic camera is configured to be portable, the size of the display screen is relatively small due to the limitation of the housing size.

  The image buffer memory 20 temporarily stores image data input from the zoom imaging unit 4 (A / D circuit 76).

  The image signal processing unit 22 performs image processing on the image data from the zoom imaging unit 4 recorded in the image buffer memory 20, and performs processing such as data compression by the compression encoding / decoding unit 24 to stop the image data. The captured image is recorded in the image / moving image / audio data memory 28 (or the external memory 37 via the external memory interface 36).

  The compression encoding / decoding unit 24 performs an encoding process on the image data (moving image and still image) input from the zoom imaging unit 4 and a decoding process of the encoded data. In the first embodiment, description will be made on the assumption that an image is encoded / decoded by the Moving Picture Experts Group (MPEG) method.

  The edit / playback control unit 26 realizes the characteristic function of the electronic camera in the first embodiment, and after editing the moving image data according to the editing operation on the moving image shot through the zoom imaging unit 4. Changing the playback order and transition screen for every predetermined unit (for example, every time, every scene, every GOP, every chapter) without converting into moving image data and recording (while leaving the original data of the encoded moving image) Reproduction control information representing effects and the like is generated and edited as a playlist and recorded in the playlist memory 30. When reproducing the moving image data, the editing / reproduction control unit 26 edits or specially processes the encoded moving image data while referring to the reproduction control information of the playlist recorded in the playlist memory 30. Is generated and output.

  The still image / moving image / audio data memory 28 stores still image data, moving image data, and audio data as respective data files. The moving image data is encoded and recorded as an MPEG file. As a file of multiplexed encoded data in which image data (still image data, moving image data) encoded by the compression encoding / decoding unit 24 and audio data encoded by the audio encoding unit 40 are multiplexed. To be recorded.

  The playlist memory 30 stores a playlist including playback control information created by the editing / playback control unit 26. Details of the playlist will be described later (FIGS. 20, 21, 22, and 23).

  The audio decoding unit 31 decodes the encoded audio data. The audio data decoded by the audio decoding unit 31 is converted into an audio signal by the A / D conversion unit 32, amplified by the amplification unit 33, and output from the speaker 34.

  The external memory interface 36 is an interface with an external memory 37 that is mounted in a slot (not shown), and performs recording or reading of various data with respect to the external memory 37 under the control of the control circuit 10.

  The audio encoding unit 40 encodes audio data output from the A / D conversion unit 41. The A / D conversion unit 41 receives an audio signal input from the microphone 43 and amplified by the amplification unit 42, performs A / D conversion, and outputs audio data.

  The power supply control unit 44 controls the power supply of the portable camera body 2, supplies power from the battery 46 to each unit, and power from the charging / PC transfer cradle unit 7 connected via the connector 50. Receiving the supply, the battery 46 is charged.

  The external I / O interface 48 is an interface with an external device connected via the connector 50, for example, the charging / PC transfer cradle unit 7, and is connected to the external device under the control of the control circuit 10. Control data input and output.

  The connector 50 is an interface connected to the connector 92 provided in the charging / PC transfer cradle unit 7, and the portable camera body 2 is placed on a holder provided in the charging / PC transfer cradle unit 7. Then connect the power line, data line, control line and so on.

Next, a detailed configuration of the zoom imaging unit 4 will be described.
In the zoom imaging unit 4, subject light received through a lens optical system 52 including a plurality of lenses, a diaphragm 53, a half mirror 54, and a shutter 55 is introduced into an imaging device 60 such as a CCD (Charge Coupled Device) and subjected to photoelectric conversion. The In the lens optical system 52, the lens position is changed by the lens driving unit 58 for focus control, and the position is changed by the lens driving unit 59 for zoom control. The photometry / ranging sensor 62 performs photometry / ranging based on light input via the lens optical system 52a.

  The zoom imaging unit 4 controls the shooting operation by a control unit 70 that operates under the control circuit 10 of the portable camera body 2. The control unit 70 includes a focus control unit 71, a zoom control unit 72, an aperture control unit 73, a shutter control unit 74, a CDS / AGC 75, an A / D circuit 76, a timing control drive unit 77, an exposure control unit 78, and a distance measurement control. Unit 79 and flash control unit 80.

The focus control unit 71 adjusts the focus by driving and controlling the lens position through the lens driving unit 58.
The zoom control unit 72 performs zoom adjustment by driving and controlling the lens position through the lens driving unit 59.

  A diaphragm control unit 73 controls a diaphragm 53 provided in the lens optical system 52.

  The shutter control unit 74 controls opening and closing of the shutter 55, and the shutter control unit 74 is controlled by the control circuit 10 according to the operation of the still image shooting half-press / shutter button 13 or the moving image shooting start / stop button 14 of the portable camera body 2. In response to the instruction, the shutter 55 is opened, and light input through the lens optical system 52 is introduced into the image sensor 60.

  The timing control drive unit 77 performs scanning drive control on the image sensor 60, converts the signal photoelectrically converted by the image sensor 60 into digital data through the CDS / AGC 75 and the A / D circuit 76, and outputs the digital data to the control circuit 10.

  The exposure control unit 78 and the distance measurement control unit 79 are used for photometry and distance measurement by the photometry / ranging sensor 62 based on the light input through the lens optical system 52 (lens optical system 52a) and the half mirror 54. The corresponding distance measurement control and exposure control are executed.

  Under the control of the control circuit 10, the flash control unit 80 causes the flash drive unit 82 to drive the flash 83 in accordance with the shutter operation for the still image shooting half-press / shutter button 13. When the strobe drive is set, the flash drive unit 82 charges the power from the battery 46 via the power supply control unit 44 to a large capacity capacitor for flash (not shown) and then drives the flash 83 to flash.

  In addition, the photometry / ranging sensor 62 is used to automatically adjust exposure conditions, aperture opening / closing amount, shutter speed, white balance, imaging sensitivity, etc., and a distance sensor such as a PSD (Position Sensitive Detector) sensor It is possible to realize an autofocus mechanism that automatically drives and controls the lens position according to the distance, and a multifocal zoom lens drive mechanism.

Next, a detailed configuration of the charging / PC transfer cradle unit 7 will be described.
The charging / PC transfer cradle unit 7 is provided with a control circuit 90 that controls the whole, and is connected to the portable camera body 2 (connector 50) via a connector 92.

  Connected to the control circuit 90 are a charge control unit 94, a power / charge / transfer indicator 96, a wireless LAN interface (I / F) 98, a LAN communication interface (I / F) 104, and a USB communication interface (I / F) 106. Has been.

  The charging control unit 94 supplies power from the AC adapter unit 8 to the portable camera main unit 2 via the connector 92 and performs control for charging the battery 46 of the portable camera main unit 2.

  The power / charge / transfer indicator 96 is an indicator for notifying the user of the operating state of the charge / PC transfer cradle unit 7, whether or not connected to the power source, and power supply (charging) to the portable camera body 2. The presence / absence of data and the presence / absence of data transfer with an external information device are notified by lighting or blinking.

  The wireless LAN I / F 98 is an interface for controlling wireless communication with an external information device via the encoding / decoding unit 99, the modulation / demodulation unit 100, and the transmission / reception unit 101.

The LAN communication I / F 104 is an interface that controls communication with another information device connected via a cable via the LAN connector 110.
The USB communication I / F 106 is an interface that controls communication with another information device connected via a cable via the USB connector 108.

  In response to a request from the control circuit 10 of the portable camera body 2, the control circuit 90 is connected to another information device such as a wireless LAN I / F 98, a LAN communication I / F 104, or a USB communication I / F 106. Communication such as transfer of image data can be executed with a personal computer (PC), a printer device, and another electronic camera.

  The AC adapter unit 8 includes a stabilization circuit 120, a smoothing circuit 122, a rectifier circuit 124, and a fuse 126. The AC adapter unit 8 converts the AC power into a DC output by the stabilization circuit 120, the smoothing circuit 122, and the rectifier circuit 124, and the charging / PC transfer cradle unit 7 (charging control unit 94) via the DC input connector 112. ).

  Next, a reproduction editing control unit 130 that creates reproduction control information (playlist) in accordance with an editing operation on a moving image (including still images, audio, etc.) in the first embodiment, and a moving image with reference to the reproduction control information Each function of the decoding reproduction control unit 140 that reproduces and outputs an image will be described.

  In the electronic camera of the first embodiment, when editing a captured moving image, instead of editing the moving image data and recording the edited moving image, the moving image is encoded and compressed and recorded by MPEG or the like. Edit contents such as changing the playback order (jump) or skipping every predetermined unit (for example, every GOP unit, every time, every scene), transition screen effect (transition effect) when switching scenes, with the original data remaining Reproduction control information to be represented is set and recorded as a reproduction control program (playlist).

  Also, when playing back and outputting moving image data, the playlist is called and the playback order is automatically switched while referring to the playback control information, or a moving image with special processing is generated and played back. To do.

  FIG. 2 is a block diagram showing the functional configuration of the playback / editing control unit 130, and FIG. 3 is a block diagram showing the functional configuration of the decoding / playback control unit 140. The playback / editing control unit 130 and the decoding / playback control unit 140 are realized by the control circuit 10, the compression encoding / decoding unit 24, and the editing / playback control unit 26. The memory media / communication paths 136 and 142 correspond to communication paths (external devices) connected via the still image / moving image / audio data memory 28 and the playlist memory 30 and, for example, the external I / O interface 48. .

  As shown in FIG. 2, the playback editing control unit 130 includes a playback method editing unit 130a, a demultiplexing / decoding unit 130b, various decoding units 130c, an intra-frame image decoding unit 130d, a reduced index image generation unit 130e, and an audio index generation unit 130f. A playlist creation unit 130g and a scene switching identification unit 130h are provided.

  The reproduction method editing unit 130a inputs an editing operation from the user to the encoded moving image data (MPEG file 132), and in response to the editing operation, the reproduction method editing unit 130a includes a reproduction control information that represents the editing content of the moving image. Edit the list. For example, the playback / editing control unit 130 includes, for example, skip, jump, and wipe along with a column of index images (for example, I pictures) for each unit (for example, GOP) to be edited by the index image creation unit 130e in the edit mode screen. Edit icons such as dissolve, fade-in, fade-out, title addition, BGM, post-recording sound, and sound effect addition are displayed. The reproduction method editing unit 130a inputs an editing operation by allowing the user to select an index image and an editing icon to be edited in the editing mode screen. Since the user only has to select an index image and an edit icon as an editing operation for a moving image, for example, a playlist can be automatically generated as if a musical note or accompaniment is added to a score.

  The demultiplexing / decoding unit 130b separates the encoded moving image data and audio data from the multiplex encoded data (multiplex encoded data) of the MPEG file 132.

  The various decoding units 130c decode the encoded image data or audio data of the image / audio file 134 prepared for editing a moving image. The audio file for editing includes, for example, audio data to be added to a moving image such as BGM, post-recording sound, and sound effect for reproduction.

  The intra-frame image decoding unit 130d performs intra-frame decoding on the image data separated by the demultiplexing / decoding unit 130b or the various decoding units 130c to generate image data.

  The reduced index image creation unit 130e is set by the user for each unit of editing processing, for example, for each GOP unit, for each scene unit, for each predetermined time unit, based on the image data generated by the intra-frame image decoding unit 130d. An index image for each chapter or the like is generated and displayed as a list on the display unit 19 in the edit mode. When displaying an index image for each scene unit, the index image creating unit 130e creates an index image according to the scene switching position in the moving image identified by the index image creating unit 130e. Note that the reproduction / edit control unit 130 not only displays the index image generated by the index image generation unit 130e for each unit to be edited, but also generates the moving image data while decoding the edit mode screen. You may make it display the clip of a reduced moving image.

  The voice index creation unit 130f creates a voice index for each unit of editing processing based on the voice data decoded by the various decoding units 130c, and causes the display unit 19 to display a list.

  The playlist creation unit 130g creates a playlist including playback control information indicating the editing content for the moving image edited by the playback method editing unit 130a, and records the playlist in the playlist memory 30 as a playlist file 136b. The playlist can be a simple program such as an XML (Extensible Markup Language) format, CSV (Comma Separated Value), for example, if the playlist can identify the correspondence between the data file in a table format or the edit code and the corresponding GOP, for example. It can be a file or text file. A detailed configuration example of the playlist will be described later.

  The scene switching identification unit 130h, for example, in accordance with the degree of correlation between images based on an image (for example, an I picture) decoded by the intra-frame image decoding unit 130d or an instruction input from the user at the time of moving image shooting A scene break position is identified according to data recorded in association with an image.

  In the editing process of the playback method by the playback editing control unit 130, the playlist file 136b is created according to the editing operation on the moving image. However, since the moving image data itself is not edited, the MPEG file 136a and the additional image are added. The audio file 136c remains the original data.

  Next, as shown in FIG. 3, the decoding / playback control unit 140 includes a playback order control unit 140a, an interframe decoding unit 140b, an intraframe decoding unit 140c, a special effect synthesis unit 140d, an intraframe decoding unit 140e, and an interframe decoding. 140f, sound effect synthesis unit 140g, and speech decoding unit 140h are provided.

  The reproduction order control unit 140a refers to the playlist file 142a (136b) created by the reproduction / editing control unit 130, and the moving image data of the MPEG file 142b, the image data of the additional image file 142c, and the additional audio file 142d. Controls the playback order of audio data. The playback order control unit 140a includes decoding by the inter-frame decoding units 140b and 140f and intra-frame decoding units 140c and 140e, synthesis of special effects (image generation) by the intra-frame decoding unit 140e, a sound effect synthesizing unit 140g and a voice decoding unit 140h. Controls the order of sound generation.

  The inter-frame decoding units 140b and 140f generate image data by inter-frame decoding for the encoded moving image data.

  The intra-frame decoding units 140c and 140e generate image data by intra-frame decoding for the encoded moving image data.

  The special effect synthesizing unit 140d has two images (for example, transition source images) decoded by the intra-frame decoding units 140c and 140e, respectively, for example, special effect images set in the playlist and added to the moving images. , Transition destination image). Here, for example, a continuous image of a transition screen composed of a plurality of images is synthesized according to a special effect.

  The sound effect synthesizing unit 140g synthesizes sound effects to be added to the moving images set in the playlist based on the sound data of the additional sound file 142d.

  The audio decoding unit 140h synthesizes audio (BGM, post-recording sound, etc.) to be added to the moving images set in the playlist based on the audio data of the additional audio file 142d.

  The reproduction output control unit 140k is configured to display images decoded by the inter-frame decoding units 140b and 140f, images decoded by the intra-frame decoding units 140c and 140e, special images according to the playlist (reproduction control information) of the playlist file 142a. By outputting the image synthesized by the effect synthesis unit 140d, the voice synthesized by the sound effect synthesis unit 140g, and the voice decoded by the voice decoding unit 140h, a moving image (image that has been subjected to editing represented by the playlist) , Voice).

  Next, the reproduction / editing by the reproduction / editing control unit 130, the reproduction output control by the decoding / reproduction control unit 140, and the detailed configuration of the compression encoding / decoding unit 24 will be described with reference to FIG.

  As shown in FIG. 4, the compression encoding / decoding unit 24 shown in FIG. 1 includes an MPEG encoder (encoder) 24a and an MPEG decoder (decoder) 24b. Note that the MPEG encoder 24a and the MPEG decoder 24b are encoder / decoders using the existing MPEG system, and a detailed description thereof will be omitted.

  The MPEG encoder 24a encodes the moving image data shot by the zoom imaging unit 4 to generate encoded image data. The MPEG encoder 24a multiplexes audio encoded data for audio input from the microphone 43, which is output from the audio encoding unit 40 at the time of moving image shooting, with image encoded data, and generates multiplexed encoded data (code Output as converted video data).

  The playback / editing control unit 130 performs demultiplex decoding on the encoded moving image data (MPEG file) with the configuration shown in FIG. 2 (demultiplexing / decoding unit 130b), for example, by intra-frame image decoding in units of GOPs. An index image is created based on the obtained image (I picture) and displayed on the edit mode screen. The playback / editing control unit 130 edits the playlist in accordance with the input operation specified on the editing mode screen by the user, that is, the specification of the playback order for the index image and the selection of the edit icon, and the playlist file 136b (142a). Record as.

  The playback order control unit 140a of the decoding / playback control unit 140 refers to the playlist (playlist files 136b and 142a) generated by the playback / editing control unit 130 and is encoded for each predetermined unit by the MPEG decoder 24b. The order of decoding the moving image data (including audio data) is controlled. Further, based on the I picture image 171 generated by intra-frame decoding by the MPEG decoder 24b, the transition image (continuous image) corresponding to the editing of the special effect is synthesized by the special effect synthesis unit 140d. The playback output control unit 140k outputs the image decoded by the MPEG decoder 24b, the continuous image representing the synthesized special effect, and the audio according to the playback control information set in the playlist (video playback output, audio). Playback output).

  Next, an outline of the hierarchical structure and the encoded / decoded frame order of MPEG moving image data generated by encoding by the compression encoding / decoding unit 24 will be described.

FIG. 5 shows a hierarchical structure of MPEG moving image data.
As shown in FIG. 5, MPEG moving image data has a six-layered structure including a sequence layer, a GOP (Group Of Pictures) layer, a picture layer, a slice layer, a macroblock layer, and a block layer.

  The sequence layer shown in FIG. 5A includes a series of screen groups having the same attribute, and includes a GOP layer. The GOP layer is a minimum unit of a screen group that is a unit of random access. The picture layer has attributes common to one screen (for example, screen coding mode (I, P, B picture types)), and the slice layer is common to small screens obtained by dividing one screen into arbitrary lengths. The information and macroblock layer are information common to the pixel blocks obtained by further dividing the slice layer, and the block layer represents the variable coefficient itself.

FIG. 6 shows the order of frame images at the time of encoding / decoding.
In MPEG, three types of images, I picture (Intra encoded image), P picture (Predictive encoded image), and B picture (Bidirectionally predictive encoded image) are defined.

  The I picture is a screen that is provided in order to maintain the independence of the GOP, and is intra-frame encoded only from the information, and is generated without using inter-frame prediction. A P picture is a screen generated by performing prediction from an I picture or a P picture, and includes an intra-frame prediction screen and a forward inter-frame prediction screen. A B picture is a screen generated by bi-directional prediction, and is generated by intra-frame prediction encoding and inter-frame prediction encoding in the forward and reverse directions.

  As shown in FIG. 6A, when the original image (input image) is input, the B picture is skipped, the next I and P pictures are encoded first, and then the B picture in between is encoded. Therefore, in the encoded image (in the recording medium), a B picture is inserted between the I picture and the P picture as shown in FIG.

  When the encoded image shown in FIG. 6B is decoded (output), the B picture is immediately decoded and displayed, and the I and P pictures are decoded and then the intermediate B picture is decoded and displayed. Display after.

  In the electronic camera according to the first embodiment, when an editing operation for a moving image is input, for example, intra-frame decoding is performed in units of GOP, and an I picture is displayed as an index image on the editing mode screen. Since the input of the editing operation is performed by designating the index image, editing (playlist creation) in GOP units can be easily and quickly performed while the moving image data is encoded.

Next, the operation of the electronic camera in the first embodiment will be described.
FIG. 7 is a flowchart for explaining the overall control flow of the electronic camera.

  First, when the mode switch of the operation input unit 15 is switched to the shooting mode, the control circuit 10 is instructed to switch the operation mode through the input circuit 12, and thereby shifts to the shooting mode (step A1, Yes). When the photographing mode is entered, the control circuit 10 sets photographing conditions (step A2), controls the distance measurement control unit 79 to execute the light metering process by the light metering / ranging sensor 62 (step A3), and zooms. The control unit 72 is controlled to execute zoom processing (step A4), and the focus control unit 71 is further controlled to execute autofocus processing (step A5).

  The control circuit 10 displays the image captured by the zoom imaging unit 4 on the display unit 19 so as to function as a finder (step A6).

  Here, when execution of still image shooting is instructed by an operation on the still image shooting half-press / shutter button 13 (step A7), the control circuit 10 executes photometry processing (step A8), and the zoom imaging unit 4 A still image is photographed according to the photographing conditions and the photometric value (step A9). The control circuit 10 temporarily records still image data captured by the zoom imaging unit 4 in the image buffer memory 20, encodes it by the compression encoding / decoding unit 24, and stores the still image / moving image / audio data memory 28 or Then, it is recorded in the external memory 37 through the external memory interface 36 (step A10). Further, the control circuit 10 stores the captured still image data in the display memory 17 and causes the display drive unit 18 to display the still image data on the display unit 19 (step A11).

  On the other hand, when execution of moving image shooting is instructed by an operation on the moving image shooting start / stop button 14 (step A14, Yes), the control circuit 10 executes photometry processing (step A15), and the zoom imaging unit 4 A moving image is photographed according to the photographing conditions and the photometric value (step A16). The control circuit 10 sequentially records the moving image data captured by the zoom imaging unit 4 in the image buffer memory 20, encodes it by the compression encoding / decoding unit 24, and stores the still image / moving image / audio data memory 28 or Then, it is recorded in the external memory 37 through the external memory interface 36 (step A17).

  FIG. 8A shows an example of a screen displayed on the display unit 19 in the moving image shooting mode for shooting a moving image.

  Here, when there is another key operation, the control circuit 10 executes display processing or the like according to the input key operation (steps A12 and A13).

  By the way, when the mode is switched to the playback mode by operating the mode switch of the operation input unit 15, the control circuit 10 activates the editing / playback control unit 26 and shifts to the playback mode (Yes in Step A22).

  When the playback mode is entered, the control circuit 10 displays a playback mode display screen for playing back a moving image for each file, for example, as shown in FIG. 8B. Here, for example, when an index display execution is instructed by an operation of moving the zoom operation lever to the (W) side (step A23, Yes), the control circuit 10 records in the still image / moving image / audio data memory 28. An index image is generated based on the image data being displayed, and a list of index images is displayed on the display unit 19 (step A24).

  FIG. 8C shows an example of the display screen of the reproduction mode displayed as an index. In the list of index images, for example, a moving image to be reproduced can be arbitrarily designated by operating a cursor key. In the index image list display, for example, by operating the zoom operation lever to the (T) side, a display screen of a playback mode for reproducing a moving image for each file shown in FIG. 8B can be displayed. .

  In the playback mode screen shown in FIG. 8B, chapter registration buttons are provided. For example, a chapter can be set for a moving image by operating the down key of the cursor key. The control circuit 10 records chapter-related data in association with moving image data in accordance with chapter specification from the user.

  When the menu display is requested by an operation on the menu key of the operation input unit 15, the control circuit 10 displays a menu as shown in FIGS. 9A and 9B on the display unit 19, for example.

  For example, a playback menu provided with a plurality of items related to playback as shown in FIG. 9A, an edit menu provided with a plurality of items related to video editing shown in FIG. 9B, and various settings. Related setting menus (not shown) are prepared. For example, the control circuit 10 switches the menu display according to the operation of the left and right cursor keys. In addition, the control circuit 10 switches the menu item to be selected in each menu according to the operation of the up / down key.

  In the playback menu, execution of playback of moving images or still images can be instructed. For example, as shown in FIG. 9A, by selecting an item “play list reproduction”, it is possible to instruct execution of moving image reproduction while performing editing according to a moving image playlist to be described later. . In the edit menu, as shown in FIG. 9B, by selecting the item “playlist editing / effect”, it is possible to instruct execution of editing (playlist creation) on a moving image to be described later. it can.

  When normal moving image reproduction is instructed in the reproduction mode (for each file shown in FIG. 8B) (steps A23, A25, A26 (No)), the control circuit 10 stores the still image / moving image / audio data memory. The moving image data selected as the reproduction target from the index image list display is read out from the plurality of moving image data recorded in 28, and decoded by the compression encoding / decoding unit 24 (step A28). The moving image data is stored in the display memory 17 and reproduced and displayed by the display unit 19 (step A29). The control circuit 10 continues the reproduction output until the processing is completed for all the moving image data to be reproduced (step A30).

  Further, when playback by a playlist is instructed from the playback menu (step A26, Yes), the control circuit 10 is stored in the playlist memory 30 by the editing / playback control unit 26 and the compression encoding / decoding unit 24. A reproduction process using the playlist is executed (step A27). Details of the reproduction processing by the playlist will be described later (see FIG. 28).

  Note that when playback of a still image is instructed from a playback menu or the like (step A31, Yes), the control circuit 10 selects a plurality of still images recorded in the still image / moving image / audio data memory 28 as a playback target. The read still image data is read out and decoded by the compression encoding / decoding unit 24 (step A32), and the decoded image data is recorded in the display memory 17 to be displayed on the display unit 19 (step A33).

  On the other hand, when the operation of the mode switch of the operation input unit 15 or the execution of the editing mode is instructed from the editing menu (step A18, Yes), the control circuit 10 performs editing for executing editing on a normal moving image or still image. The process shifts to an edit mode for executing an edit process of a playback method for creating a playlist in response to the process or edit operation input (step A19). Details of the editing process of the playback method will be described later (see FIG. 10).

  When there is another key operation, the control circuit 10 executes display processing or the like according to the input key operation (steps A20 and A21).

Next, editing processing (playlist creation) of the playback method shown in step A19 in FIG. 7 will be described with reference to the flowchart shown in FIG. 10 (see the playback editing control unit 130 shown in FIG. 2).
In the editing process of the playback method in the first embodiment, an index image is created from an I picture obtained by intra-frame decoding, for example, setting of a playback method such as editing for moving images and special effects (list editing in advance) in GOP units. Then, a list (or a clip of a reduced moving image generated while decoding moving image data may be used) can be displayed, and the playback order can be designated with the cursor key or the like for the index image on this list screen. Also, for example, a plurality of editing icons such as skip, jump, wipe, dissolve, fade-in, fade-out, title, BGM, post-recording sound, and sound effect addition are provided below the index image column, and the edit icon is selected with the cursor key. By selecting, it is possible to input edit contents for a predetermined unit of moving image corresponding to the index image. The user inputs an editing operation by selecting an index image to be edited and an editing icon, and edits and records the operation content as a playlist.

  First, in accordance with designation from the user, moving image data to be edited is selected from a plurality of moving image data (MPEG file 132) recorded in the still image / moving image / audio data memory 28 (step B1).

  The reproduction editing control unit 130 decodes the moving image data separated by the demultiplexing / decoding unit 130b by the compression encoding / decoding unit 24, generates an index image for each predetermined unit, and displays it as an editing mode screen. A list is displayed at 19 (step B2). Here, as index images in a predetermined unit, for example, index images for each time, each scene, and each GOP can be generated and displayed in a list.

  FIGS. 11A and 11B show an example in which a list of index images is displayed on the edit mode display screen. On the index image list screen, tags corresponding to “every hour”, “every scene”, and “every GOP” are provided. For example, the display is switched to the tag selection state by the display button, and the up and down keys of the cursor key are used. A tag can be arbitrarily selected. Note that the top image for each chapter set by the user in the playback mode or edit mode may be displayed as a list as an index image (not shown).

  FIG. 11A shows an example of a list display of index images when the “every hour” tag is designated. In the list display of “every hour” index images, for example, index images (or clip moving images) are generated from the top image every five minutes and displayed as a list. In addition, the timeline (time axis scale) is displayed to represent the length of the entire moving image, and also represents the range of moving images that are currently subject to index image list display. On the list display screen, the range of index images to be displayed is switched using the up / down / left / right cursor keys, and the moving image in the range corresponding to the index image to be selected (for example, the index image at the upper left position) is set using the set key. Can be played. In addition, an “edit” item is provided for specifying execution of editing by changing the playback order in units of index images and adding special effects. For example, it is possible to instruct execution of an editing operation by an operation on a display button.

  FIG. 11B shows a list display example of index images when the tag “for each GOP” is designated. In the index display of “for each GOP”, for example, an index image (or clip moving image) is generated from the top image for each GOP unit of encoded moving image data and displayed in a list. The rest of the configuration is the same as that shown in FIG.

  Although not shown, when the “for each scene” tag is designated, the scene switching position is automatically identified from the moving image data to be edited (FIG. 2, scene switching identifying unit 130h), An index image is generated based on the identified first image for each scene and displayed as a list. Details of the scene identification process for identifying the scene switching position from the moving image data will be described later (FIGS. 17 and 18).

  When an index image is displayed in a list and an execution of an editing operation is instructed by operating a display button, for example, an editing mode screen for setting a playback order in units of index images is displayed.

  FIG. 11C shows an example of a screen in which the playback order is being set. As shown in FIG. 11C, a playback order area for displaying an index image in which the playback order is specified is provided at the top of the screen, and a captured image area for displaying the index image is provided at the bottom of the screen. Yes.

  On this screen, for example, the index image displayed in a list can be changed by operating the left and right cursor keys, and the index image to be reproduced (for example, the index image at the left end) can be designated by the set key. it can. The index image selected as a reproduction target by the operation of the set key is displayed in the reproduction order area at the top of the screen. Therefore, by sequentially selecting index images from a plurality of index images according to the reproduction order, it is possible to designate the reproduction order in a predetermined unit. In addition, it is possible to cut (delete) by not selecting as a reproduction target.

  In addition, on the screen in which the playback order is being set shown in FIG. 11C, an edit mode screen for adding special effects is displayed, for example, by operating the up and down cursor keys.

  FIG. 12A shows an example of a screen in which special effect addition is being set. As shown in FIG. 12A, a playback order area for displaying an index image in which the playback order is specified is provided at the top of the screen, and an edit icon representing editing (special effect) content is displayed at the bottom of the screen. An edit icon area is provided.

  In the edit icon area, tags of “transition”, “digital effect”, and “sound effect” are provided, and for example, an arbitrary tag can be selected by operating the left and right cursor keys.

  For example, when the “transition” tag is selected, as shown in FIG. 12A, a plurality of editing icons for adding editing for applying the transition screen effect are displayed. When the “digital effect” tag is selected, as shown in FIG. 12B, a plurality of editing icons for adding editing for applying the digital effect are displayed.

  In addition, in the edit mode screen on which the edit icons shown in FIGS. 12A and 12B are displayed, any of the index images set in the playback order area to which the edit icon is added can be arbitrarily selected by a key operation or the like. Can be specified.

  For example, as shown in FIG. 13A, editing by the transition screen effect (transition effect) is performed when there are moving images and clips # 1 and # 2 corresponding to two index images, respectively. By adding a transition edit icon to the index image corresponding to 1, the image A obtained by decoding the I picture of the last GOP of the clip # 1 and the I picture of the first GOP of the clip # 2 after the transition From the decoded image B, continuous images based on various screen effects representing image transition are combined and combined between clips # 1 and # 2 as shown in FIG.

  FIG. 14 shows an example of edit icons related to the transition effect (FIGS. 14A to 14H) and screen transitions corresponding to each. For example, from a simple one such as a wipe that changes to screen B to wipe screen A (FIG. 14A), a dissolve that changes so that screen A and screen B overlap and melt (FIG. 14B) When the door of the screen A is opened, the door opens to the screen B (FIG. 14 (d)), the page peels to turn to the screen B by turning the page of the screen A (FIG. 14 (e)), and the cube rotates. However, cube spins that change screens (FIG. 14 (f)) and the like that provide complex transition screen effects that require image reduction and deformation modification processing are prepared.

  Further, for example, as shown in FIG. 15A, when digital image editing is performed, if there is a moving image corresponding to the index image, for example, clip # 1, the digital effect is applied to the index image corresponding to clip # 1. By adding the edit icon, as shown in FIG. 15B, the digital effect indicated by the edit icon is applied to the corresponding clip # 1.

  FIG. 16 shows an example of editing icons (FIGS. 16A to 16) related to digital effects, and digital effects corresponding to the respective icons. For example, various digital effects such as sepia, monochrome, mosaic, zoom in / out, negative, rotation, mirror (mirror image), slim / stretch, frame / mark, inset composition, strobe movie, still (still image playback) What to apply is prepared.

  Although not shown, when the “sound effect” tag is selected, a sound effect or a plurality of editing icons representing BGM (for example, using the note symbols shown in FIG. 12B) are used. Display.

  As shown in FIGS. 12A and 12B, when an edit icon is selected in the edit icon area, the selected edit icon is added to the lower part of the index image set in the playback order area. Thereby, the contents of the editing operation can be visually recognized.

  As described above, editing of the playback order for the index image and editing operation input by adding an editing icon to the index image selected as the playback target are performed through the editing mode screen (steps B3, B4, and B5). Since the editing operation input here only needs to specify the index image and select the editing icon, it does not require a wide display screen and can be executed only by a simple operation using keys and buttons.

  When the editing operation input is thus performed, the reproduction / edit control unit 130 (reproduction method editing unit 130a) provisionally creates an edit / playback playlist in accordance with the edit icon added to the index image (step B6).

  Here, when execution of preview display of the editing result is instructed (step B7, Yes), the temporary playlist is designated as a playlist to be referred to for reproduction of the moving image (step B8), and a moving image using this playlist is designated. Image reproduction processing is executed (step B9). Note that the details of the reproduction processing by the playlist will be described later.

  Similarly, a temporary play list corresponding to each editing operation is created by selecting an index image arbitrarily from the index image list display and performing an editing operation to which an editing icon is added.

  When an editing end instruction is input after the editing operation input is thus performed (step B10, Yes), the reproduction editing control unit 130 determines whether or not to record the editing result as a playlist, for example, a display unit The user is inquired by displaying a message at 19 (step B11). Here, when recording as a playlist is instructed, an editing / playback playlist is created based on the editing result indicated by the temporary playlist and recorded in the playlist memory 30 (step B12).

  A playlist is generated by the above editing operation and recorded in the playlist memory 30.

  Here, a scene identification process executed to generate an index image for each scene when the index image list is displayed will be described (FIG. 2, intra-frame image decoding unit 130d, scene switching identification unit 130h). ). FIGS. 17 and 18 are flowcharts for explaining scene identification processing (first and second), respectively. In the electronic camera of the first embodiment, any one of the scene identification processes shown in the flowchart of FIG. 17 or 18 is used.

First, the first scene identification process shown in FIG. 17 will be described.
First, the first GOP of the moving image data to be edited is read (step C1), the I picture in the GOP is decoded in the frame, and temporarily stored in the previous memory (step C2).

  The next GOP is read, the I picture in the GOP is decoded in the frame, and temporarily stored in the current memory (step C3).

  Then, the correlation degree R using the spatial pattern information is calculated for the I picture stored in the current memory and the I picture stored in the previous memory (step C4). For example, the degree of correlation can be calculated using a correlation coefficient generally used in statistical processing or the like, or covariance and standard deviation.

  For example, the Pearson product moment correlation coefficient (R) is expressed by the following equation for an m × n image or pattern: R = (covariance of image f and image t) / {(standard deviation of image f) · (image (standard deviation of t)}.

That is, the product moment correlation coefficient (R) is calculated by the following equation (1).

  However, f [x, y] is the density value of the x and y coordinates of the input image f (previous memory), t [x, y] is the density value of the x and y coordinates of the reference image t (current memory), fAV is an average value of density values of the input image f, and tAV is an average value of density values of the reference image t.

  If the correlation degree R is greater than or equal to a predetermined value (step C5, Yes), it is regarded as a scene continuous with the previous GOP, and if the correlation degree is lower than the predetermined value (step C5, No), the GOP is switched to a new scene. Identify as a location. Then, a new scene number is added to the image starting with the GOP identified as the new scene. Is assigned and stored (step C6).

  In order to process the next GOP, the contents of the current memory are transferred (rewritten) to the previous memory (step C7), and the process up to the last GOP is completed or until the sequence code (SEC) is detected as described above. The scene is identified for each GOP (step C8).

Next, the second scene identification process shown in FIG. 18 will be described.
In the first scene identification process, the scene switching position is identified based on the spatial pattern information of the image data of the I picture decoded in the frame. In the second scene identification process, the luminance or color difference of the I picture is determined. The scene switching position is identified based on the DC (direct current) component of the DCT coefficient.

  First, the first GOP of moving image data to be edited is read (step C11), and the DC component of the DCT coefficient (luminance or chrominance data) of the I picture in the GOP is temporarily stored in the previous memory (step C12).

  The next GOP is read, and the DC component of the DCT coefficient (luminance or color difference data) of the I picture in the GOP is temporarily stored in the current memory (step C13).

  The correlation degree R between the DC component stored in the current memory and the DC component stored in the previous memory, that is, the correlation degree using the DC (direct current) component of the DCT coefficient of the luminance information (Y) or the color difference information (Cr, Cb). R is calculated (step C14). For example, the degree of correlation can be calculated using a correlation coefficient generally used in statistical processing or the like, or covariance and standard deviation.

  For example, the Pearson product moment correlation coefficient (R) is expressed by the following equation for an m × n image or pattern: R = (covariance of DC component a and DC component b) / {(standard deviation of DC component a) (Standard deviation of DC component b)}

  That is, the product moment correlation coefficient (R) is calculated by the above-described equation (1).

  Where f [x, y] is the DC component of the x and y coordinates of the input image f (previous memory), t [x, y] is the DC component of the x and y coordinates of the reference image t (current memory), fAV is an average value of DC components of the input image f, and tAV is an average value of DC components of the reference image t.

  If the correlation degree R is greater than or equal to a predetermined value (step C15, Yes), the scene is considered to be continuous with the previous GOP. If the correlation degree is lower than the predetermined value (step C15, No), the GOP is switched to a new scene. Identify as a location. Then, a new scene No. is added to the image starting with the GOP identified as the new scene. Is assigned and stored (step C16).

  In order to process the next GOP, the current memory contents are transferred (rewritten) to the previous memory (step C17), and the processing up to the last GOP is completed or the sequence code (SEC) is detected as described above. Then, scenes are identified for each GOP (step C18).

  In the scene identification process shown in the flowcharts of FIGS. 17 and 18, the scene switching position is identified based on the correlation degree of the I picture between GOPs, but the scene switching position is determined by other methods. You may make it identify.

  For example, in accordance with an operation input at the time of shooting a moving image, data indicating a scene break position is recorded in association with the moving image data. For example, a shooting start / stop switch operation event (at least one of them) is recorded together with moving image data in association with a time code or GOP.

  Then, an index image is generated by identifying from the time or GOP when the shooting start operation is recorded to the time or GOP when the shooting stop operation is recorded as one scene.

  In addition to the operation of the shooting start / stop switch, other input operations during the shooting operation, for example, an input operation for instructing an intentional switching position from the user are accepted, and associated with the moving image data according to the input operation. Data indicating the switching position may be recorded. In this case, the moving image may be divided based on the data indicating the switching position, and an index image (for example, the head image) corresponding to each division may be generated.

  Next, the reproduction method editing process (playlist creation) according to the editing operation input in the editing mode described above will be described.

  FIG. 19 shows an example of editing a moving image. 19A is a trim edit that connects only necessary parts, FIG. 19B is a cut edit that changes the playback order of a plurality of clips, and FIG. 19C is a title character or image added to an opening or ending image. Examples of title editing to be synthesized (including addition of sound effects and BGM) are shown.

  FIG. 20 shows an example of playlist creation / playback control in trimming editing (FIG. 19A). In the following description, it is assumed that an input operation is performed on an index image in GOP units on the index image list position screen in the edit mode.

  In trimming editing, only the index image corresponding to the part to be reproduced is sequentially selected in the order of reproduction from the head direction on the input operation screen in the editing mode (FIGS. 11 and 12). In FIG. 20A, GOPs 5 to 10 corresponding to part 1 are designated by selecting the corresponding index images. Similarly, GOPs 13 to 21 corresponding to part 2 and GOPs 27 to 32 corresponding to part 3 are designated.

  The reproduction editing control unit 130 (reproduction method editing unit 130a) edits reproduction control information and records it in a playlist in accordance with an editing operation for selecting an index image (GOP).

  FIG. 20B shows a specific example of a playlist created in response to the input operation shown in FIG. As shown in FIG. 20B, the play list includes data on “edit code”, “corresponding source GOP (out)”, “time code”, “corresponding destination GOP (in)”, and “time code” as reproduction control information. Are recorded in association with each other.

  For example, a start code (“start”) is set in “edit code”, and the first GOP 5 is recorded as “corresponding destination GOP”. Further, a “time code” indicating the time from the beginning of the moving image of GOP5 is recorded. As the “edit code”, jump codes (“Jump1” and “Jump2”) indicating that the GOP not to be reproduced is skipped are set, and the GOPs (GOP10, GOP21) of the jump source are set as “corresponding source GOP” , The jump destination GOP (GOP13, GOP27) is set as “corresponding destination GOP” (the time code of each GOP is also set).

  When the playlist shown in FIG. 20B is generated, in the reproduction process using the playlist described later, as shown in FIG. 20C, GOP5 in which the start code “start” is set in the “edit code” Playback starts from. Then, after GOP 5 to 10 are reproduced, jump from GOP 9 to GOP 13 according to the jump code “Jump 1”, cut the GOP between them, and similarly reproduce and output GOP 13 to 20 corresponding to Part 2, According to the jump code “Jump2”, the jump is made from GOP20 to GOP27, and the GOP in the meantime is reproduced and output so as to cut. That is, it is possible to reproduce a trimmed and edited moving image in which only the GOP corresponding to the part selected as the reproduction target is reproduced and output by referring to the playlist, and unnecessary portions are cut off.

  FIG. 21 shows an example of playlist creation / playback control in cut editing (FIG. 19B) (assuming that an input operation is performed on an index image in GOP units).

  In cut editing, an index image corresponding to a part to be reproduced is arbitrarily selected in the reproduction order on the input operation screen in the editing mode (FIGS. 11 and 12). In FIG. 21A, GOPs 27 to 32 corresponding to clip # 3 are designated by selecting the corresponding index image. Next, GOPs 13 to 21 corresponding to clip # 2 and GOPs 5 to 10 corresponding to clip # 1 are designated in the same manner.

  The reproduction editing control unit 130 (reproduction method editing unit 130a) edits reproduction control information and records it in a playlist in accordance with an editing operation for selecting an index image (GOP).

  FIG. 21 (b1) shows a specific example of a playlist created in response to the input operation shown in FIG. 21 (a). As shown in FIG. 21 (b1), the playlist includes, as playback control information, “playback order”, “corresponding destination GOP”, “time code”, “converted time from the head”, “corresponding source GOP”, “time code”, and the like. Are recorded in association with each other.

  For example, the cut-in code “Cut In 0” is set in the “reproduction order”, and the first GOP 27 of the clip # 3 is recorded as “corresponding destination GOP”. In addition, a time code indicating the time from the beginning of the moving image of the GOP 27 and “converted time from the beginning” until the GOP is reproduced and output (initially 0) are recorded. In addition, GOP32 (final GOP of clip # 3) that is a jump source to the next clip is set in “corresponding source GOP”. For clip # 2, GOP13 is set as “corresponding destination GOP”, and the final GOP 21 of clip # 2 is set as “corresponding source GOP” (not shown). Also, the playback time of clip # 3 (total time code of each GOP) is added to the “converted time from the beginning” corresponding to GOP 27, and the obtained time corresponds to the cut-in code “Cut In 1”. Set as “converted time from the beginning”. Similarly, for “# 1”, “reproduction order”, “corresponding destination GOP”, “time code”, “converted time from the head”, “corresponding source GOP”, and “time code” are set.

  Instead of the playlist in the format shown in FIG. 21 (b1), the format shown in FIG. 21 (b2), that is, the format according to FIG. 20 (b) described in the trimming editing may be used. In FIG. 21 (b2), data of “converted time from the beginning” corresponding to each “edit code” is added to the format of FIG. 20 (b).

  As shown in FIG. 20 (b1), when a playlist is generated, playback is started from the GOP 27 in which the cut-in code “Cut In 0” is set in the “playback order” in the playback processing by the playlist described later. Then, playback is performed up to GOP 32 of “corresponding source GOP”. Next, playback is started from the GOP 13 in which the cut-in code “Cut In 1” is set, and is played back up to the GOP 21 of the “source GOP”. Similarly, the cut-in code “Cut In 2” is set. Playback is started from GOP5 that is currently set, and playback is continued up to GOP10 that is set as “corresponding source GOP”. In this way, the reproduction output corresponding to the editing operation subjected to the cut editing can be executed with reference to the playlist.

  FIG. 22 shows an example of playlist creation / playback control in title editing.

  In title editing, an image (and sound) representing a title is added to an arbitrary index image. In FIG. 22A, cut editing is performed so that parts 2, 1, and 3 are reproduced and output in this order. A detailed description of cut editing is omitted (see FIG. 21). Further, editing for combining title 1 and BGM1 is added to GOPs 61 to 75 of the first part 2, and wipeout is added to the final GOP 77 of part 2. Further, a wipe-in is added to the first GOP 5 of the part 1 and a fade-out 56 is added to the final GOP 56. Also, the first GOP 81 of part 3 is faded in and edited to synthesize title 2 with GOP 82-100.

  The reproduction editing control unit 130 (reproduction method editing unit 130a) edits reproduction control information and records it in a playlist in accordance with an editing operation (addition of editing) for selecting an index image (GOP).

  FIG. 22B shows a specific example of a playlist created in accordance with the addition of editing shown in FIG. As shown in FIG. 22B, the play list includes “edit code”, “corresponding source GOP (out)”, “time code”, “converted time from the beginning”, “corresponding destination GOP (in)” as reproduction control information. Data on “time code” and “file” are recorded in association with each other.

  Note that description of portions common to FIGS. 20B and 21B2 is omitted. In the “edit code”, codes representing the contents of editing added to the GOP, for example, title codes “Title 1” and “Title 2”, BGM code “BGM 1”, wipe code “Wipe 1”, fade code “Fade 1” Is set. Corresponding to the title code and BGM code, a file name such as image or sound (“preset A”, “sound file B”, “file C”, etc.) is set as “file”.

  When the playlist shown in FIG. 22B is generated, in the reproduction processing by the playlist described later, as shown in FIG. 22C, the GOP 60 in which the start code “start” is set in the “edit code” is shown. Playback starts from. Then, for GOP 61 to 75 in which the title code and BGM code are set, the title 1 is synthesized using the data of the “preset A” (image) file, and the “audio file B” (audio) file BGM1 is reproduced using the data and reproduced and output up to GOP77. From the last GOP 77 of part 2 to the first GOP 5 of part 1, the screen is changed by wipe 1 and reproduced and output from GOP 5 to 56. The last GOP 56 of Part 1 and the first GOP 81 of Part 3 transition the screen by Fade 1, synthesize title 2 using the file data of “File C” for GOPs 82 to 100, and reproduce and output up to GOP 101 To do. That is, in the edit mode, it is possible to reproduce a moving image by adding an edit corresponding to the edit icon added to the index image.

  FIG. 23 shows an example of playlist creation / playback control when a transition screen effect such as a transition effect is inserted.

  When the transition screen effect is inserted, as described with reference to FIG. 22, an editing icon representing editing content is added to an arbitrary index image. FIG. 23A shows a case where a wipe is added to the last GOP 21 of clip # 1. In this case, wipeout is added to GOP21, and wipein is added to the first GOP25 of clip # 2.

  As shown in FIG. 23 (b), GOP 22 is recorded as “corresponding source GOP” and GOP 15 is recorded as “corresponding destination GOP” corresponding to the edit code “Wipe 1” indicating wipe. Other details are omitted in place of the description of FIGS.

  When the playlist shown in FIG. 23 (b) is generated, in the reproduction processing by the playlist described later, as shown in FIG. 23 (c), the I picture of the GOP 22 of “corresponding source GOP” is extracted and decoded. (Transition source image A) and the I picture of GOP15 of “corresponding destination GOP” are extracted and decoded (transition destination image). Then, using the transition source image A and the transition destination image B, a continuous image representing a special effect (in this case, a wipe) is synthesized.

  Then, as shown in FIG. 23D, a screen effect transition image is inserted between clip # 1 and clip # 2 and output. Details of the composition of the transition image will be described later (FIGS. 25 to 27).

Next, the reproduction processing by the playlist shown in step A27 in FIG. 7 and step B9 in FIG. 10 will be described with reference to the flowchart shown in FIG. 24 (see the decoding reproduction control unit 140 shown in FIG. 3).
First, the playback order control unit 140a (compression encoding / decoding unit 24) reads a playlist file 142a corresponding to a moving image to be played back from the playlist memory 30 (memory medium / communication path 142) (step D1). .

  The reproduction order control unit 140a decodes and reproduces the compressed GOP / decoding unit 24 (interframe decoding units 140b and 140f and intraframe decoding units 140c and 140e) in order from the GOP for which the start is set (step D2).

  Here, when edits such as transition effects / titles are set in the playlist (step D3, Yes), I picture images (transition source image and transition destination image) of the GOP before and after transition by the intra-frame decoding units 140c and 140e. ) Is decoded and generated by the compression encoding / decoding unit 24 (step D5), and a continuous image of the screen effect is synthesized from the I pictures before and after the transition (step D5). Note that the synthesis of a continuous image having a specific screen effect will be described later (FIGS. 25, 26, and 27).

  Then, the reproduction output control unit 140k reproduces and displays a continuous image such as the synthesized transition effect (step D6).

  The reproduction order control unit 140a moves to the first GOP of the transition destination (corresponding destination GOP) recorded in the playlist, and sequentially performs decoding and reproduction in the same manner (step D7).

  Also, when digital effect editing is set in the playlist (step D13, Yes), the I picture of the GOP or the subsequent image is decoded (step C14), and the I picture or the subsequent image is zoomed in, converted to monochrome, Applicable special effects such as sepia color and mosaic are added (step D15).

  Then, the reproduction output control unit 140k reproduces and displays the continuous images subjected to the digital effect processing (Step D16).

  If jump or cut editing is set in the play list (step D17, Yes), the jump GOP moves from the last GOP of the jump source to the first GOP of the jump destination, and is sequentially decoded and reproduced (step D18).

  If no editing is added (step D17, No), the reproduction is sequentially performed according to the reproduction order set in the playlist (step D19).

  If the setting is made to add the sound effect / BGM to the playlist (step D8), the sound file (additional sound file 142d) of the corresponding sound effect or music is stored from the still image / moving image / audio data memory 28. Reading (step D9), the corresponding image is decoded and reproduced, and the corresponding audio file is decoded and reproduced by the sound effect synthesizing unit 140g or the audio decoding unit 140h (step D10).

  Here, if the reproduction process has not been completed up to the end of the playlist (No at Step D11), the decoding generation is executed according to the reproduction control information recorded in the playlist in the same manner as described above.

  When the reproduction process is completed up to the end of the playlist (step D12, Yes), the reproduction editing process is ended and the process returns to the original process.

  Next, a specific example of the composition process of the transition effect screen shown in step C5 of FIG. 24 will be described by taking as an example a case where door open / close editing is set.

  On the door open / close transition effect screen, as shown in FIG. 25, images are sequentially shifted from the door open state to the door closed state or vice versa. FIG. 26 is a diagram for explaining how the transition effect screen is synthesized, and FIG. 27 is a flowchart for explaining the transition screen synthesis processing.

  First, the I picture of the last GOP of clip # 1 before the transition is decoded in the frame and recorded as an A image (transition source image) (step D51) (FIG. 26 (a)).

  Further, the I picture of the first GOP of the clip # 2 after the transition is decoded in the frame and recorded as a B image (transition destination image) (step D52) (FIG. 26B).

  Next, a predetermined number k of continuous images M1 to Mk of a transition effect cut-out mask such as a door open effect are created (step D53) (FIG. 26C).

  Next, the image A is deformed (image conversion or the like) and pasted in accordance with the shapes of the mask portions of the mask images M1 to Mk. Alternatively, only the portion that overlaps the mask portion is left, and the portion that does not overlap is cut out (step D54) (FIG. 26 (d)).

  A predetermined number k of images B are copied, each image B is overlapped with the mask images M1 to Mn, the overlapping portions are cut out, and the remaining portions are left (step D55) (FIG. 26 (e)). That is, all the mask partial data of the mask images M1 to Mk are set to 0, the rest are all set to 1, and the B image and each mask image are ANDed.

  Next, each A image cut out by masking and the corresponding B image are combined (ORed) to create a predetermined number of continuous images (step D56) (FIG. 26 (f)).

  Through the above processing, the transition effect screen can be synthesized.

  As described above, in the electronic camera according to the first embodiment, when editing a moving image shot through the zoom imaging unit 4, the moving image data is converted into the edited moving image data and recorded instead. In addition, while maintaining the original data of the MPEG compressed moving image, it is possible to edit and record the reproduction control information indicating the reproduction order change and the transition screen effect for each clip or scene as a playlist. Then, when reproducing and outputting a moving image, a moving image that has been subjected to editing and special processing is generated and reproduced while referring to the reproduction control information of the playlist. For example, the following effects can be obtained. Obtainable.

  (1) Various edits can be made any number of times simply by changing the edit information of the playlist. Accordingly, since the recorded moving image data itself is not changed or processed, the image quality is not deteriorated even if it is edited many times.

  (2) Complex decoding, re-encoding, control of stream continuity, buffer amount, etc. are not required, so a low-speed CPU or processing circuit such as an electronic camera configured in a small portable type with a simple configuration and a small processing amount However, special effects can be easily generated and played back during playback.

  (3) Since only the playlist is newly recorded as a separate file, it can be realized even with a portable electronic camera having a small memory capacity and a small memory capacity.

  (4) Edit operations can be performed by playing back index images instead of dragging and dropping clip images on the timeline or arranging screen effect objects on the tracks between image tracks. It is only necessary to select in order and select an edit icon representing a screen effect, a title, a sound effect addition, etc. and add it to the index image. For this reason, it is possible to easily input an editing operation even with the display unit 19 having a small display size (an editing screen provided with a large number of tracks or the like cannot be displayed at the same time) or a simple operation input unit 15 with limited buttons and keys. There is no loss of operability due to hardware limitations.

  In the first embodiment described above, reproduction control information is edited and recorded as a playlist, and when a moving image is reproduced, editing and special processing are performed while referring to the reproduction control information of the playlist. Although the moving image is generated and played back, the moving image data after editing may be additionally recorded as new moving image data with reference to the playback control information of the playlist. Alternatively, not only playback editing (playlist creation), but also an editing function or an editing mode for easily performing editing processing (editing recording) on moving image data in units of GOPs by the above-described editing operation input may be provided. good. Further, it may be possible to select whether playback editing or further editing recording is performed by designating input from the user.

  Although additional editing examples of transition screen effects, titles, sound effects, BGM, etc. have been shown, other transition screen effects and sound effects, or image transformations, modifications, deformation processing, and image hues other than transition screens Add other digital effects such as black and white images, mosaic images, images with different resolutions and gradations, and other instrument sounds, imitation sounds, voice sounds, etc. You may be able to do it.

  In addition, editing operations on the timeline are difficult with small devices, but the listability and time-series order are easy to understand, so the timeline and index images on the time axis can be displayed together as a list display before and after editing. You may do it.

Next, a second embodiment of the present invention will be described.
In the first embodiment, the playlist is created and edited without actually editing the moving image data for the editing operation input by the user for the moving image, but in the second embodiment, the editing operation is performed. On the other hand, the moving image data itself is actually subjected to an editing process and recorded.

  Note that the configuration and basic operation of the electronic camera in the second embodiment are the same as those in the first embodiment, and a detailed description thereof will be omitted.

  Hereinafter, editing processing of moving image data in the second embodiment will be described with reference to the flowchart shown in FIG.

  First, in accordance with designation from the user, moving image data to be edited is selected from a plurality of moving image data (MPEG file 132) recorded in the still image / moving image / audio data memory 28 (step E1).

  The reproduction editing control unit 130 decodes the moving image data separated by the demultiplexing / decoding unit 130b by the compression encoding / decoding unit 24, generates an index image for each predetermined unit, and displays it as an editing mode screen. A list is displayed at 19 (step E2). Here, as index images in a predetermined unit, for example, index images for each time, each scene, and each GOP can be generated and displayed in a list.

  Here, in the same manner as in the first embodiment, through the editing mode screen displaying a list of index images, editing of the playback order for the index images and editing operation input by adding editing icons to the index images selected as playback targets are performed. Is performed (steps E3, E4, E5). Since the editing operation input here only needs to specify the index image and select the editing icon, it does not require a wide display screen and can be executed only by a simple operation using keys and buttons.

  When the editing operation input is thus performed, the playback / editing control unit 130 (playback method editing unit 130a) provisionally creates an edit / playback playlist according to the editing icon added to the index image (step E6).

  Here, when execution of preview display of the editing result is instructed (step E7, Yes), the temporary playlist is designated as a playlist to be referred to for reproduction of the moving image (step E8), and a moving image using this playlist is designated. Image reproduction processing is executed (step E9). Note that the details of the reproduction processing by the playlist will be described later.

  Similarly, a temporary play list corresponding to each editing operation is created by selecting an index image arbitrarily from the index image list display and performing an editing operation to which an editing icon is added.

  When an editing end instruction is input after the editing operation input is thus performed (step E10, Yes), the reproduction editing control unit 130 determines whether or not to record the editing result as a playlist, for example, a display unit The user is inquired by displaying a message at 19 (step E11). Here, when recording as a play list is instructed, an edit / playback play list is created and recorded in the play list memory 30 based on the editing result indicated by the temporary play list (step E12).

  After completion of editing the playlist, it is determined whether or not the setting for recording the edited moving image data is made (step E13, Yes). For example, after creating and editing a playlist, a message for confirming to the user such as “Do you want to record the edited data?” Is displayed. If there is a response instructing execution of this message, it is determined that the setting for recording the edited moving image data is made. Alternatively, an edit recording mode is provided separately from the creation / edit mode, and when the edit / record mode is set, it is determined that not only the playlist recording but also the edited moving image data itself is recorded.

  When recording the edited moving image data, the corresponding playlist is read (step E14), the moving image data is read in order from the GOP of the scene where the start is set, and a duplicate copy is created with the compression encoded data as it is. Then, new moving image data is created by combining in units of GOP (step E15).

  Here, when a jump or skip (cut) edit is added to the target GOP (Yes in step E24), the jump is made to the first GOP of the jump destination, and the compression encoding is performed in units of GOP in order from there. Data are combined (step E25).

In addition, when an edit for adding a transition effect or a title screen of the transition screen effect is added (step E16, Yes), the I picture of each GOP before and after the transition is decoded in the frame (step E17), a continuous image for special effects such as the corresponding screen transition is synthesized from the still images A and B (transition source image and transition destination image) of the I picture before and after the transition (step E18), and the combined transition The continuous image of the effect is encoded and combined with the adjacent GOPs before and after as new single or plural GOP moving image data (step E19). Then, the process shifts to the top of the transition destination and joins in units of GOP (step E20).
In this case, in order to adjust the buffer amount (such as VBV_delay value) and the balance of the code amount between adjacent GOPs, a continuous image synthesized for special effects is created as a new GOP and encoded. The buffer amount (VBV_delay value) is reset to a predetermined value, and the moving image data of the previous and subsequent GOPs are once decoded, and the GOP before the transition, the GOP of the combined screen effect portion, and the GOP after the transition A plurality of GOPs are collectively re-encoded to prevent excessive or insufficient code amount or imbalance (step E21).

  Here, when the end of the playlist or the playlist with the end set is reached, the combined video data is recorded as new video data different from the original video data, and editing is performed. The process ends (step E23).

  In the second embodiment, the playback order is changed in units of GOPs such as jumping and skipping (cutting). Therefore, in most editing portions, as shown in FIG. Cut and paste in units.

  As shown in FIG. 29 (b), the compression-encoded data can be edited in units of frames, but GOP correction work before and after the editing point, for example, when the cut point is a P picture frame, It is necessary to modify the DCT component of the I picture by applying motion compensation to the DCT component of the P picture at the cut point to obtain a new I picture. In GOP units, it is not necessary to perform such correction work, and editing processing can be realized with compressed data with a simple configuration and processing.

  In addition, as shown in FIG. 30, in the transition effect and special effects such as title composition, as shown in FIG. 30, a transition screen synthesized based on I-pictures decoded in frames in GOP units before and after the transition as in the first embodiment. In order to edit and record, it is necessary to re-encode together with the previous and subsequent GOPs to create a new GOP. However, since there are few decoding and re-encoding processes, even if editing and recording are performed, In addition, the memory requirement for recording the decoded baseband moving image can be small. Even if it takes some time in that part, there is little inconvenience in use because it is being edited.

  Further, since the original moving image data recorded and recorded is not changed as it is and new moving image data is newly created, the quality of the original moving image does not deteriorate no matter how many times it is edited. In addition, editing processing and memory requirements are slightly increased as compared to the creation of only a playlist for which a playback control method is set as in the first embodiment. However, unlike the first embodiment, the edited moving image during playback is different from the first embodiment. Since it is only necessary to reproduce the data as it is, control during reproduction is unnecessary and simple.

  In this way, in the second embodiment, the editing unit is a GOP unit, and most editing can be performed by editing with compression-encoded data, and special effects such as transition screen effects can be easily edited in GOP units. Since the editing process can be simplified compared to the conventional example, and the function that can edit even moving image data after compression recording is used, for example, a low-speed CPU mounted on a small electronic camera or image data processing Even a circuit can be realized sufficiently.

  In the process of the flowchart of FIG. 28 described above, a playlist is created and edited in response to an input operation from the user. However, editing is performed on the moving image data without creating the playlist, and the edited playlist is created. You may make it record moving image data.

  In the second embodiment, the moving image data is edited and recorded in response to an input operation from the user. However, a playlist is created in the same manner as in the first embodiment, and this play list is created. Editing processing may be executed in accordance with the editing content (reproduction control information) set in the list.

  Further, in each of the above-described embodiments, the description has been given with respect to a moving image photographed by the mobile camera body 2 and the zoom imaging unit 4, but for example, via the charging / PC transfer cradle unit 7, The same processing is performed on moving image data received from information devices such as other personal computers via LAN or wireless communication, and the moving image data is encoded and recorded and a playlist is created. The portable camera body 2 may be able to play back. Also, along with the moving image data encoded from other information devices, the playlist data for the moving image data is received, and the portable camera body 2 reproduces and outputs the moving image based on the playlist. it can.

  Further, the above embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent elements are deleted from all the constituent elements shown in the embodiment, at least one of the problems described in the column of problems to be solved by the invention can be solved, and described in the column of the effect of the invention. In a case where at least one of the obtained effects can be obtained, a configuration in which this configuration requirement is deleted can be extracted as an invention.

  The processing described in each of the above-described embodiments is a program that can be executed by a computer, for example, a recording medium such as a magnetic disk (flexible disk, hard disk, etc.), an optical disk (CD-ROM, DVD, etc.), and a semiconductor memory. And can be provided to various devices. It is also possible to transmit to a variety of devices by transmitting via a communication medium. The function of the computer mounted on the electronic camera reads the program recorded on the recording medium or receives the program via the communication medium, and the operation is controlled by this program, thereby executing the above-described processing.

1 is a block diagram showing a circuit system configuration of an electronic camera according to a first embodiment of the present invention. The block diagram which shows the function structure of the reproduction | regeneration edit control part 130 in 1st Embodiment. The block diagram which shows the function structure of the decoding reproduction | regeneration control part 140 in 1st Embodiment. The figure for demonstrating the detailed structure of the reproduction | regeneration edit by the reproduction | regeneration edit control part 130 in 1st Embodiment, the reproduction output control by the decoding reproduction | regeneration control part 140, and the compression encoding / decoding part 24. FIG. The figure which shows the hierarchical structure of MPEG moving image data. The figure which shows the order of the frame image at the time of encoding / decoding. 6 is a flowchart for explaining the overall control flow of the electronic camera according to the first embodiment. The figure which shows an example of the screen displayed on the display part 19 in the video recording mode which image | photographs the moving image in 1st Embodiment. The figure which shows an example of the reproduction | regeneration menu and edit menu in 1st Embodiment. The flowchart for demonstrating the edit process (creation of a play list) of the reproduction | regeneration method in 1st Embodiment. The figure which shows the example by which the list of index images is displayed on the display screen of the edit mode in 1st Embodiment. The figure which shows an example of the screen in the setting of addition of the special effect in 1st Embodiment. The figure for demonstrating the edit by a transition screen effect (transition effect). The figure which shows the example of the screen transition corresponding to the edit icon and each corresponding to the transition effect in 1st Embodiment. The figure for demonstrating the edit by a digital effect. The figure which shows the example of the digital effect corresponding to each of the edit icon relevant to the digital effect in 1st Embodiment, and each. The flowchart for demonstrating the 1st scene identification process in 1st Embodiment. The flowchart for demonstrating the 2nd scene identification process in 1st Embodiment. The figure for demonstrating an example of edit (trimming edit, cut edit, title edit) with respect to a moving image. The figure which shows the example of the play list creation / reproduction | regeneration control of trimming edit in 1st Embodiment. The figure which shows the example of the play list creation / reproduction | regeneration control of cut edit in 1st Embodiment. The figure which shows the example of play list creation / reproduction | regeneration control in the title edit in 1st Embodiment. The figure which shows the example of playlist creation / reproduction | regeneration control in the case of inserting transition screen effects, such as a transition effect, in 1st Embodiment. The flowchart for demonstrating the reproduction | regeneration processing by the play list in 1st Embodiment. The figure for demonstrating the transition effect screen of the door open / close in 1st Embodiment. The figure for demonstrating a mode that the transition effect screen in 1st Embodiment is synthesize | combined. The flowchart for demonstrating the synthetic | combination process of the transition screen in 1st Embodiment. The flowchart for demonstrating the edit processing of the moving image data in 2nd Embodiment in 2nd Embodiment. The figure for demonstrating the edit on the compression data in 2nd Embodiment. The figure for demonstrating the edit by decoding and re-encoding in GOP unit in 2nd Embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 2 ... Portable camera main-body part, 4 ... Zoom imaging part, 6 ... Connector part, 7 ... Charging / PC transfer cradle part, 10 ... Control circuit, 12 ... Input circuit, 13 ... Still image photography half press / shutter button, 14 ... Video shooting start / stop button, 15 ... operation input unit, 17 ... display memory, 18 ... display drive unit, 19 ... display unit, 20 ... image buffer memory, 22 ... image signal processing unit, 24 ... compression encoding / decoding 26 ... Editing / playback control unit 28 ... Still image / moving image / audio data memory 30 ... Playlist memory 31 ... Audio decoding unit 32 ... A / D conversion unit 33 ... Amplifying unit 34 ... Speaker, 36 ... external memory interface, 37 ... external memory, 40 ... voice encoding unit, 41 ... A / D conversion unit, 42 ... amplifying unit, 43 ... microphone, 44 ... power supply control unit, 46 ... battery, 48 ... External I / O 50, 92 ... connector, 52 ... lens optical system, 53 ... aperture, 54 ... half mirror, 55 ... shutter, 58, 59 ... lens drive unit, 60 ... image sensor, 62 ... photometry / ranging sensor, 70 ... Control unit 71 ... Focus control unit 72 ... Zoom control unit 73 ... Aperture control unit 74 ... Shutter control unit 75 ... CDS / AGC 76 ... A / D circuit 77 ... Timing control drive unit 78 ... Exposure Control unit 79 ... Distance measuring control unit 80 ... Flash control unit 82 ... Flash drive unit 83 ... Flash 90 ... Control circuit 94 ... Charge control unit 96 ... Power supply / charge / transfer indicator 97 ... Operation unit 98 ... Wireless LAN I / F, 99 ... Encoding / decoding unit, 100 ... Modulation / demodulation unit, 101 ... Transmission / reception unit, 104 ... LAN communication I / F, 106 ... USB communication I / F, 108 ... U B connector, 110 ... LAN connector, 120 ... stabilization circuit, 122 ... smoothing circuit, 124 ... rectifier circuit, 126 ... fuse, 130 ... reproduction editing control unit, 130a ... reproduction method editing unit, 130b ... demultiplexing / decoding unit, 130c ... various decoding units, 130d ... intra-frame image decoding unit, 130e ... index image creation unit, 130f ... audio index reproduction unit, 130g ... playlist creation unit, 130h ... scene switching identification unit, 132, 136a, 142b ... MPEG files, 134: Image / sound file, 136, 142 ... Memory medium / communication path, 136b, 142a ... Playlist file, 136c ... Additional image / sound file, 140a ... Playback order control unit, 140b ... Inter-frame decoding unit, 140c ... Frame Inner decoding unit, 140d ... special effect synthesis unit, 140 e ... intra-frame decoding unit, 140f ... inter-frame decoding unit, 140g ... sound effect synthesizing unit, 140h ... audio decoding unit, 140k ... reproduction output control unit, 142c ... additional image file, 142d ... additional audio file.

Claims (13)

A moving image shooting means for shooting a moving image;
Moving image recording means for recording a moving image photographed by the moving image photographing means;
An editing operation input means for inputting an editing operation on the moving image recorded by the moving image recording means;
Reproduction control information generating means for generating reproduction control information representing the editing content to be applied to the moving image in response to the editing operation input by the editing operation input means;
An electronic camera comprising: reproduction control information recording means for recording the reproduction control information generated by the reproduction control information generation means in association with the moving image recorded by the moving image recording means.   Reproduction output means for reproducing and outputting the moving image recorded by the moving image recording means while editing the moving image according to the editing content represented by the reproduction control information recorded by the reproduction control information recording means; The electronic camera according to claim 1, further comprising: A moving image shooting means for shooting a moving image;
Moving image recording means for recording a moving image photographed by the moving image photographing means;
An editing operation input means for inputting an editing operation for adding a special effect for transitioning an image to the moving image recorded by the moving image recording means;
In response to the editing operation input by the editing operation input unit, a continuous image expressing the transition of the image is generated based on the transition source image and the transition destination image, and recorded in the moving image recording unit An electronic camera comprising editing means for editing a moving image. A moving image shooting means for shooting a moving image;
Moving image recording means for recording data of moving images taken by the moving image photographing means;
Display means for generating and displaying a list of a plurality of images based on moving image data recorded in the moving image recording means;
At least one of an editing operation for designating a reproduction order for a plurality of images displayed by the display means and an editing operation for designating addition of an editing icon representing a special effect for an arbitrary image among the plurality of images. Editing operation input means to input;
An electronic camera comprising: editing means for editing moving image data recorded in the moving image recording means based on an editing operation input by the editing operation input means. A scene identification unit that calculates a correlation degree of information included in the image included in the moving image, and identifies an image having a correlation degree lower than a predetermined value as a scene switching position;
The electronic camera according to claim 4, wherein the display unit displays a representative image for each scene identified by the scene identification unit. Separation position input means for inputting an operation for designating a separation position when shooting a moving image by the moving image shooting means;
Position recording means for recording data indicating the break position input by the break position input means in association with the moving image data;
Based on the data indicating the break position recorded by the position recording means, further comprising a scene identification means for identifying a scene switching position,
The electronic camera according to claim 4, wherein the display unit displays a representative image for each scene identified by the scene identification unit. A moving image shooting means for shooting a moving image;
Moving image recording means for recording data of moving images taken by the moving image photographing means;
A dividing unit for dividing moving image data recorded in the moving image recording unit into predetermined units;
An editing operation for designating the reproduction order for the moving images for each predetermined unit divided by the dividing means, or an editing for designating addition of an editing icon representing a special effect for an arbitrary moving image in the moving images for each predetermined unit Editing operation input means for inputting at least one of operations;
An electronic camera comprising: editing means for editing moving image data recorded in the moving image recording means based on an editing operation input by the editing operation input means. Reproduction control information generating means for generating reproduction control information representing the editing content to be applied to the moving image in response to the editing operation input by the editing operation input means;
Reproduction control information recording means for recording the reproduction control information generated by the reproduction control information generating means in association with the moving image recorded by the moving image recording means,
The editing means edits the moving image data recorded by the moving image recording means based on the editing content represented by the reproduction control information recorded by the reproduction control information recording means for the moving image. The electronic camera according to claim 3, wherein the electronic camera is a digital camera.   8. The electronic camera according to claim 3, further comprising post-editing moving image recording means for recording moving image data edited by the editing means. First reconstructing means for reconstructing moving image data recorded in the edited moving image recording means by changing the order in predetermined units;
Based on the moving image data recorded in the edited moving image recording means, an image representing a special effect is generated, and the generated image is combined with moving image data in a predetermined unit to be reconstructed. The electronic camera according to claim 9, further comprising: a configuration unit.   11. The electronic camera according to claim 3, further comprising reproduction output means for reproducing and outputting the moving image edited by the editing means. A control program executed by a computer built in the electronic camera,
Computer
A moving image shooting means for shooting a moving image;
Moving image recording means for recording a moving image photographed by the moving image photographing means;
An editing operation input means for inputting an editing operation on the moving image recorded by the moving image recording means;
Reproduction control information generating means for generating reproduction control information representing the editing content to be applied to the moving image in response to the editing operation input by the editing operation input means;
An electronic camera control program for functioning as playback control information recording means for recording the playback control information generated by the playback control information generating means in association with the moving image recorded by the moving image recording means. A control program executed by a computer built in the electronic camera,
Computer
A moving image shooting means for shooting a moving image;
Moving image recording means for recording data of moving images taken by the moving image photographing means;
Display means for generating and displaying a list of a plurality of images based on moving image data recorded in the image recording means;
At least one of an editing operation for designating a reproduction order for a plurality of images displayed by the display means and an editing operation for designating addition of an editing icon representing a special effect for an arbitrary image among the plurality of images. Editing operation input means to input;
An electronic camera control program for functioning as editing means for editing moving image data recorded in the moving image recording means based on an editing operation input by the editing operation input means.

Images