JP5298930B2 - Movie processing apparatus, movie processing method and movie processing program for recording moving images - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To simultaneously record a plurality of moving pictures with a simple configuration. <P>SOLUTION: In a moving picture processing device, moving picture of at least a partial region is cut out from the moving picture data obtained by an imaging system to generate moving picture data for a sub-screen, and compositing the generated moving picture data for the sub-screen to original moving picture data and recording the composited data. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to moving image processing for recording a moving image in a digital camera, for example, a moving image processing apparatus, a moving image processing method, and a moving image processing program capable of simultaneously recording a plurality of moving images.

  2. Description of the Related Art Conventionally, a digital camera captures a desired subject, acquires a moving image, and records the moving image on a recording medium such as a memory card. In addition, a moving image recorded on the recording medium is reproduced and displayed in response to a user operation.

  In addition, in a television receiver, a plurality of video data is displayed separately for a main screen which is a normal size screen and a sub screen which is displayed as a small screen in the main screen, so-called picture-in-picture. The method is being performed.

  In still image recording, for example, a still image obtained from an imaging system and a still image obtained by partially enlarging the still image are displayed side by side, and these still images are recorded by a user operation. (See Patent Document 1).

JP 2008-22306 A

  Although it is efficient to record a plurality of moving images at the same time, it is necessary to provide an imaging system for acquiring each of the plurality of moving images. Further, when a plurality of moving images are displayed on the same screen, there is a problem that the relationship becomes difficult to understand and usability is deteriorated.

  The present invention has been made in consideration of the above points. A moving image processing apparatus, a moving image processing method, and a moving image processing program capable of simultaneously recording a plurality of moving images with a simple configuration and improving convenience. The purpose is to provide.

In order to solve the above problem, the invention of claim 1 is applied to a moving image processing apparatus, and includes at least a moving image data input unit for inputting input moving image data, and image data of frames constituting the input moving image data. cutting out a portion of the realm, the sub-screen data generating unit for generating image data for the child screen of the frame, by assigning the image data and the image data for the small picture of the frame to each parent screen and a child screen an image combining unit for generating a synthetic video data, and records on the recording medium the composite moving picture data by data compression processing, cut-out position of said at least a portion of the region corresponding to the child position and child screen of the screen And a recording unit that records the small-screen information associated with the information on the recording medium .

Further, the invention of claim 2 has an image display unit for displaying the composite video data in the configuration of claim 1 , wherein the image display unit displays the composite video data and the position of the sub-screen. A display that clearly shows the correspondence with the position of the at least a part of the region corresponding to the child screen based on the child screen information is performed .
Further, the invention of claim 3 is the configuration of claim 2, wherein the image display unit displays the synthesized moving image data and switches the position of the target child screen in accordance with a user operation. And the correspondence between the position of the at least part of the region corresponding to the child screen and a display that clearly shows the correspondence based on the child screen information.
According to a fourth aspect of the present invention, in the configuration according to any one of the first to third aspects, the recording unit multiplexes and records the small-screen information in the synthesized moving image data.

According to a fifth aspect of the present invention, in the configuration according to any one of the first to fourth aspects, the position of the at least part of the region, the size of the at least part of the region, the position of the sub-screen, and the position of the sub-screen An instruction input unit for inputting at least one instruction of size and addition of sub-screens is provided.

The invention of claim 6 has a child screen setting unit for setting a child screen based on the child screen information in the configuration of any one of claims 1 to 4, wherein the child screen setting unit includes a child screen setting unit. The position of the sub-screen and the size of the sub-screen are automatically determined according to the number of screens or the position of the subject.

Further, the invention of claim 7 is applied to a moving image processing apparatus, and includes a playback unit that plays back moving image data from a recording medium, and a display unit that displays the moving image data played back by the playing unit, The data is generated by cutting out at least a partial area as image data for a child screen for each frame image data constituting the input moving image data, and combining the image data for the child screen with the image data of the frame. The combined moving image data, the input moving image data, and sub-screen information in which the position of the sub-screen and the cut-out position of the at least a part of the region corresponding to the sub-screen are multiplexed and recorded. the display unit performs demonstrates displayed based on the corresponding child screen information and the position of said at least a portion of the region corresponding to the position and the child screen child screen.

The invention of claim 8 is applied to the moving image processing method, comprising the steps of inputting an input video data, for each image data of the frames constituting the input moving image data the input, at least a portion of the realm cut, to produce a step of generating image data for the child screen of the frame, the synthetic moving image data assigned the image data and the image data for the child screen the generated of said frame, each parent screen and a child screen A child that associates the position of the child screen with the cut-out position of the at least a part of the region corresponding to the child screen, while compressing the generated synthesized moving image data and recording it on a recording medium Recording screen information on a recording medium .

The invention of claim 9 is applied to the moving image processing method includes a step of reproducing the moving image data from a recording medium, and displaying the reproduced motion picture data, the moving image data is input moving At least a portion of each frame image data constituting the data is cut out as sub-screen image data, and the sub-screen image data is generated by combining the frame image data with the image data of the frame. Yes , the input moving image data, and sub-screen information in which the position of the sub-screen and the cut-out position of the at least part of the region corresponding to the sub-screen are associated with each other are recorded in a multiplexed manner. the step of displaying is your demonstrates displayed based on the corresponding child screen information and the position of said at least a portion of the region corresponding to the child position and child screen of the screen Including the now step.

The invention of claim 10 is applied to a moving image processing program, and a function of inputting input moving image data to a computer that controls the moving image processing apparatus, and image data of frames constituting the input moving image data. for each, at least cut out portion of the realm, the child a function of generating image data for the screen, the image data of each parent screen and the child screen image data and the sub-screen the generated of the frame of the frame to the ability to generate synthetic video data assigned, and records on the recording medium the composite video data the generated by data compression processing, the at least a portion corresponding to the position and the child screen child screen And a function of recording small-screen information associated with the cut-out position of the area on the recording medium .

The invention of claim 11 is applied to a moving image processing program, and has a function of reproducing moving image data from a recording medium and a function of displaying the reproduced moving image data on a computer that controls the moving image processing apparatus. A moving image processing program to be realized, wherein the moving image data is cut out at least a part of the image data of a frame constituting the input moving image data as image data for a child screen, and the image data for the child screen is Composite video data generated by image synthesis with frame image data , wherein the input video data is associated with a position of the sub-screen and a cut-out position of the at least part of the region corresponding to the sub-screen child screen information is recorded are multiplexed, a function of displaying the moving image data, the at least corresponds to the position and the child screen child screen The correspondence between the position of the part of the region including the function for demonstrating the display based on the slave screen information.

  According to the present invention, a plurality of moving images and the like can be recorded simultaneously with a simple configuration.

1 is a block diagram of a digital camera according to a first embodiment of the present invention. It is a figure explaining the principle of the small-screen composition mode in the digital camera of FIG. It is a function block diagram of the principal part of the digital camera of FIG. It is a chart which shows the small screen information in the digital camera of FIG. 3 is a flowchart for explaining a small screen setting process in the digital camera of FIG. 1. 4 is a flowchart for explaining association display processing in the digital camera of FIG. 1.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
(1) One Embodiment (1-1) Configuration of One Embodiment (1-1-1) Overall Configuration FIG. 1 is a block diagram of a digital camera according to an embodiment of the present invention. The digital camera 1 includes an imaging system 2, a memory card 3, a bus 4, a CPU (Central Processing Unit) 5, a frame buffer 6, an image display unit 7, a memory 8, a key input unit 9, and the like. The acquired moving image is recorded in the memory card 3.

  The imaging system 2 functions as a moving image data input unit in the digital camera 1, and includes a lens driving block 10, a lens 11, a diaphragm 12, a solid-state imaging device 13, a driver 14, a timing generator (TG: Timing Generator) 15, and a signal processing unit. 16 etc.

  The lens driving block 10 changes the focus, magnification, and aperture 12 of the lens 11 under the control of the CPU 5 via the bus 4. The lens 11 condenses incident light on the imaging surface of the solid-state imaging device 13 via the diaphragm 12 and forms an optical image to be imaged on the imaging surface of the solid-state imaging device 13.

  The solid-state imaging device 13 is an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The solid-state image sensor 13 operates in accordance with various drive signals output from the driver 14, and outputs an imaging result of an optical image formed on the imaging surface as an imaging signal. The CPU 5 controls the frame rate, charge accumulation time, and the like that are imaged by the solid-state imaging device 13. In the imaging system 2, the driver 14 generates a driving signal for the solid-state imaging device 13 in accordance with various timing signals output from the timing generator 15. The CPU 5 controls the timing generator 15 via the bus 4. Accordingly, the timing signal output to the driver 14 is also controlled by the CPU 5.

  The signal processing unit 16 performs correlated double sampling processing (CDS: Correlated Double Sampling) on the imaging signal output from the solid-state imaging device 13, and then performs automatic gain adjustment (AGC: Automatic Gain Control), analog-digital conversion processing (AD: Analog to Digital converter), and outputs the moving image data D1 to the bus 4.

  The frame buffer 6 is a working memory that temporarily stores the moving image data D1 output from the signal processing unit 16, that is, the imaging system 2. At the time of imaging, the frame buffer 6 inputs the moving image data D1 output from the imaging system 2 via the bus 4 and temporarily stores it. The CPU 5 inputs / outputs the moving image data D1 to / from the frame buffer 6, processes it, and then outputs it to the image display unit 7 for display. When receiving an instruction to start recording from the user, the CPU 5 performs processing such as compression on the moving image data D1. During reproduction, the CPU 5 reads and expands the moving image data D3 from the memory card 3, temporarily stores the expanded data D3 in the frame buffer 6, and then outputs and displays it on the image display unit 7. Various methods can be applied to the compression and decompression processing. In this embodiment, as an example, it is assumed that it conforms to MPEG2.

  The image display unit 7 is a display unit configured by a liquid crystal display panel or the like, and acquires and displays moving image data D1 from the frame buffer 6 via the bus 4 under the control of the CPU 5 during imaging. Similarly, during reproduction, the image display unit 7 acquires and displays moving image data from the frame buffer 6 via the bus 4 under the control of the CPU 5.

  The CPU 5 is a controller that controls the operation of the digital camera 1, executes a program stored in the memory 8, and responds to a user operation detected by the key input unit 9, in accordance with the digital camera 1 of the present embodiment. Are controlled, and processing of moving image data stored in the frame buffer 6 is performed. In the present embodiment, the program is described as being stored and provided in advance in the memory 8, but the present invention is not limited to this. For example, the program may be recorded on a recording medium such as a memory card and provided, or may be provided by downloading via a network. The key input unit 9 is an operator in the digital camera 1 of the present embodiment, and functions as an instruction input unit that receives various instructions from the user. Hereinafter, it is assumed that instructions from the user are input via the key input unit 9 unless otherwise specified.

  When receiving a shooting instruction from the user, the CPU 5 sequentially stores the moving image data D <b> 1 acquired by controlling the imaging system 2 in the frame buffer 6. Further, the CPU 5 executes gamma correction processing, demosaic processing, white balance processing, and the like on the moving image data D1 to store the moving image data again in the frame buffer 6, and transfers the stored moving image data to the image display unit 7. To display. Thereby, the CPU 5 displays the captured monitor image on the image display unit 7.

  When the captured moving image is displayed on the monitor, when receiving an instruction to record the captured moving image from the user, the CPU 5 sequentially reads and compresses the moving image data D1 stored in the frame buffer 6, and the memory card. Record in 3. Thereby, the CPU 5 records the captured moving image on the memory card 3.

  When receiving an instruction to reproduce the moving image recorded on the memory card 3 from the user, the CPU 5 sequentially acquires the corresponding moving image data from the memory card 3, decompresses this moving image data, and stores it in the frame buffer 6. Further, the CPU 5 sequentially transfers the moving image data from the frame buffer 6 to the image display unit 7 to display on the image display unit 7. Thus, the CPU 5 reproduces the moving image data recorded on the memory card 3 and displays it on the image display unit 7.

  In recording and reproducing these moving images, when recording the imaging result, the CPU 5 switches the operation mode to a small-screen composition mode to be described later according to the user's operation, and switches the processing of the moving image data D1 recorded on the memory card 3.

(1-1-2) Sub-screen composition mode FIG. 2 is a diagram for explaining the principle of the sub-screen composition mode. In the small-screen composition mode, the CPU 5 assigns the moving image data D1 obtained from the imaging system 2 to the parent screen. Further, a partial area of the parent screen designated by the user is cut out from the parent screen, enlarged and reduced, and assigned to the child screen. The CPU 5 combines the child screen with the parent screen and records it on the memory card 3. In addition, the monitor image of the moving image obtained by combining the images is displayed on the image display unit 7.

  Further, the CPU 5 displays a correspondence indicating a correspondence relationship between the child screen and the corresponding region on the monitor image according to a user instruction. Further, information indicating this correspondence (hereinafter referred to as sub-screen information) is repeatedly multiplexed with the moving image data D1 at a constant time interval and recorded on the memory card 3.

  In FIG. 2, each child screen is displayed with a border, the region corresponding to the parent screen is surrounded by a frame shape, and a broken line is displayed to show the border of the child screen corresponding to the frame shape of the parent screen. Are shown in association with each other. In the example of FIG. 2, the four areas are displayed in the sub-screen in accordance with the instruction to display the four areas specified by the user in the sub-screen, and one of the four sub-screens is Relationship mapping is displayed. As for the display indicating the correspondence, it is only necessary for the user to know which area of the parent screen corresponds to the area allocated to the child screen, and the corresponding child screen has a different thickness, for example, blinking display. Various display methods can be widely applied, such as switching the display to colors or displaying arrows instead of broken lines.

  FIG. 3 is a functional configuration diagram of a main part of the digital camera 1 according to the small-screen composition mode. The CPU 5 functions as functional blocks of the child screen generation unit 5A, the image compression unit 5B, the data multiplex file conversion unit 5C, and the child screen setting unit 5D in the child screen composition mode. These functional blocks 5A to 5D constitute a recording system.

  Here, the sub-screen generation unit 5A generates a sub-screen video data by cutting out and enlarging and reducing an area specified by the user from the video data D1 based on the sub-screen information D2. The sub-screen generation unit 5A combines the video data of the sub-screen with the original video data D1 and outputs the resultant. The image compression unit 5B compresses and outputs the moving image data output from the small screen generation unit 5A. Further, the compression rate of the moving image data used for the child screen may be lowered according to the user's instruction. For example, in this data compression process, the image compression unit 5B executes a data compression process for temporarily increasing the target code amount for the moving picture data for the small screen according to the user's instruction, thereby The data compression rate of the moving image data corresponding to may be temporarily reduced. As a result, the sub-screen image can be made clearer. The sub screen setting unit 5D accepts settings such as the position of the sub screen, the size of the sub screen, and the addition of the sub screen in response to the user's operation of the key input unit 9, and generates sub screen information D2. The data multiplex file forming unit 5C multiplexes and files the output data of the image compression unit 5B together with the audio data and the small screen information D2 and records them in the memory card 3.

  The CPU 5 configures functional blocks of a file read data separation unit 5E, an image decompression unit 5F, and an association display unit 5G during reproduction corresponding to the configuration of these recording systems. These functional blocks 5E to 5G constitute a reproduction system.

  The file reading data separation unit 5E reproduces and inputs the file data designated by the user from the memory card 3, and separates the data into moving image data, audio data, and sub-screen information D2 and outputs the separated data. The image decompression unit 5F decompresses and outputs the moving image data separated by the file read data separation unit 5E. The association display unit 5G displays the above-described association on the reproduction screen based on the sub-screen information D2 separated by the file read data separation unit 5E.

  FIG. 4 is a chart showing the format of the sub-screen information. In the sub-screen information D2, a sub-screen number (indicated by numerals 1 to n) for identifying the sub-screen is set, and information is recorded for each sub-screen identified by the sub-screen number. In the sub-screen information D2, information on coordinates (x, y) and size (w, h) indicating the display position on the main screen is recorded in the item “specify sub-screen”. Information on coordinates (x, y) and size (w, h) indicating the position of the corresponding area of the parent screen from which the child screen is cut out is recorded in the “designation” item.

  FIG. 4 shows a case where the user has set n sub-screens. For example, the child screen of child screen number 1 is obtained by extracting an area of size pw1, ph1 around the parent screen coordinates px1, py1 from the parent screen moving image data D1, and then centering around the parent screen coordinates cx1, cy1. It is shown that the size is set to cw1 and ch1 and is generated by pasting the area.

  The CPU 5 cuts out corresponding data according to the coordinates and size recorded in the child screen information D2 from the moving image data D1 of the parent screen, and then the ratio between the size set in the parent screen and the size set in the child screen. The moving image cut out according to the above is enlarged or reduced. Further, the CPU 5 combines the enlarged or reduced moving image with the original moving image D1 according to the coordinates recorded in the small-screen information D2, and records the synthesized image in the memory card 3.

  In accordance with an instruction from the user, the CPU 5 switches the operation mode to the child screen composition mode, and switches the operation mode from the child screen composition mode to the original operation mode. FIG. 5 is a flowchart showing the processing procedure of the CPU 5 that accepts the setting of the sub-screen in this sub-screen composition mode.

  When the CPU 5 receives an instruction to set a small screen from the user, the CPU 5 starts a small screen setting process (step S1). Specifically, the CPU 5 increments the number of sub-screens set in the sub-screen information D2 by 1 (step S2), and temporarily registers the sub-screen information in the default setting. Here, the default setting of the sub-screen information is, for example, displaying a frame shape with a certain size in the center of the main screen, and displaying the video in the area surrounded by the frame shape without any enlargement or reduction. It is set to display with a sub-screen at the corner of the screen.

  The CPU 5 sets the parent screen to the selected state (step S3). Subsequently, the CPU 5 displays the child screen and the frame shape based on the temporarily registered child screen information D2 and the selected state (step S4). Moreover, the display of the frame shape displayed on the parent screen or the border of the child screen is switched to, for example, blinking display so that the parent screen and the child screen set in the selected state can be recognized.

  Subsequently, the CPU 5 determines whether or not the selection switch key has been operated by the user (step S5). If it is determined that the selection switch key has been operated, the CPU 5 determines whether or not the parent screen is set to the selected state (step S6). If it is determined that the parent screen is set to the selected state, the selected state is switched from the parent screen to the child screen (step S7). When switching to the selected child screen, the process returns to step 4 to stop the blinking display of the frame shape of the parent screen and start the blinking display of the border of the corresponding child screen. On the other hand, if it is determined in step S6 that the parent screen is not set to the selected state, the CPU 5 switches the selected state from the child screen to the parent screen (step S8). When this selection state is switched, the process returns to step 4 to stop the blinking display in the border of the child screen and start the blinking display of the frame shape of the parent screen.

  If it is determined in step S5 that the selection switch key has not been operated by the user, the CPU 5 determines whether or not the position setting key has been operated (step S9). When it is determined that the position setting key has been operated, the CPU 5 updates the sub-screen information in response to the operation of the operator by the user, and the frame shape in the selected state or the position of the sub-screen in the selected state ( The display position is changed (step S10). When the child screen information D2 is updated, the process returns to step S4.

  On the other hand, if it is determined in step S9 that the position setting key has not been operated, the CPU 5 determines whether or not the size setting key has been operated (step S11). When it is determined that the size setting key has been operated, the CPU 5 updates the sub screen information in response to the operation of the operator by the user, and the size of the frame shape in the selected state or the sub screen in the selected state Is changed (step S12). When the child screen information D2 is updated, the process returns to step S4.

  In contrast, if it is determined in step S11 that the size setting key has not been operated, the CPU 5 determines whether or not the cancel key has been operated (step S13). If it is determined that the cancel key has been operated, the CPU 5 updates the child screen information D2 to delete one temporarily registered child screen (step S14), cancels the display of the frame shape of the parent screen, and performs this processing. After returning to the state before starting the procedure (step S15), this processing procedure is ended (step S16).

  If it is determined in step S13 that the cancel key has not been operated, the CPU 5 determines whether or not the registration key has been operated (step S17). If it is determined that the registration key has not been operated, the process returns to step S4. On the other hand, if it is determined in step S17 that the registration key has been operated, the temporarily registered small-screen information D2 is fully registered. Thereby, the CPU 5 updates the small-screen information D2 recorded in the memory card 3 according to the user setting, erases the parent-screen frame display (step S15), and ends this processing procedure (step S16).

  The digital camera 1 according to the present embodiment sequentially sets the sub-screen numbers set in the sub-screen information D2 to the selected state in accordance with an instruction from the user, changes the position and size of the frame shape, and displays the sub-screen. The position and size are changed, and the small screen information D2 is also deleted. Furthermore, the association display is started, and the association display target is sequentially and cyclically switched.

  FIG. 6 is a flowchart showing a processing procedure of the CPU 5 related to these processes. When the user instructs the display of the association, the CPU 5 starts this processing procedure (step S21) and determines whether or not the small screen information D2 is set (step S22). In this way, the CPU 5 determines whether or not the sub-screen information D2 is not set in the moving image being displayed, specifically, whether or not the number of set sub-screens is zero.

  When it is determined that the small screen information D2 is not set in the moving image being displayed, specifically, when it is determined that the number of set small screens is 0, the CPU 5 performs this processing procedure. Is finished (step S23). On the other hand, when determining that the small-screen information D2 is set in the moving image being displayed, the CPU 5 sets the information number to 1 (step S24). Here, the information number is a number corresponding to the sub-screen number recorded in the sub-screen information described with reference to FIG. 4, and is a number for specifying the sub-screen displaying the association.

  Subsequently, the CPU 5 sets the sub-screen number of the sub-screen information D2 corresponding to the information number as an association display target, and displays the frame shape corresponding to the parent screen as shown in FIG. Is displayed (step S25).

  The CPU 5 determines whether or not the user has operated the next sub-screen key, specifically, whether or not the switching key for switching the sub-screen to be displayed has been operated (step S26). If it is determined that the switching key for switching the display target child screen has been operated, the CPU 5 increments the information number by 1 (step S27), and determines whether the number of information numbers exceeds the number of child screens (step S27). S28). If it is determined that the number of information numbers exceeds the number of child screens, the CPU 5 sets the information number to 1 (step S29), and then returns to step S25. On the other hand, if it is determined in step S28 that the number of information numbers does not exceed the number of child screens, the process directly returns to step S25. Thereafter, the CPU 5 sequentially and cyclically switches the association display.

  If it is determined in step S26 that the switching key for switching the sub-screen to be displayed has not been operated, the CPU 5 determines whether or not the delete key has been operated (step S30), and determines that the delete key has been operated. Then, the CPU 5 deletes the child screen number of the child screen information D2 corresponding to the information number from the registration (step S31), resets the child screen number, and returns to step S22.

  On the other hand, if it is determined in step S30 that the delete key has not been operated, the CPU 5 determines whether or not the setting key has been operated (step S32). If it is determined here that the setting key has been operated, the CPU 5 performs sub-screen setting of the information number (step S33). Specifically, the CPU 5 executes a processing procedure similar to that in steps S5 to S17 described above, receives a change in size or the like for the child screen specified by the information number, and returns to step S25.

  On the other hand, if it is determined in step S32 that the setting key has not been operated, the CPU 5 determines whether or not the end key has been operated (step S34), and if it is determined that the end key has been operated, The processing procedure ends (step S35). On the other hand, if it is determined in step S34 that the end key has not been operated, the process returns to step S25.

  At the time of reproduction, when the CPU 5 performs the process of reproducing the file recorded in the small-screen composition mode, the display of the correspondence is displayed on the monitor image based on the small-screen information multiplexed and recorded according to the user's operation. indicate. At this time, the same processing procedure as that in FIG. 6 is executed to sequentially and sequentially switch the display target of the correspondence. Further, when the user instructs to display and display the sub screen, only the sub screen may be selectively displayed in the full screen, and the display target of the full screen display may be sequentially and cyclically switched according to the user's operation. When the child screen is displayed in full screen in this way, the parent screen may be displayed by the child screen.

(1-2) Operation of Embodiment As described above, in the digital camera 1 of this embodiment (FIG. 1), the solid-state imaging device 13 performs photoelectric conversion processing on the optical image generated by the lens 11 and images it. The resulting imaging signal is generated. The signal processing unit 16 processes the imaging signal to generate moving image data D1. In the digital camera 1 according to the present embodiment, the CPU 5 inputs the moving image data D1 through the frame buffer 6, performs processing such as data compression, and then records the data in the memory card 3. On the other hand, when receiving a reproduction instruction from the user, the CPU 5 sequentially reads the moving image data recorded on the memory card 3, decompresses the data, and displays it on the image display unit 7.

  In these processes, when receiving an instruction to record a moving image in the small-screen synthesis mode from the user, the CPU 5 cuts out the moving image data in the area instructed by the user from the moving-image data D1 obtained from the imaging system 2, and uses the small-screen image. The moving image data for the sub-screen is generated, and the moving image data for the small screen is combined with the original moving image data D1 and compressed to be recorded on the memory card 3. Specifically, the moving picture data D1 is generated by multiplexing the small-screen information D2 having the small-screen information.

  Thereby, the digital camera 1 according to the present embodiment can record a plurality of moving images simultaneously with a simple configuration. That is, the digital camera 1 of the present embodiment can process moving image data obtained from one imaging system and record a plurality of moving images without providing imaging systems corresponding to the parent screen and the child screen, respectively. And the overall configuration can be simplified. In addition, since the plurality of moving images can be processed by a single compression process, the configuration can be further simplified.

  Further, during playback, the display target of full screen display can be switched between the main screen and the sub screen, and a plurality of moving images can be selectively displayed, so that convenience for the user can be improved.

  As described above, in the digital camera 1 of the present embodiment, the sub-screen information D2 indicating the correspondence between the sub-screen and the corresponding area of the parent screen is multiplexed with the moving image data D1, compressed, and recorded. Therefore, in the digital camera 1 according to the present embodiment, since the child screen synthesized based on the child screen information D2 can be selectively displayed on the whole screen or the like on the parent screen displayed based on the moving image data D1, the user can Convenience can be improved.

  Further, in the digital camera 1 of the present embodiment, a display indicating the correspondence between the child screen and the corresponding area of the parent screen is displayed in the monitor image at the time of recording in accordance with a user instruction. Therefore, in the digital camera 1 according to the present embodiment, the user can easily and reliably grasp the correspondence between the parent screen and the child screen, so that convenience can be improved.

  Furthermore, in the digital camera 1 of the present embodiment, the user can instruct additional setting of the sub screen, the position and size of the area set on the main screen, and the setting of the position and size of the sub screen, so that the convenience for the user is further increased. improves. In other words, since the user can set the position of the area of the parent screen to which the child screen is pasted, the position of the parent screen assigned to the child screen can be variously changed and the child screen can be displayed at a desired position of the parent screen. it can. In addition, the user can set the size of the display area set on the parent screen, the size of the child screen, the position where the child screen is pasted, the number of child screens to be pasted, and the like, further improving user convenience. be able to.

  Further, at the time of reproduction, a display showing the correspondence between the sub-screen and the corresponding area of the main screen is displayed based on the sub-screen information that is multiplexed and recorded on the moving image data, as in the case of recording. As a result, the digital camera 1 can easily and reliably grasp the correspondence between the parent screen and the child screen even during reproduction.

(2) Other Embodiments In the above-described embodiment, the case where the user accepts the setting of the area or the like set in the child screen has been described. However, the present invention is not limited to this, and the position and size of the child screen are not limited thereto. For example, the size may be automatically adjusted according to the position of the subject on the parent screen, the number of child screens, and the like. Further, the target subject may be tracked by applying, for example, a method for detecting a region of interest.

  In the above-described embodiment, the case where the data compression rate is reduced in the child screen has been described. However, the present invention is not limited to this, and data compression may be performed at a constant data compression rate. The present invention can be widely applied to the case of increasing and compressing.

  Further, in the above-described embodiment, the case where the user sets the area to be set in the child screen and processes the moving image has been described. However, the present invention is not limited to this, for example, using a motion tracking method. The movement of the area set on the parent screen may be tracked, and the position and / or size of the area set on the parent screen may be dynamically changed based on the tracking result. Various methods such as a method of statistically processing motion vectors and a method of tracking feature points can be applied to the motion tracking method.

  Further, in the above-described embodiment, the case where a part of the area is cut out from the parent screen and set as the child screen has been described. However, the present invention is not limited to this, and the entire area of the parent screen is assigned to the child screen. The sub-screen may change the magnification and display the area.

  In the above-described embodiment, the case where settings of various sub-screens are received by key operation has been described. However, the present invention is not limited to this, and various interfaces such as a touch panel interface can be widely applied. .

  Furthermore, in the above-described embodiment, the case where the small screen information is multiplexed and recorded in the moving image data has been described. However, the present invention is not limited to this, and the recording of the small screen information is omitted as necessary. May be. Specifically, if the display of the association between the child screen and the parent screen is not required during reproduction, the child screen information does not have to be filed.

  As described above, according to the present invention, the monitor image displays the display indicating the correspondence between the child screen and the corresponding area of the parent screen, so that the correspondence between the child screen and the parent screen can be easily and reliably grasped. Since it can be set, user convenience can be improved.

  Furthermore, according to the present invention, when moving image data is played back on the parent screen and the child screen, the correspondence between the child screen and the corresponding area is displayed based on the child screen information, thereby improving the convenience for the user. Can do.

  Furthermore, according to the present invention, since the image for the main screen and the sub-screen information indicating the correspondence between the sub-screen and the corresponding area of the main screen can be multiplexed into the moving image data and simultaneously compressed and recorded, Neither compression processing nor decompression processing increases. Therefore, it can be realized with a simple configuration.

  In the above-described embodiment, the case where the present invention is applied to a digital camera and the moving image data D1 input from the moving image data input unit by the imaging system is processed has been described. However, the present invention is not limited thereto. For example, the present invention can be widely applied to a case where moving image data D1 provided by a network or various recording media is recorded on a recording medium such as a hard disk. Further, it can be operated by a moving image processing program that causes a computer including a CPU and a memory to control a moving image processing apparatus to function as the above-described units. The moving image processing program can be distributed via a communication line, or can be distributed by writing on a recording medium such as a CD-ROM.

  The present invention is not limited to the above-described embodiments, and various combinations of the above-described embodiments are made and various modifications are made to the above-described embodiments without departing from the spirit of the invention. It can be in the form.

    The present invention can be applied to, for example, a digital camera.

DESCRIPTION OF SYMBOLS 1 ... Digital camera, 2 ... Imaging system, 3 ... Memory card, 5 ... CPU, 6 ... Frame buffer, 13 ... Solid-state image sensor, 16 ... Signal processing part

Claims (11)

A video data input unit for inputting input video data;
For each image data of the frames constituting the input moving image data, cut out at least a part of realm, a child screen data generating unit for generating image data for the child screen of the frame,
An image combining unit for generating a synthetic video data by assigning the image data and the image data for the small picture of the frame to each parent screen and a child screen,
The composite moving image data is subjected to data compression processing and recorded on a recording medium, and sub-screen information in which the position of the sub-screen is associated with the cut-out position of the at least a part of the area corresponding to the sub-screen is recorded on the recording medium. A moving image processing apparatus having a recording unit for recording. An image display unit for displaying the synthesized moving image data;
The image display unit displays the synthesized moving image data and displays the correspondence between the position of the child screen and the position of the at least part of the region corresponding to the child screen based on the child screen information is carried out, the moving image processing apparatus according to claim 1. The image display unit displays the synthesized moving image data, and switches the target child screen according to a user operation, and the position of the child screen and the position of the at least a part of the region corresponding to the child screen, The moving image processing apparatus according to claim 2, wherein a display that clearly indicates the correspondence between the sub-screen information and the sub-screen information is performed. The recording unit, before Noriko multiplexes for recording screen information on the synthetic moving image data, the moving image processing apparatus according to any one of claims 1 to 3. An instruction input unit that inputs at least one instruction of the position of the at least part area, the size of the at least part area, the position of the child screen, the size of the child screen, and the addition of the child screen; Item 5. The moving image processing apparatus according to any one of Items 1 to 4 . A sub-screen setting unit configured to set a sub-screen based on the sub-screen information, and the sub-screen setting unit automatically sets the position of the sub-screen and the size of the sub-screen according to the number of sub-screens or the position of the subject. The moving image processing apparatus according to claim 1 , wherein the moving image processing apparatus is determined automatically . A playback unit for playing back video data from a recording medium;
A display unit for displaying moving image data reproduced by the reproduction unit,
The video data is
A combined moving image generated by cutting out at least a part of each frame image data constituting the input moving image data as image data for the child screen and combining the image data for the child screen with the image data of the frame. Data,
The input moving image data, and sub-screen information in which the position of the sub-screen and the cut-out position of the at least part of the region corresponding to the sub-screen are associated and recorded are multiplexed.
The display unit
A moving image processing apparatus for performing a display that clearly shows the correspondence between the position of the sub-screen and the position of the at least part of the region corresponding to the sub-screen based on the sub- screen information. A step of inputting input video data;
For each image data of the frames constituting the input moving image data the input, cut at least a part of realm, a step of generating image data for the child screen of the frame,
Generating a synthetic video data by assigning the image data and the image data for the child screen the generated of said frame, each parent screen and a child screen,
The generated combined video data is subjected to data compression processing and recorded on a recording medium, and sub-screen information in which the position of the sub-screen is associated with the cut-out position of the at least a part of the region corresponding to the sub-screen. And a step of recording on a recording medium . Reproducing the moving image data from the recording medium;
Displaying the reproduced moving image data,
The video data is
A combined moving image generated by cutting out at least a part of each frame image data constituting the input moving image data as image data for the child screen and combining the image data for the child screen with the image data of the frame. Data,
The input moving image data, and sub-screen information in which the position of the sub-screen and the cut-out position of the at least part of the region corresponding to the sub-screen are associated and recorded are multiplexed.
The step of displaying the moving image data includes:
A moving image processing method including a step of performing a display that clearly shows the correspondence between the position of the sub-screen and the position of the at least part of the region corresponding to the sub-screen based on the sub- screen information. In the computer that controls the video processing device,
A function to input video data,
For each image data of the frames constituting the input moving image data said input, a function of at least partially cut out of the realm, to generate image data for the child screen of the frame,
And generating a synthetic video data by assigning the image data and the image data for the child screen the generated of said frame, each parent screen and a child screen,
The generated combined video data is subjected to data compression processing and recorded on a recording medium, and sub-screen information in which the position of the sub-screen is associated with the cut-out position of the at least a part of the region corresponding to the sub-screen. A moving image processing program for realizing a function of recording on a recording medium . In the computer that controls the video processing device,
A function for reproducing moving image data from a recording medium;
A moving image processing program for realizing the function of displaying the reproduced moving image data,
The video data is
A combined moving image generated by cutting out at least a part of each frame image data constituting the input moving image data as image data for the child screen and combining the image data for the child screen with the image data of the frame. Data,
Sub-screen information in which the input moving image data and the sub-screen information in which the position of the sub-screen and the cut-out position of the at least part of the region corresponding to the sub-screen are associated with each other is multiplexed and recorded,
The function of displaying the video data is
A moving image processing program comprising a function of performing a display that clearly shows a correspondence between a position of the sub-screen and a position of at least a part of the region corresponding to the sub-screen based on the sub- screen information.

Images