JP2010199666A - Moving picture processing device, moving picture processing method and moving picture processing program for recording moving picture - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To simultaneously record a plurality of moving pictures with a simple configuration. <P>SOLUTION: In a moving picture processing device, moving picture of at least a partial region is cut out from the moving picture data obtained by an imaging system to generate moving picture data for a sub-screen, and compositing the generated moving picture data for the sub-screen to original moving picture data and recording the composited data. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to moving image processing for recording a moving image in a digital camera, for example, a moving image processing apparatus, a moving image processing method, and a moving image processing program capable of simultaneously recording a plurality of moving images.

  2. Description of the Related Art Conventionally, a digital camera captures a desired subject, acquires a moving image, and records the moving image on a recording medium such as a memory card. In addition, a moving image recorded on the recording medium is reproduced and displayed in response to a user operation.

  In addition, in a television receiver, a plurality of video data is displayed separately for a main screen which is a normal size screen and a sub screen which is displayed as a small screen in the main screen, so-called picture-in-picture. The method is being performed.

  In still image recording, for example, a still image obtained from an imaging system and a still image obtained by partially enlarging the still image are displayed side by side, and these still images are recorded by a user operation. (See Patent Document 1).

JP 2008-22306 A

  Although it is efficient to record a plurality of moving images at the same time, it is necessary to provide an imaging system for acquiring each of the plurality of moving images. Further, when a plurality of moving images are displayed on the same screen, there is a problem that the relationship becomes difficult to understand and usability is deteriorated.

  The present invention has been made in consideration of the above points. A moving image processing apparatus, a moving image processing method, and a moving image processing program capable of simultaneously recording a plurality of moving images with a simple configuration and improving convenience. The purpose is to provide.

  In order to solve the above problems, the invention of claim 1 is applied to a moving image processing apparatus, and a moving image data input unit for inputting input moving image data, and moving image data of at least a part of the region from the input moving image data are cut out. A sub-screen data generator for generating sub-screen video data, and assigning the input video data and sub-screen video data to the main screen and sub-screen, respectively, for the input video data and sub-screen An image composition unit that synthesizes the moving image data and generates combined moving image data, and a recording unit that compresses the combined moving image data and records the data on a recording medium.

  According to a second aspect of the present invention, in the configuration of the first aspect, the recording unit multiplexes sub-screen information indicating a correspondence relationship between the sub-screen and the at least a part of the region into the synthesized moving image data. Record.

  The invention according to claim 3 has an image display unit for displaying the synthesized moving image data in the configuration of claim 1 or 2, wherein the image display unit is configured to display the child movie based on the synthesized movie data. A display showing a correspondence relationship between the screen and the corresponding at least part of the area is displayed.

  According to a fourth aspect of the present invention, in the configuration of the first, second, or third aspect, the position of the at least part of the region, the size of the at least part of the region, the position of the sub-screen, and the child An instruction input unit is provided for inputting at least one instruction of screen size and sub-screen addition.

  Further, the invention of claim 5 has a sub-screen setting unit for setting a sub-screen based on the sub-screen information in the configuration of claim 2, wherein the sub-screen setting unit includes the number of sub-screens or the position of the subject. The position of the child screen and the size of the child screen are automatically determined according to the above.

  According to a sixth aspect of the invention, there is provided a moving image processing apparatus, comprising: a reproducing unit that reproduces moving image data from a recording medium; and a display unit that displays the moving image data reproduced by the reproducing unit; The data is generated by cutting out the moving image data corresponding to at least a part of the area from the input moving image data as the moving image data for the sub screen, and synthesizing the moving image data for the sub screen with the input moving image data according to the user's instruction. Composite video data, and the input video data and sub-screen information indicating a correspondence relationship between at least a part of the input video data corresponding to the sub-screen video data are multiplexed and recorded. The display unit displays a display indicating a correspondence relationship between the sub-screen and the at least a part of the region corresponding to the sub-screen information based on the sub-screen information.

  Further, the invention of claim 7 is applied to a moving image processing method, the step of inputting the input moving image data, the moving image data of at least a part of the region is cut out from the input moving image data, and the moving image for the sub-screen A step of generating data, assigning the input moving image data and the generated moving image data for the child screen to a parent screen and a child screen, respectively, and inputting the input moving image data and the moving image data for the child screen into an image Synthesizing and generating synthesized moving image data; and data compression processing of the generated synthesized moving image data and recording on a recording medium.

  The invention of claim 8 is applied to a moving image processing method, and has a step of reproducing moving image data from a recording medium, and a step of displaying the reproduced moving image data. In accordance with the instruction, the moving image data corresponding to at least a part of the area is cut out from the input moving image data as the moving image data for the sub screen, and the moving image data for the sub screen is generated by combining the input moving image data with the input moving image data. Yes, the input video data and sub-screen information indicating a correspondence relationship between at least a part of the input video data corresponding to the sub-screen video data are multiplexed and recorded, and the video data is The step of displaying the correspondence relationship between the at least a part of the region corresponding to the moving image data for the child screen based on the child screen information based on a user instruction Comprising the step of displaying a display indicating.

  The invention according to claim 9 is applied to a moving image processing program, and a function for inputting input moving image data to a computer that controls the moving image processing apparatus, and at least a part of the area from the input moving image data. A function for cutting out moving image data and generating moving image data for a child screen, and the input moving image data and the input moving image data and the generated moving image data for the child screen assigned to a parent screen and a child screen, respectively. And a function of synthesizing the moving picture data for the sub-screen and generating synthesized moving picture data, and a function of compressing the generated synthesized moving picture data and recording it on a recording medium.

  According to a tenth aspect of the present invention, there is provided a function of reproducing moving image data from a recording medium and a function of displaying the reproduced moving image data on a computer that controls the moving image processing apparatus by applying to the moving image processing program. A moving image processing program to be realized, wherein the moving image data is cut out from moving image data corresponding to at least a portion of the input moving image data as moving image data for a child screen according to a user instruction, and the moving image data for the child screen is Composite video data generated by image synthesis with the input video data, and showing a correspondence relationship between the input video data and at least a part of the input video data corresponding to the sub-screen video data Sub-screen information is multiplexed and recorded, and the function of displaying the moving image data is based on the sub-screen information according to a user instruction. Including the ability to display a display indicating the correspondence between said at least a portion of the region corresponding to the moving image data for the screen.

  According to the present invention, a plurality of moving images and the like can be recorded simultaneously with a simple configuration.

1 is a block diagram of a digital camera according to a first embodiment of the present invention. It is a figure explaining the principle of the small-screen composition mode in the digital camera of FIG. It is a function block diagram of the principal part of the digital camera of FIG. It is a chart which shows the small screen information in the digital camera of FIG. 3 is a flowchart for explaining a small screen setting process in the digital camera of FIG. 1. 4 is a flowchart for explaining association display processing in the digital camera of FIG. 1.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate.
(1) One Embodiment (1-1) Configuration of One Embodiment (1-1-1) Overall Configuration FIG. 1 is a block diagram of a digital camera according to an embodiment of the present invention. The digital camera 1 includes an imaging system 2, a memory card 3, a bus 4, a CPU (Central Processing Unit) 5, a frame buffer 6, an image display unit 7, a memory 8, a key input unit 9, and the like. The acquired moving image is recorded in the memory card 3.

  The imaging system 2 functions as a moving image data input unit in the digital camera 1, and includes a lens driving block 10, a lens 11, a diaphragm 12, a solid-state imaging device 13, a driver 14, a timing generator (TG: Timing Generator) 15, and a signal processing unit. 16 etc.

  The lens driving block 10 changes the focus, magnification, and aperture 12 of the lens 11 under the control of the CPU 5 via the bus 4. The lens 11 condenses incident light on the imaging surface of the solid-state imaging device 13 via the diaphragm 12 and forms an optical image to be imaged on the imaging surface of the solid-state imaging device 13.

  The solid-state imaging device 13 is an imaging device such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor). The solid-state image sensor 13 operates in accordance with various drive signals output from the driver 14, and outputs an imaging result of an optical image formed on the imaging surface as an imaging signal. The CPU 5 controls the frame rate, charge accumulation time, and the like that are imaged by the solid-state imaging device 13. In the imaging system 2, the driver 14 generates a driving signal for the solid-state imaging device 13 in accordance with various timing signals output from the timing generator 15. The CPU 5 controls the timing generator 15 via the bus 4. Accordingly, the timing signal output to the driver 14 is also controlled by the CPU 5.

  The signal processing unit 16 performs correlated double sampling processing (CDS: Correlated Double Sampling) on the imaging signal output from the solid-state imaging device 13, and then performs automatic gain adjustment (AGC: Automatic Gain Control), analog-digital conversion processing (AD: Analog to Digital converter), and outputs the moving image data D1 to the bus 4.

  The frame buffer 6 is a working memory that temporarily stores the moving image data D1 output from the signal processing unit 16, that is, the imaging system 2. At the time of imaging, the frame buffer 6 inputs the moving image data D1 output from the imaging system 2 via the bus 4 and temporarily stores it. The CPU 5 inputs / outputs the moving image data D1 to / from the frame buffer 6, processes it, and then outputs it to the image display unit 7 for display. When receiving an instruction to start recording from the user, the CPU 5 performs processing such as compression on the moving image data D1. During reproduction, the CPU 5 reads and expands the moving image data D3 from the memory card 3, temporarily stores the expanded data D3 in the frame buffer 6, and then outputs and displays it on the image display unit 7. Various methods can be applied to the compression and decompression processing. In this embodiment, as an example, it is assumed that it conforms to MPEG2.

  The image display unit 7 is a display unit configured by a liquid crystal display panel or the like, and acquires and displays moving image data D1 from the frame buffer 6 via the bus 4 under the control of the CPU 5 during imaging. Similarly, during reproduction, the image display unit 7 acquires and displays moving image data from the frame buffer 6 via the bus 4 under the control of the CPU 5.

  The CPU 5 is a controller that controls the operation of the digital camera 1, executes a program stored in the memory 8, and responds to a user operation detected by the key input unit 9, in accordance with the digital camera 1 of the present embodiment. Are controlled, and processing of moving image data stored in the frame buffer 6 is performed. In the present embodiment, the program is described as being stored and provided in advance in the memory 8, but the present invention is not limited to this. For example, the program may be recorded on a recording medium such as a memory card and provided, or may be provided by downloading via a network. The key input unit 9 is an operator in the digital camera 1 of the present embodiment, and functions as an instruction input unit that receives various instructions from the user. Hereinafter, it is assumed that instructions from the user are input via the key input unit 9 unless otherwise specified.

  When receiving a shooting instruction from the user, the CPU 5 sequentially stores the moving image data D <b> 1 acquired by controlling the imaging system 2 in the frame buffer 6. Further, the CPU 5 executes gamma correction processing, demosaic processing, white balance processing, and the like on the moving image data D1 to store the moving image data again in the frame buffer 6, and transfers the stored moving image data to the image display unit 7. To display. Thereby, the CPU 5 displays the captured monitor image on the image display unit 7.

  When the captured moving image is displayed on the monitor, when receiving an instruction to record the captured moving image from the user, the CPU 5 sequentially reads and compresses the moving image data D1 stored in the frame buffer 6, and the memory card. Record in 3. Thereby, the CPU 5 records the captured moving image on the memory card 3.

  When receiving an instruction to reproduce the moving image recorded on the memory card 3 from the user, the CPU 5 sequentially acquires the corresponding moving image data from the memory card 3, decompresses this moving image data, and stores it in the frame buffer 6. Further, the CPU 5 sequentially transfers the moving image data from the frame buffer 6 to the image display unit 7 to display on the image display unit 7. Thus, the CPU 5 reproduces the moving image data recorded on the memory card 3 and displays it on the image display unit 7.

  In recording and reproducing these moving images, when recording the imaging result, the CPU 5 switches the operation mode to a small-screen composition mode to be described later according to the user's operation, and switches the processing of the moving image data D1 recorded on the memory card 3.

(1-1-2) Sub-screen composition mode FIG. 2 is a diagram for explaining the principle of the sub-screen composition mode. In the small-screen composition mode, the CPU 5 assigns the moving image data D1 obtained from the imaging system 2 to the parent screen. Further, a partial area of the parent screen designated by the user is cut out from the parent screen, enlarged and reduced, and assigned to the child screen. The CPU 5 combines the child screen with the parent screen and records it on the memory card 3. In addition, the monitor image of the moving image that has been synthesized is displayed on the image display unit 7.

  Further, the CPU 5 displays a correspondence indicating a correspondence relationship between the child screen and the corresponding region on the monitor image according to a user instruction. Further, information indicating this correspondence (hereinafter referred to as sub-screen information) is repeatedly multiplexed with the moving image data D1 at a constant time interval and recorded on the memory card 3.

  In FIG. 2, each child screen is displayed with a border, the region corresponding to the parent screen is surrounded by a frame shape, and a broken line is displayed to show the border of the child screen corresponding to the frame shape of the parent screen. Are shown in association with each other. In the example of FIG. 2, the four areas are displayed in the sub-screen in accordance with the instruction to display the four areas designated by the user in the sub-screen, and one of the four sub-screens is connected to the parent screen. Relationship mapping is displayed. As for the display indicating the correspondence, it is only necessary for the user to know which area of the parent screen corresponds to the area allocated to the child screen, and the corresponding child screen has a different thickness, for example, blinking display. Various display methods can be widely applied, such as switching the display to colors or displaying arrows instead of broken lines.

  FIG. 3 is a functional configuration diagram of a main part of the digital camera 1 according to the small-screen composition mode. The CPU 5 functions as functional blocks of the child screen generation unit 5A, the image compression unit 5B, the data multiplex file conversion unit 5C, and the child screen setting unit 5D in the child screen composition mode. These functional blocks 5A to 5D constitute a recording system.

  Here, the sub-screen generation unit 5A generates a sub-screen video data by cutting out and enlarging and reducing an area specified by the user from the video data D1 based on the sub-screen information D2. The sub-screen generation unit 5A combines the video data of the sub-screen with the original video data D1 and outputs the resultant. The image compression unit 5B compresses and outputs the moving image data output from the small screen generation unit 5A. Further, the compression rate of the moving image data used for the child screen may be lowered according to the user's instruction. For example, in this data compression process, the image compression unit 5B executes a data compression process for temporarily increasing the target code amount for the moving picture data for the small screen according to the user's instruction, thereby The data compression rate of the moving image data corresponding to may be temporarily reduced. As a result, the sub-screen image can be made clearer. The sub-screen setting unit 5D accepts settings such as the position of the sub-screen, the size of the sub-screen, and the addition of the sub-screen in response to the operation of the key input unit 9 by the user, and generates sub-screen information D2. The data multiplex file forming unit 5C multiplexes and files the output data of the image compression unit 5B together with the audio data and the small screen information D2 and records them in the memory card 3.

  The CPU 5 configures functional blocks of a file read data separation unit 5E, an image decompression unit 5F, and an association display unit 5G during reproduction corresponding to the configuration of these recording systems. These functional blocks 5E to 5G constitute a reproduction system.

  The file reading data separation unit 5E reproduces and inputs the file data designated by the user from the memory card 3, and separates the data into moving image data, audio data, and sub-screen information D2 and outputs the separated data. The image decompression unit 5F decompresses and outputs the moving image data separated by the file read data separation unit 5E. The association display unit 5G displays the above-described association on the reproduction screen based on the sub-screen information D2 separated by the file read data separation unit 5E.

  FIG. 4 is a chart showing the format of the sub-screen information. In the sub-screen information D2, a sub-screen number (indicated by numerals 1 to n) for identifying the sub-screen is set, and information is recorded for each sub-screen identified by the sub-screen number. In the sub-screen information D2, information on coordinates (x, y) and size (w, h) indicating the display position on the main screen is recorded in the item “specify sub-screen”. Information on coordinates (x, y) and size (w, h) indicating the position of the corresponding area of the parent screen from which the child screen is cut out is recorded in the “designation” item.

  FIG. 4 shows a case where the user has set n sub-screens. For example, the child screen of child screen number 1 is obtained by extracting an area of size pw1, ph1 around the parent screen coordinates px1, py1 from the parent screen moving image data D1, and then centering around the parent screen coordinates cx1, cy1. It is shown that the size is set to cw1 and ch1 and is generated by pasting the area.

  The CPU 5 cuts out corresponding data according to the coordinates and size recorded in the child screen information D2 from the moving image data D1 of the parent screen, and then the ratio between the size set in the parent screen and the size set in the child screen. The moving image cut out according to the above is enlarged or reduced. Further, the CPU 5 combines the enlarged or reduced moving image with the original moving image D1 according to the coordinates recorded in the small-screen information D2, and records the synthesized image in the memory card 3.

  In accordance with an instruction from the user, the CPU 5 switches the operation mode to the child screen composition mode, and switches the operation mode from the child screen composition mode to the original operation mode. FIG. 5 is a flowchart showing the processing procedure of the CPU 5 that accepts the setting of the sub-screen in this sub-screen composition mode.

  When the CPU 5 receives an instruction to set a small screen from the user, the CPU 5 starts a small screen setting process (step S1). Specifically, the CPU 5 increments the number of sub-screens set in the sub-screen information D2 by 1 (step S2), and temporarily registers the sub-screen information in the default setting. Here, the default setting of the sub-screen information is, for example, displaying a frame shape with a certain size in the center of the main screen, and displaying the video in the area surrounded by the frame shape without any enlargement or reduction. It is set to display with a sub-screen at the corner of the screen.

  The CPU 5 sets the parent screen to the selected state (step S3). Subsequently, the CPU 5 displays the child screen and the frame shape based on the temporarily registered child screen information D2 and the selected state (step S4). Moreover, the display of the frame shape displayed on the parent screen or the border of the child screen is switched to, for example, blinking display so that the parent screen and the child screen set in the selected state can be recognized.

  Subsequently, the CPU 5 determines whether or not the selection switch key has been operated by the user (step S5). If it is determined that the selection switch key has been operated, the CPU 5 determines whether or not the parent screen is set to the selected state (step S6). If it is determined that the parent screen is set to the selected state, the selected state is switched from the parent screen to the child screen (step S7). When switching to the selected child screen, the process returns to step 4 to stop the blinking display of the frame shape of the parent screen and start the blinking display of the border of the corresponding child screen. On the other hand, if it is determined in step S6 that the parent screen is not set to the selected state, the CPU 5 switches the selected state from the child screen to the parent screen (step S8). When this selection state is switched, the process returns to step 4 to stop the blinking display in the border of the child screen and start the blinking display of the frame shape of the parent screen.

  If it is determined in step S5 that the selection switch key has not been operated by the user, the CPU 5 determines whether or not the position setting key has been operated (step S9). When it is determined that the position setting key has been operated, the CPU 5 updates the sub-screen information in response to the operation of the operator by the user, and the frame shape in the selected state or the position of the sub-screen in the selected state ( The display position is changed (step S10). When the child screen information D2 is updated, the process returns to step S4.

  On the other hand, if it is determined in step S9 that the position setting key has not been operated, the CPU 5 determines whether or not the size setting key has been operated (step S11). When it is determined that the size setting key has been operated, the CPU 5 updates the sub screen information in response to the operation of the operator by the user, and the size of the frame shape in the selected state or the sub screen in the selected state Is changed (step S12). When the child screen information D2 is updated, the process returns to step S4.

  In contrast, if it is determined in step S11 that the size setting key has not been operated, the CPU 5 determines whether or not the cancel key has been operated (step S13). If it is determined that the cancel key has been operated, the CPU 5 updates the child screen information D2 to delete one temporarily registered child screen (step S14), cancels the display of the frame shape of the parent screen, and performs this processing. After returning to the state before starting the procedure (step S15), this processing procedure is ended (step S16).

  If it is determined in step S13 that the cancel key has not been operated, the CPU 5 determines whether or not the registration key has been operated (step S17). If it is determined that the registration key has not been operated, the process returns to step S4. On the other hand, if it is determined in step S17 that the registration key has been operated, the temporarily registered small-screen information D2 is fully registered. Thereby, the CPU 5 updates the small-screen information D2 recorded in the memory card 3 according to the user setting, erases the parent-screen frame display (step S15), and ends this processing procedure (step S16).

  The digital camera 1 according to the present embodiment sequentially sets the sub-screen numbers set in the sub-screen information D2 to the selected state in accordance with an instruction from the user, changes the position and size of the frame shape, and displays the sub-screen. The position and size are changed, and the small screen information D2 is also deleted. Furthermore, the association display is started, and the association display target is sequentially and cyclically switched.

  FIG. 6 is a flowchart showing a processing procedure of the CPU 5 related to these processes. When the user instructs the display of the association, the CPU 5 starts this processing procedure (step S21) and determines whether or not the small screen information D2 is set (step S22). In this way, the CPU 5 determines whether or not the sub-screen information D2 is not set in the moving image being displayed, specifically, whether or not the number of set sub-screens is zero.

  When it is determined that the small screen information D2 is not set in the moving image being displayed, specifically, when it is determined that the number of set small screens is 0, the CPU 5 performs this processing procedure. Is finished (step S23). On the other hand, when determining that the small-screen information D2 is set in the moving image being displayed, the CPU 5 sets the information number to 1 (step S24). Here, the information number is a number corresponding to the sub-screen number recorded in the sub-screen information described with reference to FIG. 4, and is a number for specifying the sub-screen displaying the association.

  Subsequently, the CPU 5 sets the sub-screen number of the sub-screen information D2 corresponding to the information number as an association display target, and displays the frame shape corresponding to the parent screen as shown in FIG. Is displayed (step S25).

  The CPU 5 determines whether or not the user has operated the next sub-screen key, specifically, whether or not the switching key for switching the sub-screen to be displayed has been operated (step S26). If it is determined that the switching key for switching the display target child screen has been operated, the CPU 5 increments the information number by 1 (step S27), and determines whether the number of information numbers exceeds the number of child screens (step S27). S28). If it is determined that the number of information numbers exceeds the number of child screens, the CPU 5 sets the information number to 1 (step S29), and then returns to step S25. On the other hand, if it is determined in step S28 that the number of information numbers does not exceed the number of child screens, the process directly returns to step S25. Thereafter, the CPU 5 sequentially and cyclically switches the association display.

  If it is determined in step S26 that the switching key for switching the sub-screen to be displayed has not been operated, the CPU 5 determines whether or not the delete key has been operated (step S30), and determines that the delete key has been operated. Then, the CPU 5 deletes the child screen number of the child screen information D2 corresponding to the information number from the registration (step S31), resets the child screen number, and returns to step S22.

  On the other hand, if it is determined in step S30 that the delete key has not been operated, the CPU 5 determines whether or not the setting key has been operated (step S32). If it is determined here that the setting key has been operated, the CPU 5 performs sub-screen setting of the information number (step S33). Specifically, the CPU 5 executes a processing procedure similar to that in steps S5 to S17 described above, receives a change in size or the like for the child screen specified by the information number, and returns to step S25.

  On the other hand, if it is determined in step S32 that the setting key has not been operated, the CPU 5 determines whether or not the end key has been operated (step S34), and if it is determined that the end key has been operated, The processing procedure ends (step S35). On the other hand, if it is determined in step S34 that the end key has not been operated, the process returns to step S25.

  At the time of reproduction, when the CPU 5 performs the process of reproducing the file recorded in the small-screen composition mode, the display of the correspondence is displayed on the monitor image based on the small-screen information multiplexed and recorded according to the user's operation. indicate. At this time, the same processing procedure as that in FIG. 6 is executed to sequentially and sequentially switch the display target of the correspondence. Further, when the user instructs to display and display the sub screen, only the sub screen may be selectively displayed in the full screen, and the display target of the full screen display may be sequentially and cyclically switched according to the user's operation. When the child screen is displayed in full screen in this way, the parent screen may be displayed by the child screen.

(1-2) Operation of Embodiment As described above, in the digital camera 1 of this embodiment (FIG. 1), the solid-state imaging device 13 performs photoelectric conversion processing on the optical image generated by the lens 11 and images it. The resulting imaging signal is generated. The signal processing unit 16 processes the imaging signal to generate moving image data D1. In the digital camera 1 according to the present embodiment, the CPU 5 inputs the moving image data D1 through the frame buffer 6, performs processing such as data compression, and then records the data in the memory card 3. On the other hand, when receiving a reproduction instruction from the user, the CPU 5 sequentially reads the moving image data recorded on the memory card 3, decompresses the data, and displays it on the image display unit 7.

  In these processes, when receiving an instruction to record a moving image in the small-screen synthesis mode from the user, the CPU 5 cuts out the moving image data in the area instructed by the user from the moving-image data D1 obtained from the imaging system 2 and uses the small-screen image The moving image data for the sub-screen is generated, and the moving image data for the small screen is combined with the original moving image data D1 and compressed to be recorded on the memory card 3. Specifically, the moving picture data D1 is generated by multiplexing the small-screen information D2 having the small-screen information.

  Thereby, the digital camera 1 according to the present embodiment can record a plurality of moving images simultaneously with a simple configuration. That is, the digital camera 1 of the present embodiment can process moving image data obtained from one imaging system and record a plurality of moving images without providing imaging systems corresponding to the parent screen and the child screen, respectively. And the overall configuration can be simplified. In addition, since the plurality of moving images can be processed by a single compression process, the configuration can be further simplified.

  Further, during playback, the display target of full screen display can be switched between the main screen and the sub screen, and a plurality of moving images can be selectively displayed, so that convenience for the user can be improved.

  As described above, in the digital camera 1 of the present embodiment, the sub-screen information D2 indicating the correspondence between the sub-screen and the corresponding area of the parent screen is multiplexed with the moving image data D1, compressed, and recorded. Therefore, in the digital camera 1 according to the present embodiment, since the child screen synthesized based on the child screen information D2 can be selectively displayed on the whole screen or the like on the parent screen displayed based on the moving image data D1, the user can Convenience can be improved.

  Further, in the digital camera 1 of the present embodiment, a display indicating the correspondence between the child screen and the corresponding area of the parent screen is displayed in the monitor image at the time of recording in accordance with a user instruction. Therefore, in the digital camera 1 according to the present embodiment, the user can easily and reliably grasp the correspondence between the parent screen and the child screen, so that convenience can be improved.

  Furthermore, in the digital camera 1 of the present embodiment, the user can instruct additional setting of the sub screen, the position and size of the area set on the main screen, and the setting of the position and size of the sub screen, so that the convenience for the user is further increased. improves. In other words, since the user can set the position of the area of the parent screen to which the child screen is pasted, the position of the parent screen assigned to the child screen can be variously changed and the child screen can be displayed at a desired position of the parent screen. it can. In addition, the user can set the size of the display area set on the parent screen, the size of the child screen, the position where the child screen is pasted, the number of child screens to be pasted, and the like, further improving user convenience. be able to.

  Further, at the time of reproduction, a display showing the correspondence between the sub-screen and the corresponding area of the main screen is displayed based on the sub-screen information that is multiplexed and recorded on the moving image data, as in the case of recording. As a result, the digital camera 1 can easily and reliably grasp the correspondence between the parent screen and the child screen even during reproduction.

(2) Other Embodiments In the above-described embodiment, the case where the user accepts the setting of the area and the like set on the child screen has been described. However, the present invention is not limited to this, and the position and size of the child screen are not limited thereto. For example, the size may be automatically adjusted according to the position of the subject on the parent screen, the number of child screens, and the like. Further, the target subject may be tracked by applying, for example, a method for detecting a region of interest.

  In the above-described embodiment, the case where the data compression rate is reduced in the child screen has been described. However, the present invention is not limited to this, and data compression may be performed at a constant data compression rate. The present invention can be widely applied to the case of increasing and compressing.

  Further, in the above-described embodiment, the case where the user sets the area to be set in the child screen and processes the moving image has been described. However, the present invention is not limited to this, for example, using a motion tracking method. The movement of the area set on the parent screen may be tracked, and the position and / or size of the area set on the parent screen may be dynamically changed based on the tracking result. Various methods such as a method of statistically processing motion vectors and a method of tracking feature points can be applied to the motion tracking method.

  Further, in the above-described embodiment, the case where a part of the area is cut out from the parent screen and set as the child screen has been described. However, the present invention is not limited to this, and the entire area of the parent screen is assigned to the child screen. The sub-screen may change the magnification and display the area.

  In the above-described embodiment, the case where settings of various sub-screens are received by key operation has been described. However, the present invention is not limited to this, and various interfaces such as a touch panel interface can be widely applied. .

  Furthermore, in the above-described embodiment, the case where the small screen information is multiplexed and recorded in the moving image data has been described. However, the present invention is not limited to this, and the recording of the small screen information is omitted as necessary. May be. Specifically, if the display of the association between the child screen and the parent screen is not required during reproduction, the child screen information does not have to be filed.

  As described above, according to the present invention, the monitor image displays the display indicating the correspondence between the child screen and the corresponding area of the parent screen, so that the correspondence between the child screen and the parent screen can be easily and reliably grasped. Since it can be set, user convenience can be improved.

  Furthermore, according to the present invention, when moving image data is played back on the parent screen and the child screen, the correspondence between the child screen and the corresponding area is displayed based on the child screen information, thereby improving the convenience for the user. Can do.

  Furthermore, according to the present invention, since the image for the main screen and the sub-screen information indicating the correspondence between the sub-screen and the corresponding area of the main screen can be multiplexed into the moving image data and simultaneously compressed and recorded, Neither compression processing nor decompression processing increases. Therefore, it can be realized with a simple configuration.

  In the above-described embodiment, the case where the present invention is applied to a digital camera and the moving image data D1 input from the moving image data input unit by the imaging system is processed has been described. However, the present invention is not limited thereto. For example, the present invention can be widely applied to a case where moving image data D1 provided by a network or various recording media is recorded on a recording medium such as a hard disk. Further, it can be operated by a moving image processing program that causes a computer including a CPU and a memory to control a moving image processing apparatus to function as the above-described units. The moving image processing program can be distributed via a communication line, or can be distributed by writing on a recording medium such as a CD-ROM.

  The present invention is not limited to the above-described embodiments, and various combinations of the above-described embodiments are made and various modifications are made to the above-described embodiments without departing from the spirit of the invention. It can be in the form.

    The present invention can be applied to, for example, a digital camera.

DESCRIPTION OF SYMBOLS 1 ... Digital camera, 2 ... Imaging system, 3 ... Memory card, 5 ... CPU, 6 ... Frame buffer, 13 ... Solid-state image sensor, 16 ... Signal processing part

Claims (10)

A video data input unit for inputting input video data;
A sub-screen data generation unit that cuts out video data of at least a part of the area from the input video data and generates video data for a sub-screen;
An image composition unit for assigning the input moving image data and the moving image data for the child screen to the parent screen and the child screen, respectively, combining the input moving image data and the moving image data for the child screen, and generating combined moving image data; ,
A moving image processing apparatus comprising: a recording unit that compresses the synthesized moving image data and records the data on a recording medium.   The moving image processing apparatus according to claim 1, wherein the recording unit multiplexes and records sub-screen information indicating a correspondence relationship between the sub-screen and the at least some area corresponding to the synthesized moving image data. An image display unit for displaying the synthesized moving image data;
The moving image according to claim 1, wherein the image display unit displays a display indicating a correspondence relationship between the child screen and the at least a part of the region based on the synthesized moving image data. Processing equipment.   An instruction input unit that inputs at least one instruction of the position of the at least part area, the size of the at least part area, the position of the child screen, the size of the child screen, and the addition of the child screen; The moving image processing apparatus according to claim 1, claim 2, or claim 3.   A sub-screen setting unit configured to set a sub-screen based on the sub-screen information, and the sub-screen setting unit automatically sets the position of the sub-screen and the size of the sub-screen according to the number of sub-screens or the position of the subject. The moving image processing apparatus according to claim 2, which is determined automatically. A playback unit for playing back video data from a recording medium;
A display unit for displaying moving image data reproduced by the reproduction unit,
The video data is
Synthetic video generated by cutting out video data corresponding to at least a part of the area from input video data as sub-screen video data according to a user instruction, and synthesizing the sub-screen video data with the input video data Data,
The input video data and the sub screen information indicating a correspondence relationship between at least a part of the input video data corresponding to the sub screen video data are multiplexed and recorded,
The display unit
A moving image processing apparatus that displays a display showing a correspondence relationship between the image data for the small screen and the corresponding at least a part of the region based on the small screen information in accordance with a user instruction. A step of inputting input video data;
Cutting out moving image data of at least a part of the area from the input moving image data, and generating moving image data for a child screen;
The input video data and the generated sub-screen video data are assigned to a main screen and a sub-screen, respectively, and the input video data and the sub-screen video data are combined, Generating step;
And a step of compressing the generated synthesized moving image data and recording the data on a recording medium. Reproducing the moving image data from the recording medium;
Displaying the reproduced moving image data,
The video data is
Synthetic video generated by cutting out video data corresponding to at least a part of the area from input video data as sub-screen video data according to a user instruction, and synthesizing the sub-screen video data with the input video data Data,
The input video data and the sub screen information indicating a correspondence relationship between at least a part of the input video data corresponding to the sub screen video data are multiplexed and recorded,
The step of displaying the moving image data includes:
A moving image processing method including a step of displaying a display indicating a correspondence relationship between the sub-screen moving image data and the corresponding at least a part of the region based on the sub-screen information according to a user instruction. In the computer that controls the video processing device,
A function to input video data,
A function of cutting out moving image data of at least a part of the input moving image data and generating moving image data for a child screen;
The input video data and the generated sub-screen video data are assigned to a main screen and a sub-screen, respectively, and the input video data and the sub-screen video data are combined, The function to generate,
A moving image processing program that realizes a function of compressing the generated combined moving image data and recording the data on a recording medium. In the computer that controls the video processing device,
A function for reproducing moving image data from a recording medium;
A moving image processing program for realizing the function of displaying the reproduced moving image data,
The video data is
Synthetic video generated by cutting out video data corresponding to at least a part of the area from input video data as sub-screen video data according to a user instruction, and synthesizing the sub-screen video data with the input video data Data,
Sub screen information indicating a correspondence relationship between the input video data and at least a part of the input video data corresponding to the sub screen video data is multiplexed and recorded,
The function of displaying the video data is
A moving image processing program including a function of displaying a display showing a correspondence relationship between the moving image data for the small screen and the corresponding at least a part of the region based on the small screen information according to a user instruction.

Images