JP5322688B2 - Still image capturing device, moving image capturing device, and image capturing system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To associate information on imaging due to a still image capturing apparatus with a moving image being captured by a moving image capturing apparatus when cooperatively capturing images by means of the still image capturing apparatus and the moving image capturing apparatus. <P>SOLUTION: A moving image capturing and recording apparatus 2 and a still image capturing and recording apparatus 1 are configured communicable. During moving image capturing due to the moving image capturing and recording apparatus 2, the still image capturing and recording apparatus 1 transmits information on its own imaging, such as e.g., timing information of SW1, SW2 or a captured still image to the moving image capturing and recording apparatus 2. When the information on imaging is received from the still image capturing and recording apparatus 1 during moving image capturing, the moving image capturing and recording apparatus 2 associates the information on imaging with a moving image frame. <P>COPYRIGHT: (C)2010,JPO&amp;INPIT

Description

  The present invention relates to a still image capturing device, a moving image capturing device, a control method therefor, an image capturing system, and a program that are suitable for use in cooperative shooting.

  As a conventional technique, there is a product that captures a still image while recording a moving image in an image recording apparatus. This function realizes moving image shooting and still image shooting with a single imaging / recording apparatus, and can leave an image as a still image while shooting a subject as a moving image.

  Recently, there are scenes where mothers shoot videos and fathers shoot photos at a children's athletic meet, etc., and fathers move around to shoot photos with a video camera fixed on a tripod. The subject to be photographed is the same, but the composition of the subject changes because parents shoot independently with their respective photographing equipment.

  Since the above-described video camera can shoot moving images and still images, the photographer can be either a father or a mother. However, since both a moving image and a still image are shot with the same equipment, the composition of the subject is always constant, and the still image has the impression that only one frame of the moving image is cut out.

  As this type of technology, for example, Patent Document 1 discloses an electronic camera having a simultaneous photographing function for simultaneously performing a plurality of photographings and recording a plurality of photographed images obtained by the respective photographings. In this electronic camera, when recording a still image taken during moving image shooting and a moving image being shot, both images are associated with link information for specifying the other image and information indicating the shooting timing of the still image. And remember.

  At present, it is not possible to easily associate a picture taken with a separate imaging and recording device with a moving image file. Therefore, if you look at the athletic meet photo taken after you go home and find a photo that took a decisive moment, if you want to see the moving image at that time, play the moving image from the beginning and fast forward It is necessary to search while doing.

  Furthermore, in some conventional video cameras, a captured still image is linked as an index indicating a moving image file, but in many cases it simply indicates a representative image of a moving image. In that case, if it is not linked to the part to be viewed, it is necessary to search from the beginning and search while fast-forwarding in order to view the target moving image.

  Even if there is something that can be reproduced from the portion related to the still image, it becomes a monotonous index in terms of the composition of the subject.

JP 2004-304425 A

  As described above, there is no mechanism for associating images captured by separate moving image capturing / recording apparatuses and still image capturing / recording apparatuses.

  Currently, a moving image capturing / recording apparatus such as a video camera and a still image capturing / recording apparatus such as a digital camera are used in many homes. These are easily used to record children's growth and travel. Some digital cameras automatically send captured images to someone else's camera if each camera is set to recognize first. The use of this product is supposed to save you the trouble of sending the desired files at a later date by automatically sending photos taken by friends at the party.

  For example, consider a situation in which a fixed or mother records a child's figure with a moving image capturing / recording device such as a video and a father takes a decisive moment with a still image capturing / recording device such as a digital camera. Since there is no link between photos and moving images, the current situation is appreciated. Therefore, even if you want to see a moving image at that time while looking at a photograph you have taken, you need to search while playing back the image of that scene. In addition, when viewing a moving image and a still image with a single image capturing and recording apparatus, the angle of view of the subject is fixed, so that the photograph becomes a single one. The ability to take photos while changing the angle of view has the joy of taking photos. At the same time, it is possible to search and play back images recorded with moving images using images from other imaging and recording devices. Is possible.

  The present invention has been made in view of the above situation, and an object of the present invention is to make it possible to easily associate information related to shooting with a still image capturing device and a moving image capturing device.

The still image capturing apparatus according to the present invention is a still image capturing apparatus capable of communicating with a moving image capturing apparatus. The image capturing unit captures an image of a subject and obtains image data. And a transmission unit that transmits data to the moving image capturing apparatus, and when a still image capturing instruction is input by the first operating unit during moving image capturing by the moving image capturing apparatus, The imaging unit captures a still image, transmits first timing information indicating the timing at which the imaging instruction is input, to the moving image imaging apparatus by the transmission unit, and the transmission unit inputs the imaging instruction. before the timing, the second timing information indicating a timing related to imaging by the imaging means is transmitted to the moving image pickup device, said first and second timing information, before Characterized in that associated with the imaged moving image by the moving image pickup device.

  ADVANTAGE OF THE INVENTION According to this invention, the moving image imaging | photography with a moving image imaging device WHEREIN: The information regarding imaging | photography with a still image imaging device can be linked | related automatically with the moving image currently image | photographed with a moving image imaging device.

It is a figure which shows the structure of the imaging recording system which concerns on embodiment to which this invention is applied. It is an external view of a digital single-lens reflex camera capable of recording moving images and still images. It is sectional drawing of the digital single-lens reflex camera which can record a moving image recording and a still image. It is a block diagram which shows the circuit structure of the digital single-lens reflex camera which can record a moving image recording and a still image. It is a figure for demonstrating the general purpose file format in the case of recording moving image data. It is a flowchart which shows the outline | summary of the process which transmits the information regarding imaging | photography with a still image imaging recording device to a moving image imaging recording device, and associates with a moving image frame. It is a figure which shows the example of a display of the imaging | photography information from a moving image file group and a still image recording device. It is a flowchart which shows the process which transmits the imaging | photography time information to a moving image imaging recording device as imaging | photography information in a still image imaging recording device, and associates with a moving image frame. It is a figure which shows the example of an imaging | photography information transmission frame. It is a figure which shows the data structural example of an sIndex block. It is a flowchart which shows the process which transmits a still image to a moving image imaging recording device as imaging | photography information in a still image imaging recording device, and associates with a moving image frame. It is a figure for demonstrating a mode that imaging | photography information is linked | related with the moving image frame. It is a figure which shows the example of a display of a moving image file group and still image shooting information. It is a flowchart which shows the reproduction | regeneration processing in a moving image imaging recording device. It is a figure for demonstrating the outline | summary of selection of a still image, and reproduction | regeneration of the moving image frame relevant to it. It is a flowchart which shows the reproduction | regeneration processing in a moving image imaging recording device.

Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.
(First embodiment)
FIG. 1 is a diagram illustrating a configuration of an imaging system in which a still image capturing / recording apparatus (still image capturing apparatus) 1 and a moving image capturing / recording apparatus (moving image capturing apparatus) 2 according to an embodiment to which the present invention is applied perform cooperative shooting. is there. In the present embodiment, the moving image capturing / recording apparatus 2 is connected to a tripod and used for fixed-point shooting. Therefore, the moving image recording / recording apparatus 2 is installed with a composition of a subject to be photographed. Further, in the still image capturing / recording apparatus 1, an example is that the photographer operates while moving, and shoots with a desired composition while changing the angle of view with respect to the subject that is shooting the moving image. Although FIG. 1 shows a configuration in which the still image capturing / recording apparatus 1 transmits information to the moving image capturing / recording apparatus 2 by wireless communication, a configuration in which information transmission is performed by wired communication may be employed.

  FIG. 2 is an external view of the digital single lens reflex camera capable of recording moving images and still images (a rear perspective view seen from the photographer side), and FIG. 3 is a vertical sectional view of the digital single lens reflex camera. The still image capturing / recording apparatus 1 shown in FIG. 1 may be a digital single-lens reflex camera as described below or a video capable of capturing a still image. 1 may be a video or a digital single-lens reflex camera as described below.

  As shown in FIG. 2, an eyepiece window 111 for finder observation, an AE (automatic exposure) lock button 112, and an AF (automatic focus adjustment) distance measuring point selection button 113 are provided on the upper part of the camera body 100. .

  In addition, a release button 114 for performing a shooting operation (instructions related to shooting) is provided, and two-stage input of a half-pressed state (SW1) and a fully-pressed state (SW2) is possible.

  Furthermore, an electronic dial 411 for selecting various setting values, a photographing mode selection dial 117, and an external display device (LCD) 409 are provided. The shooting mode selection dial 117 can change the shooting mode by turning the dial. Examples of the shooting mode include a shutter speed priority mode, a program shooting mode, and a night view mode. The external display device 409 is composed of a liquid crystal display device, and displays shooting conditions such as shutter speed, aperture, development mode, and other information.

  On the back surface of the camera body 100, an LCD monitor device 417 for displaying a captured image, an image subjected to correction processing, various setting screens such as a menu screen, and the like is provided. Since the LCD monitor device 417 of the present embodiment is a transmission type, an image cannot be visually recognized only by driving the LCD monitor device 417, and a backlight illumination device 416 is necessarily provided on the back side as shown in FIG. It is. As described above, the LCD monitor device 417 and the backlight illumination device 416 constitute an image display device.

  Further, a monitor switch 121, a cross arrangement switch 116, and a menu button 124 for turning on / off the LCD monitor device 417 are provided. The cross placement switch 116 has four buttons arranged vertically and horizontally, and a SET button arranged in the center, and the user instructs the camera to select and execute menu items displayed on the LCD monitor device 417. Used for. The menu button 124 is a button for causing the LCD monitor device 417 to display a menu screen for performing various camera settings. For example, when changing the setting of the camera displayed on the LCD monitor device 417, after pressing this menu button 124, the upper, lower, left, and right buttons of the cross placement switch 116 are operated, and the SET mode is set with the desired mode selected. Set by pressing the button.

  As shown in FIG. 3, a photographic lens 200 that is an imaging optical system is detachably attached to the camera body 100 via a lens mount 202. Reference numeral 201 denotes a photographing optical axis.

  In the camera body 100, a quick return mirror 203 is disposed in the photographing optical path. The quick return mirror 203 has a position for guiding the subject light from the photographing lens 200 to the finder optical system (a position shown in FIG. 3, an oblique position) and a position for retreating outside the photographing optical path (referred to as a retracted position). It can be moved between.

  The subject light guided from the quick return mirror 203 to the viewfinder optical system forms an image on the focus plate 204. The condenser lens 205 is provided to improve the visibility of the finder. The pentagonal roof prism 206 guides subject light that has passed through the focusing plate 204 and the condenser lens 205 to an eyepiece lens 208 and a photometric sensor 207 for finder observation.

  Behind the quick return mirror 203, a shutter composed of a rear curtain 209 and a front curtain 210 is arranged. By opening the rear curtain 209 and the front curtain 210, an image sensor 418, which is a solid-state image sensor that photoelectrically converts a subject image, is exposed for a necessary time. A captured image converted into an electrical signal for each pixel by the image sensor 418 is processed by an A / D converter 423, an image processing circuit 425, and the like, which will be described later, and recorded on a recording medium 419 as image data. An optical low-pass filter 418a is disposed in front of the image sensor 418, and adjusts the spatial frequency of the subject image formed on the image sensor 418.

  The image sensor 418 is held on the printed circuit board 211. A display board 215 that is another printed board is disposed behind the printed board 211. An LCD monitor device 417 and a backlight illumination device 416 are disposed on the opposite surface of the display substrate 215.

  A recording medium 419 for recording image data and a battery (portable power supply) 217 are detachably attached to the camera body 100.

  FIG. 4 is a block diagram showing a circuit configuration of the digital single-lens reflex camera shown in FIGS. The microcomputer 402 controls the operation of the entire camera, including processing of image data output from the image sensor 418 and display control of the LCD monitor device 417. The nonvolatile memory 428 stores a program executed by the microcomputer 402 and the like.

  The switch (SW1) 405 is turned on when the release button 114 is half-pressed, and when the switch (SW1) 405 is turned on, a shooting preparation state is set. The switch (SW2) 406 is turned on when the release button 114 is pressed to the end (fully pressed state), and the photographing operation is started when the switch (SW2) 406 is turned on.

  The lens control circuit 407 performs communication with the photographic lens 200, drive control of the photographic lens 200 during AF, and drive control of the diaphragm blades.

  An external display control circuit 408 controls an external display device (OLC) 409 and a display device (not shown) in the finder. The switch sense circuit 410 transmits signals of a number of switches including the electronic dial 411 to the microcomputer 402.

  The strobe light emission dimming control circuit 412 is grounded via the X contact 412a and controls the external strobe. The distance measuring circuit 413 detects the defocus amount with respect to the subject for AF. The photometry circuit 414 measures the luminance of the subject.

  A shutter control circuit 415 controls the shutter and performs appropriate exposure on the image sensor 418. The LCD monitor device 417 and the backlight illumination device 416 constitute an image display device. The recording medium 419 is, for example, a hard disk drive or a semiconductor memory card that can be attached to and detached from the camera body.

  Further, the microcomputer 402 includes an A / D converter 423, an image buffer memory 424, an image processing circuit 425 including a DSP, and a pixel defect position memory storing that a predetermined pixel itself in the image sensor 418 is defective. 426 is connected. Also connected is a dust position memory 427 that stores the pixel position in the image sensor 418 causing an image defect due to dust.

  The camera can communicate with an external device via the external interface 404. Note that the external interface 404 may perform wireless communication or may perform wired communication.

  Next, a general-purpose file format for recording moving image data will be described with reference to FIG. An MP4 (MPEG-4) file format is used as a general-purpose format for recording MPEG (MPEG-2 or MPEG-4 format) image data shot by the moving image pickup recording apparatus 2. By recording as an MP4 file, compatibility is guaranteed such that it can be played back by other digital devices.

  As shown in FIG. 5A, the MP4 file basically includes an mdat box containing encoded stream image data and a moov box containing information related to the stream image data. Is done.

  The mdat box is further configured with a plurality of chunks (chunk cN) as shown in FIG. Each chunk is composed of a plurality of samples (sample sM) as shown in FIG. For example, as shown in FIG. 5 (e), each sample has a sign of I0, B-2, B-1, P3,... For sample s1, sample s2, sample s3, sample s4,. Corresponding to the converted MPEG image data. Here, I0, I1, I2,..., In are frame image data subjected to intra coding (intraframe coding). B 0, B 1, B 2,..., Bn are frame image data that is encoded (interframe encoding) with reference from both directions. P 0, P 1, P 2,..., Pn are frame image data that are encoded (inter-frame encoded) with reference from one direction (forward direction). Both are variable length code data.

  As shown in FIG. 5C, the moov box is composed of an mvhd box composed of header information in which the creation date and time are recorded, and a trak box into which information relating to stream image data stored in the mdat box is entered. Information stored in the trak box includes an sInex box shown in FIG. 5 (i), a stco box shown in FIG. 5 (h), a stsc box shown in FIG. 5 (g), and an stsz box shown in FIG. 5 (f). is there. In the stco box, as shown in FIG. 5 (h), information on the offset value to each chunk of the mdat box is stored. Information on the number of samples in each chunk is stored in the stsc box as shown in FIG. Information on the size of each sample is stored in the stsz box as shown in FIG.

  Hereinafter, cooperative shooting using the still image capturing / recording apparatus 1 and the moving image capturing / recording apparatus 2 shown in FIG. 1 will be described. FIG. 6 illustrates a process of transmitting information related to shooting with the still image capturing / recording apparatus 1 (hereinafter referred to as shooting information) to the moving image capturing / recording apparatus 2 and associating it with the moving image frame during the moving image capturing / recording apparatus 2. It is a flowchart which shows the outline | summary. The sequence of the still image capturing / recording apparatus 1 corresponds to FIG. 6A, and the sequence of the moving image capturing / recording apparatus 2 corresponds to FIG. 6B.

  First, the still image capturing / recording device 1 and the moving image capturing / recording device 2 shift to a predetermined mode in response to a user operation, perform pairing, and start cooperative shooting (steps S301 and S304). This pairing function may be performed by three or more devices. Pairing is performed for mutual address recognition necessary for communication and, in the case of wireless, exchanging shared information of the wireless network to create a communicable state and for time synchronization. In the case of a wireless LAN, if a service set ID, encryption / non-encryption, and an encryption key are necessary, key exchange is performed, and an IP address is resolved when wireless is established. In this case, an IP address is assigned by auto IP, and the destination address is stored in the imaging / recording apparatus. This portion is automatically omitted if the devices have IP addresses set in advance. As a means for automatically omitting, a Ping command is transmitted from either side, and if the packet loss is 100%, the process shifts to the auto IP function. In the case of wireless communication, a wireless LAN or Bluetooth may be used regardless of the wireless protocol. Of course, wired communication may be used. In the following description, the same processing is performed to establish communication.

  When a shooting operation is performed in the still image capturing / recording apparatus 1 (step S302), the still image capturing / recording apparatus 1 transmits information obtained by an operation on the own apparatus to the moving image capturing / recording apparatus 2 as shooting information (step S303). This process is an example of the process performed by the transmission unit.

  In the moving image recording / recording apparatus 2, moving image shooting is started by pressing an imaging switch or a recording button (step S305). When shooting information is received from the still image capturing / recording apparatus 1 during the moving image shooting (step S306), reception information processing is performed for associating the shooting information with the related moving image frame being recorded (step S307). In association processing with a moving image frame, shooting information is recorded in the sIndex box in the trak box shown in FIG. Until moving image shooting is completed (steps S308 and S309), reception information processing is repeatedly performed on the shooting information transmitted from the still image shooting and recording apparatus 1 (steps S306 and S307). This process is an example of a process performed by the reception unit and the association unit.

  As shown in FIG. 7, the image file that is cooperatively captured by the still image capturing / recording device 1 and the moving image capturing / recording device 2 is displayed as a moving image index and photographing information associated with the moving image index. This display may be displayed on the LCD monitor 417 of the moving image capturing / recording apparatus 2, or may be displayed on a PC monitor or the like using a PC application or the like after copying the data to the PC or the like. In FIG. 7, the moving image index group displays the first frame of the recorded moving image file, and displays what is recorded in the moving image file. The fact that a plurality of moving image files are recorded is shown as a horizontal row on the playback screen. The displayed image suggests the contents of the moving image file. As described above, this image may not be the first frame, but may be a predetermined representative frame image in the moving image file. Further, the vertical column is shooting information from the still image capturing / recording apparatus 1 that has performed the cooperative shooting with the moving image capturing / recording apparatus 2 that records the moving image intersecting the horizontal column. In the example of FIG. 7, shooting information is associated with two of the moving image indexes 701 to 709, that is, moving image 1 (705) and moving image 2 (702).

  FIG. 8 is a flowchart showing processing for transmitting shooting time information as shooting information in the still image shooting / recording apparatus 1, specifically, timing information of SW1 and SW2 to the moving image shooting / recording apparatus 2 and associating it with a movie frame. . The sequence of the still image capturing / recording apparatus 1 corresponds to FIG. 8A, and the sequence of the moving image capturing / recording apparatus 2 corresponds to FIG. 8B.

  First, the still image capturing / recording apparatus 1 and the moving image capturing / recording apparatus 2 are paired to start cooperative shooting (steps S401 and S409). At this point, in the case of a wireless LAN, the SSID is determined, the encryption method is determined, the encryption key is shared, and the IP address is resolved.

  Further, at the time of pairing, time synchronization is also performed based on time information of the moving image capturing / recording device 2 or the still image capturing / recording device 1. The base time may be GPS data acquired by either device. Further, as time synchronization, a time difference received by one device may be used as a correction value. This difference time may be based on a time of about 10 seconds, for example, and when the reference time is exceeded, a time setting warning may be issued to prompt the user to reset the time information of the imaging / recording apparatus.

  In this example, time information is sent from the still image capturing / recording apparatus 1 to the moving image capturing / recording apparatus 2. If the difference value of the device time is larger than the reference value, a warning is displayed and the user is prompted to set the time again. When the difference time is within the reference value, the moving image capturing / recording apparatus 2 performs the process in consideration of the difference time when using the received time of the shooting information of the other device. When performing cooperative shooting with a plurality of still image capturing / recording apparatuses 1, the moving image capturing / recording apparatus 2 stores individual information of the still image capturing / recording apparatus 1 and time difference information. As the individual information, an address used for communication may be used, or a serial number may be acquired at the time of pairing and stored in association with the serial number. However, in storing time difference information other than the communication address, it is necessary to add device individual information such as a serial number to the photographing information.

  In the moving image capturing / recording apparatus 2, moving image capturing is started by pressing an image capturing switch or a recording button (step S410). When moving image shooting is started, the moving image capturing and recording apparatus 2 continues moving image shooting and recording until a communication reception interrupt is received (steps S410 to S416).

  In the still image capturing / recording apparatus 1, the photographer determines whether or not SW1 is turned on by half-pressing the release button 114 for photometric distance measurement (step S402). When SW1 is turned on (first-stage input), as the shooting information, the content that SW1 is turned on and the time information at the time of turning on (hereinafter, this shooting information is also referred to as timing information) are captured. The data is transmitted to the recording device 2 (step S403).

  When the moving image capturing / recording apparatus 2 receives timing information from the still image capturing / recording apparatus 1 during moving image capturing (step S411), the timing information is added as a mark and associated with the moving image frame related to the time when SW1 is turned on. (Step S412). Since SW1 in the normal still image recording / recording apparatus 1 is used for AF and photometric distance measurement, it may be turned on many times. In this case, the still image capturing / recording apparatus 1 transmits timing information to the moving image capturing / recording apparatus 2 every time SW1 is turned on (repeats steps S402 to S404). Then, the moving image capturing / recording apparatus 2 marks the moving image frame every time the timing information is received (repeats steps S411 to S413).

  Next, in the still image capturing / recording apparatus 1, shooting is performed when the release button 114 is fully pressed and the SW2 is turned on. When SW2 is turned on (second-stage input) (step S405), the contents that SW2 is turned on and the time information at the time when it is turned on are transmitted to the moving image capturing / recording apparatus 2 (step S405). S406). The timing information includes information for identifying whether the timing information is transmitted when SW1 is turned on or transmitted when SW2 is turned on. Further, the still image capturing / recording apparatus 1 starts a photographing operation and performs a photographing development process (step S407).

  In the moving image recording / recording device 2, when timing information is received from the still image recording / recording device 1 (step S413), the timing information is added as a mark and associated with the moving image frame related to the time when SW2 is turned on (step S414). . The still image capturing / recording apparatus 1 transmits timing information to the moving image capturing / recording apparatus 2 until SW2 is turned off (repeats steps S406 to S408). The moving image recording / recording apparatus 2 marks a moving image frame every time timing information is received (steps S414 to S415 are repeated).

  Until moving image shooting is completed (steps S416 and S417), reception information processing is repeatedly performed on the timing information transmitted from the still image capturing and recording device 1 (steps S411 to S415).

  For example, as shown in FIG. 9, transmission of shooting information by operation of the still image capturing / recording apparatus 1 is performed using a UDP portion. UDP means a user data part, and consists of an operation identifier, a data length, and data. The contents to be put in the UDP part can be freely determined, and in this example, the shooting information is SW1 and SW2 information. Codes corresponding to SW1 and SW2 are stored in the operation identifier. In the Data length, 4 is put in the meaning of 4 bytes. Data stores the timing information of SW1 and SW2 (time information at the time of turning on). Note that the configuration of UDP and the communication frame for communicating with UDP do not have to be this configuration.

  The moving image capturing / recording apparatus 2 associates the received shooting information with the image frame. The association with the moving image frame is configured as MP-4 with the sIndex box (i) as a data area newly in the trak box of FIG. Shooting information from the still image capturing / recording apparatus 1 is marked in the sIndex box. Here, “mark” means storing data.

  FIG. 10 shows an example of the data structure of the sIndex box. The frame identifier identifies a moving image frame, and the operation identifier is the UDP operation identifier. Data 1, 2,... Vary depending on the configuration. In this example, time information is stored in data 1. Data in the sIndex box is written corresponding to each moving image frame as shown in FIG.

  The effects of the imaging / recording system according to the embodiment described above will be described by way of example. As described above, the moving image capturing / recording apparatus 2 is connected to a tripod and used for fixed-point shooting, and the still image capturing / recording apparatus 1 is operated while the photographer moves to perform image capturing on a subject that is capturing a movie. Suppose you want to shoot with your favorite composition while changing the corners. Although the subject is the same, there is a difference that the recorded image file is a moving image file and a still image file, and a difference that the angle of view is different in each imaging and recording apparatus.

  The two types of image files captured in this way are viewed while playing a moving image on a playback device when viewing a moving image file, and are printed as a photo when viewing a still image file. Or, it is displayed as a digital image on the display device for viewing.

  In this case, when there is no relationship between the moving image and the still image as in the conventional case, the moving image recorded in relation to the photograph is viewed while viewing the still image at the decisive moment or the moment of interest. If you want to play it, you have to play and search the video from the beginning. In addition, since still images and moving images are not associated with each other, they cannot be organized as index images representing the contents of moving images and playback points. Furthermore, since still images and moving images are managed separately, if they are not consciously organized, they will not be associated at all.

  As described above, when an image is intentionally captured using a plurality of image capturing / recording apparatuses, if the image capturing / recording apparatuses can collaborately capture images, it is very effective for subsequent search and display.

  When shooting with the two imaging and recording apparatuses 1 and 2, a procedure for recognizing each other in advance is performed, so that each other can recognize each other as a communication partner. When the still image capturing / recording apparatus 1 is operated, shooting information such as an image is transmitted to the moving image capturing / recording apparatus 2 in time (timing) as in an embodiment described later. The moving image capturing / recording device 2 associates the shooting information received from the still image capturing / recording device 1 by adding it directly to the moving image frame or managing it separately.

  A plurality of associated shooting information is displayed as index information when a moving image is reproduced. The index information at this time becomes shooting information of the still image pickup recording device 1 that has performed cooperative shooting with the moving image pickup recording device 2. This is used to indicate what kind of video content is in a video file, which is common in video captured with a video recording device that can capture and record video and still images. It is different from the still image index. In other words, for the purpose of representing the relationship between images taken with each other's photographic equipment, and making it easy to search and play back the content of moving images that were taken at the timing of taking a still image displayed as an index It can be used.

(Second Embodiment)
In the second embodiment, an example will be described in which a still image is transmitted to the moving image capturing / recording device 2 as shooting information in the still image capturing / recording device 1 and associated with a moving image frame. FIG. 11 is a flowchart showing a process of transmitting a still image as shooting information in the still image capturing / recording apparatus 1 to the moving image capturing / recording apparatus 2 and associating it with a moving image frame. The sequence of the still image capturing / recording apparatus 1 corresponds to FIG. 11 (a), and the sequence of the moving image capturing / recording apparatus 2 corresponds to FIG. 11 (b).

  First, the still image capturing / recording apparatus 1 and the moving image capturing / recording apparatus 2 are paired to start cooperative shooting (steps S101 and S110). At this point, in the case of a wireless LAN, the SSID is determined, the encryption method is determined, the encryption key is shared, and the IP address is resolved. Further, as described in the first embodiment, time synchronization is also performed.

  In the moving image recording / recording apparatus 2, moving image shooting is started by pressing an imaging switch or a recording button (step S111). When moving image shooting is started, the moving image capturing / recording device 2 continues moving image shooting and recording until a communication reception interruption is received (steps S111 to S118).

  In the still image capturing / recording apparatus 1, the photographer determines whether or not SW1 is turned on by half-pressing the release button 114 for photometric distance measurement (step S102). When SW1 is turned on, the contents that SW1 is turned on and the time information at the time of turning on are transmitted to the moving image capturing / recording apparatus 2 as shooting information (step S103).

  When the moving image capturing / recording apparatus 2 receives timing information from the still image capturing / recording apparatus 1 during moving image capturing (step S112), the timing information is added as a mark to a moving image frame related to the time when SW1 is turned on (step S112). S113). Since SW1 in the normal still image recording / recording apparatus 1 is used for AF and photometric distance measurement, it may be turned on many times. In that case, the still image capturing / recording apparatus 1 transmits timing information to the moving image capturing / recording apparatus 2 every time SW1 is turned on (repeats steps S102 to S104). The moving image recording / recording apparatus 2 marks the moving image frame every time the timing information is received (repeats steps S112 to S114).

  Next, in the still image capturing / recording apparatus 1, when the release button 114 is fully pressed and SW2 is turned on (step S105), a photographing operation is started and photographing and developing processing is performed (steps S106 and S107). Thereafter, the photographed still image is transmitted to the moving image capturing / recording apparatus 2 as photographing information (step S108). The photographed still image has the same meaning as the timing information of SW2, and includes the photographing time (still image file generation time).

  When receiving the still image from the still image capturing / recording device 1 (step S114), the moving image recording / recording device 2 extracts the generation time of the received still image file, and based on that time, extracts the still image file into the associated movie frame. Are associated (step S116). The still image capturing / recording apparatus 1 transmits the captured still image to the moving image capturing / recording apparatus 2 until SW2 is turned off (repeats steps S106 to S109). In addition, each time the moving image capturing / recording apparatus 2 receives a still image, the moving image recording / recording device 2 associates the still image with the moving image frame (repeats steps S115 to S117).

  Until moving image shooting is completed (steps S118 and S119), reception information processing is repeatedly performed on shooting information and still images transmitted from the still image capturing and recording device 1 (steps S112 to S117).

  FIG. 12 shows that the timing of SW1 in the still image capturing / recording apparatus 1 is transmitted to the moving image capturing / recording apparatus 2, and the moving image capturing / recording apparatus 2 associates the timing with the moving image frame A1. In addition, the still image A ′ captured by turning on SW2 in the still image capturing / recording apparatus 1 is transmitted to the moving image capturing / recording apparatus 2, and the moving image capturing / recording apparatus 2 indicates that the still image A ′ is associated with the motion image frame A4.

  For example, as shown in FIG. 9, transmission of shooting information by operation of the still image capturing / recording apparatus 1 is performed using a UDP portion. UDP means a user data part, and consists of an operation identifier, a data length, and data. The contents to be put in the UDP part can be freely determined. In this example, the shooting information is SW1 and shooting image information. In the operation identifier, SW1, the shooting time information, and a code indicating a shot image file name are stored. The Data length is set to 4 in the sense of 4 bytes when the SW1 information is notified. In Data, the timing information when SW1 is turned on is stored. In the case of a captured image file name, 16 is added to the Data length in the sense that there is a Data length of 16 bytes as the length of the shooting time information 4 bytes and the file name 12 bytes. Data includes a shooting time and an image file name. In this example, the time information is 4 bytes and the file name is 12 bytes, but the data length may naturally vary depending on the system. Note that the configuration of UDP and the communication frame for communicating with UDP do not have to be this configuration. The data structure of UDP is a combination of shooting time and file name in UDP, shooting time identifier, shooting time Data length and shooting time Data, file name identifier, file name data length and file name data. A nested structure may be used. In addition, the data structure may be created with different code systems.

  The moving image capturing / recording apparatus 2 associates the received shooting information with the image frame. The association with the moving image frame is configured as MP-4 with the sIndex box (i) as a data area newly in the trak box of FIG. Shooting information from the still image capturing / recording apparatus 1 is marked in the sIndex box. Here, “mark” means storing data.

  FIG. 10 shows an example of the data structure of the sIndex box. The frame identifier identifies a moving image frame, and the operation identifier is the UDP operation identifier. Data 1, 2,... Vary depending on the configuration. In this example, in the case of a shooting file name, shooting time information is stored in data 1 and a file name is stored in data 2. Even if the captured still image is transmitted at a different timing, it can be associated with the previously transmitted shooting time information by the file name. Data in the sIndex box is written corresponding to each moving image frame as shown in FIG.

(Third embodiment)
As a third embodiment, a case where an image is reproduced by the moving image capturing / recording apparatus 2 will be described. Information relating to the image file captured on the display unit of the moving image capturing / recording device 2 may be displayed, or the moving image capturing / recording device 2 may be connected to an external display device such as a television and information regarding the image file captured on the external display device may be displayed. It may be output and displayed.

  In the case of a moving image file, what is recorded in the file is often displayed as an image index. This image index is displayed using, for example, the first moving image frame of the moving image file shot by the moving image capturing / recording apparatus 2. In this case, information from the still image capturing / recording apparatus 1 that has been subjected to cooperative shooting is displayed as a related index of the moving image.

  FIG. 13 is a conceptual diagram displaying a moving image list and a still image group as its moving image index in the case of reproduction in the present embodiment. In FIG. 13, the moving image index group is an image diagram in which the first image of a recorded moving image is displayed and what is recorded in the moving image file. The fact that a plurality of moving image files are recorded is shown as a horizontal row on the playback screen. The displayed image suggests the contents of the moving image file. As described above, this image does not have to be the head image, and may be a representative frame image in the moving image file. The vertical column is a group of still images of the still image capturing / recording apparatus 1 that has been cooperatively photographed with the moving image capturing / recording apparatus 2 that records a moving image intersecting the horizontal column. In FIG. 13, a still image corresponding to the moving image 2 when the moving image capturing / recording device 2 and the still image capturing / recording device 1 perform cooperative shooting is displayed.

  In FIG. 13, for example, a moving image 1 is displayed as a still image in which still images 1 to 5 are associated. The still image 1 to the still image 5 are transmitted as shooting information from the still image capturing / recording apparatus 1 when coordinated shooting is performed, and indicate that the moving image capturing / recording apparatus 2 is associated with the moving image frame.

  FIG. 14 shows the flow of moving image playback. When an arbitrary still image displayed as an index is selected and designated (step S201), a reproduction request related to the selected still image is generated (step S202). The moving image frame related to the specified still image is searched from the information in the sIndex box of the trak box in FIG. 5, and the moving image frame related thereto is reproduced (step S203).

  FIG. 15 shows an overview of still image selection and associated video frame playback. C0, C1,..., C5,. The still image Index C indicates that it is associated with the moving image frame C4. Consider a case where still image Index C is selected in order to reproduce a moving image related to an image of still image Index C. In this case, the C4 frame that is the associated moving image frame is searched from the information in the sIndex box, and playback is started from C4 (playback as C4, C5,...) (See FIG. 15B).

(Fourth embodiment)
In the third embodiment, an example has been described in which, when a displayed still image is selected, a moving image frame associated with the still image is searched and reproduction is started. In the fourth embodiment, when a certain still image is selected, it is not reproduced from a moving image frame corresponding to the still image. An example will be described.

  The shooting information of the cooperative shooting includes SW1 and SW2 timing information. The timing at which the photographer of the still image was aiming for shooting is positioned at the timing of SW1.

  FIG. 16 shows the flow of moving image playback. When an arbitrary still image is selected (step S204), a reproduction request related thereto is generated (step S205). In the present embodiment, playback is started not from a moving image frame related to the selected still image but from a moving image frame marked with SW1 timing information in a previous moving image frame (steps S206 and S207). .

  In FIG. 12, the moving image frame at the timing when SW1 is turned on is A1, and the moving image frame associated with the exposure and shooting of the still image is A4. The still image A ′ becomes a still image selected for reproduction, and the moving image frame at the reproduction point is A4. When the still image A ′ is selected, an operation identifier in the sIndex box of the trak box of the moving image file structure and data 1 and data 2 that match information related to the still image A ′ are searched. At this time, information regarding SW1 of the sIndex box is also held. The stored contents are updated when new information in the sIndex box containing the SW1 information is found. When the sIndex information corresponding to A ′ is found, the corresponding moving image frame is determined from the frame identifier in the sIndex box containing the SW1 information held at that time, and is reproduced from that frame.

FIG. 15 shows an overview of still image selection and associated video frame playback. When the still image Index C is selected for reproduction, the reproduction is started from the moving image frame C1 associated with the timing of SW1 (see FIG. 15C). As described above, by reproducing the image from before the still image shooting timing, it is possible to easily see the situation of the moving image scene immediately before the photo shooting.
<Other embodiments>
Each of the above embodiments is merely an example of an embodiment to which the present invention can be applied, and the embodiments can be combined as appropriate.

  The object of the present invention can also be achieved by supplying a storage medium storing software program codes for realizing the functions of the above-described embodiments to a system or apparatus. In this case, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium.

  In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the program code itself and the storage medium storing the program code constitute the present invention.

  As a storage medium for supplying the program code, for example, a flexible disk, a hard disk, an optical disk, a magneto-optical disk, a CD-ROM, a CD-R, a magnetic tape, a nonvolatile memory card, a ROM, or the like can be used.

  Further, the functions of the above-described embodiments are not limited to being realized by executing the program code read by the computer. For example, an OS (basic system or operating system) running on the computer performs part or all of the actual processing based on an instruction of the program code, and the functions of the above-described embodiments are realized by the processing. May be.

  Further, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. In this case, after being written in the memory, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the function of the above-described embodiment is performed by the processing. Is realized.

DESCRIPTION OF SYMBOLS 1 Still image imaging and recording apparatus 2 Moving image imaging and recording apparatus 100 Camera main body 114 Release button 402 Microcomputer 404 External interface 405 Switch (SW1)
406 Switch (SW2)
407 Lens control circuit 408 External display control circuit 409 External display device 410 Switch sense circuit 411 Electronic dial 412 Strobe light emission dimming control circuit 413 Distance measurement circuit 414 Photometry circuit 416 Backlight illumination device 417 LCD monitor device 418 Image sensor 419 Recording medium 423 A / D converter 424 Image buffer memory 425 Image processing circuit 426 Pixel defect position memory 427 Dust position memory 428 memory

Claims (14)

A still image capturing device capable of communicating with a moving image capturing device,
An imaging means for imaging a subject and obtaining image data;
First operating means for inputting a still image imaging instruction by the imaging means;
Transmission means for transmitting data to the video imaging device,
When a still image capturing instruction is input by the first operation unit during moving image capturing by the moving image capturing apparatus, the image capturing unit captures a still image and indicates the timing at which the imaging instruction is input. Transmitting first timing information to the moving image capturing apparatus by the transmitting means;
Furthermore, the transmission means also transmits second timing information indicating timing related to imaging by the imaging means before the input timing of the imaging instruction to the moving image imaging apparatus,
The still image capturing apparatus, wherein the first and second timing information are associated with a moving image captured by the moving image capturing apparatus. A second operation means for inputting a still image imaging preparation instruction by the imaging means;
The still image capturing apparatus according to claim 1, wherein the second timing information indicates an input timing of the imaging preparation instruction.   When the transmission means transmits the first or second timing information, the transmission means identifies information for the moving image capturing apparatus to identify whether the transmitted information is the first timing information or the second timing information. The still image capturing apparatus according to claim 1, wherein both are transmitted.   4. The still image capturing apparatus according to claim 1, wherein the first and second timing information is time information. 5.   5. The still image capturing apparatus according to claim 1, wherein the transmission unit transmits a still image including imaging time information as the first timing information. 6. A video imaging device capable of communicating with a still image imaging device,
An imaging means for imaging a subject and obtaining image data;
A first timing information indicating a timing at which the imaging instruction of a still image is input in the still image pickup apparatus during image pickup of moving images due to the imaging unit, before the input timing of the imaging instruction of the still image, the still Receiving means for receiving, from the still image capturing device, second timing information indicating timing related to image capturing in the image capturing device;
A moving image imaging apparatus comprising: an associating unit that associates the first and second timing information received by the receiving unit with a frame of a moving image captured by the imaging unit. Display means for displaying the first or second timing information associated with the frame of the moving image by the associating means on a display unit as a moving image index;
The moving image imaging apparatus according to claim 6, further comprising: a reproducing unit that starts reproduction from a moving image frame associated with the information related to the imaging when the index is designated.   The moving image imaging apparatus according to claim 6, wherein a still image including imaging time information is received as the first timing information from the still image imaging apparatus.   The moving image imaging apparatus according to claim 6, wherein the second timing information indicates an input timing of an imaging preparation instruction in the still image imaging apparatus. An imaging system configured to allow communication between a moving image imaging device and a still image imaging device,
The still image capturing device includes:
First imaging means for imaging a subject and obtaining image data;
First operating means for inputting a still image imaging instruction by the first imaging means;
Transmission means for transmitting data to the video imaging device,
When an instruction to capture a still image is input by the first operating unit during moving image capturing by the moving image capturing apparatus, the first imaging unit captures a still image and the imaging instruction is input. Transmitting the first timing information indicating the timing to the moving image capturing apparatus by the transmitting means;
Furthermore, the transmission means also transmits second timing information indicating timing related to imaging by the first imaging means before the input timing of the imaging instruction to the moving image imaging apparatus,
The video imaging device
Second imaging means for imaging a subject and obtaining image data;
First timing information indicating a timing at which a still image capturing instruction is input in the still image capturing apparatus during capturing of a moving image by the second imaging unit, and a timing before the input timing of the still image capturing instruction Receiving means for receiving second timing information indicating timing related to imaging by the first imaging means from the still image imaging device;
An imaging system, comprising: an associating unit that associates the first and second timing information received by the receiving unit with a frame of a moving image captured by the imaging unit. A method for controlling a still image capturing device capable of communicating with a moving image capturing device,
The still image capturing apparatus includes an image capturing unit that captures an image of a subject and obtains image data, and a first operation unit that inputs a still image capturing instruction from the image capturing unit.
When a still image capturing instruction is input by the first operation unit during moving image capturing by the moving image capturing apparatus, the image capturing unit captures a still image and indicates the timing at which the imaging instruction is input. A transmission procedure for transmitting first timing information to the video imaging device;
Furthermore, in the transmission procedure, second timing information indicating timing related to imaging by the imaging unit before the input timing of the imaging instruction is also transmitted to the moving image imaging apparatus,
The method for controlling a still image capturing apparatus, wherein the first and second timing information are associated with a moving image captured by the moving image capturing apparatus. A video imaging device capable of communicating with a still image imaging device,
The moving image capturing apparatus includes an image capturing unit that captures an image of a subject and obtains image data.
A first timing information indicating a timing at which the imaging instruction of a still image is input in the still image pickup apparatus during image pickup of moving images due to the imaging unit, before the input timing of the imaging instruction of the still image, the still Receiving second timing information indicating timing related to imaging in the image capturing device from the still image capturing device;
A method of associating the received first and second timing information with a frame of a moving image captured by the imaging unit; A program for controlling a still image capturing device capable of communicating with a moving image capturing device,
The still image capturing apparatus includes an image capturing unit that captures an image of a subject and obtains image data, and a first operation unit that inputs a still image capturing instruction from the image capturing unit.
When a still image capturing instruction is input by the first operation unit during moving image capturing by the moving image capturing apparatus, the image capturing unit captures a still image and indicates the timing at which the imaging instruction is input. Causing the computer to execute transmission processing for transmitting the first timing information to the moving image capturing apparatus;
Furthermore, in the transmission process, second timing information indicating timing related to imaging by the imaging unit prior to input timing of the imaging instruction is also transmitted to the moving image imaging apparatus,
The first and second timing information is associated with a moving image captured by the moving image capturing apparatus. A program for controlling a moving image capturing device capable of communicating with a still image capturing device,
The moving image capturing apparatus includes an image capturing unit that captures an image of a subject and obtains image data.
A first timing information indicating a timing at which the imaging instruction of a still image is input in the still image pickup apparatus during image pickup of moving images due to the imaging unit, before the input timing of the imaging instruction of the still image, the still Processing to receive second timing information indicating timing related to imaging in the image capturing device from the still image capturing device;
A program for causing a computer to execute processing for associating the received first and second timing information with a frame of a moving image captured by the imaging unit.

Images