JP2015097319A - Synchronization system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a synchronization system capable of synchronizing a plurality of terminals with a simple configuration.SOLUTION: A synchronization system includes: a photoelectric conversion element for converting an optical image into an electric signal; communication means for performing communication with a network or another imaging device; receiving means for video information, voice information and their meta information; synchronization means for taking synchronization between data on the basis of meta information on data to be combined; a memory capable of storing the meta information; a plurality of imaging devices comprising the photo-electric conversion element, communication means, meta information receiving means, and memory; and synchronization means for taking synchronization among a plurality of terminals on the basis of the meta information.

Description

  The present invention relates to a system for synchronizing video information or audio information of a plurality of imaging devices.

  In general, when editing or combining video information or audio information captured by a plurality of imaging devices, the time information of each captured terminal is different and cannot be synchronized with each other, and processing is difficult as it is. It was.

  A general method for realizing synchronization among a plurality of image capturing apparatuses is a system in which cables are connected to the image capturing apparatuses. However, since such a method is difficult to realize with a consumer device, another synchronization method has been proposed.

  Patent Document 1 discloses a method for realizing synchronization of a plurality of photographing devices by using an illumination device incorporated in the photographing device. Patent Document 2 discloses a method for realizing synchronization using a universal time code obtained from GPS.

JP 2002-344800 A JP 2002-32978 A

  However, the conventional techniques disclosed in the above-mentioned patent documents cause degradation of the image and increase in cost / size of the apparatus. That is, according to the method disclosed in Patent Document 1, since it is necessary to emit light from the illumination device during shooting, only a specific frame during moving image shooting has an extremely high brightness image. In addition, it is difficult to implement in a scene where it is difficult to emit light from the lighting device (in a calm atmosphere) or outdoors where it is difficult to notice the effect of the lighting device. According to the method disclosed in Patent Document 2, since it is necessary to provide a GPS receiver in the photographing apparatus, the cost and size of the apparatus are increased. In addition, there is a problem that the internal clock of the device gradually shifts in places where it is difficult to receive GPS signals, and synchronization with a device not equipped with a GPS receiver is not possible.

  In addition, when synchronizing a plurality of audio information, it is possible to calculate the amount of deviation between the two audio information with high accuracy by using the correlation calculation method. However, since the correlation calculation is generally large, the calculation reference When the range is unnecessarily wide, there is a problem that the calculation load becomes excessive and takes time.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a synchronization system that can quickly synchronize video / audio information from a plurality of imaging devices with a simple configuration, and thus to acquire video information and audio information of a plurality of imaging devices. It makes it easy to edit and compose.

To achieve the above objective,
The present invention relates to a photoelectric conversion element that converts an optical image into an electrical signal, a communication unit that communicates with a network or another imaging device, a memory that can store meta information, and the photoelectric conversion element, the communication unit, and the memory. A plurality of imaging devices, and the communication means transmits and receives video and audio information of a plurality of imaging devices and meta information thereof, and synchronizes a plurality of terminals or a plurality of video and audio information based on the meta information. The synchronization system was configured to include the synchronization means to perform.

  Further, the meta information is stored in the frame of the video information and can be time stamped.

  Further, meta information detecting means for recognizing, storing and comparing meta information transition patterns for each frame, the synchronizing means comprises video frame synchronizing means and audio synchronizing means, and the video frame synchronizing means compares meta information transition patterns. To obtain the amount of deviation and synchronize.

  Furthermore, the voice synchronization means performs a correlation operation or the like on the voice signal synchronized in units of frames and synchronizes.

  As described above, even between photographing apparatuses whose time information is shifted, the time required for the correlation calculation is minimized by performing the synchronization to the frame unit by the meta information and the precise synchronization by the correlation calculation step by step.

  According to the present invention, it is possible to provide a synchronization system capable of quickly synchronizing a plurality of terminals and video / audio information with a simple configuration.

Block diagram of the imaging device Central section of the photographic device Flowchart showing operations at the time of photographing and composition of each photographing apparatus Schematic diagram showing the shooting situation Schematic diagram showing how to synchronize video Schematic diagram showing the shooting situation and data sharing method in the second embodiment Flowchart for explaining the operation of the photographing apparatus and the network in this embodiment Explanatory diagram of shooting scenes in this embodiment

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

  FIG. 1 is a block diagram of each photographing apparatus (corresponding to a terminal of a synchronization system) according to an embodiment of the present invention.

[Example 1]
Hereinafter, a synchronization system according to a first embodiment of the present invention will be described with reference to FIGS.

  FIG. 1 is a block diagram showing an electrical configuration of a digital camera and a lens which are photographing devices. The camera 1 has an imaging system, an image processing system, a communication system, a recording / reproducing system, and a control system. The imaging system includes an imaging optical system 3 and an imaging element 6 that is a photoelectric conversion element. The image processing system includes an A / D converter (indicated as ADC in the figure) 20 and an image processing circuit 21. The communication system is an antenna. 7 and communication means 26. The recording / reproducing system includes a recording processing circuit 22, a memory means 23, and a display means 8. The control system is a camera system control circuit 25, a focus detection unit (including an AF sensor) 12, and an exposure detection unit (including an AE sensor). 13, an operation detection circuit 27, a lens system control circuit 28, and an optical system driving unit 9. The optical system driving unit 9 includes a focus lens driving unit 9a, a shake correction driving unit 9b, an aperture driving unit 9c, and the like.

  The imaging system is an optical processing system that forms an image of light from an object on the imaging surface of the imaging device 6 via the imaging optical system 3. During a preliminary shooting operation such as aiming, a part of the light beam is also guided to the focus detection unit 12 via a mirror provided in a quick return mirror mechanism (not shown). (Refer to the central cross-sectional view of the imaging apparatus of FIG. 2 for details thereof) Further, as will be described later, the imaging optical system 3 is appropriately adjusted by the control system so that an appropriate amount of object light is captured by the imaging device. 6 and an object image is formed in the vicinity of the image sensor 6.

  The image processing circuit 21 is a signal processing circuit that processes an image signal of the number of pixels of the image sensor received from the image sensor 6 via the A / D converter 20, and includes a white balance circuit, a gamma correction circuit, and an interpolation calculation. It has an interpolation calculation circuit that performs resolution.

  The communication system can transmit / receive a signal appropriately encoded by the communication means 26 from the antenna 7. As a result, communication with other photographing devices and public communication networks (so-called Internet etc.) existing in the vicinity is possible. Data exchanged using the communication system will be described later.

  The recording processing circuit 22 outputs an image signal to the memory unit 23 and generates and stores an image to be output to the display unit 8. In addition, the recording processing circuit 22 compresses images, moving images, sounds, and the like using a predetermined method.

  The camera system control circuit 25 which is a synchronization means generates and outputs a timing signal at the time of imaging. Further, as will be described later, the camera system control circuit 25 realizes synchronization of video information and audio information among a plurality of terminals using the photometric result of the AE sensor 13. Further, the control system controls the imaging system, the image processing system, and the recording / reproducing system in response to an external operation. For example, the operation detection circuit 27 detects that a shutter release button (not shown) is pressed, and controls driving of the image sensor 6, operation of the image processing circuit 21, compression processing of the recording processing circuit 23, and the like. Further, the display means 22 controls the state of each segment of the information display device that displays information on an optical finder, a liquid crystal monitor or the like.

  The adjustment operation of the optical system of the control system will be described. A focus detection unit 12 and an exposure detection unit 13 are connected to the camera system control circuit 25, and an appropriate focus position and aperture position are obtained based on these signals. The camera system control circuit 25 issues a command to the lens system control circuit 28 via the electrical contact 10, and the lens system control circuit 28 appropriately controls the focus lens driving unit 9a and the aperture driving unit 9c. Further, a camera shake detection sensor (not shown) is connected to the lens system control circuit 28, and in the camera shake correction mode, the shake correction driving unit 9b is appropriately controlled based on a signal from the camera shake detection sensor.

  FIG. 2 shows a central cross-sectional view of the photographing apparatus that is the subject of the present invention. In FIG. 2, 1 is an imaging device, 2 is an imaging lens, 3 is an imaging optical system, 4 is an optical axis of the imaging lens, 5 is a lens barrel, 6 is an imaging device, and 7 is an imaging device 1. , 8 is a liquid crystal monitor or the like forming display means, 9 is an optical system driving means for adjusting the photographing optical system, 10 is a contact point for connecting the photographing apparatus 1 and the photographing lens 2, 11 Denotes a so-called quick return mirror mechanism, 12 denotes a focus detection unit including an AF sensor, and 13 denotes an exposure detection unit including an AE sensor.

  The imaging device 1 uses the photographing lens 2, the focus detection unit 12, and the exposure detection unit to perform focus / exposure detection, drive a part of the photographing optical system 3, and adjust the photographing optical system to obtain an image by the imaging element 6. Image in the vicinity. Further, the aperture is operated so as to achieve proper exposure.

  Further, in synchronism with the operation of a release button (not shown) by the user, subject information is obtained from the image sensor 6 and recorded in the memory means.

  Although the antenna 7 is illustrated at an appropriate position, in practice, an appropriate pattern can be formed on the flexible cable, and the above-described flexible cable can be formed in the photographing apparatus 1.

  FIG. 3 is a flowchart for explaining the operation of the photographing apparatus in the present embodiment. FIG. 3A shows a portion related to the present invention among operations at the time of shooting, and FIG. 3B shows a portion related to the present invention among operations performed after shooting (when synthesizing video and audio).

  The operation at the time of shooting will be described with reference to FIG. Step S10 corresponds to power ON at the time of shooting. In step S11, it is determined whether or not the mode is for performing synchronous shooting. When it is in the synchronous shooting mode, the process proceeds to step S12, and otherwise, the process proceeds to S13. In step S12, as will be described later, the subject exposure information is shared between the photographing apparatuses as data for synchronization. Also, format conversion and storage in the memory means described later are performed.

  Step S13 corresponds to the start of shooting, for example, corresponding to the timing when the recording start button of the shooting apparatus 1 is pressed. In step S13, it is determined whether or not the mode is for performing synchronous shooting. When it is in the synchronous photographing mode, the process proceeds to step S15, and otherwise, the process proceeds to S18.

  In step S15, the luminance of the subject is photometrically calculated as an EV value. In step S16, it is determined whether or not the EV value has changed. The process proceeds to step S17 when the subject brightness changes due to a change in the illumination conditions of the subject, and the process proceeds to step S18 otherwise. In step S18, the timing at which the EV value changes is stored as meta information. That is, meta information is time stamped and stored in a frame of video information. Step S18 corresponds to the end of shooting, for example, corresponding to the timing when the recording end button of the shooting apparatus 1 is pressed. If the end of shooting in step S18 is not detected, the process returns to step S13, and the operations from step S13 to step S18 are repeated during shooting. Step S19 corresponds to turning off the power at the time of shooting.

  The synchronization operation will be described with reference to FIG. Step S20 corresponds to the start of the synchronization operation. In step S21, a file to be synchronized is searched. This can be done while looking at appropriate tag information as will be described later. In step S22, the meta information of the synchronization source file and the synchronization destination file is scanned. Thereby, the information stored in step S17 in FIG.

  That is, the camera system control circuit 25 is meta information detection means.

  In step S23, synchronization in units of frames is realized by using the meta information obtained in step S22. (Determining the amount of delay for synchronization.) Details of the operation for realizing synchronization will be described later. In step S24, it is determined whether it is necessary to synchronize the sound. Although the video is synchronized by step S23, since the audio has a delay, it is necessary to obtain another delay amount for synthesizing the audio. When sound synchronization is necessary, the process proceeds to step S25, and when not necessary, the process proceeds to step S28. Which one to select depends on the result you want to obtain through synchronization. (Depends on the application presented to the user.) For example, if the purpose is to generate a multi-viewpoint image, audio synchronization is not necessary. On the other hand, when creating a master sound source or a directional sound source at the same time, it is necessary to synchronize the sound.

  Note that the above-described file search is executed for a predetermined search time or the number of target files set in the photographing apparatus or instructed from the network or the like, so that it may take more time than necessary. Absent.

  In step S25, the audio data of the synchronization destination and the synchronization source file is extracted. Step S26 determines the correlation calculation range of the audio signal. This may be set appropriately based on the audio delay time estimated from the size of the venue. In step S27, the audio signal is synchronized from the correlation between the audio signals. (Determining the amount of delay for synchronization) Step S28 uses the amount of delay for synchronization obtained in steps S23 and S27 to perform the necessary synthesis processing while realizing video and audio synchronization between a plurality of files. I do.

  FIG. 4 is a diagram for explaining a shooting scene in this embodiment. As shown in FIG. 4A, a case is considered in which a plurality of photographing apparatuses 1a, 1b, 1c, and 1d perform photographing in one venue. For example, a school festival or an athletic meet can be considered. In such a scene, when creating a multi-view video using images of a plurality of photographing devices, it is necessary to synchronize video signals between the photographing devices. In FIG. 4A, the case where there are four imaging devices is considered, but a plurality of imaging devices may be used, and a larger number of cases or two imaging devices may be used. In addition to the creation of multi-view video, applications that use video from a plurality of terminals include stereoscopic video shooting and creation of a master sound source / directional sound source.

  A time code associated with the internal clock of the photographing apparatus is attached to the video obtained by the photographing apparatus. Although the internal clock of the photographing apparatus is initially adjusted by the user, there are errors at this time, errors in the clock based on the internal clock, and the like, and a shift that cannot be ignored at the level of video synchronization may occur.

  In order to solve this problem, there is a method for realizing synchronization by causing a built-in strobe to emit light. However, this method has a problem in that the luminance is increased only in a specific frame and the image quality is deteriorated because strobe light is emitted during moving image shooting. In addition, there is a problem that it is difficult to apply in an environment such as outdoors where the built-in flash is difficult to catch.

  On the other hand, in an imaging apparatus equipped with GPS, the internal clock is adjusted with respect to the standard time, and therefore it is possible to perform appropriate time adjustment by referring to these. However, this method has a problem that the internal clock of the device gradually shifts in a place where it is difficult to receive a GPS signal / cannot synchronize with a device not equipped with a GPS receiver. .

  In order to solve these problems, the present invention realizes synchronization between photographing apparatuses based on meta information incorporating changes in luminance information of a subject.

  FIG. 4B is a diagram schematically showing information passing between the photographing apparatuses. FIG. 4B shows the exchange between the photographing apparatus 1a and the photographing apparatus 1b as an example, but the other terminals may be considered similarly.

  When each photographing apparatus is set to the synchronous mode after power-on (corresponding to step S10 in FIG. 3A) (corresponding to step S11 in FIG. 3A), the communication provided in each photographing apparatus. Information for sharing exposure information is exchanged with other photographing apparatuses via the means 26 and the antenna 7.

  In each photographing apparatus, the camera system control 25 operates as format conversion means. The camera system control 25 expands the received exposure information of the other photographing devices and performs an appropriate format conversion in order to fill the difference between the photographing devices. Thereafter, the exposure information is temporarily stored in the memory means 23. As described above, the exposure information received from the other photographing device stored in the memory means 23 is erased at an appropriate timing such as after a certain time has passed since the main power supply was turned off.

  In the subsequent shooting (corresponding to step S13 in FIG. 3A), subject luminance measurement is performed using the AE sensor 13. (Corresponding to step S15 in FIG. 3A) Since the luminance changes when the illumination condition of the subject changes, the information is stored as meta information. (Corresponding to step S16 and step S17 in FIG. 3A) This operation is continued until the end of photographing (corresponding to step S18 in FIG. 3A).

  Thereafter, the photographing apparatus exchanges video signals with other photographing apparatuses via the communication means 26 and the antenna 7. This delivery does not necessarily have to be performed in synchronization with shooting.

  FIG. 5 is a diagram schematically showing an operation for realizing the synchronization. FIG. 5A is a diagram schematically showing a state of searching for a file, and FIG. 5B is a diagram schematically showing how synchronization is realized in units of frames.

  As shown in FIG. 5A, the camera system control 25 is a file that is likely to be shot simultaneously with the received video signal based on the tag information stored in the meta information of the received video signal. Is selected (corresponding to step S21 in FIG. 3B).

  Next, the camera system control circuit 25 serving as meta information detection means extracts meta information of the corresponding file (corresponding to step S22 in FIG. 3B), and compares meta information. In the example of FIG. 5B, File2 and received File are compared. Since the transition of the EV value, which is exposure information as a result of subject metering, is recorded in the meta information for each frame, the frames are shifted so that the timings coincide with each other. In the example of FIG. 5B, information indicating that a change in EV value is recognized as a timing chart in gray frames (File 2 frames 2 and 9 and reception file frames 6 and 13) within a predetermined time is shown. Yes. If these timings match, the synchronization of the two images can be realized. Specifically, synchronization can be realized by delaying the File2 signal by 4 frames. The amount of delay necessary for the synchronization obtained here is stored in the memory means. (Corresponding to step S23 in FIG. 3B) Thus, the camera system control circuit 25 is a video frame synchronization means.

As a specific method for matching frames, a method of calculating a correlation between two signals can be considered. For example, the difference absolute value may be calculated while giving a delay after binarizing the signal. Taking the signal shown in FIG. 5B as an example, the signal of File 2 is “0100000010000000” and the signal of the received File is “0000010000001000”. If the bit sum is minimized while shifting the bit with respect to this signal, the minimum value can be obtained when the File2 signal is delayed by 4 frames.
Audio synchronization will be described with reference to FIG. FIG. 6A shows an example in which audio signals of two files are extracted. In FIG. 6A, the vertical axis indicates the level of the audio signal, and the horizontal axis indicates time. FIG. 6B shows the result of speech segment detection (VAD) performed on the signal of FIG. The vertical axis in FIG. 6B shows the VAD result, and the horizontal axis shows time. FIG. 6C shows an enlarged view of the audio signal after synchronization is performed according to the VAD result. In FIG. 6C, the vertical axis indicates the level of the audio signal, and the horizontal axis indicates time. Here, the horizontal axes of FIGS. 6A and 6B are shared, and FIG. 6C is enlarged, so the scale of the horizontal axis is different from FIGS. 6A and 6B.

  The audio signal of the file to be synchronized (File 2 and Receive File in the example of FIG. 6 as in FIG. 5B) is extracted. (Corresponding to step S25 in FIG. 3B.) FIG. 6A shows a part of the audio signal. The time in FIG. 6A is after the synchronization of File.2 and the received file in frame units, and File.2 and the received file are in a synchronized state in the image. In FIG. 6A, the received File audio signal is delayed with respect to the File.2 audio signal. This shows a case where the imaging device that captured the received file is located farther from the main subject. That is, the delay of the received file with respect to File.2 corresponds to the time difference from when the sound reaches the imaging device that captured File.2 until it reaches the imaging device that captured the received file. Further, as described above, the delay of the audio signal is governed by the positional relationship between the imaging devices. Therefore, if the maximum positional deviation is known in advance, the range for calculating the delay can be limited. For example, when a delay can be estimated from a GPS signal or the like, a range for calculating a correlation described later is set (corresponding to step S26 in FIG. 3B).

  Next, an appropriate VAD algorithm is applied to FIG. FIG. 6 (b) shows the result. The section where the utterance was made is shown as “H”, and the section where there was no utterance is shown as “L”. The delay amount for the synchronization is calculated in the same manner as the calculation method for performing the frame synchronization of the signal in FIG. 6B. By performing binarization using VAD, the amount of calculation can be reduced.

  Furthermore, synchronization within the VAD detection accuracy range is performed again using a part of the original audio signal. FIG. 6C is an enlarged view of a part of the audio signal shown in FIG. 6A after shifting the delay amount obtained by using VAD. The delay amount for synchronization is calculated again using this signal (corresponding to step S27 in FIG. 3B).

  Thus, the camera system control circuit 25 is an audio synchronization means.

  In this case as well, the synchronization calculation is executed for a predetermined upper limit of calculation time or the number of target files set in the photographing apparatus or instructed from the network or the like. There is no such thing.

  Using the delay amount for synchronizing the video and audio, the video and audio are synthesized by an appropriate method (corresponding to step S28 in FIG. 3B).

  In the present embodiment, the pattern of EV value change timing is recognized and compared during video information synchronization. However, not only the EV value change timing but also the stored EV values are compared. , More accurate synchronization is possible.

  In the present embodiment, the case where the timing due to the change of the exposure information as meta information (corresponding to the case where the lighting is turned bright or dark in an indoor venue) is illustrated. As other meta information for detecting the timing, when the audio AGC is changed (when the volume of the subject is suddenly changed), when the color temperature of the illumination is changed (white balance is changed), the color information is changed. In such a case (corresponding to the case where the illumination color changes in an indoor venue or the like), it can be considered as a method for realizing synchronization by passively using image information.

  In addition, the focal length information of the lens (when zoomed up according to the size of the subject), the change in the camera angle (corresponding when the camera is shaken upward or downward by a gyro sensor, etc.), the scene shooting information (the subject's information) Changeable, switching from portrait mode to landscape mode), aperture value and shutter speed as a change finer than EV value, image effect that changes the lens depth of field by changing aperture value, and subject by high-speed shutter speed It is also possible to use it as a case where the timing has changed.

  Timing charts created between devices vary depending on the type of meta information, so when narrowing down to frame units or performing audio synchronization, search for the target file and the calculation range in the file can be used. It is changed according to the contents of meta information.

  In this embodiment, information can be exchanged between the image capturing apparatuses by communication between the image capturing apparatuses (so-called peer-to-peer), or via a network.

  In the present embodiment, the photographing apparatus 1 has the antenna 7 and performs the so-called wireless communication. However, the communication may be realized by wired communication.

  In this embodiment, a so-called digital single-lens reflex camera is described as an example of a photographing apparatus. However, since the function unique to the digital single-lens reflex camera is not used, other photographing apparatuses (digital video camera, digital compact camera, It can also be implemented in a mobile phone or the like.

  As described above, according to this embodiment, it is possible to provide a synchronization system that enables synchronization of a plurality of terminals with a simple configuration.

[Example 2]
Hereinafter, a synchronization system according to a second embodiment of the present invention will be described with reference to FIGS. Since the configuration of each photographing apparatus is the same as that of the first embodiment, a description thereof will be omitted.

  FIG. 7 is a flowchart for explaining the operation of the photographing apparatus and the network in this embodiment. FIG. 7A shows a portion related to the present invention among operations at the time of shooting, and FIG. 7B shows a portion related to the present invention among operations performed after shooting (when synthesizing video and audio). In FIG. 7, the same functions as those in FIG. FIG. 7B is the same as FIG. 3B, but in this embodiment, the means for performing processing such as synchronization is a network described later.

  Only a portion of FIG. 7A different from FIG. 3A will be described. Step S112 transmits / receives exposure information to / from the network as data for synchronization, as will be described later. In step S118, the recorded video is transmitted to the network.

  FIG. 8A is a diagram for explaining a shooting scene in this embodiment. FIG. 8 shows, as an example, the exchange between the imaging device 1a and the imaging device 1b via the network 102, but other terminals may be considered in the same manner.

  In the present embodiment, each of the photographing apparatuses 101a, 101b, 101c, and 101d communicates with the network 102 via the communication unit 26 and the antenna 7. In FIG. 8A, the network is shown in a cloud shape. A large number of processing devices and storage devices are connected on the network 102 and can be virtually regarded as a large computer.

The exposure information described in the first embodiment is sent to the network 102 at the start of shooting.
When each photographing apparatus is set to the synchronous mode after the power is turned on (corresponding to step S10 in FIG. 7A) (corresponding to step S11 in FIG. 7A), the network 102 photographs at the same venue. The exposure information is processed and sent to the imaging device that seems to be. The exposure information processing is a conversion for filling in a difference in photometric sensitivity variation or the like by the photographing apparatus. The exposure information held by the photographing apparatus 101a is sent to the photographing apparatus 101b (corresponding to step S112 in FIG. 7A).

  As described in the first exemplary embodiment, the same exposure information is shared by the photographing apparatus 101a and the photographing apparatus 101b, thereby achieving synchronization.

  Each image capturing apparatus acquires the timing according to the change of exposure information and stores it as meta information in the same manner as in the first embodiment (corresponding to steps S13 to S18 in FIG. 7A. However, details of the image capturing operation are described in the embodiment. It was omitted because it was described in 1.).

  Further, each photographing apparatus transmits the video with the above-described meta information to the network 102. (Corresponding to step S118 in FIG. 7A) The network 102 analyzes these images, and realizes synchronization in units of frames using the timing based on the change of the exposure information as shown in the first embodiment ( Corresponding to the operation of FIG.

  As described above, according to this embodiment, it is possible to provide a synchronization system that enables synchronization of a plurality of terminals with a simple configuration.

  The synchronization system of the present embodiment is a system for easily editing and synthesizing a plurality of image and audio information obtained from a plurality of photographing devices and acquiring higher quality information from a cloud or a server.

1. 6. Imaging device Antenna 13. AE sensor 25. Camera system control 26. Communication means

Claims (6)

A photoelectric conversion element that converts an optical image into an electrical signal, a communication unit that communicates with a network or another imaging device, a memory that can store meta information, and a plurality of photoelectric conversion elements, a communication unit, and a memory that include the memory And the communication means send and receive video, audio information and meta information of the plurality of imaging devices, and a plurality of video or audio information obtained from the plurality of imaging devices based on the meta information. A synchronization system comprising synchronization means for performing synchronization. The synchronization system according to claim 1, wherein the meta information is stored in a frame of the video information and can be time stamped. Meta information detection means for recognizing, storing, and comparing meta information transition patterns for each frame;
The synchronizing means comprises a video frame synchronizing means and an audio synchronizing means,
2. The video frame synchronization means synchronizes by obtaining a shift amount from comparison of transition patterns of meta information, and the audio synchronization means synchronizes by performing a correlation operation or the like on an audio signal synchronized on a frame basis. The synchronization system described in 1. The synchronization system according to claim 3, wherein the calculation target range can be changed according to the content of the meta information, and an upper limit of the calculation target range is determined by a network or a device. A conversion means for expanding video captured by a plurality of terminals into a common format;
A communication means for communicating with a network or a photographing device;
Synchronization means for synchronizing video or audio captured by a plurality of terminals based on the meta information;
Synchronous system with 6. The synchronization system according to claim 5, wherein the meta information is information necessary for image exposure, audio modulation, gain control information, camera optical information, photographer position information, and the like.