JP2022006725A - Image management device, image management system, and image management method - Google Patents

Abstract

To synchronize a moving image and sound.SOLUTION: An image management device 3 includes an image data acquisition unit 332 that acquires a plurality of pieces of moving image data including time image data each of which indicates a time, a time identification unit 334 that identifies the time indicated by the time image data, a synchronization processing unit 336 that synchronizes the plurality of pieces of moving image data on the basis of the time identified by the time identification unit 334, a sound data acquisition unit 335 that acquires sound data synchronized with the time indicated by the time image data, and an image providing unit 338 that provides the plurality of pieces of moving image data and the sound data synchronized by the synchronization processing unit 336 to an information terminal.SELECTED DRAWING: Figure 7

Description

The present invention relates to an image management device, an image management system, and an image management method.

An image publishing device that publishes an image transmitted by a plurality of users via a network such as the Internet is known (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2013-191035

At events such as entrance ceremonies, athletic meet, and concerts, a large number of moving image data are created by a large number of participants taking pictures of the event. By uploading the moving image data to the server by each participant, the moving image taken by each participant can be viewed. If the moving image and the sound are played back at the same time on a terminal capable of reproducing the moving image and the sound in a state where the moving image and the sound are not synchronized, there is a problem that the sound and the moving image are shifted and it is difficult to browse.

Therefore, the present invention has been made in view of these points, and an object of the present invention is to provide an image management device, an image management system, and an image management method capable of synchronizing a moving image and a sound.

The image management device according to the first aspect of the present invention has an image data acquisition unit that acquires a plurality of moving image data including time image data indicating the time, and a time specification that specifies the time indicated by the time image data. A unit, a synchronization processing unit that synchronizes a plurality of moving image data based on the time specified by the time specifying unit, and a sound data acquisition unit that acquires sound data synchronized with the time indicated by the time image data. It has an image providing unit that provides the plurality of moving image data and the sound data synchronized by the synchronization processing unit to the information terminal.

The sound data acquisition unit acquires a plurality of the sound data associated with the plurality of moving image data, and the image providing unit provides one of the plurality of the sound data to the information terminal. May be good.

The image providing unit may provide the sound data at the earliest timing among the plurality of sound data to the information terminal.

The image providing unit may provide the information terminal with the sound data that changes at the timing closest to the timing at which the moving image data changes.

The sound data acquisition unit further acquires a plurality of sound collection position information indicating a position where each of the plurality of sound data is collected, and the image data acquisition unit obtains the position of the subject included in the moving image data. Further, the subject position information to be shown may be acquired.

The image providing unit uses the sound data associated with the sound collecting position information corresponding to the position closest to the position indicated by the subject position information among the plurality of positions indicated by the plurality of sound collecting position information. It may be provided to the terminal.

The sound data acquisition unit collects the sound data at a position closer to the subject included in the moving image data than the positions of the plurality of photographing devices that have taken the moving images corresponding to the plurality of moving image data. You may get it.

The sound data acquisition unit acquires the sound data including the sound time data indicating the time, and the synchronization processing unit obtains the time indicated by the sound time data and the time specified by the time specifying unit. The sound data including the sound time data may be synchronized with the plurality of moving image data corresponding to the time specified by the time specifying unit.

The sound data acquisition unit further has an operation reception unit that accepts a user's operation, and the sound data acquisition unit either acquires the sound data associated with the moving image data or acquires the moving image based on the operation accepted by the operation reception unit. You may choose whether to acquire the sound data that is not associated with the image data.

The image management system according to the second aspect of the present invention includes a plurality of image pickup devices and an image management device that manages moving image data generated by the plurality of image pickup devices, and each of the plurality of image pickup devices includes. The image management device has an image generation unit that generates the moving image data including time image data indicating a time, and an image transmission unit that transmits the moving image data, and the image management device acquires a plurality of the moving image data. An image data acquisition unit, a time specifying unit that specifies the time indicated by the time image data, a synchronization processing unit that synchronizes a plurality of the moving image data based on the time specified by the time specifying unit, and the time. A sound data acquisition unit that acquires sound data synchronized with the time indicated by the image data, and an image providing unit that provides the plurality of moving image data and the sound data synchronized by the synchronization processing unit to an information terminal. Have.

The image management method according to the third aspect of the present invention includes a step in which a computer acquires a plurality of moving image data including time image data indicating a time, and a step of specifying a time indicated by the time image data. , The step of synchronizing the plurality of moving image data based on the specified time, the step of acquiring the sound data synchronized with the time indicated by the time image data, and the synchronized plurality of moving image data and the sound. Perform steps to provide data and information to the information terminal.

According to the present invention, there is an effect that the moving image and the sound can be synchronized.

It is a figure which shows the structure of the image management system S1 which concerns on embodiment of this invention. It is a figure which shows the operation sequence of the image management system S1. It is a figure which shows an example of the screen when the information terminal 1 displays the 2nd QR code (registered trademark) and time information. It is a figure which shows an example of the screen when the user terminal 4 displays the presentation target image data acquired from the image management apparatus 3. It is a figure which shows the structure of the information terminal 1. It is a figure which shows the structure of the image pickup apparatus 2. It is a figure which shows the structure of the image management apparatus 3. It is a figure for demonstrating the operation which the synchronization processing unit 336 synchronizes a plurality of moving image data. It is a flowchart of the operation which the image management apparatus 3 stores the captured image data in a storage unit 32. It is a flowchart of the operation that the image management apparatus 3 synchronizes a plurality of moving image data and sends it to a user terminal 4. It is a figure for demonstrating the operation which the image providing unit 338 transmits to an information terminal in a state where a plurality of moving image data and sound data are synchronized. It is a figure which shows the structure of the image management apparatus 3 in the case of providing the music data associated with the time information included in the moving image data. It is a figure which shows the structure of the image management system S2.

[Configuration of image management system S1]
FIG. 1 is a diagram showing a configuration of an image management system S1 according to an embodiment of the present invention. The image management system S1 includes one or more information terminals 1, a plurality of image pickup devices 2 (2a, 2b, 2c), an image management device 3, and a user terminal 4. The image management system S1 manages an image by associating the captured image data captured by the plurality of image pickup devices 2 with the identification information for identifying the event generated by the information terminal 1 in an event such as an athletic meet of an elementary school or a live music. It is an image management system to store in the storage unit of the device 3. In the present specification, the case where the captured image data is moving image data is illustrated, but the captured image data may include still image data.

The information terminal 1 and the image management device 3 are connected to each other via the network N. The plurality of image pickup devices 2 and the image management device 3 are connected to each other via the network N. The user terminal 4 and the image management device 3 are connected to each other via the network N. The network N is, for example, the Internet. The information terminal 1 and the plurality of image pickup devices 2 may be connected by using a wireless communication method such as Bluetooth (registered trademark).

The information terminal 1 is, for example, a terminal used by the user of the image pickup apparatus 2. The information terminal 1 has an imaging device that captures a subject, an operation input device such as a touch panel that accepts user operations, a display that displays information based on the accepted user operations, and GPS (GPS) for specifying a current position. Global Positioning System) A terminal having a receiver for receiving from an external device such as a satellite, and a communication device for transmitting information via network N. The information terminal 1 is, for example, a tablet or a smartphone. In the following description, it is assumed that the information terminal 1 is a terminal of a representative user of a plurality of users who use a plurality of image pickup devices 2, but each of the plurality of users uses their own information terminal 1. You may.

The information terminal 1 captures, for example, an object indicating a first QR code installed at an event venue according to a user's operation. The object showing the first QR code is, for example, a poster on which the first QR code is printed, or a tablet on which the first QR code is displayed. The information terminal 1 extracts the presentation event ID, which is the presentation event identification information presented at the place where the event is held, from the first QR code captured. The presented event ID is data indicating "A elementary school athletic meet", for example, when the event is an athletic meet of A elementary school.

Subsequently, the information terminal 1 generates a second QR code including the presentation event ID and the license information preset in the information terminal 1. The license information is, for example, data indicating "A elementary school athletic meet 6th grade 1 group", and it is possible to view the captured image data only within the range indicated by the license information.

Then, the information terminal 1 displays the generated second QR code and the time image data including the time information indicating the time on the display of the information terminal 1. The time display unit of the information terminal 1 is, for example, a time acquired via a communication line, a Japanese standard time (JJY), an absolute time based on a radio wave acquired from a GPS satellite, or a relative time generated by a terminal application (predetermined time). (Time based on the timing of) is displayed. The information terminal 1 may generate a QR code including the presented event ID (that is, the first QR code) as the second QR code, and display the license information separately. Further, the information terminal 1 may transmit, for example, position information indicating the current position specified by the receiver based on the radio wave received from the GPS satellite to the image management device 3.

The information terminal 1 may generate a second QR code based on the current position and the current time without photographing the first QR code. As an example, the information terminal 1 displays the registration event ID acquired from the image management device 3 as the presentation event ID by transmitting the current position and the current time to the image management device 3. Details of the registration event ID will be described later. The information terminal 1 generates a second QR code including the received registration event ID as a presentation event ID, and displays the generated second QR code. In this case, the user of the information terminal 1 does not need to photograph the object showing the first QR code.

The image pickup device 2 is a camera capable of capturing a moving image, and is, for example, a camcorder, a digital camera, or an action camera. The image pickup apparatus 2 captures the second QR code, the time image, and the event associated with the second QR code displayed by the information terminal 1. The image pickup apparatus 2 generates moving image data generated by capturing an image of an event and a screen of an information terminal 1 on which a second QR code and a time image are displayed, for example. The image pickup device 2 transmits the generated moving image data to the image management device 3 via the network N. The image pickup device 2 has a receiver that receives radio waves for specifying the current position from an external device such as a GPS satellite, and even if the position information indicating the specified current position is transmitted to the image management device 3. good.

The image management device 3 is a server or a personal computer that manages captured image data generated by capturing an event. The image management device 3 stores the captured image data received from the image pickup device 2 and the pre-registered registration event ID in the storage unit. The registration event ID is registration event identification information for identifying an event for which the captured image data is to be managed, and is registered in the image management device 3 by, for example, the event organizer or the photographer.

The image management device 3 acquires a second QR code from the received moving image data. The image management device 3 extracts the presented event ID from the acquired second QR code. The image management device 3 sends the video image data associated with the extracted second QR code to the information terminal on condition that the extracted presentation event ID matches the registered event ID acquired in advance. It is stored in the storage unit as the presentation target image data to be presented.

The user terminal 4 is an information terminal used by a user who browses moving image data managed by the image management device 3, and is, for example, a personal computer or a smartphone. The information terminal 1 may be used as the user terminal 4. When the image management device 3 receives the browsing request information indicating the browsing request of the presentation target image data from the user terminal 4, the presentation target image data is provided on condition that the received browsing request information includes the information corresponding to the registration event ID. Is transmitted to the user terminal 4. The user terminal 4 displays the transmitted image data to be presented on the display.

FIG. 2 is a diagram showing a sequence of operations of the image management system S1. The sequence diagram shown in FIG. 2 starts from the time when the user who uses the information terminal 1 takes a picture of the first QR code at the venue of the event. In FIG. 2, the image pickup device 2a and the image pickup device 2b are illustrated as the image pickup device 2, but there may be more image pickup devices 2.

The information terminal 1 generates a second QR code based on the generated presentation event ID and the preset license information. The information terminal 1 displays the second QR code and the time information indicating the time on the display of the information terminal 1 (S11).

The image pickup device 2a is an image pickup device used by the first user, and captures a second QR code displayed on the display of the information terminal 1 (S12). The image pickup apparatus 2a generates captured image data including the second QR code and moving image data by capturing an event after capturing the second QR code and generating moving image data (S13). The image pickup device 2a transmits the moving image data P1 which is the generated captured image data to the image management device 3. In the present embodiment, it is assumed that the event is imaged following the second QR code, but the second QR code may be imaged in the middle of the image of the event.

The image pickup device 2b is an image pickup device used by the second user, and captures a second QR code displayed on the display of the information terminal 1 (S14). The image pickup apparatus 2b generates captured image data including the second QR code and moving image data by capturing an event after capturing the second QR code and generating moving image data (S15). The image pickup device 2b transmits the moving image data P2, which is the generated captured image data, to the image management device 3.

The image management device 3 receives the moving image data P1 and the moving image data P2. The image management device 3 acquires an event information image, which is an image obtained by capturing the second QR code, from each of the moving image data P1 and the moving image data P2. The image management device 3 extracts the presentation event IDs corresponding to the moving image data P1 and the moving image data P2 from the respective event information images of the acquired moving image data P1 and the moving image data P2 (S16).

Subsequently, the image management device 3 extracts the license information corresponding to each of the moving image data P1 and the moving image data P2 from the event information images of the moving image data P1 and the moving image data P2 (S17).

When the registered event ID stored in the storage unit in advance and the extracted presentation event ID match (YES in S18), the image management device 3 uses the moving image data corresponding to the extracted presentation event ID. It is stored in the storage unit in association with the license information corresponding to the moving image data (S19). On the other hand, when the registered event ID and the presented event ID do not match (NO in S18), the image management device performs processing without storing the moving image data corresponding to the presented event ID that does not match the registered event ID in the storage unit. finish.

For example, when the registration event ID and the presentation event ID included in the moving image data P1 are data indicating "A elementary school athletic meet", the image management device 3 has the moving image data included in the moving image data P1 and the moving image data. It is stored in the storage unit in association with the license information corresponding to. On the other hand, when the registered event ID is "A elementary school athletic meet" and the presented event ID is "B elementary school athletic meet", the image management device 3 does not store the captured image data included in the moving image data P2 in the storage unit. Ends the process.

FIG. 3 is a diagram showing an example of a screen when the information terminal 1 displays the second QR code and the time information. A second QR code is displayed in the upper part of the screen of the information terminal 1. In the middle of the screen of the information terminal 1, a character string indicating "A elementary school athletic meet 6th grade 1 group" which is license information is displayed. The data indicating the license information "A elementary school athletic meet 6th grade 1 group" is included in the second QR code displayed in the upper part of the screen of the information terminal 1. At the bottom of the screen of the information terminal 1, time information indicating the time (current time or relative time generated by the terminal application) is displayed.

FIG. 4 is a diagram showing an example of a screen when the user terminal 4 displays the presentation target image data acquired from the image management device 3. The screen shown in FIG. 4 is assumed to be viewed by the user using the user terminal 4 after the event is completed. The user terminal 4 is, for example, a personal computer connected to the image management device 3 via a network N. A plurality of presentation target image data P (P1, P2, P3, P4) are displayed on the screen D. Information indicating the photographer and the shooting time is simultaneously displayed in each of the plurality of presentation target image data Ps.

Each of the plurality of image data P to be presented is stored in the storage unit in association with the data indicating "A elementary school athletic meet 6th grade 1 group" which is the license information. The plurality of presentation target image data Ps are transmitted to the user terminal 4 when the image management device 3 acquires browsing request information including "A elementary school athletic meet 6th grade 1 group" from the user terminal 4.

As described above, the moving image data transmitted by the imaging device 2 includes the presentation event ID, and the image management device 3 provides the moving image data on condition that the presented event ID matches the registered event ID. To manage. Therefore, for example, when the user browses the moving image data on the user terminal 4 at home after the event ends, the person who participated in the event can browse the image taken at the event, but the person who does not participate in the event is illegal. The image transmitted to is not displayed on the user terminal 4.

Further, the moving image data includes the license information, and the image management device 3 does not provide the moving image data to the user terminal 4 of the user who is not included in the range indicated by the license information. By operating the image management device 3 in this way, it is possible to prevent people outside the license range (for example, people other than "A elementary school athletic meet 6th grade 1 group") from seeing the moving image data.

[Configuration of information terminal 1]
FIG. 5 is a diagram showing the configuration of the information terminal 1. The information terminal 1 has an image pickup unit 10, a storage unit 11, a display unit 12, a communication unit 13, and a control unit 14. The control unit 14 has a license information generation unit 141 and a code generation unit 142.

The image pickup unit 10 includes an image pickup element that captures an image of a subject. The image pickup unit 10 captures, for example, a first QR code and generates captured image data including the first QR code. The image pickup unit 10 inputs the created captured image data to the code generation unit 142.

The storage unit 11 has a storage medium such as a ROM (Read Only Memory), a RAM (Random Access Memory), and a hard disk. The storage unit 11 stores a program executed by the control unit 14. The storage unit 11 stores the license information set by the user by operating the display unit 12.

The display unit 12 is, for example, a liquid crystal display, an organic EL (Electro-Luminescence) display, or the like. The display unit 12 displays the second QR code and the current time based on the control of the control unit 14. Further, the display unit 12 has, for example, a touch panel on the surface of the display unit 12, specifies the operation content of the user based on the signal indicating the position touched by the user, and the operation content of the specified user is code generation unit. Notify 142.

The communication unit 13 includes a communication device for transmitting information via the network N. The communication device is, for example, a wireless LAN controller.

The control unit 14 is, for example, a CPU (Central Processing Unit). The control unit 14 functions as the license information generation unit 141 and the code generation unit 142 by executing the program stored in the storage unit 11. The control unit 14 causes the display unit 12 to display time information indicating the current time or the relative time generated by the terminal application.

The license information generation unit 141 generates license information indicating the usable range of the image data to be presented, based on the operation content of the user acquired from the display unit 12. The license information generation unit 141 stores the generated license information in the storage unit 11 and notifies the code generation unit 142.

The code generation unit 142 generates a second QR code based on the presentation event ID extracted from the first QR code included in the captured image data generated by the image pickup unit 10 and the license information generated by the license information generation unit 141. Generate. The code generation unit 142 causes the display unit 12 to display the generated second QR code.

[Structure of image pickup device 2]
FIG. 6 is a diagram showing the configuration of the image pickup apparatus 2. The image pickup apparatus 2 includes an image pickup unit 21, a communication unit 22, a storage unit 23, and a control unit 24. The image pickup unit 21 includes an image pickup element such as a CCD (Charge Coupled Device) sensor and a CMOS (Complementary Metal Oxide Semiconductor) sensor, and a lens. The image pickup unit 21 converts the optical signal input to the lens into an electric signal, then converts the electric signal into digital data and outputs the electric signal to the control unit 24. The digital data is, for example, moving image data or still image data.

The communication unit 22 includes a communication device for transmitting information via the network N. The communication device is, for example, a wireless LAN controller. The communication unit 22 transmits the moving image data generated by the control unit 14 to the image management device 3 by using, for example, a wireless LAN communication network.

The storage unit 23 has a storage medium such as a ROM, RAM, and a hard disk. The storage unit 23 stores a program executed by the control unit 24, which will be described later. The storage unit 23 is an information terminal 1 that displays a second QR code imaged by the image pickup unit 21 and time information indicating the current time or a relative time generated by the terminal application, based on the control of the control unit 24. Stores moving image data based on the screen image of.

The control unit 24 is, for example, a CPU. The control unit 24 functions as an image generation unit 241 and an image transmission unit 242 by executing a program stored in the storage unit 23.

The image generation unit 241 generates moving image data associated with the presentation event ID, which is the presentation event identification information presented at the venue where the event is held. Further, the image generation unit 241 generates moving image data including time image data indicating the time. The image generation unit 241 generates, for example, moving image data in which the image data and time image data of the second QR code including the presentation event ID are included in a predetermined time (for example, the time zone at the beginning of the moving image data). do. The image generation unit 241 may generate moving image data in which the image data of the second QR code is included for the first time and the time image data is included for the second time before and after the first time. .. The image generation unit 241 outputs the generated captured image data to the image transmission unit 242.

The image transmission unit 242 transmits the moving image data acquired from the image generation unit 241 to the image management device 3 via the communication unit 22.

[Configuration of image management device 3]
FIG. 7 is a diagram showing the configuration of the image management device 3. The image management device 3 has a communication unit 31, a storage unit 32, and a control unit 33.

The communication unit 31 is an interface for connecting to a communication network such as the Internet, and includes a communication device for transmitting and receiving information via the network N. The communication device is, for example, a LAN controller. The communication unit 31 receives moving image data from a plurality of image pickup devices 2. The communication unit 31 receives the registration event ID associated with the received moving image data from the information terminal 1.

The storage unit 32 has a storage medium such as a ROM, RAM, and a hard disk. The storage unit 32 stores a program executed by the control unit 33, which will be described later. The storage unit 32 stores moving image data acquired from a plurality of image pickup devices 2. Further, the storage unit 32 stores the registration event ID acquired in advance, the presentation event ID received from the information terminal 1, and the license information.

The control unit 33 is, for example, a CPU. By executing the program stored in the storage unit 32, the control unit 33 executes the identification information acquisition unit 331, the image data acquisition unit 332, the permission information acquisition unit 333, the time identification unit 334, the sound data acquisition unit 335, and the synchronization. It functions as a processing unit 336, an image management unit 337, an image providing unit 338, and an operation receiving unit 339.

The identification information acquisition unit 331 acquires a registration event ID, which is registration event identification information for identifying an event for which the captured image data is to be managed. The identification information acquisition unit 331 notifies the image management unit 337 of the acquired registration event ID. The identification information acquisition unit 331 notifies the image management unit 337 of the registration event ID by storing it in a storage unit 32 accessible by the image management unit 337, for example.

The image data acquisition unit 332 acquires moving image data from the image pickup device 2. The image data acquisition unit 332 analyzes the event information image data, which is the information storage image data included in the moving image data acquired from the image pickup device 2, to obtain the presentation event ID included in the event information image data. Identify. The event information image data is image data including a second QR code.

The image data acquisition unit 332 identifies the presentation event ID included in the second QR code. The image data acquisition unit 332 outputs the specified presentation event ID and moving image data to the image management unit 337.

Further, the image data acquisition unit 332 acquires moving image data including time image data indicating the time. The time image data may be the same as that included in the event information image, for example. The image data acquisition unit 332 outputs time image data indicating the acquired time to the time identification unit 334.

The license information acquisition unit 333 acquires license information indicating the usable range of the image data to be presented. Specifically, the license information acquisition unit 333 acquires the license information by analyzing the event information image. More specifically, the license information acquisition unit 333 acquires the license information included in the second QR code. For example, when the event information image contains the digital watermark data, the license information acquisition unit 333 may acquire the license information by analyzing the digital watermark data. The license information acquisition unit 333 notifies the image management unit 337 of the extracted license information.

The license information acquisition unit 333 acquires the license information from the information terminal 1 associated with the image pickup device 2 that has transmitted the moving image data to the image data acquisition unit 332, and acquires the moving image data and the information terminal 1. You may store it in the storage unit 32 in association with the license information. By operating in this way, the image management device 3 can associate the moving image data with the license information even after the moving image data is stored in the storage unit 32, for example.

Further, the license information acquisition unit 333 acquires the imaging position information, which is the first position information indicating the position where the imaging device 2 has captured the captured image corresponding to the moving image data, from the imaging device 2 that has transmitted the moving image data. You may. The license information acquisition unit 333 notifies the image management unit 337 of the acquired image pickup position information.

The time specifying unit 334 specifies the time when the moving image data corresponding to the time image data is captured by specifying the time indicated by the time image data acquired from the image data acquisition unit 332. Specifically, the time specifying unit 334 specifies the time by, for example, extracting an image of a numerical value captured in the time image data and then converting the extracted numerical image into numerical data. The time specifying unit 334 notifies the synchronization processing unit 336 of the specified time.

The sound data acquisition unit 335 acquires sound data synchronized with the time indicated by the time image data included in each of the plurality of moving image data. The sound data synchronized with the time indicated by the time image data is sound data collected at the same time and place as the imaging of the moving image data. For example, in the imaging of "relay competition of A elementary school athletic meet", "relay start". It is a sound data including a voice sound and a cheering sound. The sound data acquisition unit 335 outputs the acquired sound data to the synchronization processing unit 336. The sound data may be transmitted from the image pickup device 2 to the image management device 3 by the same data frame as the moving image data. In this case, for example, the sound data is separated in the image data acquisition unit 332, and the sound data is input from the image data acquisition unit 332 to the sound data acquisition unit 335.

The sound data acquisition unit 335 may specify the presentation event ID associated with the acquired sound data. For example, when the acquired sound data includes an announcement that "a relay competition of A elementary school athletic meet will be held", the sound data acquisition unit 335 specifies the presented event ID as "A elementary school athletic meet". The sound data acquisition unit 335 notifies the synchronization processing unit 336 of the specified presentation event ID in association with the acquired sound data.

When the image data acquisition unit 332 acquires a plurality of moving image data including the time image data, the synchronization processing unit 336 acquires a plurality of moving image data based on the respective times of the moving image data specified by the time specifying unit 334. Synchronize. The synchronization processing unit 336 synchronizes the moving image data corresponding to the specified time with other data which is image data different from the moving image data, based on the time specified by the time specifying unit 334. The other data is, for example, moving image data transmitted by an imaging device 2 different from the imaging device 2 that transmitted the moving image data. The synchronization processing unit 336 includes the moving image data and others, provided that the presentation event ID associated with the moving image data acquired by the image data acquisition unit 332 and the presentation event ID associated with the other data match. Synchronize with the data.

Here, a process in which the synchronization processing unit 336 synchronizes a plurality of moving image data will be described with reference to FIG. FIG. 8 is a diagram for explaining an operation in which the synchronization processing unit 336 synchronizes a plurality of moving image data. FIG. 8A shows a plurality of moving image data P (P1, P2) acquired by the image data acquisition unit 332. The moving image data P1 is, for example, moving image data generated by the imaging device 2a, and the moving image data P2 is, for example, moving image data generated by the imaging device 2b.

Each of the moving image data P includes event information image data and event image data including the content of the event. For example, the moving image data P1 includes event information image data and event image data A11 to A14. Each of the event image data A11 to A14 is, for example, an image frame constituting the moving image data. When the frame rate of the moving image data is 60 fps, the length of one event image data is about 0.17 seconds. FIG. 8 illustrates a case where the length of one event image data is 0.17 seconds.

The moving image data P acquired by the image data acquisition unit 332 is, for example, a moving image having a larger time interval in which the frame is updated than the minimum unit of time indicated by the time image data included in the event information image data. Specifically, in the case of moving image data having 60 event image data per second, the minimum unit of time indicated by the time image data is less than 1/60 second, for example, 0.01 second. In this way, by acquiring the time that changes in a unit smaller than the time interval in which the frame is updated, the synchronization processing unit 336 can improve the accuracy of specifying the time of the event image data.

In the event information image, the time is switched at the position of the solid arrow. The synchronization processing unit 336 acquires the time indicated by each solid line arrow of the moving image data P from the time specifying unit 334 as the time specified by the time specifying unit 334. The synchronization processing unit 336 specifies the imaging time corresponding to each of the event image data as shown by the broken line arrow by counting the time by the internal timer with the time indicated by the solid line arrow as a reference. The synchronization processing unit 336 specifies, for example, that the start time of the event image data A11 of the moving image data P1 is "10:30:49:17".

The synchronization processing unit 336 may specify the imaging time for each event image data, or may specify the imaging time for each of a plurality of event image data. Further, when the time changes every 0.01 seconds in the time image data, the synchronization processing unit 336 may specify the imaging time of each frame in units of 0.01 seconds, and is an integer of the time interval of the changing time. The imaging time of each frame may be specified for each fold.

The synchronization processing unit 336 aligns the time positions of the event image data included in each of the moving image data P based on the respective imaging times of the event image data specified by the synchronization processing unit 336.

FIG. 8B shows a state in which the synchronization processing unit 336 aligns the time positions of the event image data included in each of the moving image data P. The presentation target image data P1 corresponds to the moving image data P1, and the presentation target image data P2 corresponds to the moving image data P2.

The synchronization processing unit 336 specifies, for example, that the imaging time of the event image data A12 and the event image data B11 is “10:30:49:34”. Then, the time positions are aligned so that the event image data A12 and the event image data B11 are output from the image management device 3 at the same time. In this way, the synchronization processing unit 336 synchronizes a plurality of moving image data.

For example, when a plurality of image pickup devices image the "first competition of the A elementary school athletic meet" and generate moving image data, it is highly possible that the generation of each of the plurality of moving image data is started from different times. As described above, when moving image data having different start times are simultaneously reproduced on the user terminal 4, different scenes are displayed for each moving image data.

Therefore, the synchronization processing unit 336 synchronizes the moving image data corresponding to the time specified by the time specifying unit 334 with other data, so that a plurality of different moving image data corresponding to the same scene is displayed on the user terminal 4. Will be done. Further, the synchronization processing unit 336 synchronizes the presentation event IDs included in each of the plurality of moving image data on the condition that they match, so that the moving image data of the same event is displayed on the user terminal 4. ..

By the way, the image management device 3 manages moving image data taken at a plurality of different event venues. The image management device 3 manages, for example, the moving image data of "the second competition of A elementary school" and the moving image data of "the first competition of B elementary school". When the moving image data taken at different event venues are not displayed on the user terminal 4 at the same time, the synchronization processing unit 336 does not need to synchronize the moving image data taken at different event venues. Therefore, the synchronization processing unit 336 associates the image management device 3 with the moving image data in order to synchronize the plurality of moving image data generated by imaging the same competition when the moving image data of a plurality of events is managed. Using the presented event ID, a plurality of moving image data to be synchronized are specified.

Further, the synchronization processing unit 336 may synchronize a plurality of moving image data based on the sound data associated with the moving image data acquired by the sound data acquisition unit 335. For example, the sound data acquisition unit 335 acquires the first sound data associated with the moving image data and the second sound data associated with other data different from the moving image data. The synchronization processing unit 336 synchronizes a plurality of moving image data so that the image frames corresponding to the characteristic sounds (for example, pistol sounds) contained in both the first sound data and the second sound data match. May be good.

Further, the synchronization processing unit 336 may use the first sound data and the second sound data to determine whether or not to synchronize a plurality of moving image data. The sound data acquisition unit 335 uses the first presentation event ID, which is the first presentation event identification information for identifying the event performed at the place where the moving image data is captured, based on the sound included in the first sound data. To identify. Subsequently, the sound data acquisition unit 335 identifies the second event ID, which is the second presentation event identification information for identifying the event performed at the place where the other data is captured. The sound data acquisition unit 335 notifies the synchronization processing unit 336 of the specified first event ID and second event ID. The synchronization processing unit 336 synchronizes the moving image data with other data on condition that the first event ID acquired from the sound data acquisition unit 335 and the second event ID match.

Further, when the image data acquisition unit 332 further acquires the display data associated with the time, the synchronization processing unit 336 synchronizes the moving image data with the display data associated with the time specified by the time identification unit 334. You may generate the composite moving image data synthesized in the state of being. The display data is, for example, text data associated with a time, CG image data, or the like, and is, for example, text data indicating an event title.

Returning to FIG. 7, the image management unit 337 manages the moving image data transmitted by the image pickup device 2. The image management unit 337 acquires image data in association with the presentation event ID, provided that the presentation event ID notified from the identification information acquisition unit 331 matches the registration event ID stored in the storage unit 32. The moving image data acquired by the unit 332 is stored in the storage unit 32 as the presentation target image data to be presented to the user terminal 4.

That is, the image management unit 337 stores the moving image data including the image data of the second QR code including the same presentation event ID as the registration event ID in the storage unit 32 as the presentation target image data. The image management unit 337 does not store the captured image data that does not include the image data of the second QR code including the presentation event ID that is the same as the registration event ID in the storage unit 32 as the presentation target image data. When the moving image data is stored in the storage unit 32 as the presentation target image data, the image management unit 337 stores the license information acquired from the license information acquisition unit 333 in the storage unit 32 in association with the moving image data.

By operating the image management unit 337 in this way, the image management device 3 limits the users who can browse by determining whether to transmit the captured image data based on the license information and the browsing request information. can do.

When the presented event ID includes the event position information which is the second position information indicating the position of the event, the image management unit 337 calculates the difference between the event position information and the image pickup position information corresponding to the moving image data. You may. In that case, the image management unit 337 stores the moving image data in the storage unit 32 when the difference between the calculated event position information and the imaging position information is within a predetermined range. The predetermined range is, for example, a range equal to or less than the diameter of the smallest circle including the site of A elementary school when the event is "A elementary school athletic meet".

By operating the image management unit 337 in this way, the image management device 3 can associate and store the moving image data captured at the place where the event is held. Further, the image management device 3 can suppress the provision of the captured image data to the user terminal 4 even if the captured image data obtained by capturing an event different from the event indicated by the presented event ID is associated with the image management device 3.

Further, the image management unit 337 may store the moving image data in the storage unit 32 on condition that the moving image data is transmitted at a time within a predetermined range from the time zone in which the event was performed. In order to do so, for example, the image management unit 337 acquires the event time zone information indicating the time zone in which the event corresponding to the presented event ID is performed from the information terminal 1, and sets the time when the moving image data is received. The indicated reception time is acquired from the image data acquisition unit 332. The image management unit 337 stores the moving image data in the storage unit 32 when the acquired reception time is within a predetermined range based on the time zone indicated by the acquired event time zone information.

The predetermined range based on the time zone indicated by the event time zone information is, for example, the time zone from the time when the "athletic meet of A elementary school" starts to the time when one hour has passed since the end of the "athletic meet of A elementary school". Is. By operating the image management unit 337 in this way, it is possible to prevent the moving image data not related to the event from being erroneously stored in the storage unit 32.

The image providing unit 338 transmits the image to be presented to the user terminal 4 via the network N. For example, when the image providing unit 338 receives the browsing request information indicating the browsing request of the presentation target image data from the user terminal 4, the presentation target image is provided on the condition that the browsing request information includes the information corresponding to the registration event ID. The data is transmitted to the user terminal 4.

Specifically, for example, when the registered event ID is "A elementary school athletic meet" and the browsing request information is "A elementary school athletic meet 6th grade 1 group", the browsing request information includes information corresponding to the registered event ID. In this case, the image providing unit 338 transmits the image data to be presented to the user terminal 4 based on the "A elementary school athletic meet 6th grade 1 group" which is the browsing request information received from the user terminal 4.

When the browsing request information is received from the information terminal, the image providing unit 338 sends the presentation target image data to the information terminal on condition that the browsing request information indicates that the viewing request information is within the usable range indicated by the license information. You may send it. Specifically, for example, when the license information is "A elementary school 6th grade 1 group" and the browsing request information is "A elementary school athletic meet 6th grade 1 group", the image providing unit 338 uses the license information. It is determined that the viewing request information indicates the possible range. As a result, the image data to be presented is transmitted to the information terminal based on the "A elementary school athletic meet 6th grade 1 group" which is the browsing request information received from the information terminal.

When transmitting a plurality of presentation target image data to the user terminal 4, the image providing unit 338 provides the user terminal 4 with a plurality of moving image data synchronized by the synchronization processing unit 336 and other data. When the synchronization processing unit 336 synchronizes the moving image data and the sound data, the image providing unit 338 provides the user terminal 4 with the plurality of moving image data and the sound data synchronized.

When the image providing unit 338 provides a plurality of moving image data and sound data to the user terminal 4 in a synchronized state, for example, one is selected from a plurality of sound data, and the selected sound data and a plurality of moving images are selected. Provided in association with image data. Since the propagation speed of sound is slower than that of light, the delay time of sound data with respect to moving image data differs depending on the shooting location. By providing one sound data to the user terminal 4 by the image providing unit 338, it is possible to prevent a plurality of sounds slightly deviated from being reproduced and becoming difficult to hear. The image providing unit 338 selects, for example, the sound data at the earliest timing among the plurality of sound data. By operating in this way, the image providing unit 338 can provide the user terminal 4 with a sound having a small delay in the sound with respect to the moving image data.

Further, the image providing unit 338 may select, for example, sound data that changes at the timing closest to the timing at which the moving image data acquired by the image data acquisition unit 332 changes. For example, it is assumed that the moving image data includes an image indicating a signal of "relay start". In that case, among the sound data including the voice sound of "relay start", the sound data including the voice sound of "relay start" at the timing closest to the time of the image indicating the signal of "relay start" (that is,). , The sound data acquired by the image pickup device 2 at the position closest to the person who is saying "relay start") is selected. By operating in this way, the image management device 3 can make it difficult for the user to feel a sense of discomfort due to the time difference between the moving image data and the sound data.

Further, the image providing unit 338 obtains sound data associated with the sound collecting position information corresponding to the position closest to the position indicated by the acquired subject position information among the plurality of positions indicated by the acquired plurality of sound collecting position information. You may choose. In order to do so, the image providing unit 338 acquires a plurality of sound collecting position information indicating the position where each of the plurality of sound data acquired by the sound data acquisition unit 335 is collected. The image providing unit 338 acquires subject position information indicating the position of the subject included in the moving image data acquired by the image data acquisition unit 332. By operating in this way, it is possible to provide a sound with less delay in the sound with respect to the moving image data. This configuration is suitable, for example, when collecting the singing voice of a singer on the stage of a concert venue.

When the presentation target image data, which is the moving image data stored in the storage unit 32, includes the time image data, the image providing unit 338 provides the user terminal 4 with partial moving image data excluding the time image data among the moving image data. You may. For example, when the time image data is included in the beginning portion of the moving image data, the image providing unit 338 provides the moving image data of the portion where the event is taken after the time image data is completed to the user terminal 4. Further, when the synchronization processing unit 336 generates the composite moving image data in which the text data or the CG image data is combined with the moving image data, the image providing unit 338 transmits the generated synthesized moving image data to the user terminal 4. do.

The operation reception unit 339 accepts a browsing request from the user terminal 4 and an operation performed to acquire the image to be presented. The operation reception unit 339 notifies the image providing unit 338 of the received request and operation.

When the sound data is associated with the image data to be presented, the operation reception unit 339 acquires the sound data associated with the moving image data which is the image data to be presented, or the sound data not associated with the moving image data. Accepts whether to get. The operation reception unit 339 notifies the sound data acquisition unit 335 of the received operation.

[Operation of image management unit 337]
FIG. 9 is a flowchart of an operation in which the image management device 3 stores moving image data in the storage unit 32. The image management unit 337 acquires, for example, a registered event ID from the event organizer and stores it in the storage unit 32 (S31). The image management unit 337 may store information such as the event holding time zone and the event holding place in the storage unit 32 in association with the registered event ID. After that, when the event starts, the image data acquisition unit 332 starts acquiring the moving image data. When the image data acquisition unit 332 receives the moving image data from the image pickup device 2 (YES in S32), the image data acquisition unit 332 extracts the presentation event ID included in the moving image data (S33).

Subsequently, the license information acquisition unit 333 acquires the license information indicating the usable range of the image to be presented (S34). The license information acquisition unit 333 acquires the license information by analyzing the information storage image included in the moving image data acquired by the image data acquisition unit 332. When the digital watermark data is included in the information storage image included in the moving image data, the license information acquisition unit 333 acquires the license information by analyzing the digital watermark data.

Subsequently, the image management unit 337 determines whether or not the registered event ID stored in the storage unit 32 and the presented event ID extracted from the moving image data match (S35). When the registered event ID and the presented event ID match (YES in S35), the image management unit 337 proceeds to the process of specifying the position where the acquired moving image data is captured. When the registered event ID and the presented event ID do not match (NO in S35), the image management unit 337 ends the process without storing the moving image data corresponding to the presented event ID in the storage unit 32.

Subsequently, the license information acquisition unit 333 acquires the image pickup position information, which is the first position information indicating the position corresponding to the image captured image corresponding to the moving image data, from the image pickup device 2. The image management unit 337 calculates the difference between the position indicated by the image pickup position information and the position indicated by the event position information which is the second position information indicating the position of the event included in the presentation event ID.

When the difference between the position indicated by the image pickup position information and the position indicated by the event position information is equal to or less than the threshold value and the imaged position is within a predetermined range (YES in S36), the image management unit 337 may use the acquired moving image data. Proceed to the process of specifying the time when the image was taken. When the difference between the position indicated by the imaging position information and the position indicated by the event position information is not equal to or less than the threshold value (NO in S36), the image management unit 337 stores the moving image data corresponding to the presentation event ID including the imaging position information. The process ends without being stored in the unit 32.

Subsequently, the image management unit 337 acquires a reception time indicating the time when the moving image data is received. Then, when the acquired reception time is within a predetermined range determined based on the time zone indicated by the event time zone information stored in advance in the storage unit 32, the image management unit 337 receives the data (YES in S37). The moving image data associated with the presentation event ID corresponding to the time is stored in the storage unit 32 (S38).

In the flowchart shown in FIG. 9, in S36 and S37, the image management unit 337 stores the moving image data in the storage unit 32 on condition that the shooting position and the shooting time zone are within a predetermined range. The image management unit 337 does not have to execute S36 and S37. That is, the image management unit 337 may store the moving image data in the storage unit 32 in response to the determination that the registered event ID and the presented event ID match.

[Operation of synchronization processing unit 336]
As described above, the images captured by the plurality of event participants have different times when the imaging starts. Therefore, even if the user who sent the viewing request can view only the moving image data associated with the event, the sound and the moving image will be shifted due to the difference in the start time of each of the moving image data. It is difficult to view the statue at the same time. Next, a process in which the synchronization processing unit 336 synchronizes a plurality of moving image data in order to facilitate viewing of a plurality of moving images at the same time will be described.

FIG. 10 is a flowchart of an operation in which the image management device 3 synchronizes a plurality of moving image data and transmits the moving image data to the user terminal 4. The image data acquisition unit 332 acquires a plurality of moving image data from the plurality of image pickup devices 2 (S41). The image data acquisition unit 332 acquires event information image data included in each of the acquired plurality of moving image data (S42). The image data acquisition unit 332 identifies a plurality of presentation event IDs included in each of the plurality of event information image data by analyzing each of the acquired plurality of event information image data (S43).

The image data acquisition unit 332 outputs the acquired plurality of event information image data to the time specifying unit 334. The time specifying unit 334 specifies the time indicated by each of the acquired plurality of event information image data.

When the synchronization processing unit 336 determines that the plurality of presented event IDs specified by the image data acquisition unit 332 match (YES in S44), each of the plurality of moving image data specified by the time specifying unit 334 is captured. The time set is acquired (S45).

The synchronization processing unit 336 specifies the imaging time corresponding to the event image data included in each of the plurality of moving image data acquired by the image data acquisition unit 332 based on the time specified by the time specifying unit 334 (S46). .. The synchronization processing unit 336 identifies an image frame having the same time in a plurality of image frames included in each of the plurality of moving image data based on the specified imaging time (S47).

The synchronization processing unit 336 aligns the positions of a plurality of moving image data so that the image frames having the same time are reproduced at the same timing (S48). The image management unit 337 stores a plurality of moving image data whose positions are aligned in the storage unit 32 (S49).

[Sound data selection]
FIG. 11 is a diagram for explaining an operation in which the image providing unit 338 transmits a plurality of moving image data and sound data to the user terminal 4 in a synchronized state. FIG. 11A shows a plurality of moving image data P (P1, P2) acquired by the image data acquisition unit 332 and a plurality of sound data M (M1, M2) acquired by the sound data acquisition unit 335. .. The plurality of moving image data P are the same as those in FIG. 8 (a).

The sound data M1 is sound data associated with the moving image data P1. The sound data acquisition unit 335 acquires the event sound data E11 at the same time as the time when the image data acquisition unit 332 acquires the event image data A11 included in the moving image data P1, for example.

The sound data M2 is sound data associated with the moving image data P2. The sound data acquisition unit 335 acquires the event sound data F11 later than the time when the image data acquisition unit 332 acquires the event image data B11 included in the moving image data P2, for example.

FIG. 11B shows a plurality of moving image data and sound data transmitted by the image providing unit 338 to the user terminal 4. The presentation target image data P1 corresponds to the moving image data P1, the presentation target image data P2 corresponds to the moving image data P2, and the presentation target sound data M1 corresponds to the sound data M1.

The image providing unit 338 selects, for example, sound data having a small delay with respect to the moving image data from among the plurality of sound data M synchronized with the plurality of moving image data P acquired from the synchronization processing unit 336. In FIG. 11B, the delay between the sound data M2 and the moving image data P2 is larger than the delay between the sound data M1 and the moving image data P1. Therefore, the image providing unit 338 selects the sound data M1, associates it with the plurality of presentation target image data, and outputs the sound data to the user terminal 4.

By the way, the image providing unit 338 may select the sound data based on the time of the moving image data associated with the sound data. Specifically, the sound data associated with the moving image data whose specified time is earlier may be selected from the plurality of moving image data and output to the user terminal 4.

When a plurality of moving image data are simultaneously captured at the start timing of a performance song, for example, in a concert, the image providing unit 338 outputs sound data with a small delay between the moving image data and the sound data. select. On the other hand, when the timing of starting imaging differs depending on the photographer, such as at an athletic meet, sound data associated with moving image data having an earlier start time is selected. The image providing unit 338 may determine what kind of sound data is selected based on, for example, the operation content received by the operation receiving unit 339. By operating the image providing unit 338 in this way, it is possible to select sound data suitable for the content of the event in which the moving image data is captured.

[Modification 1]
In the above description, the case where the image management device 3 is used for providing the moving image data taken at an event venue such as an athletic meet and the sound data associated with the moving image data to the user terminal 4 is exemplified. However, the application of the image management device 3 is not limited to this. The event is for one or more users to shoot a song or dance at their home or store, and the home or store may be the event venue. In this case, the image management device 3 is used for providing the user terminal 4 with moving image data and music data in which a moving scene in which a person sings in time with music is captured. The music data is, for example, the same as the music included in the sound data associated with the event identification information for identifying the event held independently by the user or the moving image data, and is recorded or collected at a place different from the shooting place. It is the sounded data. The music data may be stored in the storage unit 32 in advance by the image management device 3.

FIG. 12 is a diagram showing a configuration of an image management device 3 in the case of providing music data associated with time information included in moving image data. The image management device 3 of FIG. 12 is different from the image management device 3 shown in FIG. 7 in that it has a music data acquisition unit 340, and is the same in other respects.

The music data acquisition unit 340 acquires the first music data associated with the time. The synchronization processing unit 336 synchronizes the acquired first music data with the moving image data. The synchronization processing unit 336 synchronizes the moving image data with the first music data associated with the time specified by the time specifying unit 334, and synthesizes the first music data and the moving image data. Generate content. The image providing unit 338 outputs the synthetic moving image content generated by the synchronization processing unit 336 to the user terminal 4.

Hereinafter, the image management device 3 provides the user terminal 4 with synthetic video content in which music data is synthesized with moving image data generated by capturing an image of a scene singing along with music, accompaniment, or instrumental music, for example. The processing in the case of doing so will be described.

The user who is the photographer of the moving image data is the same as the first music data on his / her own information terminal 1 when a predetermined relative time with respect to the reference time set by the user himself / herself or a third party arrives in the image pickup apparatus 2. The second music data associated with the time information is set to be played. The reference time is, for example, the time when the user activates the music playback function of the predetermined application software on the information terminal 1, and when the music playback function is activated, the information terminal 1 has -10 seconds, -9 seconds, ... Display the relative time that changes as in. The information terminal 1 starts playing music after a predetermined time has elapsed from the start of displaying the relative time. For example, when 10 seconds have elapsed from the start of displaying the relative time and the relative time reaches 0 seconds, the reproduction of the second music data is started.

After activating the music creation function, the user captures the time image data displayed on the information terminal 1 of the user with the image pickup device 2, and then sets the image pickup device 2 so that the user himself / herself is photographed. When the information terminal 1 starts playing the second music data, the user starts singing along with the music based on the second music data while the user himself / herself is being photographed. The image pickup apparatus 2 transmits the moving image data thus created to the image management apparatus 3.

Since such moving image data includes environmental sounds other than music based on the second music data, the quality of the moving image content is low as it is. Therefore, the image management device 3 generates high-quality synthetic moving image content by replacing the sound data associated with the moving image data with the first music data corresponding to the second music data.

When the moving image data is transmitted from the image pickup device 2 to the image management device 3, the image data acquisition unit 332 is an information terminal before playing the music based on the second music data corresponding to the first music data on the information terminal 1. Acquire moving image data including the time image data displayed in 1. The time image data used here is synchronized with the start timing of the first music data and the second music data. The second music data is music data possessed by the information terminal 1 and reproduced by the information terminal 1. The synchronization processing unit 336 creates the composite image content in a state in which the moving image data and the first music data associated with the time synchronized with the second music data specified by the time specifying unit 334 are synchronized.

Specifically, the synchronization processing unit 336 specifies the imaging start time of image data (that is, an image frame) other than the time image data included in the moving image data based on the time image data, and captures the specified moving image frame. The composite image content is created by synchronizing the moving image data and the first music data so that the first music data starts at the start time. The image providing unit 338 provides the composite image content created in this way to the user terminal 4.

[Modification 2]
In the above description, the case where the information terminal 1 displays the absolute time based on the radio wave received from the GPS satellite or the relative time generated by the terminal application is illustrated, but the time displayed by the information terminal 1 is determined in the event. It may be a relative time with respect to the reference time. For example, each of the plurality of information terminals 1 belongs to one of a plurality of groups, and the display unit 12 functioning as a time display unit is associated with the time indicated by the time image data acquired by the information terminals 1 belonging to the same group. Display the time that was set.

For example, one of the plurality of information terminals 1 belonging to the same group, one information terminal 1 periodically transmits time information indicating the time when the internal clock is generated to the other information terminals 1 belonging to the same group. The other information terminal 1 displays time image data based on the received time information.

[Modification 3]
In the above description, the case where the image pickup device 2 transmits the image data of the second QR code including the presentation event ID and the moving image data including the time image data to the image management device 3 has been illustrated, but the information has been illustrated. The terminal 1 has the function of the image pickup device 2, the information terminal 1 captures the first QR code, and the moving image data including the image data and the time image data of the first QR code is transferred to the image management device 3. You may send it.

FIG. 13 is a diagram showing a configuration of the image management system S2 according to the modification 3. As shown in FIG. 13, each of the information terminals 1a to 1c transmits the moving image data created by photographing the first QR code including the presented event ID to the image management device 3. Further, the information terminals 1a to 1c can also function as the user terminal 4 shown in FIG. 1, and the information terminals 1a to 1c can also display moving image data managed by the image management device 3. As described above, in the image management system S2, the information terminal 1 and the image management device 3 can be used to realize the same processing as the configuration shown in FIG.

In the image management system S2, the information terminal 1 may execute various processes that can be executed by the image pickup apparatus 2. For example, the license information acquisition unit 333 of the image management device 3 acquires the imaging position information indicating the position where the information terminal 1 has captured the captured image corresponding to the moving image data from the information terminal 1 that has transmitted the moving image data. May be good.

The information terminal 1 generates the first QR code and the time image data, and also generates the moving image data, and the moving image data including the generated image data and the time image data of the first QR code is used as the image management device 3. May be sent to.

[Modification 4]
The information terminal 1 may generate a QR code that does not include the presentation event ID and includes the license information. The information terminal 1 generates, for example, a QR code indicating license information corresponding to the group to which the user of the information terminal 1 belongs. In this case, the image management unit 337 is a target for presenting the moving image data acquired by the image data acquisition unit 332 to the user terminal 4 on condition that the license information registered in advance is included in the moving image data. It may be stored in the storage unit 32 as a certain presentation target image data. Then, when the browsing request information is received from the user terminal 4, the image providing unit 338 displays the image data to be presented on the condition that the browsing request information indicates that the viewing request information is within the usable range indicated by the license information. It is transmitted to the user terminal 4. By operating the information terminal 1 and the image management device 3 in this way, it becomes possible to share moving image data among members of the same group without using the registration event ID.

[Modification 5]
In the above description, the synchronization processing unit 336 performs a plurality of moving images based on the characteristic sound timing commonly included in the plurality of sound data associated with the plurality of moving image data including the time image data. Although the case of synchronizing the image data is illustrated, the synchronization processing unit 336 may synchronize the plurality of moving image data with the sound data not associated with the moving image data.

The sound data includes sound time data indicating the time, and the synchronization processing unit 336 includes a plurality of moving image data including the time image data based on the time indicated by the time image data and the time indicated by the sound time data. And the sound data including the sound time data may be synchronized. The sound time data is data indicating a time specified based on, for example, a time signal sound collected at an event venue or a radio wave acquired from a GPS satellite or the like. The sound time data may be audible sound, or may be data representing sounds in a band other than the audible band. Further, the sound time data may be data obtained by converting sound into a digital value, or may be text data representing time.

The sound data acquisition unit 335 or the synchronization processing unit 336 acquires sound data including sound time data, and specifies the time indicated by the sound time data included in the acquired sound data. The synchronization processing unit 336 corresponds to the sound data including the sound time data and the time specified by the time specifying unit 334 based on the time indicated by the sound time data and the time indicated by the time image data specified by the time specifying unit 334. Synchronize with multiple moving image data.

Specifically, the synchronization processing unit 336 specifies the time of each frame in the moving image data based on the time indicated by the time image data, and each of the sound data is based on the time indicated by the sound time data. Specify the time of the part of. The synchronization processing unit 336 associates the frame in the moving image data corresponding to the same time with the sound data.

By operating in this way, the image management device 3 can generate a plurality of moving image data including the sound data and the time image data even if the sound data is synchronized with the time indicated by the time information different from the time image data. Can be synchronized. The image management device 3 can also acquire sound data including sound time data synchronized with the time indicated by the time image data generated by the information terminal 1 displaying the time image data from the information terminal 1. Such a configuration is suitable, for example, in an environment where only sound data can be collected near a sound source.

[Effect of the image management device 3 of this embodiment]
As described above, the image management device 3 includes an image data acquisition unit 332 that acquires moving image data including time image data indicating the time, a time specifying unit 334 that specifies the time indicated by the time image data, and a time specifying unit. The unit 334 has a synchronization processing unit 336 that synchronizes a plurality of moving image data based on a specified time. Then, the sound data acquisition unit 335 acquires the sound data synchronized with the time indicated by the time image data, and the image providing unit 338 provides the sound data and the plurality of moving image data to the user terminal 4 in a synchronized state. do.

As a result, the user of the user terminal 4 can browse a plurality of moving images synchronized with the sound data, so that it becomes easy to browse the plurality of moving images.

Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and changes can be made within the scope of the gist thereof. be. For example, all or part of the device can be functionally or physically distributed / integrated in any unit. Also included in the embodiments of the present invention are new embodiments resulting from any combination of the plurality of embodiments. The effect of the new embodiment produced by the combination has the effect of the original embodiment together.

1 Information terminal 2 Image pickup device 3 Image management device 4 User terminal 10 Image pickup section 11 Storage section 12 Display section 13 Communication section 14 Control section 21 Imaging section 22 Communication section 23 Storage section 24 Control section 31 Communication section 32 Storage section 33 Control section 141 License information generation unit 142 Code generation unit 241 Image generation unit 242 Image transmission unit 331 Identification information acquisition unit 332 Image data acquisition unit 333 License information acquisition unit 334 Time specification unit 335 Sound data acquisition unit 336 Synchronization processing unit 337 Image management unit 338 Image Providing unit 339 Operation reception unit 340 Music data acquisition unit

Claims (10)

An image data acquisition unit that acquires a plurality of moving image data including time image data, each of which indicates a time, and an image data acquisition unit.
A time specifying unit that specifies the time indicated by the time image data, and
A synchronization processing unit that synchronizes a plurality of moving image data based on a time specified by the time specifying unit, and a synchronization processing unit.
A sound data acquisition unit that acquires sound data synchronized with the time indicated by the time image data, and a sound data acquisition unit.
An image providing unit that provides the information terminal with the plurality of moving image data and the sound data synchronized by the synchronization processing unit.
Image management device with. The sound data acquisition unit acquires a plurality of the sound data associated with the plurality of the moving image data, and obtains the plurality of the sound data.
The image providing unit provides one of the plurality of the sound data to the information terminal.
The image management device according to claim 1. The image providing unit provides the sound data at the earliest timing among the plurality of sound data to the information terminal.
The image management device according to claim 2. The image providing unit provides the information terminal with the sound data that changes at the timing closest to the timing at which the moving image data changes.
The image management device according to claim 2 or 3. The sound data acquisition unit further acquires a plurality of sound collection position information indicating a position where each of the plurality of sound data is collected.
The image data acquisition unit further acquires subject position information indicating the position of the subject included in the moving image data.
The image providing unit uses the sound data associated with the sound collecting position information corresponding to the position closest to the position indicated by the subject position information among the plurality of positions indicated by the plurality of sound collecting position information. Provide to the terminal,
The image management device according to any one of claims 2 to 4. The sound data acquisition unit collects the sound data at a position closer to the subject included in the moving image data than the positions of the plurality of photographing devices that have taken the moving images corresponding to the plurality of moving image data. get,
The image management device according to any one of claims 1 to 5. The sound data acquisition unit acquires the sound data including the sound time data indicating the time, and obtains the sound data.
The synchronization processing unit corresponds to the sound data including the sound time data and the time specified by the time specifying unit based on the time indicated by the sound time data and the time specified by the time specifying unit. Synchronize with the moving image data of
The image management device according to any one of claims 1 to 6. It also has an operation reception unit that accepts user operations.
The sound data acquisition unit acquires the sound data associated with the moving image data or acquires the sound data not associated with the moving image data based on the operation received by the operation receiving unit. Select
The image management device according to any one of claims 1 to 7. A plurality of image pickup devices and an image management device for managing moving image data generated by the plurality of image pickup devices are provided.
Each of the plurality of image pickup devices
An image generation unit that generates the moving image data including time image data indicating the time, and
An image transmission unit that transmits the moving image data, and
Have,
The image management device is
An image data acquisition unit that acquires a plurality of the moving image data,
A time specifying unit that specifies the time indicated by the time image data, and
A synchronization processing unit that synchronizes a plurality of the moving image data based on the time specified by the time specifying unit, and a synchronization processing unit.
A sound data acquisition unit that acquires sound data synchronized with the time indicated by the time image data, and a sound data acquisition unit.
An image providing unit that provides the information terminal with the plurality of moving image data and the sound data synchronized by the synchronization processing unit.
Have,
Image management system. The computer
A step to acquire multiple moving image data including time image data, each of which indicates the time, and
The step of specifying the time indicated by the time image data and
The step of synchronizing the plurality of moving image data based on the specified time, and
The step of acquiring the sound data synchronized with the time indicated by the time image data, and
A step of providing the plurality of synchronized moving image data and the sound data to an information terminal, and
Image management method to execute.

Images