JP2018032408A - Transmission device, distribution server, method of controlling transmission device, method of controlling distribution server, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the sense of presence felt from captured images using low data transmission rates.SOLUTION: According to an aspect of the present invention, a transmission device 2 is provided. A transmission device 2 comprises: an image capturing unit 41 configured to acquire a captured image G; a music information acquisition unit 42 configured to acquire music identification information of a musical piece delivered by a distribution server 12 in synchronization with sounding of the musical piece inside an establishment A where the transmission device 2 resides; and a transmission unit 44 configured to transmit the captured image G and the music identification information attached thereto to a reception device.SELECTED DRAWING: Figure 5

Description

The present invention relates to a transmission device, a distribution server, a transmission device control method, a distribution server control method, and a program .

  Conventionally, a portable terminal applied to a communication network system is known as this type of receiving apparatus (see Patent Document 1). This mobile terminal receives a photo mail transmitted from another mobile terminal, and displays image data of a photo attached to the photo mail on an LCD display. In this way, a photograph (photographed image) is exchanged (transmitted / received) by the mail function, and the photograph can be browsed on the mobile terminal on the receiving side.

JP 2002-91880 A

By the way, in this kind of portable terminal, it is considered that voice data of music is added to the shot image and exchanged, and the added voice data is reproduced while the shot image is displayed on the portable terminal on the receiving side. It was. For example, audio data of music flowing at the shooting site is added to the shot image and exchanged. The mobile terminal on the receiving side reproduces the audio data while displaying the captured image. Thus, the user on the receiving side can recognize the music that was playing at the shooting site while browsing the shot image, and thus can improve the live feeling in the shot image.
However, in such a configuration, since voice data is exchanged together with the photographed image, the amount of data transmission increases, and there is a problem in that the exchange of the photographed image is delayed or incomplete.

It is an object of the present invention to provide a transmission device, a distribution server, a transmission device control method, a distribution server control method, and a program that can improve a live feeling in a captured image with a small amount of data transmission.

  According to an aspect of the present invention, a transmission device is provided. The transmission device includes an image capturing unit that acquires a captured image, a music information acquisition unit that acquires music identification information of the music distributed by the distribution server in synchronization with the sound emission of the music to the facility where the transmission device exists, A transmission unit that transmits the photographed image and the music identification information added to the photographed image to the reception device.

  The transmission device further includes means for transmitting facility information obtained by selecting the facility to the distribution server, and the music information acquisition unit may acquire the music identification information distributed by the distribution server based on the facility information. Good.

  The transmission apparatus may further include means for receiving a user operation for selecting a facility.

  The transmission apparatus may further include means for selecting a facility by performing a check-in process in the facility.

According to another aspect of the present invention, the music identification information of the music is received from the audio reproduction device that emits the music in the facility, and the music identification information is synchronized with the sound emission of the music in the transmission device existing in the facility. Is provided .

According to another aspect of the present invention, music identification information of a song is received from a plurality of sound reproducing devices that respectively emit music in a plurality of facilities, and the transmission device is transmitted to a transmission device existing in any of the plurality of facilities. There is provided a distribution server that distributes music identification information in synchronization with sound emission of music to a facility where the music exists .

  According to another aspect of the present invention, a step of acquiring a photographed image, a step of acquiring music identification information of the music distributed by the distribution server in synchronization with the sound emission of the music into the facility where the transmission device exists, and And a step of transmitting the photographed image and the music identification information added to the photographed image to the receiving device.

  According to another aspect of the present invention, the step of receiving the music identification information of the music from the sound reproducing device that emits the music in the facility, and the music in sync with the sound emission of the music in the transmitting device existing in the facility. A method of controlling the distribution server, including the step of distributing the identification information.

According to another aspect of the present invention, a program for causing a computer to function as a transmission device is provided. This program has a function of acquiring a photographed image, a function of acquiring song identification information of a song distributed by a distribution server in synchronization with sound emission of a song to a facility where a transmission device exists, a shot image, and a shot image. And a function of transmitting the music identification information added to the receiver to the receiving device .

According to another aspect of the present invention, a program for causing a computer to function as a distribution server is provided. This program has a function of receiving music identification information of a music from a sound reproducing device that emits the music in the facility, and a function of distributing music identification information to the transmitting device existing in the facility in synchronization with the sound emission of the music And make it happen on a computer .

1 is a system configuration diagram illustrating a music presentation system according to an embodiment. It is the block diagram which showed the structure of the portable terminal. It is the functional block diagram which showed the function structure of the transmission side portable terminal and the receiving side portable terminal. (A) is the figure which showed the display screen of the picked-up image in embodiment, (b) is the figure which showed the display screen of the music purchase site. (C) is the figure which showed the display screen of the picked-up image in a modification. It is the flowchart which showed the music presentation operation | movement by a music presentation system.

  Hereinafter, a reception apparatus, a control method for the reception apparatus, and a program according to an embodiment of the present invention will be described with reference to the accompanying drawings. In this embodiment, the music presentation system provided with the portable terminal used as the receiver of this invention is illustrated. This music presentation system is a system for exchanging shot images between a plurality of mobile terminals, and a system for presenting music that has been played at the shooting site (shooting environment) when displaying a shot image on a mobile terminal on the receiving side. It is. The music presenting system has a configuration that reduces the data transfer amount by not exchanging music audio data itself.

  As shown in FIG. 1, the music presentation system 1 includes a plurality of portable terminals 2 and a facility-side system 3 provided in each facility A serving as a shooting site. The plurality of mobile terminals 2 are configured to be able to communicate with each other via a communication network NW such as the Internet or a mobile phone network. On the other hand, each facility-side system 3 is configured to be able to communicate with the mobile terminal 2 in each facility A. Each facility A is assumed to be a facility that plays some kind of music, such as an amusement facility, a club or a shopping mall. That is, it includes at least a facility that plays music as BGM (Back Ground Music) and a facility that plays music as an acoustic performance such as DJ (Disc Jockey) play.

  The facility-side system 3 includes an audio reproduction device 11 that reproduces music and a distribution server 12 (local server) that distributes music identification information of the music. The audio playback device 11 includes, for example, a personal computer, an amplifier, and a speaker, and plays back music and emits music from the speaker. As a result, music is played in the facility A.

  The distribution server 12 distributes music identification information (information for identifying music) of the music to the mobile terminal 2 in the facility A in synchronization with the sound emission of the music into the facility A. Specifically, the music identification information of the music being emitted is distributed to the mobile terminal 2 in the facility A by wireless LAN communication such as WiFi (registered trademark) communication. Here, as the music identification information, the music metadata (including at least the title T and the artist name) and the URL (Uniform Resource Locator) of the music purchase site S of the music are distributed.

  As shown in FIG. 2, each mobile terminal 2 is configured by a smartphone or the like, and includes a communication interface 21, a liquid crystal display 22, a touch panel 23, an imaging camera 24, a speaker 25, a memory 26, and these. And a control mechanism 27 for controlling.

  The communication interface 21 transmits / receives data to / from other portable terminals 2 via the communication network NW, and acquires the music identification information distributed from the distribution server 12.

  The liquid crystal display 22 displays various information to the user, and displays a captured image G and various screens. The touch panel 23 is disposed so as to overlap the liquid crystal display 22 and is an operation device for performing various user operations on the screen of the liquid crystal display 22.

  The imaging camera 24 is configured by a camera including an image sensor such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD), and can capture (capture) still images and moving images.

  The speaker 25 outputs (sounds) various sounds and enables sound output of music. The memory 26 is a non-volatile storage medium that stores various data including the captured image G in a rewritable manner.

  The control mechanism 27 includes a CPU (Central Processing Unit) 31, a ROM (Read Only Memory) 32, and a RAM (Random Access Memory) 33. The CPU 31 is a central processing unit and performs various arithmetic processes. The ROM 32 stores a control program for the CPU 31 to perform various arithmetic processes, and the RAM 33 is used as a work area when the CPU 31 performs various arithmetic processes. The mobile terminal 2 executes a predetermined program stored in the memory 26 by the control mechanism 27 to transmit and capture the captured image G, and a receiving device that receives and displays the captured image G Function.

  Next, with reference to the functional block diagram of FIG. 3, each function of the portable terminal 2 concerning a music presentation operation | movement is demonstrated. Here, in the music presentation operation, the portable terminal 2 that functions as a transmission device will be described as a transmission-side portable terminal 2a, and the portable terminal 2 that functions as a reception device will be described as a reception-side portable terminal 2b. In the present embodiment, it is assumed that each mobile terminal 2 can function as both the transmission-side mobile terminal 2a and the reception-side mobile terminal 2b. Whether it becomes the side portable terminal 2a or the reception side portable terminal 2b changes appropriately.

  As shown in FIG. 3, the transmission-side portable terminal 2 a includes an image photographing unit 41, a music information acquisition unit 42, an image storage unit 43, and an image transmission unit 44. The image capturing unit 41 acquires a captured image G by capturing a still image (photograph) in the facility A with the imaging camera 24 as a main part. The music information acquisition unit 42 uses the communication interface 21 as a main part, and acquires music identification information distributed by the distribution server 12 by a dedicated application.

  The image storage unit 43 uses the memory 26 as a main part, and stores a shot image G obtained by shooting, and stores corresponding music identification information in addition to the shot image G. That is, the image storage unit 43 stores each photographed image G in association with music identification information distributed at the time of photographing.

  The image transmission unit 44 uses the communication interface 21 as a main part, and transmits the captured image G and the music identification information added to the captured image G to the reception-side portable terminal 2b. The image transmission unit 44 transmits the photographed image G and the music identification information by a mail function or SNS (social networking service). That is, the configuration may be such that the photographed image G and the music identification information are uploaded to an upload server, and the received portable terminal 2b downloads them to transmit them.

  The receiving-side portable terminal 2b includes an image receiving unit (receiving unit) 51, an image display unit 52, a music presence / absence determination unit 53, a purchase site display unit 54, and a music playback unit 55. The “operation unit” in the claims includes an image display unit 52, a purchase site display unit 54, and a music playback unit 55.

  The image receiving unit 51 receives the captured image G and the music identification information with the communication interface 21 as a main part. That is, the image receiving unit 51 receives the photographed image G and the music identification information added to the photographed image G transmitted from the transmission-side portable terminal 2a. Similar to the image transmission unit 44, the image reception unit 51 receives the photographed image G and the music identification information by the mail function or SNS. That is, the transmission side portable terminal 2a may receive the captured image G and the music identification information uploaded to the upload server by downloading them.

  The image display unit 52 uses the liquid crystal display 22 as a main part to display the photographed image G and displays the title T of the music based on the music identification information superimposed on the photographed image G (see FIG. 4A). . That is, the title T included in the metadata of the music identification information is extracted, and a composite image in which the title T is superimposed on the lower part of the captured image G is generated and displayed. As shown in FIG. 4A, in addition to the title T, a reproduction mark M indicating that the tap operation on the photographed image G is a reproduction operation is displayed superimposed on the photographed image G.

  The music presence / absence determination unit 53 uses the control mechanism 27 as a main part, and determines whether or not the user of the reception-side mobile terminal 2b holds the music based on the music identification information (whether or not it has been purchased). That is, the music presence / absence determination unit 53 searches the memory 26 based on the metadata of the music identification information, and determines whether or not the audio data of the target music is stored in the memory 26.

  The purchase site display unit 54 displays the music purchase site S based on the music identification information with the liquid crystal display 22 as a main part (see FIG. 4B). That is, the purchase site display unit 54, based on the URL of the music purchase site S of the music identification information, when the music presence determination unit 53 determines that the user does not have the music based on the music identification information. The music purchase site S on the web is displayed.

  The music playback unit 55 plays back (sound outputs) music based on the music identification information with the speaker 25 as a main part. That is, the music playback unit 55, when the music presence / absence determination unit 53 determines that the user has a music based on the music identification information, the music playback unit 55, based on the searched audio data of the target music, Play.

  Here, with reference to FIG. 5, the music presentation operation | movement by the music presentation system 1 is demonstrated. As shown in FIG. 5, first, the transmitting-side portable terminal 2a captures a still image by the image capturing unit 41 and acquires a captured image G in accordance with a user operation (imaging operation) (S1). When the captured image G is acquired, the music information acquisition unit 42 acquires the music identification information distributed from the distribution server 12 (acquires the latest information) (S2). Thus, the photographed image G and the music identification information are acquired due to the photographing operation. Then, the acquired photographed image G and music identification information are stored (recorded) in the image storage unit 43 (S3). At this time, the acquired music identification information is added to the acquired photographed image G and stored.

  Next, the transmission-side portable terminal 2a transmits the photographed image G and the music identification information by the image transmission unit 44 according to the user operation (transmission operation) (S4). That is, the captured image G and music identification information stored in the image storage unit 43 are transmitted to the receiving-side mobile terminal 2b.

  On the other hand, the reception-side portable terminal 2b receives the photographed image G and the music identification information transmitted from the transmission-side portable terminal 2a by the image reception unit 51 (S5: reception step). When the photographed image G and the music identification information are received, the received photographed image G is displayed by the image display unit 52 according to the user operation (display operation), and the title T of the music based on the received music identification information is displayed. (S6). Thereafter, in a state in which the captured image G is displayed, the state shifts to a waiting state for a tap operation on the captured image G (S7). As described above, the tap operation on the captured image G is a music reproduction operation by the user.

  When the user performs a tap operation on the photographed image G (S7: Yes), the music presence / absence determination unit 53 determines whether or not the user owns the music based on the received music identification information ( S8). When the music presence / absence determination unit 53 determines that the user does not have the music (S8: No), the reproduction of the music is canceled, and the purchase site display unit 54 displays the music purchase site S of the music. Is displayed (S9).

  On the other hand, when the music presence / absence determining unit 53 determines that the user has the music (S8: Yes), the music reproducing unit 55 reproduces the music (S10). Thus, the present music presentation operation is terminated. As described above, in the present music presentation operation, while displaying the photographed image G, the music title T based on the music identification information is displayed as the music presentation operation based on the music identification information, and the music purchase site based on the music identification information is displayed. S is displayed or the music based on the music identification information is reproduced (operation step).

  According to the configuration as described above, while displaying the captured image G, the music associated with the displayed captured image G is displayed or displayed by audio output, thereby improving the live feeling in the captured image G. Can do. The amount of data transmission can be reduced by exchanging the music identification information of the music instead of the audio data of the music. Therefore, the flow up to the presentation of music can be executed comfortably and smoothly. Furthermore, by not exchanging music audio data itself, copyright problems associated with transmission and reception of music audio data can be eliminated.

  Further, by performing an operation of displaying the title T of the music based on the music identification information on the photographed image G as the music presentation operation, the music can be recognized only visually.

  Furthermore, as a music presentation operation, an operation of displaying the music purchase site S based on the music identification information can be performed, and the user's willingness to purchase can be stimulated.

  Furthermore, the live feeling in the photographed image G can be improved by voice by performing the voice output operation of the music based on the music identification information as the music presentation operation.

  In addition, the transmitting-side mobile terminal 2a acquires the music identification information distributed in synchronization with the sound emission of the music by the distribution server 12, so that the music identification information of the music actually flowing at the shooting site (shooting environment) Can be added to the captured image G. Therefore, the user on the receiving side can recognize the music actually played at the shooting site while browsing the shot image G. Therefore, the live feeling in the captured image G can be further improved.

  In the present embodiment, the title T is displayed on the photographed image G, but information other than the title T in the metadata (artist name, genre, etc.) is displayed on the photographed image G. It may be a configuration. Moreover, the structure which displays a part or all of the said lyrics on the picked-up image G including the part or all of the lyrics of a music may be sufficient as metadata. Further, the display of these titles T and the like may be configured to enable display / non-display settings.

  In the present embodiment, the music purchase site S is displayed when the user does not have the target music, but the music purchase site S is displayed regardless of whether the user has the target music. It may be configured to do so. Further, although the music purchase site S is displayed when the captured image G is tapped, the music purchase site S may be displayed simultaneously with the display of the captured image G. Further, instead of displaying the music purchase site S, a configuration in which the music can be auditioned for a certain period of time may be used.

  Furthermore, in the present embodiment, when the captured image G is tapped, the target music is output as audio. However, the target music may be output as audio simultaneously with the display of the captured image G. .

  Furthermore, in the present embodiment, a local server provided for each facility A is used as the distribution server 12, and the distribution server 12 distributes the music identification information by wireless LAN communication. 12, the Internet server may be used, and the distribution server 12 may be configured to distribute the music identification information via the Internet. In such a case, the distribution of the music identification information in the plurality of facilities A can be performed by the single distribution server 12.

  In addition, when the music distribution information is distributed in a plurality of facilities A by a single distribution server 12 (Internet server), the distribution server 12 distributes the music identification information to the mobile terminal 2. It is necessary to grasp information on which facility A exists (hereinafter referred to as facility information). Here, the portable terminal 2 side selects the facility A in which the mobile terminal 2 exists, and transmits the facility information obtained by the selection to the distribution server 12. Then, when the distribution server 12 receives this, the distribution server 12 grasps the facility information. In such a configuration, the selection method of the facility A on the portable terminal 2 side is assumed to be manual selection and automatic selection. As a selection method by manual selection, for example, a method of displaying a list of facilities A on a website on the Internet and selecting the facility A from the list by a user operation is conceivable. Further, a method of selecting the facility A by displaying a map on the Web site and designating the position of the facility A on the map by a user operation is conceivable.

  On the other hand, in the selection method by automatic selection, check-in processing is performed in the facility A, and the facility A is automatically selected. For example, a method of identifying the facility A where the mobile terminal 2 exists by using the GPS mounted on the mobile terminal 2 and automatically selecting the facility A (automatic setting) is conceivable. In addition, the code authentication function installed in the portable terminal 2 performs code authentication (acquisition of facility identification information) for the QR code or the like provided in the facility A, thereby identifying the facility A in which it exists, and the facility A It is conceivable to select automatically. Furthermore, facility identification information is acquired from a reader (reader / writer) provided in the facility A by a non-contact proximity sensor such as FeliCa or NFC (both are registered trademarks) mounted on the portable terminal 2, A method of identifying the facility A in which it exists and automatically selecting the facility A is conceivable. Still further, a method of receiving facility identification information distributed by a wireless LAN such as specific WiFi or Bluetooth (both are registered trademarks), identifying the facility A where the device itself exists, and automatically selecting the facility A is conceivable. . In addition, a method of specifying the facility A where the user exists by using a check-in service in SNS such as forsquare (registered trademark) and automatically selecting the facility A can be considered.

  Although not mentioned in the above embodiment, when the distribution server 12 distributes the music identification information to the mobile terminals in the facility A in synchronism with the sound emission of the music in the facility A, audio playback is performed. It is necessary to specify information on which music is being emitted (reproduced) by the device 11 and to update the information as needed. This is done using the following method. In other words, the audio playback device 11 and the distribution server 12 are connected to a network via a router or the like, and the audio playback device 11 transmits the music identification information (only metadata here) of the music being played back in real time. Transmit to the distribution server 12. Then, the distribution server 12 receives the music identification information from the audio playback device 11 to identify information indicating which music is being emitted. This configuration is also applied to the above-described case where “single distribution server 12 (Internet server) distributes music identification information in a plurality of facilities A”. Further, when the distribution server 12 provided for each facility A is used, the distribution server 12 manages only the music flowing in the corresponding one facility. On the other hand, when a single distribution server 12 is shared by a plurality of facilities A, the distribution server 12 simultaneously manages music flowing in the plurality of facilities A.

  In the present embodiment, the music title T is displayed superimposed on the photographed image G. In addition to this, as shown in FIG. The location information R (facility name of the facility A that has become the shooting site) may be superimposed and displayed. In such a case, the transmitting-side portable terminal 2a acquires the location information and transmits location information R in addition to the captured image G and the music identification information. On the other hand, the receiving side mobile terminal 2b receives them, displays the captured image G, and displays the title T and the location information R superimposed on the captured image G.

  Moreover, in this embodiment, although it was the structure which produces | generates and displays the composite image which superimposed the music identification information on the picked-up image G with the receiving side portable terminal 2b, It may be configured to generate a composite image. In such a case, the captured image G and the music identification information are exchanged (transmitted / received) in the state of the composite image. According to such a configuration, the amount of data transmission can be further reduced.

  Furthermore, in the present embodiment, the music identification information is exchanged together with the photographed image G and the music associated with the photographed image G is presented while the photographed image G is displayed. (Sound Effect) The configuration may be such that the identification information is exchanged and the SE associated with the captured image G is presented (mainly voice output) while displaying the captured image G. For example, when there is an SE that matches the sound that can be heard at the shooting site (shooting environment), the transmission-side portable terminal 2a transmits the SE identification information of the SE to the reception-side portable terminal 2b together with the shot image G. Then, the SE on the basis of the SE identification information is reproduced (sound output) while the captured image G is displayed on the reception-side portable terminal 2b. According to such a configuration, it is possible to improve the live feeling even in the shot image G shot at the shooting site where music is not flowing. If it is determined whether or not music is flowing, and it is determined that music is flowing, exchange of music identification information and presentation of music are performed, and if it is determined that music is not flowing Further, the configuration may be such that SE identification information is exchanged and SE is presented.

  Furthermore, instead of the music identification information, music data such as MIDI data may be exchanged. In such a case, the receiving-side portable terminal 2b receives the performance data added to the captured image G together with the captured image G, and reproduces the music based on the performance data while displaying the captured image G.

  2a: transmitting side mobile terminal, 2b: receiving side mobile terminal, 51: image receiving unit, 52: image display unit, 54: purchase site display unit, 55: music playback unit, G: photographed image, S: music purchase site, T: Title

Claims (10)

  A transmitting device,
  An image capturing unit for acquiring captured images;
  A music information acquisition unit that acquires music identification information of the music distributed by the distribution server in synchronization with sound emission of the music into the facility where the transmission device exists;
  A transmission unit that transmits the photographed image and the music identification information added to the photographed image to a receiving device;
  A transmission device comprising:   Means for transmitting facility information obtained by selection of the facility to the distribution server;
  The transmission apparatus according to claim 1, wherein the music information acquisition unit acquires the music identification information distributed by the distribution server based on the facility information.   The transmission apparatus according to claim 2, further comprising means for receiving a user operation for selecting the facility.   The transmission apparatus according to claim 2, further comprising means for selecting the facility by performing a check-in process in the facility.   A distribution server that receives the music identification information of the music from a sound reproducing device that emits the music in the facility, and distributes the music identification information to the transmission device in the facility in synchronization with the sound emission of the music.   The music identification information of the music is received from a plurality of sound reproducing devices that emit music in each of the plurality of facilities, and the transmission device existing in any of the plurality of facilities is transferred to the facility where the transmission device exists. A distribution server that distributes the music identification information in synchronization with sound emission of the music.   A method for controlling a transmitting device, comprising:
  Acquiring a captured image;
  Obtaining the music identification information of the music distributed by the distribution server in synchronization with the sound emission of the music into the facility where the transmission device exists;
  Transmitting the photographed image and the music identification information added to the photographed image to a receiving device;
  A control method for a transmission device, including:   Receiving the music identification information of the music from an audio playback device that emits the music in the facility;
  Delivering the music identification information to a transmitting device existing in the facility in synchronization with sound emission of the music;
  Including a distribution server control method.   A program for causing a computer to function as a transmission device,
  A function to acquire a captured image;
  A function of acquiring music identification information of the music distributed by the distribution server in synchronization with sound emission of the music into the facility where the transmission device exists;
  A function of transmitting the photographed image and the music identification information added to the photographed image to a receiving device;
  A program that makes a computer realize.   A program for causing a computer to function as a distribution server,
  A function of receiving music identification information of the music from an audio playback device that emits the music in the facility;
  A function of delivering the music identification information to a transmitting device existing in the facility in synchronization with sound emission of the music;
  A program that makes a computer realize.

Images