KR102171479B1 - Method and system for digital audio co-play service - Google Patents

Abstract

Disclosed are a digital sound co-play service method and a system therefor. According to one aspect of the present invention, the digital sound co-play service method performed by a mobile device comprises the steps of: receiving external sound and recording the external sound for a predetermined period of time; transmitting sound information corresponding to the recorded sound to a service device; receiving searched information from a service server searching for original sound and a playback section corresponding to the sound information; and playing the original sound from a playback point corresponding to the playback section.

Description

Method and system for digital audio co-play service

The present invention relates to a method and system for a joint playback service for digital audio output from a personal mobile device.

Recently, as the demand for multimedia has increased rapidly, various services for listening to or viewing MP3 music and video in a download or streaming method using mobile devices carried by users are provided.

However, since a mobile device carried by a user has a size of a size that can be held in a hand, there is a limitation in the size of a speaker and thus there is a hardware limitation in volume and sound quality. If multiple listeners are listening to music on a single mobile device, those far away will not hear well. There are many users who use an external speaker with a Bluetooth communication function to supplement speaker performance of such mobile devices. In this case, there is an inconvenience of carrying an external speaker, and there is a limit to the volume due to a size limitation due to portability.

Korean Patent Registration No. 10-1925217 (Registration date November 28, 2018) Gachang expression transplant system

Accordingly, the present invention has been conceived to solve the above-described problem, and is to provide a digital audio joint playback service method and system in which a plurality of mobile devices can simultaneously reproduce digital audio and listen to it together.

In addition, the present invention is to provide a digital audio joint playback service method and system that synchronizes playback times to minimize sound ringing.

Other objects of the present invention will become more apparent through preferred embodiments described below.

According to an aspect of the present invention, there is provided a digital audio joint playback service method performed in a mobile device, the method comprising: receiving external audio and recording for a predetermined time period; Transmitting audio information corresponding to the recorded audio to the service device; Receiving the searched information from the service server searching for an original audio and a reproduction section corresponding to the audio information; And reproducing the original audio from a playback time corresponding to the playback section, and a recording medium on which a program for executing the method is recorded.

Here, the playback time may be determined in consideration of the time taken to receive the searched information from the service server after recording is completed.

Further, the reproducing of the original audio may include receiving and recording external audio while lowering the output volume to a predetermined level or lower for a predetermined period of time; Comparing the recorded audio and the output audio; And changing the reproduction section according to the comparison result.

In addition, when the measured location information is transmitted to the service server and location information for another mobile device that outputs the original audio corresponding to the location information is received from the service server, the location information for the other mobile device The difference between the and the measured location information can be further used to determine the playback time.

In addition, the audio information may be DNA data, which is characteristic information by analysis of the recorded audio.

And, in the digital audio joint playback service method performed in a service server coupled through a mobile device and a communication network, from at least one first mobile device that reproduces digital audio, playback audio identification information, playback time information, device time information and location Receiving and registering audio reproduction information including information about a; When a joint playback service is requested from a second mobile device, searching for audio playback information corresponding to the location information of the second mobile device based on a location; Calculating a current playback time based on the retrieved audio playback information using the current time, the playback time information, and device time information; And providing information on the calculated current playback time and playback audio to the second mobile device so that the second mobile device reproduces the digital audio from a playback time corresponding to the current playback time. An audio joint reproduction service method and a recording medium in which a program for executing the method is recorded are provided.

Here, the separation distance may be calculated using the location information of the first mobile device and the location information of the second mobile device, and the separation distance may be reflected in the calculation of the current playback time.

In addition, at least one of a device time difference corresponding to a model of each of the first mobile device and the second mobile device and a time required for data communication with the second mobile device may be reflected in the calculation of the current playback time.

According to another aspect of the present invention, when reproducing digital audio, a first mobile device for generating and uploading audio reproduction information including reproduction audio identification information, reproduction time information, device time information, and location information; A service server that registers and stores the audio reproduction information; And a second mobile device that accesses the service server and requests a joint playback service for the digital audio, wherein the service server receives audio playback information corresponding to the location information of the second mobile device based on the location. Based on the searched and retrieved audio reproduction information, the current time, the reproduction time information, and the device time information are used to calculate the current reproduction time, and the second mobile device reproduces the digital audio corresponding to the current reproduction time. A digital audio joint playback service system is provided that provides information on the calculated current playback time and playback audio to the second mobile device so as to be played back from the beginning.

Here, the service server may calculate a separation distance using the location information of the first mobile device and the location information of the second mobile device, and reflect the separation distance in the calculation of the current playback time.

In addition, when the second mobile device cannot obtain related information corresponding to the joint playback service from the service server, the second mobile device receives external audio and records it for a predetermined period of time, and provides audio information corresponding to the recorded audio to the service. The original audio corresponding to the audio information and the searched information may be received from the service server that searched for a playback section, and the original audio can be played from a playback time corresponding to the playback section.

In addition, the service server transmits control information corresponding to a command from the first mobile device to the second mobile device, and the second mobile device may control reproduction or output of audio according to the control information. .

According to the present invention, digital audio can be simultaneously played by a plurality of mobile devices so that several people can listen to it together.

In addition, according to the present invention, it is possible to minimize the inconvenience caused by the sound by synchronizing the temporal location of the playback in a manner based on the location of the mobile device, taking into account the device processing time and the communication time.

1 is a block diagram schematically showing a digital audio joint playback service system according to an embodiment of the present invention.
2 is a flowchart illustrating a process of providing a joint playback service by analyzing recorded audio according to an embodiment of the present invention.
3 is a flowchart illustrating a process of adjusting a playback sync performed in a mobile device according to an embodiment of the present invention.
4 is a flowchart illustrating a location-based audio joint playback service process performed in a service server according to an embodiment of the present invention.
5 is a table illustrating audio reproduction information managed by a server for a location-based audio joint reproduction service according to an embodiment of the present invention.

In the present invention, various modifications may be made and various embodiments may be provided, and specific embodiments will be illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all changes, equivalents, or substitutes included in the spirit and scope of the present invention.

When a component is referred to as being "connected" or "connected" to another component, it is understood that it may be directly connected or connected to the other component, but other components may exist in the middle. Should be. On the other hand, when a component is referred to as being "directly connected" or "directly connected" to another component, it should be understood that there is no other component in the middle.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component. For example, terms such as a first threshold value and a second threshold value to be described later may be pre-designated as threshold values that are substantially different or some of the same values, but when expressed in the same word as the threshold value, there is a confusion. There is room, so for the convenience of classification, terms such as first and second will be added together.

The terms used in the present specification are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this specification, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or a combination thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance.

In addition, components of the embodiments described with reference to each drawing are not limited to the corresponding embodiments, and may be implemented to be included in other embodiments within the scope of maintaining the technical spirit of the present invention. Even if the description is omitted, it is natural that a plurality of embodiments may be implemented again as a unified embodiment.

In addition, in the description with reference to the accompanying drawings, the same or related reference numerals are assigned to the same components regardless of the reference numerals, and redundant descriptions thereof will be omitted. In describing the present invention, when it is determined that a detailed description of related known technologies may unnecessarily obscure the subject matter of the present invention, a detailed description thereof will be omitted.

1 is a block diagram schematically illustrating a digital audio joint playback service system according to an embodiment of the present invention.

Referring to FIG. 1, the entire system according to the present embodiment includes a first mobile device 10 that reproduces digital audio, and a second user who wants to output the same audio as the first mobile device 10. A mobile device 20 and a service server 30 for providing a joint playback service are included. According to an example, the service server 30 may operate a digital audio DB 40.

The first mobile device 10 and the second mobile device 20 may be a communication terminal device such as a smart phone, and are devices having a media function for reproducing and outputting audio (including video including audio). .

The second mobile device 20 uses the service server 30 to reproduce and output the same audio as the audio being played by the first mobile device 10 located nearby. At this time, when the first mobile device 10 and the second mobile device 20 play the same audio (for example, the same music), the first mobile device 10 and the second mobile device 20 synchronize with the playback sync (that is, the playback time, which is the temporal position at which they are played). ) Is important. The playback synchronization method will be described in detail later with reference to a related drawing (FIG. 2 or 3).

The first mobile device 10 or the second mobile device 20 can reproduce digital audio in a streaming method or a download method, and the audio providing service (hereinafter referred to as sound source service) may be provided by a separate server (not shown). In the following description, for convenience of explanation, the service server 30 provides a sound source service together using the digital audio DB 40. For example, the service server 30 provides a sound source service to a mobile device, and also provides a joint playback service for a corresponding sound source.

2 is a flowchart showing a process of providing a joint playback service by analyzing recorded audio according to an embodiment of the present invention, and FIG. 3 is a playback sync control performed in a mobile device according to an embodiment of the present invention. It is a flow chart showing the process.

Referring to FIG. 2 according to an example, the second mobile device 20 receives external audio and records for a predetermined time (eg, 1 to 3 seconds, etc.) (S210). For example, a user who wants to use the joint playback service for music heard in the surroundings uses his or her mobile device (which becomes the second mobile device 20), and the second mobile device 20 is a microphone, etc. It is to record external sound using the sound source input means.

The second mobile device 20 generates audio information corresponding to the recorded audio (S220). According to an example, characteristic information (hereinafter referred to as DNA data) is read by analyzing the frequency and waveform of the recorded audio. Since extracting and using feature information from any information is used in various fields (fingerprint recognition, image filtering, etc.), the concept of extracting unique DAN data that is characteristic in audio will be apparent to those skilled in the art, and all methods are It will be more apparent through the following description that the same can be applied.

The second mobile device 20 transmits audio information including feature information to the service server 30 and requests a joint playback service (S230).

The service server 30 requested for the joint playback service searches and recognizes the original audio having DNA data according to the received audio information in the digital audio DB 40 provided, and searches for a corresponding playback section in the searched original audio. (S240). For example, it is recognized that the original audio is music of A, and the corresponding DNA data is a playback section of music A of [01:30:012 ~ 01:33:018] up to the millisecond unit.

The service server 30 provides the searched information (original audio identification information, reproduction section information, etc.) to the second mobile device 20 (S250).

Then, the second mobile device 20 reproduces the original audio from a playback time corresponding to the playback section (S260). For example, music A is played with [01:33:019] as the playback time, because the last time of the playback section of the recorded audio is [01:33:018], so will be.

At this time, according to an example, the delay time (for example, communication time with the service server 30, search time in the service server 30, playback in the device itself) at [01:33:018], which is the last time of the playback section. The playback time may be determined in consideration of the processing time and the like). For example, if the delay time is 10 milliseconds, [01:33:29] may be the playback time. That is, the second mobile device 20 further considers a playback section and a time taken to receive original audio information from the service server 30 after recording is completed to determine a playback time point.

In addition, referring to FIG. 3 according to an example, when the second mobile device 20 starts to play the original audio, the second mobile device 20 lowers the output volume to a certain level or less for a certain period of time (for example, 3 seconds, etc.). It receives the input and records (S310).

Then, the recorded audio and the output audio are compared (S320).

The playback section is changed according to the comparison result (S330). For example, if the output audio is later than the recorded audio, the playback position is delayed by that amount. Even at this time, the change time of the playback section can be determined in consideration of the delay time required for output and the delay time required for recording processing.

Further, although not shown in the drawing, according to another example, the second mobile device 20 transmits the measured location information to the service server 30, and outputs the original audio corresponding to the location information from the service server 30 When receiving the location information of the first mobile device 10, which is another mobile device, the difference between the location information of the first mobile device 10 and the measured location information of the first mobile device 10 is further used to determine the playback time. That is, when the second mobile device 20 knows the separation distance from the first mobile device 10, the second mobile device 20 determines the playback time in consideration of the delay time of the sound transmitted by the distance. That is, by determining the playback time in consideration of the transmission time of the sound (that is, calculated in consideration of the speed of the sound), the user of the second mobile device 20 transmits the audio according to the same playback section at about the same time. You will be able to listen through the device.

According to the joint playback service described so far, the user of the second mobile device 20 can recognize the recognition target audio regardless of the device or application program being played, and can play the same audio on his device. There is this.

4 is a flowchart showing a location-based audio co-play service process performed by the service server 30 according to an embodiment of the present invention, and FIG. 5 is a flow chart showing a location-based audio co-play service according to an embodiment of the present invention. This table is an example of audio reproduction information managed by the server.

First, referring to FIG. 1, a method of registering information on audio being played by the first mobile device 10 in the service server 30 in advance may be used.

That is, when reproducing digital audio, the first mobile device 10 generates and uploads audio reproduction information including reproduction audio identification information, reproduction time information, device time information, and location information.

Then, the service server 30 searches for audio playback information corresponding to the location information of the second mobile device 20 based on the location, and the current time and playback time information and device time information based on the found audio playback information. The current playback time is calculated using, and the calculated information on the current playback time and the playback audio is calculated so that the second mobile device 20 plays the digital audio from the playback time corresponding to the current playback time. 20).

Referring to FIG. 4 showing an embodiment according to this, the service server 30 provides information on playback audio identification information, playback time information, device time information and location from one or more first mobile devices 10 that reproduce digital audio. Audio reproduction information including information is received and registered (S410).

Referring to FIG. 5 showing information registered in the service server 30, location information, playback audio identification information, playback time information, and device time information of the first mobile device 10 may be registered as audio playback information. Also, as shown in the drawing, the model information of the first mobile device 10 may be further registered. This is because device time information may be different for each mobile device type, and data processing speed may be different, and accordingly, the playback time may be calculated differently.

Referring to FIG. 4 again, when the service server 30 receives a request for a joint playback service from the second mobile device 20 (S420), the service server 30 plays audio corresponding to the location information of the second mobile device 20 based on the location. Information is searched (S430). That is, to search for audio reproduction information registered by the first mobile device 10 located in the vicinity of the second mobile device 20, and if there are a plurality of pieces within a certain distance (eg, 100 meters), 2 It will be possible to provide all of them to the mobile device 20 so that they can select one.

The service server 30 calculates the current playback time using the current time, registered playback time information, and device time information based on the retrieved audio playback information (S440). For example, if the current time is 30 seconds later than the device time information, the current play time may be a time when 30 seconds are added to the play time information (this may be reflected when the server time and the device time are different).

Here, the service server 30 calculates the separation distance using the location information of the first mobile device 10 and the location information of the second mobile device 20, and calculates the separation distance calculated as described above at the current playback time. Can be reflected in the calculation of.

In addition, the device time difference corresponding to each model of the first mobile device 10 and the second mobile device 20, or the time required for data communication with the second mobile device 20 may be reflected in the calculation of the current playback time. I can.

Then, the service server 30 provides the calculated information on the current playback time and playback audio to the second mobile device 20 (S450).

Then, the second mobile device 20 reproduces the corresponding digital audio from a reproduction time corresponding to the current reproduction time. For example, based on the current playback point, the playback point is determined by reflecting the above-described delay time, and the digital audio is reproduced from the playback point.

According to the present embodiment, there is an advantage that audio that only individuals listen to, such as a place with heavy ambient noise and earphones, can be enjoyed in the same manner without misrecognition.

Here, although not shown in the drawing, according to an example, when the service server 30 fails to search for audio reproduction information corresponding to the location of the second mobile device 20 (ie, the first mobile device at the corresponding location ( If there is no information previously registered by 10)), the second mobile device 20 may use an external audio analysis method as illustrated in FIG. 2.

That is, when the second mobile device 20 cannot obtain related information corresponding to the joint playback service from the service server 30, the second mobile device 20 receives the external audio and records it for a predetermined time, and the audio corresponding to the recorded audio The information is transmitted to the service device, the original audio corresponding to the audio information and the searched information are received from the service server 30 that has searched for the reproduction section, and the original audio is reproduced from the reproduction time corresponding to the reproduction section.

According to the embodiment of FIG. 4, since the service server 30 recognizes the location of each mobile device, it is possible to provide various location-based audio playback services.

According to an example, based on the location of each mobile device using a joint playback service that is playing the same digital audio together, not only control the volume output from each mobile device, but also control whether to play or not (for example, the service server 30 ) Will be able to remotely control each mobile device through the network with control.

According to a specific example, based on the first mobile device 10 that initially outputs the original audio, the second mobile devices 20 located on the left are configured to output the bass sound higher, and the second mobile device located on the right ( 20) may perform control such as outputting by applying a specific effect, and this may be performed by a command from the first mobile device 10.

Although the above has been described with reference to preferred embodiments of the present invention, those of ordinary skill in the relevant technical field can variously modify the present invention within the scope not departing from the spirit and scope of the present invention described in the following claims. It will be appreciated that it can be modified and changed.

10: first mobile device
20: second mobile device
30: service server
40: Digital audio DB

Claims (13)

In the digital audio joint playback service method performed in a mobile device,
Receiving external audio and recording for a predetermined time;
Transmitting audio information corresponding to the recorded audio and the measured location information to a service server;
The service server identifies the original audio and playback section by analyzing the audio information, and checks the location information of another mobile device, which is the location information previously transmitted by another mobile device that outputs the original audio corresponding to the location information. Receiving information on the playback section and location information of the other mobile device from the service server;
After the recording of the external audio is completed, the time taken to receive the information on the playback section from the service server and the delay time according to the separation distance between the location information of the other mobile device and the measured location information are in the playback section. Applying and calculating a reproduction time point; And
And reproducing the original audio from the calculated reproduction time point.
delete The method according to claim 1,
Reproducing the original audio,
Receiving and recording external audio while lowering the output volume below a predetermined level for a predetermined time;
Comparing the recorded audio and the output audio; And
And changing a reproduction period according to the comparison result.
delete The method according to claim 1,
The audio information is DNA data, which is characteristic information by analysis of recorded audio.
A recording medium containing instructions executable by a computer, such as an application or program module executed by a computer for performing the method of claim 1.
delete delete delete When playing digital audio, a first mobile device that generates and uploads audio reproduction information including information on reproduction audio identification information and location;
A service server that registers and stores the audio reproduction information; And
A joint playback service for digital audio by receiving the audio output from the first mobile device and recording it for a predetermined period of time, and transmitting audio information corresponding to the recorded audio and measured location information to the service server. Including a second mobile device requesting,
The service server identifies the original audio and the playback section by analyzing the audio information, checks the location information of the first mobile device outputting the original audio corresponding to the location information of the second mobile device, and plays the playback Providing section information and location information of the first mobile device to the second mobile device,
The second mobile device plays the delay time according to the distance between the location information of the first mobile device and the location information of the first mobile device and the time taken to receive the information on the playback section from the service server after the recording is completed. A digital audio joint playback service system for calculating a playback time by applying it to a section and playing the original audio from the calculated playback time.
delete delete The method of claim 10,
The service server transmits control information corresponding to the command from the first mobile device to the second mobile device,
The second mobile device controls audio reproduction or output according to the control information.

Images