RU2783261C1 - Method and system for information exchange between devices - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: technical solution generally relates to computing and, more specifically, to systems and methods for implementing the exchange of information between two or more mobile computing devices and/or other computing systems. This communication method allows two or more devices to pinpoint each other's location using the built-in microphone to analyze small samples of the surrounding sound space and generate a hash. These hashes are sent back to the cloud database where they are compared and matched against other hashes, and if there is a match, the devices can immediately exchange whatever information they wish. The hash can be done both on the device and on the server. Numerous open source and commercial perceptual hashing algorithms can be used to sample and match audio hashes.

EFFECT: technical result is an increase in the efficiency of the rapid exchange of financial information between mobile devices, terminals and other gadgets.

8 cl, 5 dwg

Description

FIELD OF TECHNOLOGY

[001] The present technical solution generally relates to computing and, more specifically, to systems and methods for implementing the exchange of information between two or more mobile computing devices and/or other computing systems.

BACKGROUND OF THE INVENTION

[002] In today's interconnected world, mobile applications are experiencing explosive growth, and communication between users' mobile devices is becoming increasingly important. For various mobile applications, it is necessary to maximize the speed and efficiency of information exchange between mobile devices, terminals and other gadgets, as the world becomes more and more interconnected. The exchanged information can be different, ranging from the contact details of users, their photos, videos, files, ending with financial transactions and other commercial data. On the other hand, there are still many obstacles and unresolved problems in this exchange of information, especially when interacting mobile devices use different operating systems.

[003] It will be appreciated by one of ordinary skill in the art, in view of the foregoing and other shortcomings of conventional technology, that new and improved systems and methods for implementing information exchange between two or more mobile computing devices and/or other computing systems will be highly sought after in the market.

[004] Existing ways of exchanging financial information can be divided according to the technologies used: NFC, dynamic QR, ultrasound and manual data entry. This technical has significant advantages over each of them. NFC technology as a payment instrument is limited geographically by device manufacturers and operating system developers, and also requires a specialized module, which is not available in many smartphones, for the technology to work. This invention requires only standard smartphone modules, which makes it possible to use it on absolutely any smartphone and without territorial restrictions. The technology of information exchange using dynamic QR requires a special display on the payment terminal or cash register for the technology. This invention is sufficient to have a microphone, which is often built into modern payment terminals, and also, its cost is much cheaper than the display. The main disadvantage of the ultrasonic method of information exchange is the instability of its work, because The quality of data transfer is affected by many factors, such as the location of devices relative to each other, the presence of sound-reflecting objects, the smartphone model and OS settings. Manual data entry is significantly inferior to this technical solution in terms of ease of use and data exchange time.

ESSENCE

[005] The technical problem or technical problem solved in this technical solution is the implementation of a system and a method for ensuring the exchange of information between devices.

[006] The technical result is an increase in the efficiency of the rapid exchange of financial information between mobile devices, terminals and other gadgets. Without the use of this technical solution in such an exchange, many obstacles and problems remain in the prior art, especially if these devices use different operating platforms (operating systems).

[007] This technical result is achieved due to the following factors:

• successful operation of the technology does not require additional equipment (Bluetooth, NFC, geolocation, etc.), except for a microphone and access to the Internet;

• sound recording and typos are made on the device, only the hash gets into the cloud, which cannot be converted back into playable audio, guaranteeing privacy.

• fingerprints are compared between hashes generated in short time intervals (from 1 to 10 seconds), which allows you to safely pair mobile devices with terminals and cash registers, and make it impossible to change the audio environment through remote access.

[008] The specified technical result is achieved through the implementation of a computer-implemented method that runs on a computerized system containing a mobile audio capture device, a central processor, a display device and a memory, and the computer-implemented method includes capturing an audio signal based on a predetermined criterion using a mobile user audio capture devices, wherein the audio signal comprises an ambient sound sample in the ambient sound space of the mobile device; using the CPU to generate a hash function of the captured audio signal; transmitting the hash of the captured audio signal to the server; performing by the server comparing the hash of the captured signal and at least one hash stored in the cloud database, and create the first long audio signature; verifying the candidate audio track by bit by bit comparing the hash of the long audio signature of the captured audio signal with the hash of the long audio signature stored in the cloud database and available to the server using the verification index; receiving a comparison result from a server identifying a second user device, the second device being associated with a similar audio signal hash similar to the audio hash obtained from the ambient sound sample in the second device's ambient sound space; and authorizing the transfer of financial data between the mobile audio capture device and the second device associated with the same hash of the audio signal, based on the result of the comparison.

[009] In some implementations, the predetermined criterion is the presence of an audio signal.

[0010] In some implementations, the predetermined criterion is the volume of the captured audio signal that is above a predetermined threshold.

[0011] In some embodiments, when the audio signal does not meet the predetermined criteria, random audio is generated using an audio generating device.

[0012] In some embodiments, the implementation further includes transmitting the captured audio signal to a remote server, wherein the hash is generated at the remote server.

[0013] In some embodiments, the remote server is a cloud server.

[0014] In some embodiments, the ambient sound space of the second device is located in a different geographic location than the ambient sound space of the audio capture device.

[0015] In some embodiments, the server contains a database that stores a plurality of hashes of the audio signal.

[0016] The method of claim 1, further comprising performing a data communication with the second device based on the identifier of the second device.

[0017] In some embodiments, the hash of the captured audio signal and the similar hash of the audio signal comprise perceptual hashes.

DETAILED DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings, which are included in and form part of this specification, illustrate embodiments of the present invention and, together with the description, serve to explain and illustrate the principles of operation of the technical solution.

[0019] FIG. 1 illustrates an exemplary embodiment of a computerized system for exchanging information between mobile computing devices.

[0020] FIG. 2 illustrates another alternative embodiment of a computerized system for exchanging information between mobile computing devices.

[0021] FIG. 3 illustrates an exemplary flowchart of an embodiment of a method performed by a computerized system according to the invention for exchanging information between the mobile computing devices shown in FIG. 1 and 2.

[0022] FIG. 4 illustrates an exemplary embodiment of a computerized mobile system that can be used as a user's mobile computing device in conjunction with the computerized system for exchanging information between the mobile computing devices shown in FIG. 1 and 2.

[0023] FIG. 5 illustrates an exemplary computerized server system that can be used, for example, as a server of the computerized system according to the invention for exchanging information between the mobile computing devices shown in FIG. 1 and 2.

DETAILED DESCRIPTION OF THE INVENTION

[0024] As used herein, a "server" is a computer program that runs on appropriate hardware and is capable of receiving requests (eg, from devices) over a network and fulfilling or causing those requests to be executed. The hardware may be one physical computer or one physical computer system, but neither is required for the present technology. In the present context, the use of the term "server" does not imply that every task (e.g. received instructions or requests) or any particular task will be received, executed or called for execution by the same server (i.e. the same software software and/or hardware); this expression implies that any number of software elements or hardware devices can be involved in receiving / sending, executing or calling to perform any task or request, or the consequences of any task or request; and all of this software and hardware may be a single server or multiple servers, both of which are included in the expression "at least one server".

[0025] As used herein, a "device" is any computer hardware that is capable of executing software that is appropriate for the relevant task at hand. Thus, some (non-limiting) examples of devices include personal computers (desktops, laptops, netbooks, etc.), smartphones and tablets, and network equipment such as routers, switches, and gateways. It should be noted that a device acting as a device in the present context is not excluded from being able to act as a server for other devices. The use of the term "device" does not preclude the use of multiple devices when receiving/sending, performing or causing the execution of any task or request, or the consequences of any task or request, or the steps of any method described herein.

[0026] As used herein, a "database" is any structured collection of data, regardless of its specific structure, database administration software, or computer equipment on which data is stored, implemented, or otherwise made available for use. The database may reside on the same hardware as the process that stores or uses the information stored in the database, or it may reside on separate hardware such as a dedicated server or multiple servers.

[0027] In the context of the present description, the term "information" includes information of any nature or kind that can be stored in the database in any way. Thus, information includes, but is not limited to, audiovisual works (images, films, sound recordings, presentations, etc.), data (location data, numerical data, etc.), text (opinions, comments , questions, messages, etc.), documents, spreadsheets, word lists, etc.

[0028] As used herein, unless specifically stated otherwise, the term "component" is intended to include software (as appropriate in a particular hardware context) that is both necessary and sufficient to implement a particular function(s), on which is being referenced.

[0029] As used herein, the expression "media used by a computer" is intended to include media of any kind and kind, including RAM, ROM, disks (CD-ROMs, DVDs, floppy disks, hard disk drives, etc.) etc.), USB keys, SSDs, tape drives, etc.

[0030] In the context of the present description, the words "first", "second", "third", etc. are used as adjectives only to allow the nouns they modify to be distinguished from each other, and not to describe any particular relationship between such nouns. Thus, for example, it should be understood that the use of the terms "first server" and "third server" does not imply any particular order, type, chronology, hierarchy or ranking (for example) of such servers, nor does their use (in itself ) does not mean that some "second server" must necessarily exist in any given situation. Also, as discussed elsewhere in this document, reference to a "first" element and a "second" element does not exclude that the two elements are in fact the same real world element. Thus, for example, in some cases the "first" server and the "second" server may be the same software and/or hardware, in other cases they may be different software and/or hardware.

[0031] In the following detailed description, reference will be made to the accompanying drawings, in which the same functional elements are designated by the same numbers. The above accompanying drawings are given by way of illustration and not by way of limitation, that is, specific embodiments consistent with the principles of the present invention. These implementations are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that other implementations may be used and that structural changes and/or substitutions of various elements may be made without departing from the scope and spirit of the present invention. Therefore, the following detailed description should not be construed in a limited sense. In addition, the various embodiments of the invention described may be implemented as software running on a general purpose computer, as dedicated hardware, or as a combination of software and hardware.

[0032] To address the above and other problems associated with conventional technology, one or more of the embodiments described herein implement various exemplary systems and methods for implementing information exchange between two or more mobile computing devices and/or other computing systems. . One aspect of the inventive concepts described herein provides a method for exchanging information that allows two or more computing devices, such as smartphones or other gadgets, to pinpoint each other's location using their built-in microphone and their CPU to capture and analyze small samples of information surrounding the sound space. In one or more embodiments, a hash is generated based on the captured audio sample. This generated hash is sent back to the cloud database where it is compared and matched against the other hashes stored in it. If a match is found, the devices can immediately exchange any necessary information. In various embodiments, the aforementioned hash may be generated on the mobile device itself or on a remote (eg cloud) server.

[0033] The hash identification engine, as described herein, functions to solve the problems described above by providing individual clients with real-time, server-side identification of media objects included in a streaming media broadcast. These media items include, for example, songs, commercials, musical intros, station call signs, and so on. In general, the media identification means operates by providing at least one server that provides real-time identification of media objects in a broadcast media stream to individual clients by comparing hashes computed by each client based on parts of the media stream. information taken as samples, with a database of known hashes. In general, the computed hashes are sent by each client to the server and compared, either directly or via a server-side hash cache, with a large database of stored precomputed hashes whose identities are known.

[0034] In FIG. 1 shows an exemplary embodiment of a computerized system 100 for exchanging financial information between mobile devices 101 and 102 that are spatially adjacent to each other, such as may be used to pay for services or goods. As shown in FIG. 1, the mobile audio capture device 101 includes a microphone 103 that captures an ambient sound sample that includes sound emitted by a second user device 102, which in an embodiment may be a POS terminal, for example. The captured raw audio signal 104 is then sent to the processor 105 of the mobile audio capture device 101, which creates a hashed version of it using any hashing algorithm now known or later developed. The hash of the captured audio signal 106 is then sent to the server 107 where it is compared by the server processor (see element 108 in FIG. 1) against a set of stored hashes 109. Perceptual hash algorithms describe a class of functions for generating comparable hashes. They use various image properties to construct an individual hash. In the future, these hashes can be compared with each other. If the hashes are different, then the data is different. If the hashes match, then the data is most likely the same (since there is a possibility of collisions, the same hashes do not guarantee the same data). In various embodiments, server 107 may be deployed in the cloud. The hash comparison result 110 is returned to the mobile audio capture device 101, and the identification result is displayed to the user using the user interface 111 that is generated on the corresponding display device. In various embodiments, the result 110 of the hash comparison indicates to the user the identifier of the second user device 102. Thus, the mobile audio capture device 101 can accurately identify the second user device 102 as being nearby. Thus, the mobile audio capture device 101 in the implementation example does not even need to be applied to the POS terminal 102 in order to exchange financial information and carry out a financial transaction. This identification may be used to authorize and/or initiate data communication between mobile devices 101 and 102.

[0035] During the first step of the comparison procedure, quick cloud candidate selection, the server 107 finds a candidate audio signal using the first short audio signature (incoming audio short audio signature or hash) in the pruning index. The candidate audio signal may also be an audio track that contains at least one chroma word that exactly matches the chroma word identified in the incoming audio signal.

[0036] During the second step of the comparison procedure, the verification step, the server 107 creates a first long audio signature. The first long audio signature may describe a portion of the audio signal up to the first 120 seconds of the incoming audio signal.

[0037] Thereafter, the server 107 checks the hash of the candidate audio track by bit-by-bit comparison of the long audio signature of the incoming audio signal with the hash of the long audio signature stored in the cloud database and available to the server 107 using the check index. At this stage, the server 107 can take into account the different quality of the compared audio signals, as well as the time offset between the compared audio signals.

[0038] At this stage, the server 107 excludes from the list of candidates those indexed audio signals whose length differs from the length of the incoming audio signal by a predetermined time, for example, 10 seconds, 30 seconds, a minute, and so on. An audio match is established when their audio signatures match at a predetermined level. For example, in some embodiments, bit matching in the long audio signatures of the incoming audio signal and the candidate audio signal need not be 100% matched. In such cases, the long audio signature bytes of the audio signals match 90%, 80%, 70%, and so on.

[0039] It should be noted that specific embodiments of audio signatures are not limited in any way and, in such a case, various prior art approaches to creating audio signatures can be used. An example of such an approach that can be used to generate an audio signature is provided in US Pat. No. 7,013,301 "Audio Signature System and Method".

[0040] Another approach to creating an audio signature is described in the article "How does a chromaprint work?" by Lukas Lalinsky published at https://oxyqene.sk/2011/01/how-does-chromaprint-work/. "Chromaprint" is a client library that implements a custom algorithm for extracting audio signatures from an audio source. Hyperlinks to the source code are published at https://acoustid.org/chromaprint. The source code is available for authorization under the LGPL2.1+ (Less General Public License).

[0041] Technology "Chromaprint" allows you to represent the audio signal as a spectrogram, which shows how the intensity changes over time at specific frequencies. To create a spectrogram, the audio signal is divided into many overlapping intervals, which are processed using a Fourier transform function.

[0042] In the case of a chroma print, the incoming audio signal can be converted at 11025 Hz and 4096 intervals (0.371 s) with a 2/3 overlap. The chromaprint further processes the information, converting the frequencies into notes, resulting in 12 cells, each containing a corresponding note. The result is an audio representation that is resilient to change.

[0043] Accordingly, a predefined set of 16 filters are applied to each of them, with these filters capturing differences in note intensity over time.

[0044] As will be appreciated by those skilled in the art, other methods for generating audio signatures may be used. In addition, the described methods with modifications can be used.

[0045] However, it should be understood that in some embodiments, once a particular audio signature generation method has been selected, it is used to generate both short and long audio signatures. In other embodiments, different approaches may be taken to create short and long audio signatures.

[0046] As a non-limiting example, a search can be performed using a pruning index. The pruning index is used to quickly select candidate audio signals in the database. In some embodiments, a short audio signature is stored in a pruning index that is generated for a plurality of short audio signatures.

[0047] The search can be performed as follows: first, the server 107 extracts the first multibyte sequence of the corresponding chromaword of the first short audio signature and finds the key index corresponding to the first multibyte sequence. If the chromaword of the first short audio signature looks like 5214-5255-5008-5047, then the corresponding key index is the key index 5214-5255-5008.

[0048] The set of lists of collocations corresponds to the index key. The server 107 retrieves this set of posting lists. The server 107 then extracts the subsequent byte of the corresponding chromaword of the first short audio signature and finds the sort key corresponding to the subsequent byte of the first short audio signature chromaword.

[0049] For example, if the chromaword of the first short audio signature looks like 5214-5255-5008-5047, the corresponding sort key would be the sort key (5047). Using this sort key, server 107 retrieves a list of postings for chromaword 5214-5255-5008-5047.

[0050] Then, in the next step, with the list of word positions already retrieved, the server 107 can find and select audio signals in which the chroma word 5214-5255-5008-5047 appears. As a non-limiting example, a candidate audio signal can be found by using the audio signal identifier contained in the posting list of the corresponding indexed chromaword. Since the same chromaword can be found in several different indexed audio signals, several corresponding indexed audio signals with several corresponding track IDs can be selected as candidate audio signals. Thus, for one chromaword of the first short audio signature, several candidate audio signals can be selected.

[0051] In step, the server 107 repeats the previous steps for each chromaword of the first short audio signature. When an additional candidate audio signal is determined in response to a search performed considering each next chromoword of the first short audio signature, these additional audio signals must be added to the plurality of candidate audio signals. Thus, a plurality of candidate audio tracks may comprise all audio signals, with each audio signal having at least one chroma word that matches any of the plurality of chroma words of the first short audio signature. It is possible that some candidate audio signals contain a higher or lower number of chromawords corresponding to the chromaword of the first short audio signature.

[0052] The second user device 102 operates in a substantially similar manner. Its microphone 112 picks up the ambient sound signal and converts it into electronic form. The raw audio signal 113 is sent to the processor 114, which creates a hash of the audio signal 115, which is subsequently sent to the processor 108 of the server 107. The processor compares the received hash of the audio signal 115 with the audio hashes stored in the hash store 109 and returns the comparison result 116 to the second user device 102. The comparison result is displayed to the user in the user interface 117, for example, in the form of a green checkmark, after which the financial transaction is carried out directly.

[0053] In one or more embodiments, numerous currently known, commercially available, or later developed perceptual hashing algorithms may be used to select and match hashes, including but not limited to FDMF; musicURI; jHears; Schmidt (part of Tuneprint); audio scout; OpenFP; echoprint, libFoolD; freetantrum; advertise; Last.fm AudiolD; Shazam; RoviMediaRecognitionService (part of AMGLASSO); Gracenote MusicID; Philips; (JSC and The Song Tapper.

[0054] Without limitation, the following frameworks may be used more broadly.

[0055] Open Source

• Finding hashes in Kurt Rosenfeld's FDMF.

• MusicURI, part of the Mpeg-7 Audio DB project.

• jHears is an acoustic hashing framework based on the design of FutureProofFingerPrint by Jeff Schmidt (formerly of Tuneprint). jHears is developed by Juha Heljoranta.

• Audio Scout. Based on the pHash hash library developed by the same authors. Uses the Philips Robust Hashing algorithm.

• OpenFP.

• echoprint. Audio hash solution developed by Echo Nest.

• libFoolD. Sound hash library used and developed for foosic.

[0056] Partially Open Source

• Last.fm. The client side audio hash library (fplib) is open source. Uses the "Computer Vision for Music Identification" algorithm.

[0057] Commercial software.

• Audible Magic.

• AudioLD. Developed by the Fraunhofer Institute for Integrated Circuits IIS and the Fraunhofer Institute for Digital Media Technology IDMT, currently provided by mufin GmbH.

• Shazams. Patented music hashing system currently offered directly to users over the phone.

• Rovi Media Recognition Service (formerly AMG LASSO).

• Gracenote MusicID.

• Philips. One of the first scalable audio hash algorithms.

• USC.

[0058] It should be noted that in the exemplary embodiment of the system shown in FIG. 1, the user does not need to enter any identifiers (eg, phone number, alias, etc.) into the mobile device in order to identify it to another mobile device. Appropriate mobile devices must be identified as being next to each other based on their sound environment, after which financial information can be securely sent between mobile devices via the cloud and a financial transaction can be carried out similar to ApplePay, where the technology works on NFC wireless data transmission. In this solution, the principle of the exchange of sound information for synchronization operates. In various embodiments, any device with an integrated microphone may be used in connection with the system 100, including, but not limited to, mobile phones and smart phones, banking terminals, self-service terminals, and the like.

[0059] FIG. 2 illustrates another exemplary computerized system 200 for exchanging information between mobile computing devices 101 and 102 that are spatially adjacent to each other. In this embodiment, the mobile audio capture device 101 operates in the same way as the mobile audio capture device 101 shown in FIG. 1. On the other hand, in this embodiment 200, the second user device 102 does not have the processing capability to create a hash function of the captured audio signal, as may be the case with older mobile phones. Therefore, instead of generating the hash locally, the second user device 102 sends the captured raw audio signal 201 to the server 107, which uses its more powerful processor to generate the audio hash 202 and compare the generated audio hash with the stored audio hash. The comparison results 116 are again returned to the user's second device 102 where they are displayed to the user via the user interface 117 and/or used to authorize or initiate communication between the mobile devices 101 and 102.

[0060] In various embodiments, hash storage 109 may be deployed as a database. In various embodiments, this database may be implemented on any currently known or later developed type of database management system, such as a relational database management system, including but not limited to MySQL, Oracle, SQL Server, DB2, SQL Anywhere , PostgreSQL, SQLite, Firebird, Redis, MongoDB, Hadoop, and/or MaxDB, which are well known to those skilled in the art. In an alternative embodiment, instead of hash storage 109, a cloud-based distributed database such as the Amazon Relational Database Service (Amazon RDS), well known to those skilled in the art, may also be used. The database 109 can store a large number of digital audio signals, such as 10,000, 100,000, 1,000,000, and the like. A plurality of digital audio signals may be collected and stored in the database 109 in many different ways. However, by way of example only, a variety of digital audio signals may be recorded by the people responsible for evaluating and/or crowdsourced.

[0061] FIG. 3 illustrates an exemplary flowchart of an embodiment of an implementation method executed on a computerized system according to the invention for exchanging information between the mobile computing devices shown in FIG. 1 and 2. First, in step 301 performed on the mobile audio capture device 101 or the second user device 102 of the user, ambient sound is captured using the microphone 103 or 112 of the respective mobile audio capture device 101 or 102. The captured audio signal is analyzed at step 302, and generated corresponding hash. This can be done locally on the respective mobile audio capture device 101 or 102, or remotely on the server 107 which has increased processing performance.

[0062] If silence or low-volume sounds are detected, step 303 is started on the mobile device or server, the process operation proceeds to step 308 where a random sound is generated, and the process subsequently returns to the above-mentioned step 301. Otherwise, the process proceeds to step 304, whereupon a hash of the captured audio sample is generated (on the mobile computing device or server) using one of the aforementioned hashing algorithms. Subsequently, at step 305, the generated hash is sent to the cloud server 107. This step is performed only if the previous steps are performed on a mobile computing device. At step 305, executed on the server server 107, the generated hash is used by the user to search the hash store 109 for a similar hash. If a matching hash is found in the hash store 109, the corresponding device ID (DeviceID) is obtained from the hash store, see step 307, which can be performed either on the server 107 or on the corresponding mobile computing device.

[0063] In another embodiment of the invention, database searches are accelerated by using bit vector indexing, as described in the US patent application "Multidimensional Data Object Searching Using Bit Vector Indices" , which has a filing date of June 23, 2003, and is assigned serial number 10/602,438, and the contents of which are incorporated herein by this reference. This bit vector indexing approach is also described in a publication entitled "Indexing High Dimensional Rectangles for Fast Multimedia Identifications" by John C. Piatt and Christopher J. C. Burges, published in Microsoft Technical Report MSR-TR-2003-38, dated October 28, 2003, and the contents of which are incorporated herein. description through this link.

[0064] In general, the bit vector indexing methods described in the aforementioned US patent application and related Microsoft technical report address the problem of rapidly performing point queries on multidimensional regions. Such queries are useful in the increasingly important tasks of media identification and retrieval in which different database elements have different similarity scores. As opposed to regular database indexing, which focuses on indexing for multidimensional nearest neighbor and epsilon interval queries, point query indexing for multidimensional regions provides an efficient indexing method that relies on a combination of redundancy and bit vector indexing. It has been noted that such methods provide significant performance improvements over real-world linear enumeration methods.

[0065] Negative database search results, i.e. no trace match with database hashes, are not sent back to the client. Positive results of the database lookup, i.e. the trace and hash match data, are sent back to the client to identify the media object from which the trace was computed. As noted above, a cache lookup to identify matching "footprints" is performed before a hash database lookup. Therefore, if a match is found with the "trace" stored in the cache, then no further database lookup is required, since the "trace" stored in the cache will either contain the identity of the media object associated with it, or will simply indicate, that there is no matching hash. In the case where the "trace" matches a hash in the database, any identification information associated with that matching hash will be sent back to the client.

[0066] One advantage of the server-side cache provided by the media identifier is that the cache size is relatively small, making cache lookups extremely fast. In particular, the cache size is a function of the maximum expected delay between clients receiving a point in the media stream and the number of unique media streams for which identification services are provided. Also, note that cache size is not a function of the number of users.

[0067] In particular, each particular broadcast media stream is approximately synchronized with each client that receives that particular media stream.

[0068] For this reason, there is no need to keep "traces" in the media cache for a period of time exceeding some small multiple of the maximum expected delay time (moreover, the delay time also includes the maximum expected delay per client for calculating and transmitting the "trace "to the server). In other words, this delay is simply the maximum time between the server receiving the first trace for a given part of a particular media object and the server receiving the last trace (from the slowest device) for that part of the same media object. Therefore, traces need only be cached for a short period of time, approximately a few seconds. As a result, there is no need for a large cache to store all the unique traces that come to the server.

[0069] In a related embodiment of the invention, instead of trying to determine or estimate maximum latency values for establishing cache trace lifetimes, a simpler solution is proposed which is to simply remove cache traces after they did not match any incoming "trails" for some short period of time of about one second or so. In additional alternative embodiments of the invention, other caching schemes may also be used, including, for example, the use of a fixed cache size that removes traces based on traditional oldest-used item removal logic.

[0070] In view of the foregoing summary of the invention, it should be clear that the media identification tool described herein provides a unique system and method for providing real-time media identification for a large number of concurrent client requests over synchronous media broadcast streams. . In addition to the benefits just described, other advantages of the system will become apparent from the detailed description that follows, when considered in conjunction with the accompanying drawings.

[0071] As will be appreciated by those skilled in the art, there are currently several ways to determine that two or more devices are located next to each other. All of these methods use location services (geolocation), WiFi, Bluetooth, NFC, etc., and they all have their limitations. On the other hand, the technology described here can work even in silence. If there is not enough sound to create a distinctive sound environment, the described device can play a melody or a ringtone to interfere with the sound and find a match. It can also be used for the user to speak a few words into the microphone with two or more devices in close proximity to each other.

[0072] In one or more implementations, the initiation of hash recording as described above may vary: the described technique may be initiated by the device's gyroscope (shaking or tilting the device), by pressing a button in a specially designed application running in the background, and any other suitable way. In one or more implementations, enabling location services as well as Geo IP and Mac address significantly narrows the search radius and speeds up the process of determining the geographic proximity of devices.

[0073] A description will be given below in an implementation with an artificial intelligence algorithm, that is, a neural network.

[0074] Generally speaking, a defined neural network (hereinafter - NN) consists of an interconnected group of artificial "neurons" that process information using a connectionist approach to computing. NNs are used to model complex relationships between inputs and outputs (without actually knowing those relationships) or to look for patterns in data. NNs are first trained during a training phase during which they are provided with some known set of "inputs" and information to adapt the NN to generate the proper output (for some particular situation they are trying to model). During this learning phase, this NN adapts to the situation being learned and changes its structure so that the given NN can provide reasonable predictive output for certain inputs during some new situation (based on what has been learned). Thus, instead of trying to define complex statistical schemes or mathematical algorithms for some specific situation, a defined NN tries to give an "intuitive" answer based on the "perception" of the situation. Thus, a certain NN is a kind of trained "black box" that can be used in a situation where what is in this "box" is not important; what matters is that the "box" gives reasonable answers to certain inputs.

[0075] NNs are commonly used in many of these situations in which it is important to know only the output based on some specific input, but how that output was obtained is of little or no importance. For example, NNs are commonly used to optimize the distribution of web traffic between servers and data processing, including filtering, clustering, signal separation, compression, vector generation, speech recognition, and the like.

[0076] It should be understood that NNs can be classified into different classes of NNs. One of these classes contains Recurrent Neural Networks (RNNs). These particular NNs are adapted to use their "internal states" (stored memory) to process input sequences. This makes RNNs well suited for tasks such as non-segmented handwriting recognition and speech recognition. These internal states of the RNN can be manipulated and are referred to as gated states or gated memories.

[0077] It should also be noted that RNNs themselves can also be classified into various subclasses of RNNs. For example, RNNs can be implemented as Long Short Term Memory Networks (LSTMs), Managed Recurrent Units (GRUs), Bidirectional RNNs (BRNNs), and the like.

[0078] LSTM networks are deep learning systems that can be trained on tasks that require, in a sense, "memories" of events that happened previously over very short and discrete time intervals. The topologies of LSTM networks can change depending on the specific tasks they are "trained" to perform. For example, LSTM networks can be trained to perform tasks where relatively long delays occur between events, or when events occur together at low and high frequency.

[0079] In some embodiments of the present technology, it is contemplated that NN 400 may be implemented as a given LSTM network. It can be said that the NN 400 can be implemented in some embodiments with some network topology that allows "memories" of events that have occurred previously to be stored.

[0080] Summarizing, the implementation of NN by the communication system between devices can be generally divided into two phases: a learning phase and a usage phase.

[0081] First, NNs are trained during the training phase. During the training phase, the server 107 may perform a large number of training iterations on the NN. Generally speaking, during a certain training iteration, the NN is introduced with feature sets associated with a common digital audio signal, and it is, in a sense, “learned” which of these feature sets corresponds to the segment of that digital audio signal during which the user utterance or audio recording ended in this digital audio signal (using the corrected end time of the utterance corresponding to the timestamp as its representative variable).

[0082] Then, during the usage phase, when the NN knows what data to expect as input (eg, feature sets) and what data to provide as output (eg, feature set predictions), the NN actually works, using data in the process of use (in-use data). Generally speaking, at use time, feature sets associated with the digital audio signal in use (such as digital audio signal) are entered into the NN and used to determine the likelihood of the corresponding segment of the digital audio signal in use during which the user utterance ended.

[0083] Next, how the server 107 can iterate NN training will be described. Assume that a given NN training iteration is performed on the data associated with a digital audio signal. However, it should be noted that for NN, during its training phase, more training iterations are performed, as mentioned above, using data associated with other audio signals from a plurality of digital audio signals, similarly to performing a given training iteration.

[0084] For a given NN training iteration, server 107 may retrieve training data from a database. The training data contains (i) a plurality of feature sets for the digital audio signal and (ii) corresponding associated labels. As mentioned above, in some embodiments of the present technology, the training data may be predetermined by the server 107, i.e., the server 107 may have determined and stored this training data prior to the training phase of the NN.

[0085] The server 107 is configured to input the plurality of feature sets into the NN in the same sequence as the sequence in which corresponding segments of the plurality of segments occur in the digital audio signal. In other words, during a given training iteration, the server 107 is configured to input a plurality of feature sets in the following order: first feature set, second feature set, third feature set, fourth feature set. [0086] When the server 107 inputs the first feature set to the NN, the NN is configured to output a first output value indicating the probability that the hash in the digital audio signal completed during the first segment of the digital audio signal. Suppose the first output value is "0.1" (or 10% for example). We can say that NN determines the first output value "0.1" for the first segment based on the first set of features.

[0087] The server 107 then inserts the second set of features into the NN. The NN is configured to output a second output value indicating the probability that the hash in the digital audio signal has completed during the second segment of the digital audio signal. Suppose the second output value is "0.3".

[0088] As previously mentioned, the NN may have a network topology that allows the NN to store "memories" of events that have occurred previously. In other words, the NN may be configured to output a second output value of "0.3" based on (i) the second feature set, and also based on the "memory" that the previously input feature set is (ii) the first feature set. This means that the NN can determine the second output value "0.3" for the second segment based on both (i) the second feature set and (ii) the first feature set.

[0089] The server 107 then inputs the third feature set to the NN, and the NN outputs a third output value of "0.1". The NN may determine the third output value "0,1" for the third segment based on (i) the third feature set, (ii) the second feature set, and (iii) the first feature set.

[0090] The server 107 then inputs the fourth feature set to the NN, and the NN outputs a fourth output value of "0.4". The NN may determine a fourth output value of "0.4" for the fourth segment based on (i) the fourth feature set, (ii) the third feature set, (iii) the second feature set, and (iv) the first feature set.

[0091] Thus, as explained above, during a given training iteration, the NN predicts that:

[0092] there is a probability of "0.1" that the user utterance ended during the first segment of the digital audio signal;

[0093] there is a probability of "0.3" that the user utterance ended during the second segment of the digital audio signal;

[0094] there is a probability of "0.1" that the user utterance ended during the third segment of the digital audio signal;

[0095] There is a probability of "0.4" that the user utterance ended during the fourth segment of the digital audio signal.

[0096] Then, during a given training iteration, the server 107 may be configured to perform comparison analysis between the output values provided by the NN and the labels associated with the respective feature sets from the plurality of feature sets. Based on this comparison, server 107 can be configured to determine multiple individual penalty values.

[0097] Therefore, the server 107 is configured to prepare the NN such that the NN outputs output values in response to feature sets such that differences between the values of the respective labels and the respective output values are minimized. The greater the difference between the values of the respective labels and the respective output values, the higher the combined penalty value will be, and hence the greater the adjustment of connections between the artificial "neurons" NN can be. Similarly, the smaller the difference between the values of the respective labels and the respective output values, the lower the combined penalty value will be, and hence the smaller the adjustment of connections between the artificial "neurons" NN can be.

[0098] With a "false positive" prediction, NN determines that there is a high probability that some segment that occurs before the completion (end) of the user utterance or an audio signal in a digital audio signal contains this completion of said user utterance. Reducing the number of false positives can be beneficial during use because a false positive can cause the system to erroneously determine that the user's utterance ended at the time of the most recently received part of the digital audio signal, when the user has not, in fact, stopped speaking yet.

[0099] For example, false positive predictions may be more likely to occur for segments during which there is a pause in the user's utterance. In other words, a false positive prediction is more likely to occur during segments in which the user pauses the user's utterance (eg, during a pause between two words the user speaks).

[00100] The number of NN false positives can be reduced by assigning weights to the individual penalty values associated with the digital audio signal segments that occur before the adjusted utterance end time to increase their contribution to the combined penalty value.

[00101] Next, an embodiment of the technology platform will be described.

[00102] FIG. 4 illustrates an exemplary embodiment of a computerized mobile system that can be used as a user's mobile computing device in conjunction with the computerized system for exchanging information between the mobile computing devices shown in FIG. 1 and 2. In one or more embodiments, computerized mobile system 400 may be implemented in the form factor of a mobile computing device, such as a smartphone, iPAD, tablet computer, or smart watch, that is widely available commercially and well known to those skilled in the art. .

[00103] The computerized system 400 may include a data bus 404 or other data transfer method or communication mechanism for transferring information between and between various hardware components of the mobile computerized system 400, as well as a central processing unit (CPU or simply a processor) 401 coupled to data bus 404 to process information and perform other computing and control tasks. Computerized system 400 also includes memory 412, such as random access memory (RAM) or other dynamic storage device, coupled to data bus 404 to store various information as well as instructions to be executed by processor 401. Memory 412 may also include permanent storage devices such as magnetic disk, optical disk, solid state flash memory or other non-volatile solid state storage devices.

[00104] In one or more embodiments, memory 412 may also be used to store temporary variables or other intermediate information during execution of instructions by processor 401. Additionally, computerized system 400 may further include read-only memory (ROM or EPROM) 402 or other static a storage device coupled to the data bus 404 for storing static information and instructions for the processor 401, such as the firmware needed to operate the computerized system 400, the basic input/output system (BIOS), and various configuration settings for the computerized system 400.

[00105] In one or more embodiments, computerized system 400 may include a display device 409 that may also be connected to data bus 404 to display various information to a user of computerized system 400, such as the user interface screens shown in FIG. 4 and 5. Alternatively, display device 409 may be associated with a graphics controller and/or graphics processor (not shown). The display device 409 may be implemented in the form of a liquid crystal display (LCD) manufactured using, for example, thin film transistor (TFT) technology or organic light emitting diode (OLED) technology, which are well known to those skilled in the art. In various embodiments, the display device 409 may be built into the same common housing as the rest of the computerized system 400. In an alternative embodiment, the display device 409 may be located outside such a housing.

[00106] In one or more embodiments, computerized system 400 may further include a microphone 425 coupled to data bus 404 and configured to capture ambient sound and convert it into electronic form.

[00107] In one or more embodiments, the computerized system 400 may include one or more input devices, such as a touch screen interface 410 for receiving tactile commands, a camera 411 for receiving still images and videos of various objects, and a keyboard 406, which may be connected to the aforementioned data bus 404 to transmit information, including but not limited to images and video, as well as selection of user commands to the processor 401. In an alternative embodiment, the input devices may include a user eye tracking system (not shown) , which can be used to indicate to the computerized system 400 the user's command selection.

[00108] In one or more embodiments, computerized system 400 may further include a location signal receiver 403 configured to perform a scan of the GPS signal or beacons and transmit the scan data described above to processor 401 via data bus 404.

[00109] In one or more embodiments, computerized system 400 may further include a communications interface, such as a network interface 405, coupled to data bus 404. The network interface 405 may be configured to establish a connection between the computerized system 400 and the Internet 424 using at least one of the WIFI interfaces 407 and a cellular network adapter (GSM or CDMA) 408 . The network interface 405 may be configured to provide two-way communication between the computerized system 400 and the Internet 424. The WIFI interface 407 may operate in accordance with the 802.11a, 802.11b, 802.11g, and/or 802.11n protocols, as well as the well-known Bluetooth protocol. specialists in this field of technology. In an exemplary implementation, the WIFI interface 407 and the cellular network adapter (GSM or CDMA) 408 send and receive electrical or electromagnetic signals that carry digital data streams representing various types of information.

[00110] In one or more embodiments, the implementation of the Internet 424 typically provides data transmission through one or more subnets to other network resources. Thus, computerized system 400 is capable of accessing various network resources located anywhere on the Internet 424, such as web servers, other content servers, and other network storage resources. In one or more embodiments, computerized system 400 is configured to send and receive messages, multimedia, and other data, including application program code, over various networks, including the Internet 424, via network interface 405. In the Internet example, when computerized system 400 acts as a network client, it may request code or data for an application program running on computerized system 400. Similarly, it may send various data or computer code to other network resources.

[00111] In one or more embodiments, the computerized system 400 uses a network interface 405 to send request(s) over the Internet 424, such as HTTP requests, to servers and obtain various information, including, but not limited to, the aforementioned displayed information to a user using the user interface shown in Fig. 4 and 5.

[00112] In one or more embodiments, the functionality described herein is implemented by computerized system 400 in response to processor 401 executing one or more sequences of one or more instructions contained in memory 412. Such instructions may be read into memory 412 from another computer. Execution of the sequences of instructions contained in the memory 412 causes the processor 401 to perform the various steps of the method described here. In alternative embodiments, instead of software commands, or in combination with software instructions, hardware circuitry may be used to implement embodiments of the invention. Thus, embodiments of the invention are not limited to any particular combination of hardware circuits and software.

[00113] The term "computer-readable medium" as used herein refers to any medium that participates in providing instructions to the processor 401 for execution. A computer-readable medium is just one example of a computer-readable medium that may contain instructions for implementing any of the methods and/or techniques described herein. Such an environment may take various forms, including, but not limited to, a non-volatile environment and a volatile environment.

[00114] Common forms of permanent computer-readable media include, for example, floppy disk, floppy disk, hard disk, magnetic tape or any other magnetic media, CD, any other optical media, punched cards, paper tape, any other physical media with a set of holes, RAM, PROM, EPROM, FLASH-EPROM, flash drive, memory card, any other memory chip or cartridge, or any other media that a computer can read. Various forms of computer-readable media may be used to carry one or more sequences of one or more instructions to the processor 901 for execution. For example, instructions may initially be transferred to a magnetic disk from a remote computer. Alternatively, the remote computer may download the instructions into its dynamic memory and send the instructions over the Internet 424. In particular, the computer instructions may be downloaded into the memory 412 of the computerized system 400 from the aforementioned remote computer over the Internet 424 using a variety of network communication protocols well known in the art. this field of technology.

[00115] In one or more embodiments, memory 412 of computerized system 400 may store any of the following programs, applications, or modules:

[00116] 1. Operating system (OS) 413, which may be a mobile operating system for implementing basic system services and managing various hardware components of computerized system 400. Exemplary embodiments of operating system 413 are well known to those skilled in the art, and may include iOS , Android, Windows Mobile or any other known or future mobile operating system(s).

[00117] 2. Mobile applications 414 may include, for example, a set of software applications executed by processor 401 of computerized system 400 that causes computerized mobile system 400 to perform certain predefined functions such as capturing an audio signal, generating a hash from it, and transmitting it. to the server 107 to receive information for display to the user and to display this information to the user using a graphical user interface displayed on the display device as described above. In one or more embodiments, mobile applications 414 may include, for example, an audio capture application 415 for capturing an audio signal using a microphone 425, a hashing application 416 for generating a hash of the captured audio signal using a hashing algorithm, and user interface generation.

[00118] 3. Data store 418 may be used to store a hash of an audio signal, as well as other suitable statistical information, for example.

[00119] FIG. 5 illustrates an exemplary computerized server system that can be used, for example, as a server of the computerized system according to the invention for exchanging information between the mobile computing devices shown in FIG. 1 and 2. It should be noted that other components of the described system, such as the database engine, can also be deployed on the computerized server system 500.

[00120] In one or more embodiments, the computerized server system 500 may include a data bus 504 that may be substantially the same and may perform substantially the same functions as the data bus 504 of the computerized system 400 shown in FIG. 4. In various embodiments, data bus 504 may use the same or a different connection and/or communication protocol as data bus 404. The display device 509 and keyboard 506 of the computerized server system 500 may also be substantially the same as the corresponding processor 401, network interface 405, EPROM/firmware storage 402, display device 409, and keyboard 406 of the computerized system 400, except that the first components are deployed in server platform configuration. In various implementations, one or more processors 501 may have significantly increased processing power compared to processor 401.

[00121] In addition to the input device (keyboard) 506, the computerized server system 500 may further include a cursor control device 510, such as a mouse, trackball, or cursor direction keys, to communicate direction information and command selection to the processor 501 and to control moving the cursor on the display device 509 . This input device typically has two degrees of freedom in two axes, a first axis (eg x) and a second axis (eg y), which allows the device to define positions on a plane.

[00122] The LAN/ISDN adapter 507 of the computerized server system 500 may be implemented, for example, using an Integrated Services Digital Network (ISDN) card or modem to provide a data connection to an appropriate type of telephone line that is interfaced to the Internet 424 using ISP hardware (not shown). As another example, the LAN/ISDN adapter 507 may be a local area network interface card (LAN NIC) to provide a data connection to a compatible LAN and the Internet 424. To store various data files, the computerized server system 500 may be provided with a data store 508 for storing various statistical data to be displayed to the user by the storage controller 503 . The camera 511 may be used to acquire images and/or video of various objects.

[00123] In one or more embodiments, memory 512 of computerized server system 500 may store any of the following programs, applications, modules, and/or data:

[00124] 1. Server operating system (OS) 513, which may be an operating system for implementing basic system services and managing various hardware components of computerized server system 500. Exemplary embodiments of server operating system 513 are well known to those skilled in the art and may include Windows Server, Mac OS, Unix, AIX, FreeBSD, Linux, and any operating system currently known or later developed.

[00125] 2. The network communication module 514 may include, for example, one or more network protocol stacks that are used to establish a network connection between the computerized server system 500 and various Internet network entities 424, such as the computerized mobile system 400, using a network 505 adapter that works in conjunction with the 507 LAN/ISDN adapter.

[00126] 3. Server applications 515 may include, for example, a set of software applications executed by one or more processors 501 of computerized server system 500 that cause computerized server system 500 to perform certain predefined functions or tasks, such as generating a hash of a captured audio signal. and compare this hash with the contents of the hash store. In one or more embodiments, server applications 515 may include a hash application 516 to generate an audio hash, the functionality of which has been detailed above. Additionally, a database management system 517 may be provided for storing and managing access to various information as detailed above, such as hash storage data. A hash comparison application 518 may be provided to calculate the distance (eg, difference) between two hashes and to find matching hashes.

[00127] 4. Data store 519 may be used, for example, to store hash data or compare against a generated hash, which may be stored in one or more database tables containing database records.

[00128] The method for detecting and exchanging financial information between nearby devices using this technology will be described in detail below.

[00129] Devices suitable for using the technology are laptops, smartphones, wearable devices (e.g., watches, glasses, rings), cash registers (including mobile devices), personal computers, System on a Chip (SOC) devices, single board computers (SBC) and so on.

[00130] The device must be equipped with a microphone of any type, an analog-to-digital converter (ADC) with a sampling rate of at least 8 kHz, a processor and a communication module operating on the Wi-Fi protocol, Bluetooth, Ethernet, EDGE, 3G, LTE, 4G, 5G or another that allows two-way data exchange at a speed of 16 kbps or higher.

[00131] When the discovery mode is started, the device establishes a connection to the server, for example, via the WebSockets communication protocol, but not limited to, and performs connection initialization, which consists of authorization and time synchronization with the server. The time synchronization algorithm determines the average packet delivery time by making 5 or more network requests and adjusts the time received from the server to the average packet delivery time. In case of successful initialization, the device sends a message to the server with public transaction data that allows you to identify the device and initialize the payment, such as: cash desk ID, cash desk name, user id, user full name, payment amount, payment ID, as well as any other data on the discretion of the company using the technology.

[00132] Surround sound is captured using a microphone, converted to a digital audio signal using an ADC, and then sent to the processor for processing, while the device must have the following characteristics:

• sampling frequency not less than 8kHz;

• signal width not less than 8 bits;

• total delay when capturing an audio stream is no more than 1 sec.

[00133] The audio signal, as it accumulates, is transformed into an amplitude spectrum using the windowed Fourier transform (STFT), where the Hamming window is used as the window function, and the Fast Fourier Transform (FFT) is used as the Fourier function. The result of the transformation is the amplitude spectrum, which is divided into 32 equal frequency intervals. The average amplitude of each frequency interval is checked for exceeding a threshold value, where the result of the check is 1 or 0 in binary representation. Thus, the result of each iteration of the window function is represented as a 32-bit number (hash element), and the audio environment hash consists of 100 or more such elements.

[00134] The device sends the new audio environment hashes to the server in real time using, but is not limited to, the bidirectional WebSockets communication protocol. In the event of a connection loss, hashes are accumulated in the device's RAM and sent when the connection is resumed.

[00135] The data packet is in JSON format, but is not limited to it. The package contains the exact recording time of the original audio signal and the hash of the audio environment, consisting of an array of hash elements, from 100 to 1000 elements long.

[00136] When receiving a data packet with a hash, the server creates a hash index and replicates the index and the original hash to all servers in the cluster (nodes), where they are stored for 1 minute. Thus, each node has a complete set of hashes of all devices for the last minute. The server sends a task to a random node to search for matches with a new hash. Thus, the server evenly distributes the load in the cluster.

[00137] Horizontal server scaling is performed automatically depending on the average load on the nodes in the cluster. Upon reaching 60%, an additional node is added to the cluster, when the load drops below 40%, the node is removed from the cluster.

[00138] The hash index is a complete inverted lookup index, where each key refers to a location in the matching hash. The keys in the index are built using the sampling expansion algorithm: each hash element is multiplied by 8 different bit masks, the resulting 8 keys are added to the index, indicating the mask used.

[00139] The search for matching hashes occurs in several steps. 1) using the index, hashes with the largest number of matching keys are selected. 2) the shift in hashes relative to the desired hash is determined and the intersection area of the hashes is determined. 3) For each candidate, the error rate is calculated as the ratio of the number of bit differences between the hash and the desired hash to the length of the hash intersection area. 4) Candidates whose error rate is below the threshold are considered a match.

[00140] It is worth noting that the threshold value can be configured for various use cases, for example, to change the detection radius of devices, reduce the detection time.

[00141] If the hashes match, the server sends to the corresponding devices public transaction data that allows you to identify the device and initiate the payment, such as: cash desk ID, cash desk name, user id, user full name, payment amount, payment ID, as well as any other data , at the discretion of the company using the technology.

[00142] The device performs the payment using the technology of this patent. The technology does not get access to the transaction data and does not affect the security of the transaction.

[00143] Finally, it should be understood that the processes and methods described herein are not inherently associated with any particular device and may be implemented using any suitable combination of components. In addition, in accordance with the ideas described in this document, various types of general purpose devices can be used. It may also be advantageous to design a specialized device for carrying out the steps of the method described here. The present invention has been described in terms of specific examples, which are intended in all respects to be illustrative and not restrictive. Those skilled in the art will appreciate that many different combinations of hardware, software, and firmware are suitable for practicing the present invention. For example, the described software may be implemented in a wide variety of programming or scripting languages such as assembly language, C/C++, Objective-C, perl, shell, PHP, Java, or any programming language now known or later developed, or scripting language.

[00144] Currently, proprietary technologies limited to their ecosystem (GooglePay, ApplePay, and so on) are used to implement a seamless contactless payment (financial information exchange) experience. To implement a cross-platform solution, dynamic QR codes are used, which increases the requirements for devices (the presence of a screen on the receiving device and a camera on the payment device), or systems for generating and entering one-time codes are used, which worsens the user experience.

[00145] In the case of this technical solution, in the vast majority of cases, the identification of a pair of devices participating in the process of exchanging financial information occurs automatically, without human intervention and does not place high demands on the devices involved in the exchange.

[00146] In addition, other embodiments of the invention will be apparent to those skilled in the art from a consideration of the description and practice of the invention described herein. Various aspects and/or components of the described embodiments may be used singly or in any combination in systems and methods for implementing information exchange between two or more mobile computing devices and/or other computing systems. It is intended that the description and examples be regarded as illustrative only, and that the true scope and spirit of the invention be indicated in the following claims.

Claims (18)

1. A computer-implemented method for detecting and exchanging financial information between nearby devices, executing on a computerized system containing a mobile audio capture device, a central processing unit, a display device, and a memory, the computer-implemented method comprising: - capturing an audio signal based on a predetermined criterion using a user's mobile audio capture device, wherein the audio signal contains an ambient sound sample in the ambient sound space of the mobile device; - using the CPU to generate the hash function of the captured audio signal; - transmission of the hash of the captured audio signal to the cluster server; - moreover, when receiving a data packet with a hash of the captured audio signal by means of the server, a hash index is created and the index and the received hash are replicated to all servers in the cluster (nodes), where each node has a complete set of hashes of the audio signals of all devices for the last minute; - implementation by means of a server of comparing the hash of the captured audio signal and at least one hash stored in the cloud database, which sends a task to a random node to search for matches with the hash of the captured audio signal, moreover -- server finds a candidate audio signal using the short audio signature in the index; -- create a long audio signature; -- verifying the candidate audio signal by bitwise comparing the hash of the long audio signature of the captured audio signal with the hash of the long audio signature stored in the cloud database and available to the server using the verification index; receiving a comparison result from a server identifying a second user device, the second device being associated with a similar audio hash similar to the audio hash obtained from the ambient sound sample in the second device's ambient sound space; as well as - authorizing the transfer of financial data between the mobile audio capture device and the second device associated with the same hash of the audio signal, based on the result of the comparison. 2. The method according to claim 1, characterized in that the predetermined criterion is the presence of a sound signal. 3. The method according to claim 1, characterized in that the predetermined criterion is the loudness of the captured audio signal exceeding the predetermined threshold. 4. The method according to claim 1, characterized in that when the audio signal does not meet the predetermined criterion, random audio is generated using the audio generating device. 5. The method according to claim 1, characterized in that the remote server is a cloud server. 6. The method according to claim 1, characterized in that the server contains a database that stores a plurality of hashes of the audio signal. 7. The method of claim 1, further comprising performing data communication with the second device based on the identifier of the second device. 8. The method according to claim 1, characterized in that the hash of the captured audio signal and the similar hash of the audio signal contain perceptual hashes.

Images