EP3002960A1 - System und Verfahren zur Erzeugung von Surround-Sound - Google Patents

System und Verfahren zur Erzeugung von Surround-Sound Download PDF

Info

Publication number
EP3002960A1
EP3002960A1 EP14461574.7A EP14461574A EP3002960A1 EP 3002960 A1 EP3002960 A1 EP 3002960A1 EP 14461574 A EP14461574 A EP 14461574A EP 3002960 A1 EP3002960 A1 EP 3002960A1
Authority
EP
European Patent Office
Prior art keywords
sound
loudspeakers
location
event
computer
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
EP14461574.7A
Other languages
English (en)
French (fr)
Inventor
Jacek Paczkowski
Tomasz Nalewa
Krzysztof Kramek
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Patents Factory Ltd Sp zoo
Original Assignee
Patents Factory Ltd Sp zoo
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Patents Factory Ltd Sp zoo filed Critical Patents Factory Ltd Sp zoo
Priority to EP14461574.7A priority Critical patent/EP3002960A1/de
Publication of EP3002960A1 publication Critical patent/EP3002960A1/de
Withdrawn legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04S—STEREOPHONIC SYSTEMS 
    • H04S7/00—Indicating arrangements; Control arrangements, e.g. balance control
    • H04S7/30—Control circuits for electronic adaptation of the sound field
    • H04S7/308—Electronic adaptation dependent on speaker or headphone connection

Definitions

  • the present invention relates to a system and method for generating surround sound.
  • the present invention relates to surround environment independent from number of loudspeakers and configuration/placement of the respective loudspeakers.
  • Reflections may be used to generate virtual surround sound. This is the case in so-called sound projectors (an array of loudspeakers in a single casing - a so called sound bar).
  • Ambisonics system which is a full-sphere surround sound technique: in addition to the horizontal plane, it covers sound sources above and below the listener.
  • the aim of the development of the present invention is a surround system and method that is independent from number of loudspeakers and configuration/placement of the respective loudspeakers.
  • An object of the present invention is a signal comprising at least one sound event data, wherein the sound event comprises: time of event information; information regarding location in space with respect to a reference location point; a movement trajectory in space; orientation information; spatial characteristic of the source of the event; information on sampling frequency; information on signal resolution; and a set of acoustic samples of the sampling frequency and at the signal, resolution.
  • Another object of the present invention is a method for generating surround sound the method comprising the steps of: receiving a sound data signal stream according to the present invention; obtaining access to a database of loudspeakers records the records comprising information regarding location and characteristics of loudspeakers available; executing calculating, which loudspeakers may be used from the available loudspeakers so as to achieve the effect closest to a perfect arrangement according to the at least one sound event data; calculating an angular difference between sound source location and positions of the candidate loudspeakers in spherical coordinates; selecting a set of loudspeakers that have the lowest distance from the sound event location; in case of insufficient number of physical loudspeakers, creating one or more virtual loudspeakers by utilizing reflection of sound; and generating data streams that are to be sent to physical loudspeakers.
  • the step of calculating is be executed for each sound event.
  • the closest loudspeakers will be used in order to simulate a virtual loudspeaker, located where the sound source is located, by applying a superposition principle.
  • the selected loudspeakers are located at opposite sides, when facing the reference location of a user, with respect to the sound event location.
  • the information regarding location and characteristics of loudspeakers is described with s points whereas u describes a shape of the sound beam in the horizontal plane while v respective shape in the vertical plane wherein such characteristics is determined using an array of microphones.
  • Another object of the present invention is a computer program comprising program code means for performing all the steps of the computer-implemented method according to the present invention when said program is run on a computer.
  • Another object of the present invention is a computer readable medium storing computer-executable instructions performing all the steps of the computer-implemented method according to the present invention when executed on a computer.
  • these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated in a computer system.
  • these signals are referred to as bits, packets, messages, values, elements, symbols, characters, terms, numbers, or the like.
  • a computer-readable (storage) medium typically may be non-transitory and/or comprise a non-transitory device.
  • a non-transitory storage medium may include a device that may be tangible, meaning that the device has a concrete physical form, although the device may change its physical state.
  • non-transitory refers to a device remaining tangible despite a change in state.
  • the present invention is independent from loudspeakers placement due to the fact that an acoustic stream is not divided into channels but rather sound events present in a three-dimensional space.
  • Fig. 1 presents a diagram of a sound event according to the present invention.
  • the sound event 101 represents the fact of presence of a sound source in an acoustic space.
  • Each such event has an associated set of parameters such as: time of event 102, location in space with respect to a reference location point 103.
  • the location may be given as x,y,z coordinates (alternatively spherical coordinates r, ⁇ , ⁇ may be used).
  • the sound event 101 comprises further a movement trajectory in space (for example in case of a vehicle changing its location) 104.
  • the movement trajectory may be defined as n, ⁇ t1, x1, y1, z1, ⁇ 1, ⁇ 1, ⁇ t2, x2, y2, z2, ⁇ 2, ⁇ 2, ..., ⁇ tn, xn, yn, zn, ⁇ s, ⁇ s which is a definition of a curve on which the sound source moves.
  • n is a number of points of the curve while the xi, yi, zi are points in space and ⁇
  • ⁇ is temporary orientation of the sound source (azimuth and elevation) and ⁇ t is an increase in time.
  • the sound event 101 comprises further orientation ( ⁇ , ⁇ - direction in which the highest sound amplitude is generated; azimuth and elevation are defined relative to orientation of a coordination system) 105.
  • the sound event 101 comprises spatial characteristic of the source of the event (a shape of a curve of the sound amplitude with respect to emission angle - zero angle means emission in the direction of the highest amplitude) 106.
  • This parameter may be provided as s, ⁇ 1, u1, v1, ⁇ 2, u2, v2, ⁇ 3, u3, v3, ⁇ 3, 53, ..., ⁇ s, us, vs where the characteristic is symmetrical and described with s points whereas u i describe a shape of the sound beam in the horizontal plane while v i respective shape in the vertical plane.
  • the sound event 101 comprises further information on sampling frequency (in case it is different from the base sampling frequency of the sound stream) 107, signal resolution (the number of bits per sample; this parameter is present if a given source has a different span than a standard span of the sound stream) 108 and a set of acoustic samples 109 of the given frequency, resolution.
  • a plurality of sound events will typically be encoded into an output audio data stream.
  • the loudspeakers may be located in an arbitrary way however preferably they should not be all placed in a single place, for example a single wall.
  • the plurality of loudspeakers may be considered a cloud of loudspeakers. The more the loudspeakers the better spatial effect may be achieved.
  • the loudspeakers are scattered in the presentation location, preferably on different walls of a room.
  • the loudspeakers may be either wired or wireless and be communicatively coupled to a sound decoder according to the present invention.
  • the decoder may use loudspeakers of other electronic devices as long as communication may be established with controllers of such speakers (eg, bluetooth or wi-fi communication with loudspeakers of a TV set or mobile device).
  • the sound decoder may obtain information on location and characteristic of a given loudspeaker by sending to its controller a test sound stream and subsequently recording the played back test sound stream and analyzing the relevant acoustic response.
  • an array of omnidirectional microphones for example spaced from each other by 10cm and positioned on vertices of a cube or a tetrahedron.
  • the characteristics of a given loudspeaker may be obtained by analyzing recorded sound at different frequencies.
  • the sound decoder executes sound location analysis aimed at using reflective surfaces (such as walls) to generate reflected sounds. All sound reflecting surfaces are divided into triangles and each of the triangles is treated by the decoder as a virtual sound source. Each triangle has an associated function defining dependence of a sound virtually emitted by this triangle on sounds emitted by physical loudspeakers. This function defines the amplitude as well as spatial characteristics of emission, which may be different for each physical loudspeaker. In order for the system to operate properly it is necessary to place, at a sound presentation location, microphones used by the sound decoder for constant measurements of compliance of the emitted sounds with expected sounds and for fine tuning the system.
  • Such a function is a sum of reflected signals emitted by all loudspeakers in a room, wherein a signal reflected from a given triangle depends on the triangle location, loudspeaker(s) location(s), loudspeaker(s) emission characteristics, acoustic pressure emitted by the loudspeaker(s).
  • the signal virtually emitted by the triangle will be a sum of reflection generated by all loudspeakers.
  • a spatial acoustic emission characteristics of such triangle will depend on physical loudspeakers whereas each physical loudspeaker will influence it partially. Such characteristics may be discrete, comprising narrow beams generated by different loudspeakers.
  • an appropriate loudspeaker or a linear combination of loudspeakers (appropriate means in line with the acoustic target eg. generating, from a given plane, a reflection in the direction of the listener such that other reflections do not ruin the effect).
  • the most important module of the system is a local sound renderer. This means that the renderer receives separate sound events and composes from them acoustic output streams that are subsequently sent to loudspeakers.
  • the renderer shall select a speaker or speakers, which is/are closest to the location in space where the sound was emitted from.
  • speakers adjacent to this location shall be used, preferably speakers located at opposite sides of the location so that they may be configured in order to create an impression for the listener that the sound is emitted from its original location in space.
  • More than two loudspeakers may be used for one sound event in particular when a virtual sound source is to be positioned between them.
  • the reference point location may be differently selected for a given sound rendering location or room. For example on may listen to the music in an armchair and watch television sitting on a sofa. Therefore, there are two different reference locations depending on circumstances. Consequently, the coordinates system changes.
  • the reference location may be automatically obtained by different sensors such as an infrared camera or manually input by the listener. Such solution is possible only because of local sound rendering.
  • Fig. 1B An exemplary normalized characteristics of a physical loudspeaker is shown in Fig. 1B .
  • the characteristic is usually symmetrical and described with s points whereas u describes a shape of the sound beam in the horizontal plane while v respective shape in the vertical plane.
  • Such characteristics may be determined using an array of microphones as previously described.
  • characteristic can be asymmetrical and discontinuous.
  • Fig. 2 presents a diagram of the method according to the present invention.
  • the method starts, after receiving a sound data stream according to Fig. 1 , at step 201 from accessing a database of loudspeakers present at sound presentation location. Subsequently, at step 202, there is executed calculating, which loudspeakers may be used from the available loudspeakers so as to achieve the effect closest to a perfect arrangement. This may be effected by location thresholding based on the database of loudspeakers records.
  • Such calculation needs to be executed for each sound event because sound events may run in parallel and the same loudspeaker(s) may be needed to emit them.
  • Data for each loudspeaker has to be added by applying superposition approach (all sound events at a given moment of time that affect a selected loudspeaker).
  • a loudspeaker In case a loudspeaker is close to a location in which a sound source is located, this loudspeaker will be used. In case the sound source is located between physical loudspeakers then the closest loudspeakers will be used in order to simulate a virtual loudspeaker, located where the sound source is located. A superposition principle may be applied for this purpose. It is necessary to take into account, during this process, the emission characteristics of the loudspeakers.
  • the physical loudspeakers selected for simulating a virtual loudspeaker will emit sound in direction of the listener at predefined angles of azimuth and elevation. For these angles there is to be read attenuation level from the emission characteristic of the loudspeaker (the characteristics is normalized and therefore it will be a number from a range of 0 ... 1) and multiplied by emission strength of the loudspeaker (acoustic pressure). Only after that, superposition may be executed.
  • the signals are to be added by assigning weights to loudspeakers, the weights arising from location of a virtual loudspeaker with respect to these used to its generation (based on proportionality rule).
  • the calculations shall include not only the direction from which a sound event is emitted but also a distance from the listener (i.e. a delay of the signal in such a way so as to simulate the correct distance from the listener to the sound event).
  • the properly selected loudspeakers surround the sound event location. There may be more than two selected loudspeakers that will emit a particular sound event data.
  • step 203 there is calculated an angular difference between sound source location and positions of the candidate loudspeakers in spherical coordinates.
  • the sound event location is:
  • a set of loudspeakers that have the lowest distance from the sound event location are selected at step 204.
  • the loudspeakers are to be located at opposite sides (when facing the reference location of a user) with respect to the sound event location so that the listener has an impression that the sound arrives from the sound event location.
  • step 205 in case of insufficient number of physical loudspeakers there may be created one or more virtual loudspeaker(s). Reflection of sound is utilized for this purpose. The reflections are generated by physical loudspeakers so that they imitate a physical loudspeaker in a given location of the sound presentation location. The generated sound will reflect from a selected surface and be directed towards the listener.
  • a straight line is to be virtually drawn from the listener to this location and further to a reflective plane (such as a wall).
  • a point indicated as an intersection of this line with the reflective plane will indicate a triangle on the reflective plane, which is to be used in order to generate a reflected sound.
  • From the characteristics of emission of that triangle it needs to be read which physical loudspeakers are to be used.
  • These data stream are to be added to other data emitted by the respective loudspeakers 207.
  • Fig. 3 presents a diagram of the system according to the present invention.
  • the system may be realized using dedicated components or custom made FPGA or ASIC circuits.
  • the system comprises a data bus 301 communicatively coupled to a memory 304. Additionally, other components of the system are communicatively coupled to the system bus 301 so that they may be managed by a controller 305.
  • the memory 304 may store computer program or programs executed by the controller 305 in order to execute steps of the method according to the present invention.
  • the system comprises a sound input interface 303, such as an audio/video communication connector eg. HDMI or communication connector such as Ethernet.
  • the received sound data is processed by a sound renderer 302 managing the presentation of sounds using the listener's premises loudspeakers setup.
  • the management of the presentation of sounds includes virtual loudspeakers management that is effected by a virtual loudspeakers module 307 operating according to the method described above.
  • the present invention related to recording, encoding and decoding of sound in order to provide for surround playback independent of loudspeakers setup at the sound presentation location. Therefore, the invention provides a useful, concrete and tangible result.
  • the aforementioned method for generating surround sound may be performed and/or controlled by one or more computer programs.
  • Such computer programs are typically executed by utilizing the computing resources in a computing device.
  • Applications are stored on a non-transitory medium.
  • An example of a non-transitory medium is a non-volatile memory, for example a flash memory or volatile memory, for example RAM.
  • the computer instructions are executed by a processor.
  • These memories are exemplary recording media for storing computer programs comprising computer-executable instructions performing all the steps of the computer-implemented method according the technical concept presented herein.
EP14461574.7A 2014-10-04 2014-10-04 System und Verfahren zur Erzeugung von Surround-Sound Withdrawn EP3002960A1 (de)

Priority Applications (1)

Application Number Priority Date Filing Date Title
EP14461574.7A EP3002960A1 (de) 2014-10-04 2014-10-04 System und Verfahren zur Erzeugung von Surround-Sound

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
EP14461574.7A EP3002960A1 (de) 2014-10-04 2014-10-04 System und Verfahren zur Erzeugung von Surround-Sound

Publications (1)

Publication Number Publication Date
EP3002960A1 true EP3002960A1 (de) 2016-04-06

Family

ID=51726472

Family Applications (1)

Application Number Title Priority Date Filing Date
EP14461574.7A Withdrawn EP3002960A1 (de) 2014-10-04 2014-10-04 System und Verfahren zur Erzeugung von Surround-Sound

Country Status (1)

Country Link
EP (1) EP3002960A1 (de)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109286888A (zh) * 2018-10-29 2019-01-29 中国传媒大学 一种音视频在线检测与虚拟声像生成方法及装置

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070253561A1 (en) * 2006-04-27 2007-11-01 Tsp Systems, Inc. Systems and methods for audio enhancement
WO2011020065A1 (en) * 2009-08-14 2011-02-17 Srs Labs, Inc. Object-oriented audio streaming system
EP2451196A1 (de) * 2010-11-05 2012-05-09 Thomson Licensing Verfahren und Vorrichtung zur Erzeugung und Decodierung von Schallfelddaten einschließlich Ambisonics-Schallfelddaten höher als drei
US20140112484A1 (en) 2012-10-24 2014-04-24 Google Inc. Automatic Detection of Loudspeaker Characteristics

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070253561A1 (en) * 2006-04-27 2007-11-01 Tsp Systems, Inc. Systems and methods for audio enhancement
WO2011020065A1 (en) * 2009-08-14 2011-02-17 Srs Labs, Inc. Object-oriented audio streaming system
EP2451196A1 (de) * 2010-11-05 2012-05-09 Thomson Licensing Verfahren und Vorrichtung zur Erzeugung und Decodierung von Schallfelddaten einschließlich Ambisonics-Schallfelddaten höher als drei
US20140112484A1 (en) 2012-10-24 2014-04-24 Google Inc. Automatic Detection of Loudspeaker Characteristics

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109286888A (zh) * 2018-10-29 2019-01-29 中国传媒大学 一种音视频在线检测与虚拟声像生成方法及装置
CN109286888B (zh) * 2018-10-29 2021-01-29 中国传媒大学 一种音视频在线检测与虚拟声像生成方法及装置

Similar Documents

Publication Publication Date Title
CN112567767B (zh) 用于交互式音频环境的空间音频
JP6455686B2 (ja) 分散型無線スピーカシステム
US9888333B2 (en) Three-dimensional audio rendering techniques
RU2625953C2 (ru) Посегментная настройка пространственного аудиосигнала к другой установке громкоговорителя для воспроизведения
JP6082160B2 (ja) 任意ｎ角形のメッシュとして編成されたスピーカーを用いたオーディオレンダリング
US20160337777A1 (en) Audio processing device and method, and program therefor
CN111164673B (zh) 信号处理装置、方法和程序
KR20150047334A (ko) 다채널 오디오 신호 생성 방법 및 이를 수행하기 위한 장치
CN108379842B (zh) 游戏音频处理方法、装置、电子设备及存储介质
RU2020120210A (ru) Способ и система для обработки глобальных переходов между положениями прослушивания в среде виртуальной реальности
JP6550473B2 (ja) スピーカの配置位置提示装置
WO2021158273A1 (en) Augmented reality virtual audio source enhancement
US10869151B2 (en) Speaker system, audio signal rendering apparatus, and program
Beig et al. An introduction to spatial sound rendering in virtual environments and games
US10462596B2 (en) Audio apparatus, method of processing audio signal, and a computer-readable recording medium storing program for performing the method
EP2552130B1 (de) Verfahren zur Klangsignalverarbeitung und Computerprogramm zur Implementierung des Verfahrens
EP3002960A1 (de) System und Verfahren zur Erzeugung von Surround-Sound
WO2023083876A2 (en) Renderers, decoders, encoders, methods and bitstreams using spatially extended sound sources
EP3013072B1 (de) System und Verfahren zur Erzeugung von Surround-Sound
Vorländer et al. Virtual room acoustics
Kim et al. Immersive virtual reality audio rendering adapted to the listener and the room
US11330391B2 (en) Reverberation technique for 3D audio objects
KR20230139772A (ko) 오디오 신호 처리 장치 및 오디오 신호 처리 방법
KR20240008241A (ko) 녹음 거리 파라미터 기반 오디오 렌더링 방법 및 이를 수행하는 장치

Legal Events

Date Code Title Description
PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

17P Request for examination filed

Effective date: 20150605

AK Designated contracting states

Kind code of ref document: A1

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

AX Request for extension of the european patent

Extension state: BA ME

17Q First examination report despatched

Effective date: 20161108

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN

18D Application deemed to be withdrawn

Effective date: 20170321