CN109873933A - Apparatus for processing multimedia data and method - Google Patents
Apparatus for processing multimedia data and method Download PDFInfo
- Publication number
- CN109873933A CN109873933A CN201711269168.2A CN201711269168A CN109873933A CN 109873933 A CN109873933 A CN 109873933A CN 201711269168 A CN201711269168 A CN 201711269168A CN 109873933 A CN109873933 A CN 109873933A
- Authority
- CN
- China
- Prior art keywords
- sound
- source
- image
- collecting device
- multimedia data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Abstract
A kind of apparatus for processing multimedia data, including acquiring unit, image collecting device, audio collecting device and processing unit, acquiring unit are used to obtain the relative positional relationship between source of sound and image collecting device；Image collecting device is for the image data in the acquisition preset range of relationship depending on that relative position, the audio data that audio collecting device is issued for source of sound in the acquisition preset range of relationship depending on that relative position, processing unit are used to the image data in collected preset range establishing corresponding association with audio data.The present invention also provides a kind of multimedia data processing methods.By obtaining the relative positional relationship between source of sound and image collecting device, and then source of sound is positioned.It is thusly-formed the voice data with sense of direction, the experience for greatly promoting user is felt.
Description
Technical field
The present invention relates to a kind of apparatus for processing multimedia data and methods.
Background technique
Usually when user images, sound and image are separate collections.When being recorded a video, video recording needs to obtain figure
As data and voice data, however, existing voice data is all acquisition of all sound regardless of orientation, when finally playing,
Just can't hear it is any have relief sound perception, in this way, user experience is poor.
Summary of the invention
In view of the foregoing, it is necessary to which a kind of apparatus for processing multimedia data and method are provided.
A kind of multimedia data processing method, the method includes the steps:
Obtain the relative positional relationship between source of sound and image collecting device；
The audio data that image data in relationship acquisition preset range and source of sound are issued depending on that relative position；And
Image data in collected preset range is established into corresponding association with audio data.
Preferably, the step of relative positional relationship obtained between source of sound and image collecting device specifically includes:
Obtain the positioning signal of the output at source of sound；And
The relative positional relationship between source of sound and image collecting device is determined according to the positioning signal received.
Preferably, the step of relative positional relationship for obtaining source of sound and image collecting device specifically includes:
The mobile angular momentum of source of sound is acquired by gyroscope；And
The angular momentum is converted into the corresponding azimuth information of source of sound to determine the phase between source of sound and image collecting device
To positional relationship.
Preferably, the step of relative positional relationship for obtaining source of sound and image collecting device specifically includes:
Displacement and/or the acceleration of image collecting device are acquired by linear accelerator；And
The displacement and/or acceleration are converted into the corresponding azimuth information of image collecting device to determine source of sound and image
Relative positional relationship between acquisition device.
Preferably, the step of relative positional relationship for obtaining source of sound and image collecting device specifically includes:
By the angular momentum and displacement and/or acceleration of gyroscope and linear accelerator acquisition image collecting device, and will
The angular momentum of institute's acquired image acquisition device and displacement and/or acceleration are converted into first orientation information；
The mobile angular momentum of source of sound and displacement and/or acceleration are acquired by gyroscope and linear accelerator, and will be adopted
The source of sound collected mobile angular momentum and displacement and/or acceleration are converted into second orientation information；And
The phase between source of sound and image collecting device is determined according to the first orientation information and the second orientation information
To positional relationship.
Preferably, the method also includes steps:
Determine that user watches the visual angle of image；And
Play the corresponding audio data in the visual angle.
Preferably, the method also includes steps:
Determine that user watches visual angle and the distance of image；And
It is adjusted according to the visual angle and apart from progress volume weighting and direction.
A kind of apparatus for processing multimedia data, described device include:
Acquiring unit, for obtaining the relative positional relationship between source of sound and image collecting device；
Image collecting device, for the image data in the acquisition preset range of relationship depending on that relative position；
Audio collecting device, the audio number issued for source of sound in the acquisition preset range of relationship depending on that relative position
According to；And
Processing unit, for the image data in collected preset range to be established corresponding association with audio data.
Preferably, the apparatus for processing multimedia data further includes a storage unit, and the storage unit will be for that will establish
Corresponding associated image data is stored with audio data.
Preferably, the playing device in the apparatus for processing multimedia data is associated more for playing the foundation correspondence
Media data, and the direction and visual angle of image are watched for detecting user, and be performed in accordance with according to the visual angle and distance of user
Volume weighting and direction adjustment.
Above-mentioned apparatus for processing multimedia data and method are by obtaining the relative position between source of sound and image collecting device
Relationship, and then source of sound is positioned.In this way, forming the voice data with sense of direction, the experience of user can be greatly promoted
Feel.
Detailed description of the invention
Fig. 1 is the block diagram of a better embodiment of apparatus for processing multimedia data.
Fig. 2 is the schematic diagram of a better embodiment of apparatus for processing multimedia data.
Fig. 3 is the flow chart of a better embodiment of multimedia data processing method.
Fig. 4 is the flow chart of a better embodiment of multimedia data playing method.
Main element symbol description
| Apparatus for processing multimedia data | 100 |
| Source of sound | 10、12、14 |
| Acquiring unit | 20 |
| Image collecting device | 30 |
| Audio collecting device | 40 |
| Processing unit | 50 |
| Storage unit | 60 |
The present invention that the following detailed description will be further explained with reference to the above drawings.
Specific embodiment
Following will be combined with the drawings in the embodiments of the present invention, and technical solution in the embodiment of the present invention carries out, clear, complete
Site preparation description.
In order to make the objectives, technical solutions, and advantages of the present invention clearer, below with reference to attached drawing and embodiment party
Formula, in the present invention apparatus for processing multimedia data and method is described in further detail and related description.
As shown in Figures 1 and 2, a preferred embodiment of the invention provides a kind of apparatus for processing multimedia data 100.
The apparatus for processing multimedia data 100 is used to determine the azimuth information of multiple sources of sound 10,12,14.Specific real
Shi Zhong, the source of sound 10,12,14 can be respectively the sound that 3 different personages (such as performer) are issued.In the present embodiment,
The number of source of sound 10,12,14 can be more or less than 3, for example, at least 1 for 3.
The apparatus for processing multimedia data 100 includes acquiring unit 20, image collecting device 30, audio collecting device
40, processing unit 50 and storage unit 60.
The acquiring unit 20 is used to obtain the relative positional relationship between source of sound 10,12,14 and image collecting device 30.
Wherein, the relative positional relationship includes the direction between source of sound 10,12,14 and image collecting device 20 and distance.
Described image acquisition device 30 is for the image data in the acquisition preset range of relationship depending on that relative position.?
In the present embodiment, described image acquisition device 30 includes multiple pick-up lens, to be respectively used to the picture number in preset range
According to being acquired.
The audio collecting device 40 is for source of sound 10,12,14 in the acquisition preset range of relationship depending on that relative position
The audio data issued.
The processing unit 50 is used to establish the image data in collected preset range with audio data corresponding
Association.
The storage unit 60 is stored for that will establish corresponding associated multi-medium data.
In a preferred embodiment, when source of sound 10,12,14 and image processing apparatus 30 do not move, the acquisition is single
Member 20 exports positioning signal to image processing apparatus 30 by the way that the positioning device at each source of sound 10,12,14 is arranged in, and so may be used
With according to received positioning signal determine the relative positional relationship between source of sound 10,12,14 and image processing apparatus 30.At this
In embodiment, the positioning device is ultrasonic unit, global positioning system (Global Positioning System, GPS)
The one of which of device, Wireless Fidelity (Wireless-Fidelity, WiFi) device.
In another preferred embodiment, described to obtain when source of sound 10,12,14 is mobile and image processing apparatus 30 does not move
Take unit 20 to acquire the mobile angular momentum of source of sound 10,12,14 by positioning device, by the angular momentum be converted to source of sound 10,12,
14 corresponding azimuth informations are to determine the relative positional relationship between source of sound and image collecting device 30.In the present embodiment, institute
Stating positioning device can be gyroscope.
In another preferred embodiment, described to obtain when image collecting device 30 is mobile and source of sound 10,12,14 does not move
Unit 20 is taken to acquire displacement and/or the acceleration of image collecting device 30 by positioning device, and by the displacement and/or acceleration
Degree is converted to the corresponding azimuth information of image collecting device 30 to determine between source of sound 10,12,14 and image collecting device 30
Relative positional relationship.In the present embodiment, the positioning device can be linear accelerator.
In another preferred embodiment, when image collecting device 30 and source of sound 10,12,14 move, the acquisition is single
Member 20 acquires the angular momentum and displacement and/or acceleration of image collecting device 30 by positioning device, and by a collected figure
As the angular momentum of acquisition device 30 and displacement and/or acceleration are converted into first orientation information.Similarly, the acquiring unit 20 is logical
Cross positioning device acquisition source of sound 10,12,14 mobile angular momentum and displacement and/or acceleration, and by the collected source of sound 10 of institute,
12,14 mobile angular momentums and displacement and/or acceleration are converted into second orientation information respectively.The processing unit 50 is according to
One azimuth information and second orientation information determine the relative positional relationship between source of sound 10,12,14 and image collecting device 30.
In a preferred embodiment, the multi-medium data stored in the storage unit 60 can be transmitted to playing device
It is used in (not shown), wherein transmission mode may include but be not limited to: duplication being carried out by storage medium or wireless network passes
The modes such as defeated.
When user uses playing device, the direction of playing device detecting user's viewing, and determine that user watches image
Visual angle, and establish associated multi-medium data according to acquired and play the corresponding audio data in the visual angle.
In a preferred embodiment, the playing device is also used to watch the visual angle of image according to user and apart from progress sound
Amount weighting and direction adjustment.
Specifically, when user uses playing device (such as VR head-mounted display), just facing towards certain in image
When one visual angle, the visual angle corresponds to associated sound and will transmit in front of user, and corresponding with other visual angles in image
Associated sound will be transmitted from the left back of user and right back.Also, the sound that each orientation transmits can be weighted and be adjusted
Volume.
Referring to FIG. 3, multimedia data processing method the following steps are included:
Step S100 obtains the relative positional relationship between source of sound and image collecting device, can specifically pass through such as lower section
Formula is realized:
When source of sound is mobile and image processing apparatus does not move, the mobile angular momentum of source of sound is acquired by positioning device, it will
The angular momentum is converted to the corresponding azimuth information of source of sound to determine the relative positional relationship between source of sound and image collecting device.
In a particular embodiment, positioning device can realize the acquisition of Angular Momentum by gyroscope.
When source of sound and image processing apparatus do not move, by the way that the positioning device output positioning at each source of sound is arranged in
Signal to image processing apparatus, so can according to received positioning signal determine the phase between source of sound and image processing apparatus
To positional relationship.In a particular embodiment, positioning device can pass through ultrasonic unit, global system positioning device, Wireless Fidelity
One of device realizes the positioning to source of sound.
When image collecting device is mobile and source of sound does not move, the displacement of image collecting device is acquired by positioning device
And/or acceleration, and the displacement and/or acceleration are converted into the corresponding azimuth information of image collecting device to determine source of sound
Relative positional relationship between image collecting device.In a particular embodiment, positioning device can be by linear accelerator with reality
The displacement of existing acquisition device and acceleration.
When image collecting device and source of sound move, angular momentum and the position of image collecting device are acquired by positioning device
Shifting and/or acceleration, and the angular momentum of institute's acquired image acquisition device and displacement and/or acceleration are converted into first party
Position information.Similarly, the mobile angular momentum of source of sound and displacement and/or acceleration are acquired by positioning device, and institute is collected
The mobile angular momentum of source of sound and displacement and/or acceleration are converted into second orientation information respectively.The processing unit is according to first
Azimuth information and second orientation information determine the relative positional relationship between source of sound and image collecting device.
Step S102, relationship acquires image data in preset range respectively depending on that relative position and source of sound is issued
Audio data.
Image data in collected preset range is established corresponding association with audio data by step S104.Specifically
For, determining relative positional relationship is combined with corresponding audio data and image data, it is associated more to generate correspondence
Media data.
Referring to FIG. 4, multimedia data playing method the following steps are included:
Step S200, the direction of detecting user's viewing, and determine that user watches the visual angle of image, according to acquired foundation
Associated multi-medium data plays the corresponding audio data in the visual angle.
Specifically, being associated with when first visual angle of the user using playing device viewing image according to acquired foundation
Multi-medium data, play the first visual angle corresponding audio data in the multi-medium data to determine.When user sees
When seeing the second visual angle of image, associated multi-medium data is established according to acquired, to determine that playing second visual angle exists
Corresponding audio data in the multi-medium data.And so on, user is at the visual angle for watching image different, it will according to institute
It states and establishes associated multi-medium data to play audio data corresponding to different perspectives.
Step S202, according to user watch image visual angle and distance difference and to audio data carry out volume weighting and
Direction adjustment.
Specifically, when user using playing device and just facing towards a certain visual angle in image when, the visual angle correspondence
Associated sound will transmit in front of user, and in image with associated sound is corresponded in other visual angles will be from user's
Left back and right back are transmitted.Also, user can adjust volume by weighting in the sound that each orientation transmits.
Above-mentioned apparatus for processing multimedia data and method are by obtaining the relative position between source of sound and image collecting device
Relationship, and then source of sound is positioned.In this way, forming the voice data with sense of direction, the experience of user can be greatly promoted
Feel.
Those skilled in the art it should be appreciated that more than embodiment be intended merely to illustrate the present invention,
And be not used as limitation of the invention, as long as within spirit of the invention, it is to the above embodiments
Appropriate change and variation are all fallen within the scope of protection of present invention.
Claims (10)
1. a kind of multimedia data processing method, which is characterized in that the method includes the steps:
Obtain the relative positional relationship between source of sound and image collecting device；
The audio data that image data in relationship acquisition preset range and source of sound are issued depending on that relative position；And
Image data in collected preset range is established into corresponding association with audio data.
2. multimedia data processing method as described in claim 1, which is characterized in that the acquisition source of sound and image collector
The step of relative positional relationship between setting, specifically includes:
Obtain the positioning signal of the output at source of sound；And
The relative positional relationship between source of sound and image collecting device is determined according to the positioning signal received.
3. multimedia data processing method as described in claim 1, which is characterized in that the acquisition source of sound and image collector
The step of relative positional relationship set, specifically includes:
The mobile angular momentum of source of sound is acquired by gyroscope；And
The angular momentum is converted into the corresponding azimuth information of source of sound to determine the opposite position between source of sound and image collecting device
Set relationship.
4. multimedia data processing method as described in claim 1, which is characterized in that the acquisition source of sound and image collector
The step of relative positional relationship set, specifically includes:
Displacement and/or the acceleration of image collecting device are acquired by linear accelerator；And
The displacement and/or acceleration are converted into the corresponding azimuth information of image collecting device to determine source of sound and Image Acquisition
Relative positional relationship between device.
5. multimedia data processing method as described in claim 1, which is characterized in that the acquisition source of sound and image collector
The step of relative positional relationship set, specifically includes:
By the angular momentum and displacement and/or acceleration of gyroscope and linear accelerator acquisition image collecting device, and will be adopted
The angular momentum of the image collecting device collected and displacement and/or acceleration are converted into first orientation information；
The mobile angular momentum of source of sound and displacement and/or acceleration are acquired by gyroscope and linear accelerator, and will be collected
The mobile angular momentum and displacement and/or acceleration of source of sound be converted into second orientation information；And
The opposite position between source of sound and image collecting device is determined according to the first orientation information and the second orientation information
Set relationship.
6. multimedia data processing method as described in claim 1, which is characterized in that the method also includes steps:
Determine that user watches the visual angle of image；And
Play the corresponding audio data in the visual angle.
7. multimedia data processing method as claimed in claim 6, which is characterized in that the method also includes steps:
Determine that user watches visual angle and the distance of image；And
It is adjusted according to the visual angle and apart from progress volume weighting and direction.
8. a kind of apparatus for processing multimedia data, which is characterized in that described device includes:
Acquiring unit, for obtaining the relative positional relationship between source of sound and image collecting device；
Image collecting device, for the image data in the acquisition preset range of relationship depending on that relative position；
Audio collecting device, the audio data issued for source of sound in the acquisition preset range of relationship depending on that relative position；
And
Processing unit, for the image data in collected preset range to be established corresponding association with audio data.
9. apparatus for processing multimedia data as claimed in claim 8, which is characterized in that the apparatus for processing multimedia data is also
Including storage unit, the storage unit is stored for that will establish corresponding associated image data with audio data.
10. apparatus for processing multimedia data as claimed in claim 8, which is characterized in that the apparatus for processing multimedia data
In playing device for playing the corresponding associated multi-medium data of foundations, and the direction for detecting user's viewing image
And visual angle, and volume weighting and direction adjustment are performed in accordance with according to the visual angle of user and distance.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711269168.2A CN109873933A (en) | 2017-12-05 | 2017-12-05 | Apparatus for processing multimedia data and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711269168.2A CN109873933A (en) | 2017-12-05 | 2017-12-05 | Apparatus for processing multimedia data and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109873933A true CN109873933A (en) | 2019-06-11 |
Family
ID=66916567
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711269168.2A Pending CN109873933A (en) | 2017-12-05 | 2017-12-05 | Apparatus for processing multimedia data and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109873933A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111883186A (en) * | 2020-07-10 | 2020-11-03 | 上海明略人工智能(集团)有限公司 | Recording device, voice acquisition method and device, storage medium and electronic device |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050147257A1 (en) * | 2003-02-12 | 2005-07-07 | Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. | Device and method for determining a reproduction position |
| CN105263085A (en) * | 2011-01-13 | 2016-01-20 | 高通股份有限公司 | Variable beamforming with a mobile platform |
| CN106027933A (en) * | 2016-06-21 | 2016-10-12 | 维沃移动通信有限公司 | Video recording method, video playing method and mobile terminal |
| CN106162206A (en) * | 2016-08-03 | 2016-11-23 | 北京疯景科技有限公司 | Panorama recording, player method and device |
| CN106200945A (en) * | 2016-06-24 | 2016-12-07 | 王杰 | Content reproduction apparatus, the processing system with this replay device and method |
| CN106993249A (en) * | 2017-04-26 | 2017-07-28 | 深圳创维－Rgb电子有限公司 | A kind of processing method and processing device of the voice data of sound field |
| CN107026974A (en) * | 2017-03-06 | 2017-08-08 | 浙江大学 | A kind of web camera audio enhancing and control method and system |
-
2017
- 2017-12-05 CN CN201711269168.2A patent/CN109873933A/en active Pending
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050147257A1 (en) * | 2003-02-12 | 2005-07-07 | Fraunhofer-Gesellschaft Zur Forderung Der Angewandten Forschung E.V. | Device and method for determining a reproduction position |
| CN105263085A (en) * | 2011-01-13 | 2016-01-20 | 高通股份有限公司 | Variable beamforming with a mobile platform |
| CN106027933A (en) * | 2016-06-21 | 2016-10-12 | 维沃移动通信有限公司 | Video recording method, video playing method and mobile terminal |
| CN106200945A (en) * | 2016-06-24 | 2016-12-07 | 王杰 | Content reproduction apparatus, the processing system with this replay device and method |
| CN106162206A (en) * | 2016-08-03 | 2016-11-23 | 北京疯景科技有限公司 | Panorama recording, player method and device |
| CN107026974A (en) * | 2017-03-06 | 2017-08-08 | 浙江大学 | A kind of web camera audio enhancing and control method and system |
| CN106993249A (en) * | 2017-04-26 | 2017-07-28 | 深圳创维－Rgb电子有限公司 | A kind of processing method and processing device of the voice data of sound field |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111883186A (en) * | 2020-07-10 | 2020-11-03 | 上海明略人工智能(集团)有限公司 | Recording device, voice acquisition method and device, storage medium and electronic device |
| CN111883186B (en) * | 2020-07-10 | 2022-12-23 | 上海明略人工智能(集团)有限公司 | Recording device, voice acquisition method and device, storage medium and electronic device |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10679676B2 (en) | Automatic generation of video and directional audio from spherical content | |
| US9940969B2 (en) | Audio/video methods and systems | |
| US7577316B2 (en) | System and method for creating, storing and utilizing images of a geographic location | |
| US10681276B2 (en) | Virtual reality video processing to compensate for movement of a camera during capture | |
| CN106162206A (en) | Panorama recording, player method and device | |
| WO2018106605A1 (en) | Distributed audio capturing techniques for virtual reality (vr), augmented reality (ar), and mixed reality (mr) systems | |
| US9654762B2 (en) | Apparatus and method for stereoscopic video with motion sensors | |
| WO2013055980A1 (en) | Method, system, and computer program product for obtaining images to enhance imagery coverage | |
| CN106131531A (en) | Method for processing video frequency and device | |
| CN106165402A (en) | Information reproduction apparatus, information regeneration method, information record carrier and information recording method | |
| You et al. | Internet of Things (IoT) for seamless virtual reality space: Challenges and perspectives | |
| KR20200067981A (en) | Method for processing vr audio and corresponding equipment | |
| US20140294366A1 (en) | Capture, Processing, And Assembly Of Immersive Experience | |
| US20200358415A1 (en) | Information processing apparatus, information processing method, and program | |
| JP6242011B2 (en) | Video management system and method for identifying a photographing terminal photographing an arbitrary area | |
| CN115668913A (en) | Stereoscopic display method, device, medium and system for field performance | |
| US20170111678A1 (en) | Method and apparatus for processing broadcast data by using external device | |
| CN109873933A (en) | Apparatus for processing multimedia data and method | |
| JP2015037242A (en) | Reception device, reception method, transmission device, and transmission method | |
| US20110316885A1 (en) | Method and apparatus for displaying image including position information | |
| US20190335153A1 (en) | Method for multi-camera device | |
| US10979806B1 (en) | Audio system having audio and ranging components | |
| WO2018027067A1 (en) | Methods and systems for panoramic video with collaborative live streaming | |
| KR101390811B1 (en) | Method and apparatus for receiving multiview camera parameters for stereoscopic image, and method and apparatus for transmitting multiview camera parameters for stereoscopic image | |
| KR20080006925A (en) | The method and system to get the frame data of moving shot with camera on a vehicle and the location data from location base service or gps and the direction data of the vehicle to send to server through wireless internet by real time and to be used that by another vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20190611 |