CN108076304A - A kind of built-in projection and the method for processing video frequency and conference system of camera array - Google Patents
A kind of built-in projection and the method for processing video frequency and conference system of camera array Download PDFInfo
- Publication number
- CN108076304A CN108076304A CN201611003597.0A CN201611003597A CN108076304A CN 108076304 A CN108076304 A CN 108076304A CN 201611003597 A CN201611003597 A CN 201611003597A CN 108076304 A CN108076304 A CN 108076304A
- Authority
- CN
- China
- Prior art keywords
- unit
- signal processor
- digital signal
- camera array
- array
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 13
- 238000012545 processing Methods 0.000 title claims abstract description 13
- 230000003993 interaction Effects 0.000 claims abstract description 5
- 230000033001 locomotion Effects 0.000 claims description 4
- 238000001514 detection method Methods 0.000 claims description 2
- 230000004807 localization Effects 0.000 claims description 2
- 230000005540 biological transmission Effects 0.000 abstract description 5
- 230000003287 optical effect Effects 0.000 abstract description 4
- 238000009434 installation Methods 0.000 description 3
- 230000011218 segmentation Effects 0.000 description 3
- 238000003491 array Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 230000006872 improvement Effects 0.000 description 2
- 230000009467 reduction Effects 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000004458 analytical method Methods 0.000 description 1
- 230000015572 biosynthetic process Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000012423 maintenance Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 238000003786 synthesis reaction Methods 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/14—Systems for two-way working
- H04N7/15—Conference systems
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N9/00—Details of colour television systems
- H04N9/12—Picture reproducers
- H04N9/31—Projection devices for colour picture display, e.g. using electronic spatial light modulators [ESLM]
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Studio Devices (AREA)
Abstract
The present invention discloses the method for processing video frequency and conference system of a kind of built-in projection and camera array, including microcontroller, connects network element, storage unit, man-machine interaction unit and digital signal processor on a microcontroller；Microphone unit, loudspeaker unit, camera array and the projecting cell being connected on digital signal processor；Projecting cell includes projector controller, light source unit, digital micromirror array and projection lens, and light source unit and digital micromirror array are controlled by the projector controller for being connected to digital signal processor.The multiple images that the digital signal processor exports camera array are spliced, and synthesize a wide angle picture, are needed cutting image according to user, are then done image coding, then pass through network transmission to other side.The video data that digital signal processor exports is converted into optical signal and shows output by the projecting cell.The present invention improves the ease for use and user experience of video conferencing system.
Description
Technical field
The present invention relates to field of multimedia communication, the video processing side of more particularly to a kind of built-in projection and camera array
Method and conference system.
Background technology
Video conference is as a kind of multimedia communication means, it is necessary to realize the acquisition of audio, video data, coding, transmission, solution
Code and display.Video conferencing system is made of several parts such as video terminal, microphone, camera and display device.Ordinary circumstance,
Video terminal only completes audio/video encoding/decoding function, and microphone realizes audio collection, and camera can be external, can also be integrated into video
Lane terminal, display device is using external projecting apparatus or large-screen tv wall etc..Various external equipments are needed through numerous and diverse line
Cable connects, meeting-place there are wiring confusion, difficult in maintenance, labial synchronization difficulty is high, echo processing is difficult the problems such as.
When video terminal needs to realize speaker tracking, close-up shot when functions.Conventional method be according to sound or
Image realizes the positioning of speaker, and the camera with holder is then controlled to focus on speaker.Shortcoming is when speaker carrys out switchback
It needs to rotate camera repeatedly when changing, since holder is mechanical structure, reacts slow there are speaker tracking, frequently uses and shorten the longevity
The shortcomings of life.Additionally, due to the limited angle of single camera, when video conference needs to cover the owner in entire meeting-place, often
Rule method is using wide-angle lens, and at this moment the marginal distortion of image is big.
By above-mentioned analysis, the video conference terminal wiring installation that prior art is realized is complicated, of high cost, makes
With it is more inconvenient the shortcomings of.Therefore, it is necessary to develop the user experience of new technological improvement video conferencing system.
The content of the invention
The present invention provides the method for processing video frequency and conference system of a kind of built-in projection and camera array, skill of the invention
Art scheme is as follows：
A kind of built-in projection and the method for processing video frequency and conference system of camera array, including microcontroller, are connected to micro-control
Network element, storage unit, man-machine interaction unit and digital signal processor on device processed；It is connected on digital signal processor
Microphone unit, loudspeaker unit, camera array and projecting cell；Projecting cell include projector controller, light source unit,
Digital micromirror array and projection lens, light source unit and digital micromirror array are controlled by the projection for being connected to digital signal processor
Device controls.The multiple images that the digital signal processor exports camera array are spliced, and synthesize a wide angle picture, root
Cutting image is needed according to user, image coding is then done, then passes through network transmission to other side.The projecting cell is by number
The video data of signal processor output is converted into optical signal and shows output.
It is a feature of the present invention that including：One microphone unit, for gathering the sound in meeting-place and giving digital signal
Processor processing.
It is a feature of the present invention that including：One loudspeaker unit, for playing the sound of Digital Signal Processing output.
It is a feature of the present invention that including：More than two cameras, circumference are placed.
It is a feature of the present invention that including：A built-in projecting cell, for the video for exporting digital signal processor
Data conversion is into optical signal and shows output.
Present invention additionally comprises human-computer interaction module, for user's control video terminal.
According to an aspect of the present invention, a kind of method that electronic platform is realized using camera array is provided, includes step
Suddenly：Multiple image is gathered simultaneously using camera array, is then spliced into a wide angle picture, based on Lip Movement Recognition speaker,
Then cut and obtain the close-up image of speaker.
According to another aspect of the present invention, a kind of method for generating panoramic picture is provided, includes step：From camera array
A wide angle picture is obtained, is encoded according to image, a multiple image is synthesized, is then encoded by image, network transmission arrives
Other side.
Overcome the deficiencies in the prior art of the present invention directly displays the figure of digital signal processor output using projection module
Picture, technical solution provided by the invention have many advantages, such as low latency, portable easy-to-use, reduction installation wiring, reduction mounting complexity.
Built-in multiple cameras need by the image that digital signal processing unit synthesis needs distal end to be supplied to export according to business,
Quick in real time reliable, output distal end picture delay is minimum in real time for built-in projection.Splicing broad angular image and auditory localization replacement make
The camera realized with mechanical holder tracks speaker, the switch speed of close-up shot when improving speaker's switching.The present invention
Making video conferencing system, audio-video signal processing synchronously completes without external video input apparatus and video display apparatus, without
The complicated wiring of conventional terminal, is convenient for carrying, and the organic cooperation of audio and video greatly improves user experience.
Description of the drawings
Fig. 1 is the system block diagram of first embodiment of the invention；
Fig. 2 is the position of microphone unit, loudspeaker unit, camera array and projecting cell that first embodiment of the invention provides
Put distribution map；
Fig. 3 is user's usage scenario of first embodiment of the invention；
Fig. 4 is the flow chart that first embodiment of the invention realizes speaker tracking；
Fig. 5 is the stitching image and speaker tracking image of first embodiment of the invention output；
Fig. 6 is the flow chart of second embodiment of the invention generation panoramic picture；
Fig. 7 is the panoramic picture of second embodiment of the invention output.
The embodiments will be further described with reference to the accompanying drawings for the realization, the function and the advantages of the object of the present invention.
Specific embodiment
Below in conjunction with the accompanying drawings and specific embodiment is described further with regard to technical scheme.It should be appreciated that this
Locate described specific embodiment to be only used to explain the present invention, be not intended to limit the present invention.
Fig. 1 is the system block diagram of the present invention：Including microcontroller 110, man-machine interaction unit on a microcontroller is connected
120th, network element 130, storage unit 140 and digital signal processor 180；It is connected to taking the photograph on digital signal processor 180
As head array 150, microphone unit 160, loudspeaker unit 170 and projecting cell 190；Projecting cell 190 includes projection and controls
Device 191, light source unit 192, digital micromirror array 194 and projection lens 193, light source unit 192 and digital micromirror array 194 by
The projector controller 191 for being connected to digital signal processor 180 controls.The digital signal processor 180 is by camera battle array
The multiple images splicing that row 150 export, synthesizes a wide angle picture, needs cutting image according to user, then does image volume
Code, then pass through network transmission to other side.The video data that the projecting cell 190 exports digital signal processor 180 turns
It changes optical signal into and shows output.Image output directly controls built-in projecting cell 190 and exports, so as to reach video in meeting
The low latency output of audio avoids the shortcomings that conventional video terminal labial synchronization is difficult and signal distortion is big.
Fig. 2 is microphone unit 210, projecting cell 230,220 and of camera array that first embodiment of the invention provides
The location map of loudspeaker unit 240.210 typically used as 3 211,212,213 circumference of directional microphone of microphone unit
It puts, realizes 360 degree of pickups.Loudspeaker unit 240 is centrally disposed position.Camera array 220 includes 2 ~ 6 cameras,
Typical case is：Each camera horizontal direction covers 60 degree, and 3 221,222,223 circumference of camera layouts are looped around and raise one's voice
Device unit surrounding realizes 180 degree covering.The video of distal end is directly projected out by built-in projection module 230 simultaneously.Microphone list
Member 210 tentatively judges to send out by information such as multiple microphone pickup intensity and phases by conventional microphone array location algorithm
Say the position of people, camera array 220 is by conventional recognition of face and motion detection so as to more accurately provide speaker
Position, by the rotary splicing of embedded digital signal processing unit so as to including speaker in image approximately mid way between.
Fig. 3 is user's usage scenario of first embodiment of the invention.
Fig. 4 is the flow chart that first embodiment of the invention realizes speaker tracking.The most important condition of image mosaic is two width
Image possesses common picture material, i.e. overlay region.Merging algorithm for images seeks to determine the overlapping region between two images,
And the position of two images is determined under a unified coordinate system according to the position of overlay region.In order to ensure stitching algorithm just
Often operation, the installation site of camera array needs to ensure the image of adjacent camera, and there are overlapping regions.This method includes：
Step S401 camera arrays export more sub-pictures simultaneously.
All image mosaics are obtained a secondary wide angle picture by step S402.Panoramic picture is obtained as splicing multiple image
There is very ripe algorithm, such as based on the relevant stitching algorithm in region, the relevant stitching algorithm of feature based.
Step S403 is according to the orientation of Lip Movement Recognition speaker.
Step S404 exports the close-up image of speaker.The mechanical complexity of camera is reduced, machinery need not be passed through
Cloud platform rotation solves the problems, such as speaker tracking.
Fig. 5 is the stitching image and speaker tracking image of first embodiment of the invention output.Black surround image is finally to send
To the image of coding.Various outputs are quickly handled using algorithm, are realized without conventional mechanical cloud platform rotation.
Fig. 6 is the flow chart of second embodiment of the invention generation panoramic picture.This method includes：
Step S601 camera arrays export more sub-pictures simultaneously.
All image mosaics are obtained a secondary wide angle picture by step S602.
Step S603 is to wide angle picture according to portrait segmentation figure picture.
Step S604 generates panoramic picture according to user's control.For example segmentation image scaling is generated entirely with more visual formats
Scape image；The excess margins that segmentation figure picture is either removed to nobody's picture splice according still further to 720P the 1080P sizes of standard
A complete image is obtained, simulates the natural angular field of view of people, dynamic generation image is output to distal end.So as to solve convention video
Requirement of the conference system to participant position.
Fig. 7 is the panoramic picture that second embodiment of the invention is exported with more visual formats.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the invention, for the skill of this field
For art personnel, the invention may be variously modified and varied.Within the spirit and principles of the invention, that is made any repaiies
Change, equivalent substitution, improvement etc., should all be included in the protection scope of the present invention.
Claims (5)
1. a kind of built-in projection and the method for processing video frequency and conference system of camera array, including：Microcontroller is connected to micro-
Network element, storage unit, man-machine interaction unit and digital signal processor on controller, it is characterised in that：
Microphone unit, loudspeaker unit, camera array and projecting cell are connected to digital signal processor；
Projecting cell includes projector controller, light source unit, digital micromirror array and projection lens, light source unit and digital micro-mirror
Array is controlled by the projector controller for being connected to digital signal processor.
2. video conferencing system as described in claim 1, which is characterized in that projecting cell built in terminal inner and loud speaker list
Member, the image and sound that real-time synchronization output distal end is come.
3. video conferencing system as described in claim 1, which is characterized in that the quantity of camera array is 2 ~ 6, and circumference is put
It puts, corresponding microphone unit is also that circumference is put.
4. video conferencing system as described in claim 1, which is characterized in that video and audio decoder synchronously complete, and solve yardage
Built-in audio and video projection are driven when factually, synchronism output is transferred without external other equipment.
5. a kind of speaker's localization method based on microphone and camera array, which is characterized in that camera array input and
The organic cooperation of microphone input unit by conventional microphone array location algorithm, is received according to multiple directions microphone and believed
Number intensity and time delay rough estimate go out the orientation of speaker, obtain by conventional recognition of face and motion detection algorithm and then accurately
The orientation of main speaker simultaneously needs to export the close-up image of speaker or is needed to splice multiple take the photograph according to business according to business
As the image of head, obtain a secondary wide angle picture and give distal end conference facility output.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611003597.0A CN108076304A (en) | 2016-11-15 | 2016-11-15 | A kind of built-in projection and the method for processing video frequency and conference system of camera array |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201611003597.0A CN108076304A (en) | 2016-11-15 | 2016-11-15 | A kind of built-in projection and the method for processing video frequency and conference system of camera array |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN108076304A true CN108076304A (en) | 2018-05-25 |
Family
ID=62162459
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201611003597.0A Pending CN108076304A (en) | 2016-11-15 | 2016-11-15 | A kind of built-in projection and the method for processing video frequency and conference system of camera array |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108076304A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108718404A (en) * | 2018-05-26 | 2018-10-30 | 明基智能科技（上海）有限公司 | Image correcting method and image correcting system |
| CN108924522A (en) * | 2018-06-22 | 2018-11-30 | 四川长虹电器股份有限公司 | A kind of automatic focusing mechanism and its application method of ultrashort out-of-focus projection's equipment |
| CN109089087A (en) * | 2018-10-18 | 2018-12-25 | 广州市盛光微电子有限公司 | The audio-visual linkage of multichannel |
| CN112887654A (en) * | 2021-01-25 | 2021-06-01 | 联想(北京)有限公司 | Conference equipment, conference system and data processing method |
| CN113037994A (en) * | 2019-12-09 | 2021-06-25 | 茂杰国际股份有限公司 | Video conference panoramic image unfolding method |
| CN114584750A (en) * | 2022-03-17 | 2022-06-03 | 宏立信（厦门）科技有限公司 | Projector and projection method |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101534413A (en) * | 2009-04-14 | 2009-09-16 | 深圳华为通信技术有限公司 | System, method and apparatus for remote representation |
| CN102368816A (en) * | 2011-12-01 | 2012-03-07 | 中科芯集成电路股份有限公司 | Intelligent front end system of video conference |
| CN102833476A (en) * | 2012-08-17 | 2012-12-19 | 歌尔声学股份有限公司 | Camera for terminal equipment and implementation method of camera for terminal equipment |
| CN102998880A (en) * | 2011-09-13 | 2013-03-27 | 宇达电脑（上海）有限公司 | Projector with function of camera and shooting method |
| CN103841357A (en) * | 2012-11-21 | 2014-06-04 | 中兴通讯股份有限公司 | Microphone array sound source positioning method, device and system based on video tracking |
| CN205071183U (en) * | 2015-09-16 | 2016-03-02 | 南京城市职业学院 | A equipment for multi -user video conferencing |
| US20160105643A1 (en) * | 2008-07-28 | 2016-04-14 | Centurylink Intellectual Property Llc | System and Method for Projection Utilizing a Wireless Device |
| CN105592268A (en) * | 2016-03-03 | 2016-05-18 | 苏州科达科技股份有限公司 | Video conferencing system, processing device and video conferencing method |
-
2016
- 2016-11-15 CN CN201611003597.0A patent/CN108076304A/en active Pending
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20160105643A1 (en) * | 2008-07-28 | 2016-04-14 | Centurylink Intellectual Property Llc | System and Method for Projection Utilizing a Wireless Device |
| CN101534413A (en) * | 2009-04-14 | 2009-09-16 | 深圳华为通信技术有限公司 | System, method and apparatus for remote representation |
| CN102998880A (en) * | 2011-09-13 | 2013-03-27 | 宇达电脑（上海）有限公司 | Projector with function of camera and shooting method |
| CN102368816A (en) * | 2011-12-01 | 2012-03-07 | 中科芯集成电路股份有限公司 | Intelligent front end system of video conference |
| CN102833476A (en) * | 2012-08-17 | 2012-12-19 | 歌尔声学股份有限公司 | Camera for terminal equipment and implementation method of camera for terminal equipment |
| CN103841357A (en) * | 2012-11-21 | 2014-06-04 | 中兴通讯股份有限公司 | Microphone array sound source positioning method, device and system based on video tracking |
| CN205071183U (en) * | 2015-09-16 | 2016-03-02 | 南京城市职业学院 | A equipment for multi -user video conferencing |
| CN105592268A (en) * | 2016-03-03 | 2016-05-18 | 苏州科达科技股份有限公司 | Video conferencing system, processing device and video conferencing method |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108718404A (en) * | 2018-05-26 | 2018-10-30 | 明基智能科技（上海）有限公司 | Image correcting method and image correcting system |
| CN108924522A (en) * | 2018-06-22 | 2018-11-30 | 四川长虹电器股份有限公司 | A kind of automatic focusing mechanism and its application method of ultrashort out-of-focus projection's equipment |
| CN109089087A (en) * | 2018-10-18 | 2018-12-25 | 广州市盛光微电子有限公司 | The audio-visual linkage of multichannel |
| CN109089087B (en) * | 2018-10-18 | 2020-09-29 | 广州市盛光微电子有限公司 | Multi-channel audio-video linkage device |
| CN113037994A (en) * | 2019-12-09 | 2021-06-25 | 茂杰国际股份有限公司 | Video conference panoramic image unfolding method |
| CN112887654A (en) * | 2021-01-25 | 2021-06-01 | 联想(北京)有限公司 | Conference equipment, conference system and data processing method |
| CN114584750A (en) * | 2022-03-17 | 2022-06-03 | 宏立信（厦门）科技有限公司 | Projector and projection method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108076304A (en) | A kind of built-in projection and the method for processing video frequency and conference system of camera array | |
| CN102084650B (en) | Telepresence system, method and video capture device | |
| US11688034B2 (en) | Virtual lens simulation for video and photo cropping | |
| CN106210703A (en) | The utilization of VR environment bust shot camera lens and display packing and system | |
| TWI440361B (en) | Apparatus and method for overlaying image in video presentation system having embedded operating system | |
| CN102368816A (en) | Intelligent front end system of video conference | |
| WO2005002201A2 (en) | Visual teleconferencing apparatus | |
| WO2012100114A2 (en) | Multiple viewpoint electronic media system | |
| CN110333837B (en) | Conference system, communication method and device | |
| CA3190886A1 (en) | Merging webcam signals from multiple cameras | |
| US11006072B2 (en) | Window system based on video communication | |
| CN114827517A (en) | Projection video conference system and video projection method | |
| US20150304559A1 (en) | Multiple camera panoramic image capture apparatus | |
| CN115118880A (en) | XR virtual shooting system based on immersive video terminal is built | |
| CN207397518U (en) | Wireless mobile objects projection system | |
| CN106210701A (en) | A kind of mobile terminal for shooting VR image and VR image capturing apparatus thereof | |
| JP2005117285A (en) | Information input device, communication terminal and communication method | |
| CN208623757U (en) | A kind of image mosaic emerging system | |
| US11665391B2 (en) | Signal processing device and signal processing system | |
| US20220303320A1 (en) | Projection-type video conference system and video projecting method | |
| US10045000B1 (en) | Method and system for omni-directional audio signal acquisition and playback | |
| CN114286121A (en) | Method and system for realizing picture guide live broadcast based on panoramic camera | |
| CN206807670U (en) | The servomechanism live for panorama | |
| Kuchelmeister et al. | Immersive mixed media augmented reality applications and technology | |
| CN113271482A (en) | Portable full-width image scratching blackboard |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180525 |
|
| WD01 | Invention patent application deemed withdrawn after publication |