TWI739585B - Full fov conference camera device - Google Patents
Full fov conference camera device Download PDFInfo
- Publication number
- TWI739585B TWI739585B TW109130638A TW109130638A TWI739585B TW I739585 B TWI739585 B TW I739585B TW 109130638 A TW109130638 A TW 109130638A TW 109130638 A TW109130638 A TW 109130638A TW I739585 B TWI739585 B TW I739585B
- Authority
- TW
- Taiwan
- Prior art keywords
- conference
- mode
- image
- presentation
- full
- Prior art date
Links
Images
Landscapes
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
- Studio Devices (AREA)
Abstract
Description
本發明涉及一種全視角會議攝影裝置,更詳而言之,為一具備多個攝影鏡頭,可針對會議室360°的視角,搭配目標追蹤(ROI)與影像處理,使會議發言人可在簡報模式(Presentation Mode)下,改善現有線上會議的攝影系統。 The present invention relates to a full-view conference photographing device. More specifically, it is a camera with multiple photographic lenses, which can aim at a 360° viewing angle of a meeting room, with target tracking (ROI) and image processing, so that conference speakers can present presentations In the Presentation Mode, improve the existing online conference photography system.
近年來,由於遠距辦公的興起,為了使分散在遠端的工作者即便不在一固定的地點辦公,也能快速的參與工作會議,因此線上視訊會議應運而生。所謂的線上視訊會議,為一種將相隔多地的終端,經由影像、語音與資料輸出入設備以及網絡連結,使會議參加者可以達成如同在同一會議地點,迅速確實地做資料交換、意見溝通的一種系統。視訊會議為達成多方溝通的目的,會議室均配備有攝影機、屏幕、顯示器、麥克風,與喇叭等裝備,並經由終端裝置相互連接。一般而言,會議中的攝影機除了會拍攝參加者和主持人外,各終端也須能傳輸書面資料以及電子資料,以達溝通之目的。 In recent years, due to the rise of remote offices, in order to enable workers dispersed at remote locations to quickly participate in work meetings even if they are not working in a fixed location, online video conferencing has emerged as the times require. The so-called online video conference is a kind of terminal that is separated from many places, through video, voice and data input and output equipment and network connection, so that conference participants can reach the same meeting place, quickly and reliably exchange data and communicate opinions. A system. In order to achieve the purpose of multi-party communication, the meeting rooms are equipped with cameras, screens, monitors, microphones, and speakers, and are connected to each other through terminal devices. Generally speaking, in addition to taking pictures of the participants and the host, each terminal must also be able to transmit written and electronic data for the purpose of communication.
在過往的會議視訊系統中,若一地,或多地的終端包含多個會議參加者,其在各視訊會議的畫面上,會出現不同的攝影機拍攝的影像視窗,以分別顯示出各個會議參加者的情況,然而此種畫面顯示方式,並未考量到參加者之間的互動關係與環境連結,導致參加者缺乏彼此間的聯繫感,常造成參加者並非參與同一場會議,甚至開會時產生事不關己的錯覺。此外,過往的會議視訊系統通常缺乏整合,在同一會議地點若參加會議的人數較多,則所架設多部的攝影機、麥克風、傳輸線路四散各處,除了前述畫面顯示問題外,每當會議的發言人離開攝影機的視角範圍時,也可能產生會議中斷,需要人工調整攝影機的位置,使得過往的視訊系統不論是攜帶、架設,與使用的過程皆顯得繁瑣不便。 In the past conference video system, if the terminal in one place or multiple places contains multiple conference participants, on the screen of each video conference, there will be image windows taken by different cameras to show the participants of each conference. However, this screen display method does not take into account the interactive relationship and environmental connection between the participants, resulting in a lack of a sense of connection between the participants, which often results in participants not participating in the same meeting, or even during the meeting. The illusion that it's nothing to do. In addition, the past conference video system usually lacks integration. If there are more people attending the meeting at the same meeting place, multiple cameras, microphones, and transmission lines are scattered all over the place. In addition to the aforementioned screen display problems, whenever the meeting is When the speaker leaves the camera's field of view, the meeting may also be interrupted, requiring manual adjustment of the camera's position, which makes the process of carrying, setting up, and using the previous video system seem cumbersome and inconvenient.
基於上述問題,在目前時點的市場上,遂有廠商推出一種整合式的會議視訊系統,將上述的攝影機、麥克風、傳輸線路、傳輸模組等整合為一可攜式的裝置,以提高其安裝與使用的便利性。例如,由Owl Labs公司所提出的日本專利JP2018521593A中,即展示了一具有數個攝影鏡頭、麥克風的整合型視訊裝置,其可依據攝影鏡頭、麥克風所取得的音頻資料定義在會議時,自動跟蹤參加者或主持人的功能,以及依據參與會議的人數分割或縫合影像的功能。又例如,Polycom公司所提出的日本專利JP2011244455A,其同樣展示了一種會議視訊系統,並具有針對會議參加者整合或分割影像,以及追蹤主持人的功能。 Based on the above problems, in the current market, some manufacturers have launched an integrated conference video system, which integrates the above-mentioned camera, microphone, transmission line, transmission module, etc. into a portable device to improve its installation. And ease of use. For example, in Japanese patent JP2018521593A proposed by Owl Labs, it shows an integrated video device with several camera lenses and microphones, which can automatically track during the meeting according to the audio data obtained by the camera lens and microphone. The function of participants or moderators, and the function of dividing or stitching images according to the number of participants in the meeting. For another example, the Japanese patent JP2011244455A proposed by Polycom Corporation also shows a conference video system, and has the functions of integrating or splitting images for conference participants and tracking the host.
然而,由於會議地點的環境不同,其可能包含小會議室,或較大型的講堂,目前市場上的會議視訊系統依然未見有同時具備依據會議地點的不同,而可以針對影像的解析度、幀率(Frames per second,fps)進行選擇,以及強調攝影畫面在追蹤主持人或參加者時(ROI,Region of Interest),可隨其講題的不同而能具有多個縫合、拼貼所需影像模式的功能。此外,現有ROI功能一般多透過機械馬達移動的攝影鏡頭,在追蹤過程中需要物理上的轉動鏡頭,可能導致影像畫面的不穩定,更有甚者,許多的會議視訊系統依然採用分散式設計而未將其整合為一體成型的形式,是以,基於以上缺失,現有視訊系統依然具有進一步改進的空間。 However, due to the different environment of the meeting place, which may include small meeting rooms or larger lecture halls, there is still no meeting video system on the market that has the ability to target the resolution and frame of the image depending on the meeting place. Select the frame rate (Frames per second, fps), and emphasize that when the photographic screen is tracking the host or participant (ROI, Region of Interest), it can have multiple stitching and collage images according to the different topics. Mode function. In addition, the existing ROI function generally uses a camera lens that is moved by a mechanical motor. The camera needs to be physically rotated during the tracking process, which may cause the image to be unstable. What's more, many conference video systems still adopt a decentralized design. It has not been integrated into a one-piece form, so based on the above shortcomings, the existing video system still has room for further improvement.
為解決上述問題,本發明提出一種全視角會議攝影裝置,藉由一處理模組,協調系統的運作與管理,其詳細的架構包含:殼體,用以乘載或配置有關之電子元件以及機構;n個攝影單元,拍攝會議影像;影音處理模組,依據應用之需要,裁切或拼貼前述n個攝影單元所拍攝的會議影像。其中,影音處理模組進一步包含:目標追蹤單元,以追蹤會議參加者所在的位置;以及,分割單元,依據應用之所需將會議影像分割為全景模式(Panorama mode)、專注模式(Focus mode),與多畫面模式。其中,所述的多畫面模式可將影像分為具有上下分割畫面的分割模式(Top-Down mode),以及多人畫面的影格模式(Grid Mode)。其中,根據本發明最佳化的實施例,攝影單元的數量為4個(亦即,n=4),此外,於本案中,為便於說明,所述的第一攝影單元、第二攝影單元、第三攝影單元, 可被統稱為攝影單元,於此先行予以敘明。其中上述n個攝影單元等距安排在殼體上的一虛擬圓周線上。 In order to solve the above problems, the present invention proposes a full-view conference camera device. A processing module is used to coordinate the operation and management of the system. The detailed structure includes a housing for carrying or disposing related electronic components and mechanisms. ; N photographing units to shoot conference images; the audio-visual processing module, according to the needs of the application, cuts or collages the conference images taken by the aforementioned n photographing units. Among them, the audio-visual processing module further includes: a target tracking unit to track the location of the meeting participants; and a segmentation unit to segment the meeting image into Panorama mode and Focus mode according to the needs of the application , And multi-screen mode. Among them, the multi-screen mode can divide the image into a top-down mode (Top-Down mode) with a top-down split screen, and a grid mode (Grid Mode) with a multi-person screen. Among them, according to the optimized embodiment of the present invention, the number of photographing units is 4 (that is, n=4). In addition, in this case, for ease of description, the first photographing unit and the second photographing unit , The third photography unit, They can be collectively referred to as photographic units, which are described here first. The above n photographing units are arranged equidistantly on a virtual circle on the housing.
根據本發明內容,影音處理模組包含簡報單元,其中,簡報單元內安裝有一簡報模式(Presentation Mode),將會議主持人或發言人所在位置的會議影像,將所需背景的會議影像(例如,會議室的白板、投影幕,或平面牆壁),加以拼接為簡報影像。 According to the content of the present invention, the audio-visual processing module includes a presentation unit, wherein a presentation mode (Presentation Mode) is installed in the presentation unit. Whiteboards, projection screens, or flat walls in the meeting room) are spliced into presentation images.
根據本發明內容,當線上會議開始時,可於不同地理區域(例如:第一講堂、第二講堂、第三講堂,或是會議參加者的家中等等)配置至少兩個以上的全視角會議攝影裝置,可組成一雲端會議系統,並透過傳輸模組與雲端網路連結各個終端,透過傳輸模組,雲端會議系統將每個全視角會議攝影裝置的攝影單元、影音處理模組加以整合,使各終端均能接收到會議影像,及對應的電子資料與書面資料。 According to the content of the present invention, when an online conference starts, at least two full-view conferences can be configured in different geographic areas (for example: the first lecture hall, the second lecture hall, the third lecture hall, or the homes of conference participants, etc.) The camera device can form a cloud conference system, and connect each terminal with the cloud network through the transmission module. Through the transmission module, the cloud conference system integrates the camera unit and audio-visual processing module of each full-view conference camera device. So that each terminal can receive the conference video, and the corresponding electronic data and written data.
根據本發明之內容,所述的全景模式(Panorama mode),為分割單元將n個攝影單元所拍攝,角度範圍為90°-360°的會議影像,處理為一可在平面顯示的全景影像。其中,在本發明的實施例中,每一個攝影單元的視角範圍(Field of view,FOV),為30°-100°。 According to the content of the present invention, in the Panorama mode, a conference image with an angle range of 90°-360° captured by n photographing units by a dividing unit is processed into a panoramic image that can be displayed on a plane. Wherein, in the embodiment of the present invention, the field of view (FOV) of each photographing unit is 30°-100°.
根據本發明之內容,所述的專注模式(Focus mode),為將目標追縱單元所追蹤並聚焦各個會議的參加者所在的位置後,以分割單元將各個參加者預定範圍內的專注影像處理後呈現。 According to the content of the present invention, the focus mode is to track and focus on the location of the participants of each meeting by the target tracking unit, and then process the focused images within the predetermined range of each participant by the segmentation unit After rendering.
根據本發明之內容,全視角會議攝影裝置的外觀為一體成形,其中所述的n個攝影單元採用一體式架構,以改善習知會議視訊系統的攝影機四散各處的缺失。 According to the content of the present invention, the appearance of the full-view conference camera device is integrally formed, wherein the n camera units adopt an integrated structure to improve the lack of scattered cameras in the conventional conference video system.
根據本發明一實施例,攝影單元的設置為固定式,具有光學變焦或數位變焦進行會議影像的上、下、左、右調整與縮放(左右調整Pan;上下調整 Tile;縮放Zoom。英語簡寫:PTZ模式),改善習知會議視訊系統的攝影機需以PTZ調整會議影像時,需藉由機械馬達,可能使攝影機產生晃動,使會議影像不穩定的缺失。 According to an embodiment of the present invention, the camera unit is set to a fixed type, with optical zoom or digital zoom for up, down, left, and right adjustment and zooming of conference images (left and right adjustment Pan; up and down adjustment Tile; zoom zoom. English abbreviation: PTZ mode). To improve the conventional conference video system, when the camera needs to use PTZ to adjust the conference image, a mechanical motor is required, which may cause the camera to shake and make the conference image unstable.
根據本發明內容,全視角會議攝影裝置包含一傳輸模組,以將會議影像、全景影像、簡報影像、專注影像傳輸至外部終端。其中,在本發明一實施例中,該傳輸模組的規格可為但不限於USB Mini、USB Micro、USB Type A、USB Type B、USB Type C、LPT、RS232、PS/2,或以上之組合,並可依據應用的需要替換任意所需的傳輸規格。 According to the content of the present invention, the full-view conference camera device includes a transmission module to transmit conference images, panoramic images, presentation images, and focused images to external terminals. Wherein, in an embodiment of the present invention, the specification of the transmission module can be but not limited to USB Mini, USB Micro, USB Type A, USB Type B, USB Type C, LPT, RS232, PS/2, or any of the above Combination, and can replace any required transmission specifications according to the needs of the application.
以上所述係用以說明本發明之目的、技術手段以及其可達成之功效,相關領域內熟悉此技術之人可以經由以下實施例之示範與伴隨之圖式說明及申請專利範圍更清楚明瞭本發明。 The above descriptions are used to illustrate the purpose, technical means and achievable effects of the present invention. Those familiar with this technology in the relevant field can get a clearer understanding of the present invention through the demonstration of the following examples and accompanying schematic descriptions and the scope of patent applications. invention.
100:全視角會議攝影裝置 100: Full-view conference photography installation
200:雲端會議系統 200: Cloud Conference System
201:處理模組 201: Processing Module
203:第一攝影單元 203: The first photography unit
205:第二攝影單元 205: The second photography unit
207:第三攝影單元 207: The third photography unit
209:影音處理模組 209: Audio and Video Processing Module
209a:目標追蹤單元 209a: Target tracking unit
209c:分割單元 209c: segmentation unit
209e:簡報模式單元 209e: Presentation mode unit
211:傳輸模組 211: Transmission Module
213:殼體 213: Shell
300a:全景模式 300a: Panorama mode
300b:多畫面模式 300b: Multi-picture mode
300c:分割模式 300c: Split mode
300d:影格模式 300d: Frame mode
300f:專注模式 300f: Focus mode
500a:簡報模式 500a: Presentation mode
如下所述之對本發明的詳細描述與實施例之示意圖,應使本發明更被充分地理解;然而,應可理解此僅限於作為理解本發明應用之參考,而非限制本發明於一特定實施例之中。 The detailed description of the present invention and the schematic diagrams of the embodiments described below should make the present invention more fully understood; however, it should be understood that this is only used as a reference for understanding the application of the present invention, and does not limit the present invention to a specific implementation. In the case.
圖1顯示全視角會議攝影裝置的正面外觀。 Figure 1 shows the front appearance of a full-view conference camera device.
圖2係說明全視角會議攝影裝置的系統架構。 Figure 2 illustrates the system architecture of a full-view conference camera device.
圖3顯示分割單元的細部系統架構。 Figure 3 shows the detailed system architecture of the segmentation unit.
圖4A說明分割單元以全景模式將會議影像處理為全景影像。 FIG. 4A illustrates that the dividing unit processes the conference image into a panoramic image in a panoramic mode.
圖4B說明分割單元以分割模式將會議影像處理為具有上下分割畫面的分割影像。 FIG. 4B illustrates that the dividing unit processes the conference video into a divided video with top and bottom divided screens in a divided mode.
圖4C說明分割單元以分割模式將會議影像處理為影格形式的分割影像。 FIG. 4C illustrates that the dividing unit processes the conference video into a divided video in the form of a frame in a dividing mode.
圖4D說明分割單元以專注模式將會議影像處理為專注影像。 Figure 4D illustrates that the segmentation unit processes the conference image into a focused image in a focused mode.
圖5顯示簡報模式單元的細部系統架構。 Figure 5 shows the detailed system architecture of the presentation mode unit.
圖6A說明簡報單元對於簡報模式的影像處理方式。 Fig. 6A illustrates the image processing method of the presentation unit for the presentation mode.
圖6B說明簡報單元對於簡報模式的影像處理方式。 FIG. 6B illustrates the image processing method of the presentation unit for the presentation mode.
圖6C說明簡報單元對於簡報模式的影像處理方式。 Fig. 6C illustrates the image processing method of the presentation unit for the presentation mode.
圖6D說明雲端會議系統於各個不同地點透過雲端網路進行線上會議的使用情形。 Figure 6D illustrates the use of the cloud conference system for online conferences in various locations through the cloud network.
本發明將以較佳之實施例及觀點加以詳細敘述。下列描述提供本發明特定的施行細節,俾使閱者徹底瞭解這些實施例之實行方式。然該領域之熟習技藝者須瞭解本發明亦可在不具備這些細節之條件下實行。此外,本發明亦可藉由其他具體實施例加以運用及實施,本說明書所闡述之各項細節亦可基於不同需求而應用,且在不悖離本發明之精神下進行各種不同的修飾或變更。本發明將以較佳實施例及觀點加以敘述,此類敘述係解釋本發明之結構,僅用以說明而非用以限制本發明之申請專利範圍。以下描述中使用之術語將以最廣義的合理方式解釋,即使其與本發明某特定實施例之細節描述一起使用。 The present invention will be described in detail with preferred embodiments and viewpoints. The following description provides specific implementation details of the present invention, so that the reader can thoroughly understand the implementation of these embodiments. However, those skilled in the field must understand that the present invention can also be implemented without these details. In addition, the present invention can also be applied and implemented by other specific embodiments. The details described in this specification can also be applied based on different needs, and various modifications or changes can be made without departing from the spirit of the present invention. . The present invention will be described in terms of preferred embodiments and viewpoints. Such description is to explain the structure of the present invention, and is only for illustration and not to limit the scope of patent application of the present invention. The terms used in the following description will be interpreted in the broadest reasonable manner, even if they are used in conjunction with the detailed description of a specific embodiment of the present invention.
本發明之第一目的,在於改善習知會議視訊系統僅能將部署在會議室各處,或各個終端的攝影機所拍攝的影像單純的加以顯示,而未有將所拍攝的會議的各個參加者與主持人的會議影像,加以整合的功能,使得參加者缺乏彼此間的聯繫感,而沒有參加同一場會議的錯覺。本發明之第二目的,在於改善習 知會議視訊系統缺乏整合,在同一會議地點若參加會議的人數較多,則所架設多部的攝影機、麥克風、傳輸線路四散各處,除了前述畫面顯示問題外,每當會議的發言人離開攝影機的視角範圍時,也鮮少具備目標追蹤(ROI,Region of Interest)的相關功能,需要人工調整攝影機的位置,使得系統不論是攜帶、架設的過程皆顯得不便。本發明之第三目的,在於改善現有的會議視訊系統所附攝影機的ROI功能通常藉由機械馬達調整攝影機的拍攝角度,在追蹤過程中需要物理上的轉動鏡頭,可能導致影像畫面的不穩定。最後,本發明之第四目的,在於改善習知攝影機的影像解析度與禎率,使得線上會議的過程中,可依據不同的使用情況調整會議影像的品質,達到藉由線上會議溝通意見或交換資料,增加遠距離辦公的效率。為達上述目的,本發明詳細的技術手段與實施方式,將詳述如後。 The first purpose of the present invention is to improve the conventional video conference system that can only display the images taken by the cameras deployed in various places in the conference room, or by the cameras of each terminal, instead of displaying each participant of the taken meeting. The function of integrating the meeting video with the host makes the participants lack the sense of connection with each other, and does not have the illusion of participating in the same meeting. The second purpose of the present invention is to improve the It is known that the conference video system lacks integration. If there are more people participating in the meeting at the same meeting place, multiple cameras, microphones, and transmission lines will be scattered everywhere. In addition to the aforementioned screen display problems, whenever the speaker of the meeting leaves the camera In the range of angle of view, there are few functions related to ROI (Region of Interest), and the position of the camera needs to be adjusted manually, which makes the process of carrying and erecting the system inconvenient. The third object of the present invention is to improve the ROI function of the camera attached to the existing conference video system. The shooting angle of the camera is usually adjusted by a mechanical motor. During the tracking process, the lens needs to be physically rotated, which may lead to instability of the image. Finally, the fourth objective of the present invention is to improve the image resolution and frame rate of conventional cameras, so that during the online meeting, the quality of the meeting images can be adjusted according to different usage conditions, so as to achieve communication or exchange of opinions through online meetings. Data, increase the efficiency of remote office. To achieve the above objectives, the detailed technical means and implementation manners of the present invention will be described in detail later.
請參閱圖1,為解決上述問題,本發明提出一種具有一體成型架構的全視角會議攝影裝置(100),藉由一處理模組(201),耦接系統中的各個元件,協調系統的運作與管理。請參閱圖2,本發明詳細的系統架構,包含:一殼體(213),用以乘載或配置本系統之電子元件以及機構;n個攝影單元,拍攝會議影像;影音處理模組(209),依據應用之需要,裁切或拼貼前述n個攝影單元所拍攝的會議影像。其中上述n個攝影單元等距安排在上述殼體(213)之虛擬水平圓周線上,俾使其具有相同高度且等距配置,上述殼體(213)之截面可為方形、矩形、圓形、多邊形等等。其中,影音處理模組(209)進一步包含:目標追蹤單元(209a),以追蹤會議參加者所在的位置;以及,分割單元(209c),依據應用之所需將會議影像分割為全景模式(300a,Panorama mode)、專注模式(300c,Focus mode),與多畫面模式(300b)。其中,所述的多畫面模式可將影像分為具有上下分割畫面的分割模式(Top-Down mode),以及多人畫面的影格模式(Grid Mode)。此外,在本發明中,所述處理模組(201),通常包含處理晶片(Microprocessor Control Unit,MCU)、記憶體、顯示卡、網路卡、作業系統及應用程式等等,以通常已知方式相互連接,執行運算、暫存、顯示及資料傳輸,提供全視角會議攝影裝置(100)之運作與管理協調等功能,其中,該作業系統可支援的類型,可為但不限於Linux、Windos、Android、Mac OS或iOS,基於處理模組(201)屬於通常已知的架構,故在此即不再贅述。 Please refer to FIG. 1. In order to solve the above-mentioned problems, the present invention proposes a full-view conference camera device (100) with an integrated structure. A processing module (201) is used to couple the various components in the system to coordinate the operation of the system. And management. Please refer to FIG. 2, the detailed system architecture of the present invention includes: a housing (213) for carrying or configuring the electronic components and mechanisms of the system; n photographing units for shooting conference images; and an audio-visual processing module (209) ), according to the needs of the application, crop or collage the conference images taken by the aforementioned n photographing units. Wherein, the n photographing units are arranged equidistantly on the virtual horizontal circumference of the housing (213), so that they have the same height and are arranged equidistantly. The section of the housing (213) can be square, rectangular, circular, Polygons and so on. Wherein, the audio-visual processing module (209) further includes: a target tracking unit (209a) to track the location of the conference participants; and a segmentation unit (209c) to divide the conference image into a panoramic mode (300a) according to the needs of the application , Panorama mode), focus mode (300c, Focus mode), and multi-screen mode (300b). Among them, the multi-screen mode can divide the image into a top-down mode (Top-Down mode) with top and bottom split screens, and a grid mode (Grid Mode) with multi-person screens. In addition, in the present invention, the processing module (201) usually includes a processing chip (Microprocessor Control Unit, MCU), memory, display card, network card, operating system and application programs, etc., which are generally known It can be connected to each other in ways to perform calculation, temporary storage, display and data transmission, and provide functions such as the operation and management coordination of the full-view conference camera device (100). Among them, the type of operating system that can be supported can be, but not limited to, Linux, Windos , Android, Mac OS or iOS, based on the processing module (201) belongs to the generally known architecture, so it will not be repeated here.
請參閱圖1,根據本發明一實施例,n個攝影單元以環型排列的方式,平均設置於全視角會議攝影裝置(100)的容置殼體(213)之上,使每一個攝影單元能夠含概360°/n的拍攝視角。例如,在本發明較佳地實施例中,具有4個攝影單元,則每一個攝影單元的拍攝視角為90°。其中,應當注意者為,攝影單元的鏡頭可以依據應用的需要,如針對所需擺設的會議室或講堂的面積大小,選擇不同的鏡頭直徑、視角範圍與鏡頭數量,以達到最佳化地拍攝效果。例如,在上述的實施例中,當全視角會議攝影裝置(100)具有4個攝影單元時,其所拍攝出的會議影像,其解析度與對應的幀率,可以達到下列之標準:3840 X 2160,10-50fps;3840 X 640,10-60fps;1920 X 1080,10-70fps。 Please refer to FIG. 1, according to an embodiment of the present invention, n photographing units are arranged in a ring-shaped arrangement and are evenly arranged on the housing shell (213) of the full-view conference photographing device (100), so that each photographing unit It can include a shooting angle of 360°/n. For example, in a preferred embodiment of the present invention, there are 4 photographing units, and the shooting angle of view of each photographing unit is 90°. Among them, it should be noted that the lens of the camera unit can be selected according to the needs of the application, such as the size of the meeting room or lecture hall that needs to be set up, choose different lens diameters, viewing angles and lens numbers to achieve optimal shooting Effect. For example, in the above-mentioned embodiment, when the full-view conference camera device (100) has 4 camera units, the resolution and corresponding frame rate of the conference video shot by it can reach the following standard: 3840 X 2160, 10-50fps; 3840 X 640, 10-60fps; 1920 X 1080, 10-70fps.
請參閱圖1與圖3,根據本發明內容,影音處理模組(209)包含一目標追蹤單元(209a),提供全視角會議攝影裝置(100)一目標追蹤(ROI,Region of Interest,或稱感興趣區域)的功能。在本發明中,所述之目標追蹤,為根據n個攝影單元所拍攝的會議影像裡,以自動或手動地定義出會議參加者或主持人的臉部、身體(例如攝影單元中每一象素中的型態、顏色),或其周遭一定面積大小的畫面,使影音處理模組(209)可以依照當前會議的參加者或主持人在攝影單元中的位置,持續藉由上、下、左、右調整與縮放會議影像的畫面(左右調整Pan;上下調整Tile;縮放Zoom。英語簡寫:PTZ模式),使分割單元(209c)能依照線上會議的需要,調整畫面的顯示模式,使目標追蹤單元(209a)在進行目標追蹤及聚焦,而需要移動拍攝的區域時時,無須透過機械馬達轉動鏡頭,使其達到更佳的畫面穩定性,達到改善會議影像品質的目的。 1 and 3, according to the content of the present invention, the audio-visual processing module (209) includes a target tracking unit (209a), providing a full-view conference camera device (100), a target tracking (ROI, Region of Interest, or ROI) Region of interest). In the present invention, the target tracking is to automatically or manually define the faces and bodies of meeting participants or moderators (for example, each image in the photographing unit) in the meeting images taken by n photographing units. The video processing module (209) can continue to use up, down, Adjust and zoom the conference video screen from left and right (Adjust Pan left and right; Adjust Tile up and down; Zoom Zoom. English abbreviation: PTZ mode), so that the split unit (209c) can adjust the display mode of the screen according to the needs of the online meeting to make the target When the tracking unit (209a) performs target tracking and focusing and needs to move the shooting area, there is no need to rotate the lens through a mechanical motor, so that it can achieve better picture stability and achieve the purpose of improving the image quality of the meeting.
承上述,請參閱圖3,在本發明一較佳地實施例中,分割單元(209c)包含數種的會議影像處理方式,以依據應用之所需將會議影像透過傳輸模組(211)加以顯示。上列所述的會議影數處理方式包含:全景模式(Panorama mode,300a)、專注模式(Focus mode,300f),與多畫面模式(300b),其中多畫面模式進一步包含分割模式(Top-Down mode,300c),以及的影格模式(Grid Mode,300d)。請參閱圖4A-圖4D,其分別說明了全景模式(300a)、分割模式(300c)、影格模式(300d), 以及專注模式(300f)的處理方式。其中,所述的全景模式(300a),為分割單元(209c)將第一攝影單元(203)、第二攝影單元(205)、第三攝影單元(207)等n個攝影單元所拍攝角度範圍為90°-360°的會議影像,即會議的參加者、主持人所在的會議室或講堂中的會議影像,處理為一可在平面顯示的全景影像,而其每一個攝影單元的視角範圍,為30°-100°。 In view of the above, please refer to FIG. 3. In a preferred embodiment of the present invention, the dividing unit (209c) includes several conference image processing methods to process the conference images through the transmission module (211) as required by the application. show. The above-mentioned conference video processing methods include: Panorama mode (300a), Focus mode (300f), and multi-screen mode (300b). The multi-screen mode further includes the split mode (Top-Down). mode, 300c), and the grid mode (Grid Mode, 300d). Please refer to Figures 4A-4D, which respectively illustrate the panorama mode (300a), the split mode (300c), and the frame mode (300d), And the processing method of focus mode (300f). Among them, the panoramic mode (300a) is a division unit (209c) that divides the photographing angle range of n photographing units such as the first photographing unit (203), the second photographing unit (205), and the third photographing unit (207). It is a 90°-360° meeting image, that is, the meeting image in the meeting room or lecture hall where the participants and the host are located, and is processed into a panoramic image that can be displayed on a plane. The viewing angle range of each camera unit is It is 30°-100°.
此外,請參閱圖4B,所述的分割模式(300c),為藉由前述的目標追蹤單元(209a)定義出會議中各個參加者的位置後,分割單元(209c)將全景影像依照適當的大小平均分割為兩個分別為90°-180°視角,或兩個以上可上下排列的分割影像,使會議室或講堂中的會議情形可以透過一個畫面即可一覽無遺,以增加會議參加者的參予感。接著,請參閱圖4C,根據本發明內容,所述的影格模式(300d),為分割單元(209c)將上述的全景影像進一步分割為4個以上,視角為50°-90°,左右並排且上下交迭的分割影像。最後,請參閱圖4D,所述的專注模式(300f),為當會議的參加者眾多時,可將全景影像再度進一步的放大與聚焦參加者,並限縮視角,使專注影像能最大幅度的聚焦顯示參加者的情況,根據本發明一較佳的實施例,以專注模式(300f)顯示的專注影像,能顯示的參加者數量較佳的效果為6-14人,並最多能顯示14人。其中,上述的分割模式(300c)、影格模式(300d),以及專注模式(300f)的處理方式,除了在參加者人數眾多時,能使畫面盡可能顯示參加者以外,亦能在不損失會議影像品質的前提下,藉由將不需要被顯示的畫面或無關緊要,如雜物、背景等無關會議內容的目標剃除,以降低網路頻寬佔用率和減少會議影像日後所需的存儲空間。 In addition, please refer to FIG. 4B. The segmentation mode (300c) is that after the location of each participant in the meeting is defined by the aforementioned target tracking unit (209a), the segmentation unit (209c) adjusts the panoramic image to an appropriate size Evenly split into two 90°-180° viewing angles, or two or more split images that can be arranged up and down, so that the meeting situation in the meeting room or lecture hall can be seen through one screen, so as to increase the participation of meeting participants. I feel. Next, please refer to FIG. 4C. According to the content of the present invention, the frame mode (300d) is a dividing unit (209c) that further divides the aforementioned panoramic image into 4 or more, with a viewing angle of 50°-90°, side by side and side by side. Split images that overlap top and bottom. Finally, please refer to Figure 4D. The focus mode (300f) described is for when there are a large number of participants in the meeting, the panoramic image can be further enlarged and focused on the participants, and the viewing angle can be reduced, so that the focused image can be maximized. Focusing on the display of participants. According to a preferred embodiment of the present invention, the number of participants that can be displayed in the focus mode (300f) is 6-14, and the maximum number of participants is 14 people. . Among them, the above-mentioned split mode (300c), frame mode (300d), and focus mode (300f) processing methods, in addition to the large number of participants, the screen can display the participants as much as possible, but also can not lose the meeting Under the premise of image quality, by cutting out the images that do not need to be displayed or irrelevant, such as clutter, background, and other irrelevant meeting content, it can reduce the network bandwidth occupancy rate and reduce the storage of meeting images in the future. space.
請參閱圖5、圖6A-6C根據本發明內容,影音處理模組(209)包含簡報單元(209e),其中,簡報單元(209e)內安裝有一簡報模式(500a,Presentation Mode)。根據本發明一實施例,簡報模式(500a)的作用,為透過目標追蹤單元(209a),將會議主持人、簡報者或所欲關注的參加者,從攝影單元中的會議影像中獨立出來,並將所需背景的會議影像(例如,會議室的白板、投影幕,或平面牆壁),加以拼接為一簡報影像,簡報模式(500a)並可以選擇將簡報者置中配置。舉例來說,當第一攝影單元(203)拍攝圖6A中的主持人時,第二攝影單元(205)拍攝到一會議室白板的背景時,簡報單元(209e)可透過目標追蹤單元(209a)將主持人的影像, 與會議室白板背景的影像加以獨立出來後加以拼合(merge),使得最終輸出至另一終端的簡報影像,可包含了主持人與會議室的白板背景,或是由傳輸模組(211)所輸入的書面資料、電子資料等內容。而上述之目標追蹤單元(209a)會驅使相關之第一攝影單元(203)隨者簡報者或主持人而移動或轉動鏡頭。 Please refer to FIG. 5 and FIGS. 6A-6C. According to the content of the present invention, the audio-visual processing module (209) includes a presentation unit (209e), wherein a presentation mode (500a, Presentation Mode) is installed in the presentation unit (209e). According to an embodiment of the present invention, the function of the presentation mode (500a) is to separate the meeting host, presenter, or participant to be followed from the meeting image in the photographing unit through the target tracking unit (209a). And splicing the conference image of the required background (for example, the whiteboard, projection screen, or flat wall of the conference room) into a presentation image, the presentation mode (500a), and you can choose to place the presenter in the center configuration. For example, when the first photographing unit (203) photographs the host in FIG. 6A and the second photographing unit (205) photographs the background of a whiteboard in a conference room, the presentation unit (209e) can pass through the target tracking unit (209a). ) The host’s image, The image of the whiteboard background of the conference room is separated and merged, so that the final presentation image output to another terminal can include the background of the host and the whiteboard of the conference room, or be combined by the transmission module (211) The inputted written materials, electronic materials, etc. The above-mentioned target tracking unit (209a) will drive the related first photographing unit (203) to move or rotate the lens along with the presenter or host.
承上述,在本發明之另一實施例中,請參閱圖6C,當第一攝影單元(203)拍攝會議的主持人或簡報者時,第二攝影單元(205)、第三攝影單元(207)或其餘的n-1個攝影單元可能用做拍攝會議的其他的場景或參加者,並透過影音處理模組(209)處理為全景影像、分割景象、專注影像等畫面,此時簡報單元(209c)可透過所述簡報模式(500a)的功能,將簡報影像,與上述全景影像、分割景象、專注影像加以拼合唯一畫面加以輸出。以圖6C中的說明來舉例,其中的會議主持人、會議室中的白板、會議的電子資料,以及白板上的環景影像,皆為透過簡報模式(500a),將n個攝影單元的會議影像,及傳輸模組(211)中的電子資料合成為一畫面,所最終獲得的簡報影像。 In view of the above, in another embodiment of the present invention, please refer to FIG. 6C. When the first photographing unit (203) photographs the host or presenter of the meeting, the second photographing unit (205) and the third photographing unit (207) ) Or the remaining n-1 photography units may be used to shoot other scenes or participants of the meeting, and processed into panoramic images, split scenes, focused images, etc. through the audio-visual processing module (209). At this time, the presentation unit ( 209c) Through the function of the presentation mode (500a), the presentation image can be combined with the aforementioned panoramic image, segmented scene, and focused image to be output as a single screen. Take the description in Figure 6C as an example. The meeting host, the whiteboard in the meeting room, the electronic data of the meeting, and the surround view image on the whiteboard are all in the presentation mode (500a), which combines n photographic units of the meeting The image and the electronic data in the transmission module (211) are combined into one screen, and the final presentation image is obtained.
根據本發明內容,傳輸模組(211)的規格可為但不限於USB Mini、USB Micro、USB Type A、USB Type B、USB Type C、LPT、RS232、PS/2,或以上之組合,並可依據應用的需要替換任意所需的傳輸規格。此外,在本發明之實施例中,全視角會議攝影裝置(100)除可透過傳輸模組(211)傳輸會議影像、全景影像、分割影像、專注影像、簡報影像等畫面至外部的終端外,外部的終端,如智慧型手機、平板電腦、電腦、智慧型穿戴裝置等均可藉由傳輸模組(211),傳輸控制指令予處理模組(201),以控制全視角會議攝影裝置(100)的運作,其相容的作業系統,可為但不限於Linux、Windos、Android、Mac OS或iOS,使其得以具備較佳的系統相容性。 According to the content of the present invention, the specifications of the transmission module (211) can be but not limited to USB Mini, USB Micro, USB Type A, USB Type B, USB Type C, LPT, RS232, PS/2, or a combination of the above, and Any required transmission specifications can be replaced according to the needs of the application. In addition, in the embodiment of the present invention, the full-view conference camera device (100) can transmit conference images, panoramic images, split images, focused images, presentation images, etc. screens to external terminals through the transmission module (211). External terminals, such as smart phones, tablets, computers, smart wearable devices, etc., can use the transmission module (211) to transmit control commands to the processing module (201) to control the full-view conference camera device (100 ) Operation, its compatible operating system can be but not limited to Linux, Windos, Android, Mac OS or iOS, so that it can have better system compatibility.
請參閱圖4A、4B、6B、6C與6D,根據本發明內容,本發明提出一種雲端會議系統,包含:數個全視角會議裝置,配置於至少兩個不同地理區域,其中各個全視角會議裝置具有數個攝影單元,以拍攝會議影像;影音處理模組,裁切或拼貼上述攝影單元所拍攝的會議影像,其中影音處理模組更包含目標追蹤單元,追蹤至少一目標位置(例如會議的參加者、主持人、簡報者);分割單 元,依據目標位置,將會議影像分割為多畫面模式,或拼貼成為一全景模式(Panorama mode)。其中,影音處理模組更包含簡報模式(Presentation Mode),將簡報者由會議影像中分割,並與簡報電子資料或書面資料拼接為簡報影像;以及傳輸模組,用於連結雲端遠端,以執行雲端會議 Please refer to Figures 4A, 4B, 6B, 6C and 6D. According to the content of the present invention, the present invention proposes a cloud conference system, including: a number of full-view conference devices, which are arranged in at least two different geographic areas, wherein each full-view conference device There are several camera units to shoot conference images; the audio-visual processing module crops or collages the conference images taken by the aforementioned camera unit, wherein the audio-visual processing module further includes a target tracking unit to track at least one target location (such as Participants, moderators, presenters); split list Meta, according to the target location, the conference image is divided into a multi-screen mode, or collaged into a panoramic mode (Panorama mode). Among them, the audio-visual processing module also includes a presentation mode (Presentation Mode), which divides the presenter from the conference image, and stitches the presentation electronic data or written data into the presentation image; and the transmission module is used to connect to the remote end of the cloud to Run cloud meetings
承上述,根據本發明一應用的實施例,當線上會議開始時,前述的數個會議參加者、主持人,或簡報者實際上可能位於不同的地點,因此為了應付具有多個地點的線上會議,有多套的全視角會議攝影裝置(100)可被佈署於所述的數個地點,組成一雲端會議系統(200),並透過傳輸模組(211)與雲端網路連結各個終端。當會議開始時,雲端會議系統(200)的攝影單元將所拍到的各個會議影像,透過前述的影音處理模組(209),將參加者的畫面處理為全景影像、專注影像,或分割影像。此外,當會議主持人或簡報者開始發言時,以目標追蹤單元(209a),自動或手動地定義出會議參加者或主持人的臉部、身體(例如攝影單元中每一象素中的型態、顏色),或其周遭一定面積大小的畫面,使影音處理模組(209)可以依照當前會議的參加者或主持人在攝影單元中的位置,持續藉由PTZ模式的調整,使分割單元(209c)能依照線上會議的需要,以上下左右之方向,或縮放的方式追蹤主持人或簡報者的位置。進一步地來說,當線上會議的議程需要時,可藉由簡報單元(209e)啟動簡報模式(500a),以將主持人、簡報者或所欲關注的參加者,從攝影單元中的會議影像中分割出來,並將所需背景的會議影像(例如,會議室的白板、投影幕,或平面牆壁),加以拼接為一簡報影像。請參閱圖6C,其為簡報模式(500a)啟動後之一實施例,其中,主持人、白板、全景影像,甚或電子資料,均可為來自不同會議地點,由不同的攝影單元所拍攝的會議影像,並由影音處理模組(209)將其裁切或拼貼為一簡報影像,以達到增加線上會議的溝通效率,及會議參加者的參予感之目的。 In view of the above, according to an application of the present invention, when an online meeting starts, the aforementioned several meeting participants, moderators, or presenters may actually be located in different locations. Therefore, in order to cope with an online meeting with multiple locations There are multiple sets of full-view conference camera devices (100) that can be deployed in the several locations to form a cloud conference system (200), and connect each terminal to the cloud network through the transmission module (211). When the meeting starts, the camera unit of the cloud meeting system (200) will process the images of the participants into panoramic images, focused images, or split images through the aforementioned audio-visual processing module (209). . In addition, when the meeting host or presenter starts to speak, the target tracking unit (209a) is used to automatically or manually define the face and body of the meeting participant or host (such as the shape of each pixel in the camera unit). The video processing module (209) can continuously adjust the PTZ mode to divide the unit according to the position of the participant or host of the current meeting in the photographing unit. (209c) According to the needs of the online meeting, the position of the host or presenter can be tracked in the up, down, left, and right directions, or zoom. Furthermore, when the agenda of the online meeting is needed, the briefing unit (209e) can be used to activate the briefing mode (500a), so that the host, the presenter or the participants you want to follow can be viewed from the meeting image in the photographing unit The conference images of the required background (for example, the whiteboard, projection screen, or flat wall of the conference room) are spliced into a presentation image. Please refer to Figure 6C, which is an embodiment after the presentation mode (500a) is activated, in which the host, whiteboard, panoramic images, or even electronic data can be meetings taken by different photography units from different meeting locations The image is cut or collaged by the audio-visual processing module (209) into a presentation image to achieve the purpose of increasing the communication efficiency of the online meeting and the sense of participation of the meeting participants.
以上敘述係為本發明為達到第一~第四目的的較佳實施例。此領域之技藝者應得以領會其係用以說明本發明而非用以限定本發明所主張之專利權利範圍。其專利保護範圍當視後附之申請專利範圍及其等同領域而定。凡熟悉此領域之技藝者,在不脫離本專利精神或範圍內,所作之更動或潤飾,均屬於本發明所揭示精神下所完成之等效改變或設計,且應當被包含在下述之申請專利 範圍內。 The above description is a preferred embodiment of the present invention to achieve the first to fourth objectives. Those skilled in this field should be able to understand that it is used to illustrate the present invention rather than to limit the scope of the claimed patent rights of the present invention. The scope of its patent protection shall be determined by the scope of the attached patent application and its equivalent fields. Anyone familiar with the art in this field, without departing from the spirit or scope of this patent, makes changes or modifications that are equivalent changes or designs completed under the spirit of the present invention, and should be included in the following patent applications Within range.
200:全視角會議攝影裝置 200: Full-view conference photography installation
201:處理模組 201: Processing Module
203:第一攝影單元 203: The first photography unit
205:第二攝影單元 205: The second photography unit
207:第三攝影單元 207: The third photography unit
209:影音處理模組 209: Audio and Video Processing Module
209a:影音追蹤單元 209a: Audio-visual tracking unit
209c:分割單元 209c: segmentation unit
209e:簡報模式單元 209e: Presentation mode unit
211:傳輸模組 211: Transmission Module
Claims (10)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW109130638A TWI739585B (en) | 2020-09-07 | 2020-09-07 | Full fov conference camera device |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
TW109130638A TWI739585B (en) | 2020-09-07 | 2020-09-07 | Full fov conference camera device |
Publications (2)
Publication Number | Publication Date |
---|---|
TWI739585B true TWI739585B (en) | 2021-09-11 |
TW202211667A TW202211667A (en) | 2022-03-16 |
Family
ID=78778018
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
TW109130638A TWI739585B (en) | 2020-09-07 | 2020-09-07 | Full fov conference camera device |
Country Status (1)
Country | Link |
---|---|
TW (1) | TWI739585B (en) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI817301B (en) * | 2021-12-29 | 2023-10-01 | 宏碁股份有限公司 | Wide-angle video apparatus and controlling method thereof |
Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010118685A1 (en) * | 2009-04-14 | 2010-10-21 | 华为终端有限公司 | System, apparatus and method for remote representation |
US20120314015A1 (en) * | 2011-06-10 | 2012-12-13 | Microsoft Corporation | Techniques for multiple video source stitching in a conference room |
TW201517631A (en) * | 2013-08-29 | 2015-05-01 | Vid Scale Inc | User-adaptive video telephony |
US9172909B2 (en) * | 2013-10-29 | 2015-10-27 | Cisco Technology, Inc. | Panoramic video conference |
US20160134838A1 (en) * | 2014-11-06 | 2016-05-12 | Cisco Technology, Inc. | Automatic Switching Between Dynamic and Preset Camera Views in a Video Conference Endpoint |
US20160173821A1 (en) * | 2014-12-15 | 2016-06-16 | International Business Machines Corporation | Dynamic video and sound adjustment in a video conference |
-
2020
- 2020-09-07 TW TW109130638A patent/TWI739585B/en active
Patent Citations (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2010118685A1 (en) * | 2009-04-14 | 2010-10-21 | 华为终端有限公司 | System, apparatus and method for remote representation |
US20120314015A1 (en) * | 2011-06-10 | 2012-12-13 | Microsoft Corporation | Techniques for multiple video source stitching in a conference room |
TW201517631A (en) * | 2013-08-29 | 2015-05-01 | Vid Scale Inc | User-adaptive video telephony |
US9172909B2 (en) * | 2013-10-29 | 2015-10-27 | Cisco Technology, Inc. | Panoramic video conference |
US20160134838A1 (en) * | 2014-11-06 | 2016-05-12 | Cisco Technology, Inc. | Automatic Switching Between Dynamic and Preset Camera Views in a Video Conference Endpoint |
US20160173821A1 (en) * | 2014-12-15 | 2016-06-16 | International Business Machines Corporation | Dynamic video and sound adjustment in a video conference |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
TWI817301B (en) * | 2021-12-29 | 2023-10-01 | 宏碁股份有限公司 | Wide-angle video apparatus and controlling method thereof |
Also Published As
Publication number | Publication date |
---|---|
TW202211667A (en) | 2022-03-16 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US10171771B2 (en) | Camera system for video conference endpoints | |
US7475112B2 (en) | Method and system for presenting a video conference using a three-dimensional object | |
US8638354B2 (en) | Immersive video conference system | |
WO2018214746A1 (en) | Video conference realization method, device and system, and computer storage medium | |
US20130093838A1 (en) | Methods and systems for establishing eye contact and accurate gaze in remote collaboration | |
US9143727B2 (en) | Dual-axis image equalization in video conferencing | |
CN102404545A (en) | Two-way video conferencing system | |
CN104935848A (en) | Projector capable of shooting | |
US7551199B2 (en) | Computer camera system and method for reducing parallax | |
US9253442B1 (en) | Holopresence system | |
CN104144315A (en) | Displaying method of multipoint videoconference and multipoint videoconference system | |
US20230283888A1 (en) | Processing method and electronic device | |
EP4106326A1 (en) | Multi-camera automatic framing | |
US12101573B2 (en) | System for capturing and projecting images, use of the system and method for capturing, projecting and inserting images | |
JP7424076B2 (en) | Image processing device, image processing system, imaging device, image processing method and program | |
US11831454B2 (en) | Full dome conference | |
TWI739585B (en) | Full fov conference camera device | |
TWI488503B (en) | Conference photography device and the method thereof | |
US9445052B2 (en) | Defining a layout for displaying images | |
WO2011124066A1 (en) | Remote representation system and camera group thereof | |
CN112887653B (en) | Information processing method and information processing device | |
US20220264156A1 (en) | Context dependent focus in a video feed | |
CN212392942U (en) | Immersive elevating gear and system for video conference | |
JP7322510B2 (en) | Information processing device, information processing method and program | |
KR102619761B1 (en) | Server for TelePresentation video Conference System |