TWI584050B - Panoramic stereoscopic image synthesis method, apparatus and mobile terminal - Google Patents

Panoramic stereoscopic image synthesis method, apparatus and mobile terminal Download PDF

Info

Publication number
TWI584050B
TWI584050B TW104137748A TW104137748A TWI584050B TW I584050 B TWI584050 B TW I584050B TW 104137748 A TW104137748 A TW 104137748A TW 104137748 A TW104137748 A TW 104137748A TW I584050 B TWI584050 B TW I584050B
Authority
TW
Taiwan
Prior art keywords
image
frame
information
speed
frames
Prior art date
Application number
TW104137748A
Other languages
Chinese (zh)
Other versions
TW201701051A (en
Inventor
李玉峰
陳宏偉
付琴琴
Original Assignee
深圳創銳思科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳創銳思科技有限公司 filed Critical 深圳創銳思科技有限公司
Publication of TW201701051A publication Critical patent/TW201701051A/en
Application granted granted Critical
Publication of TWI584050B publication Critical patent/TWI584050B/en

Links

Description

全景立體圖像的合成方法、裝置及其移動終端 Method, device and mobile terminal for synthesizing panoramic stereo image

本發明涉及影像處理技術領域,尤其涉及一種全景立體圖像的合成方法、裝置及其移動終端。 The present invention relates to the field of image processing technologies, and in particular, to a method and an apparatus for synthesizing a panoramic stereoscopic image and a mobile terminal thereof.

圖像是由照相機、攝像機等成像設備捕捉實際的畫面而產生的圖形或影像。 An image is a graphic or image produced by an actual image captured by an imaging device such as a camera or a video camera.

隨著人們對於圖像的要求越來越高,通常希望看到全景圖像或者立體圖像,全景圖像是指圖像可以達到符合雙眼正常有效視角(大約水平90度,垂直70度)或包括雙眼餘視角(大約水平180度,垂直90度)以上,乃至360度完整視角範圍所拍攝出的圖像,立體圖像是指具有兩個帶有位置差的畫面代表左右雙眼觀看的內容,利用左右眼視差讓觀察人產生帶有深度感覺效果的圖像。 As people's requirements for images become higher and higher, it is usually desirable to see a panoramic image or a stereoscopic image. The panoramic image means that the image can reach the normal effective viewing angle of both eyes (about 90 degrees horizontally and 70 degrees vertical). Or include images taken from the binocular residual angle (approximately 180 degrees horizontally, 90 degrees vertically) or even 360 degrees of complete viewing angle. The stereo image refers to two pictures with position difference representing left and right eyes. The content of the left and right eye parallax allows the observer to produce an image with a deep sensory effect.

而儘管隨著成像設備的發展,具有大視角的鏡頭,例如廣角鏡頭、魚眼鏡頭等得到廣泛應用,可以實現超大視角的拍攝,但是仍不能拍攝出全景圖像,需要採用圖像合成的方式來形成。 Although with the development of imaging devices, lenses with large viewing angles, such as wide-angle lenses, fisheye lenses, etc., are widely used to achieve large viewing angles, but still cannot capture panoramic images, and image synthesis is required. form.

由於現有三維顯示裝置的局限,用戶不能很方便體驗到通過二維攝像所見即所得的效果,導致二維圖像合成 立體圖像的技術並沒有得到廣泛的應用。但隨著虛擬實境、增強現實以及裸眼三維技術的成熟及普及,人們對二維圖像合成立體圖像的需求會大大增加。 Due to the limitations of existing three-dimensional display devices, users cannot easily experience the effects of WYSIWYG through two-dimensional imaging, resulting in two-dimensional image synthesis. The technology of stereoscopic images has not been widely used. However, with the maturity and popularity of virtual reality, augmented reality and naked-eye three-dimensional technology, the demand for two-dimensional image synthesis stereoscopic images will increase greatly.

發明人在研究中發現,現有技術中並沒有一種有效的圖像合成技術,能夠形成全景且立體的圖像,以滿足使用者的需求。 The inventors found in the research that there is no effective image synthesis technology in the prior art, which can form a panoramic and stereoscopic image to meet the needs of users.

本發明提供一種全景立體圖像合成方法,可以合成不同視角的立體圖像,以實現全景立體圖像,滿足用戶需求。在第一方面,本發明提供一種全景立體圖像的合成方法,包括:獲取圖像資訊,圖像資訊為一組第一多幀連續圖像;從多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像;分別將抽取的兩幀圖像進行立體圖像合成,以獲得一組不同視角的第二多幀立體圖像。 The invention provides a method for synthesizing panoramic stereo images, which can synthesize stereo images of different viewing angles to realize panoramic stereo images and meet user requirements. In a first aspect, the present invention provides a method for synthesizing a panoramic stereoscopic image, comprising: acquiring image information, the image information being a set of first multi-frame continuous images; and extracting from a plurality of consecutive frames to satisfy a predetermined Two frame images of the frame interval; respectively, the extracted two frames of images are subjected to stereoscopic image synthesis to obtain a second plurality of stereoscopic images of different views.

在第二方面,本發明還提供一種全景立體圖像合成裝置,其中,包括:獲取單元,用於獲取圖像資訊,圖像資訊為一組第一多幀連續圖像;抽取單元,用於從多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像;合成單元,用於分別將抽取的兩幀圖像進行立體圖像合成,以獲得一組不同視角的第二多幀立體圖像。 In a second aspect, the present invention further provides a panoramic stereoscopic image synthesizing apparatus, comprising: an acquiring unit, configured to acquire image information, wherein the image information is a set of first multi-frame continuous images; and an extracting unit is configured to: Extracting, from the multi-frame continuous image, two frames of images satisfying a predetermined frame interval; and combining means for respectively performing stereoscopic image synthesis on the extracted two frames to obtain a second plurality of frames of different viewing angles Stereo image.

在協力廠商面,本發明又提供一種移動終端,包括:中央處理器,用於獲取圖像資訊,圖像資訊為一組第一多幀連續圖像;從第一多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像,以及分別將抽取的兩幀圖像進行立體圖像合成,以獲得一組不同視角的第二多幀立體圖像;顯示螢幕,用於顯示第二多幀立體圖像。 The invention further provides a mobile terminal, comprising: a central processor, configured to acquire image information, wherein the image information is a set of first multi-frame continuous images; and from the first multi-frame continuous image, Extracting two frames of images that satisfy a predetermined frame interval, and separately combining the extracted two frames of images to obtain a second multi-frame stereo image of a different set of angles; and displaying a screen for displaying the second image Multi-frame stereo image.

在本發明實施例中,通過獲取一組多幀連續圖像;並從多幀連續圖像中分別抽取滿足預定幀間隔的兩幀圖像進行立體圖像合成,以獲得一組不同視角的多幀立體圖像,解決了現有技術中沒有一種有效的圖像合成技術,填補了現有技術的空白。實現了全景立體圖像合成,滿足了用戶對全景立體圖像的觀看需求。 In the embodiment of the present invention, a set of multi-frame continuous images is acquired; and two frames of images satisfying a predetermined frame interval are separately extracted from the multi-frame continuous image for stereoscopic image synthesis to obtain a plurality of different viewing angles. The frame stereo image solves the gap in the prior art without solving an effective image synthesis technique in the prior art. The panoramic stereoscopic image synthesis is realized, which satisfies the user's viewing requirements for the panoramic stereoscopic image.

為讓本發明之上述和其他目的、特徵和優點能更明顯易懂,下文特舉較佳實施例並配合所附圖式做詳細說明如下。 The above and other objects, features, and advantages of the present invention will become more apparent from the description of the appended claims.

61‧‧‧獲取單元 61‧‧‧Acquisition unit

62‧‧‧抽取單元 62‧‧‧ extraction unit

63‧‧‧合成單元 63‧‧‧Synthesis unit

64‧‧‧運動補償單元 64‧‧‧Motion compensation unit

65‧‧‧顯示單元 65‧‧‧Display unit

66‧‧‧人眼位置獲取單元 66‧‧‧Human eye position acquisition unit

67‧‧‧角度資訊獲取單元 67‧‧‧ Angle Information Acquisition Unit

68‧‧‧顯示圖像確定單元 68‧‧‧Display image determination unit

71‧‧‧顯示螢幕 71‧‧‧ Display screen

73‧‧‧中央處理器 73‧‧‧Central processor

S11、S13、S15、S17、S41、S42、S51、S52‧‧‧步驟 S11, S13, S15, S17, S41, S42, S51, S52‧‧

圖1為本發明實施例一種全景立體圖像的合成方法流程示意圖;圖2為本發明實施例沿水平方向的移動拍攝俯視示意圖;圖3為本發明實施例以拍攝對象為圓心的圓周移動拍攝俯視示意圖;圖4為本發明實施例根據人眼位置資訊顯示多幀立體圖像流程示意圖;圖5為本發明實施例根據旋轉角度資訊顯示多幀立體圖像流程示意圖;圖6為本發明實施例一種全景立體圖像合成裝置示意圖;圖7為本發明實施例一種移動終端示意圖。 1 is a schematic flow chart of a method for synthesizing a panoramic stereoscopic image according to an embodiment of the present invention; FIG. 2 is a schematic plan view showing a moving movement in a horizontal direction according to an embodiment of the present invention; FIG. FIG. 4 is a schematic diagram showing a flow of displaying a multi-frame stereo image according to a position information of a human eye according to an embodiment of the present invention; FIG. 5 is a schematic flowchart showing a multi-frame stereo image according to a rotation angle information according to an embodiment of the present invention; A schematic diagram of a panoramic stereoscopic image synthesizing device; FIG. 7 is a schematic diagram of a mobile terminal according to an embodiment of the present invention.

下面結合附圖和實施例對本發明進行詳細說明。 The invention will now be described in detail in conjunction with the drawings and embodiments.

需要說明的是,如果不衝突,本發明實施例以及 實施例中的各個特徵可以相互結合,均在本發明的保護範圍之內。另外,雖然在裝置示意圖中進行了功能模組劃分,在流程圖中示出了邏輯順序,但是在某些情況下,可以以不同於裝置中的模組劃分,或流程圖中的循序執行所示出或描述的步驟。 It should be noted that, if not conflicting, the embodiments of the present invention and The various features of the embodiments can be combined with each other and are within the scope of the invention. In addition, although the functional module partitioning is performed in the device schematic, the logical sequence is shown in the flowchart, but in some cases, it may be divided by modules different from the device, or sequentially executed in the flowchart. The steps shown or described.

下面對本發明實施例的全景立體圖像的合成方法進行簡要描述。在本發明實施例中,通過高速連拍或視頻拍攝獲得一組多幀連續圖像,從多幀連續圖像中分別抽取滿足預定幀間隔的兩幀圖像。其中,多幀連續圖像中存在多組滿足預定幀間隔的兩幀圖像,比如,如果預定幀間隔為3,則第1幀與第4幀圖像為一組,第2幀與第5幀圖像為一組,依此類推。將多幀連續圖像中滿足預定幀間隔的兩幀圖像分別進行立體圖像合成,從而獲得一組不同視角的多幀立體圖像。 A method for synthesizing a panoramic stereoscopic image of an embodiment of the present invention will be briefly described below. In the embodiment of the present invention, a set of multi-frame continuous images is obtained by high-speed continuous shooting or video shooting, and two frames of images satisfying a predetermined frame interval are respectively extracted from the multi-frame continuous images. Wherein, there are multiple sets of two frames of images satisfying a predetermined frame interval in the multi-frame continuous image. For example, if the predetermined frame interval is 3, the first frame and the fourth frame image are a group, and the second frame and the fifth frame are Frame images are a group, and so on. The two frames of images in the multi-frame continuous image satisfying the predetermined frame interval are respectively subjected to stereoscopic image synthesis, thereby obtaining a plurality of multi-frame stereo images of different viewing angles.

在本發明實施例中,為了確定幀間隔,比如,可以把拍照或錄製視頻時的旋轉角度和移動速度保存到每一張照片或每一幀視頻的檔頭資訊裡,然後通過每一張照片或每一幀視頻檔頭資訊裡記錄的旋轉角度及速度來確定幀間隔。在本發明實施例中,速度越大應取的幀間隔越小;角度越大應取的幀間隔越小;速度越大而且角度越大應取的幀間隔也越小。同時,為了方便用戶觀看全景立體圖像,在觀看合成的立體圖像時,根據觀看者觀看螢幕的視角、或根據螢幕的旋轉角度呈現立體圖像,從而實現全景立體圖像顯示。 In the embodiment of the present invention, in order to determine the frame interval, for example, the rotation angle and the moving speed when taking a picture or recording a video may be saved in the header information of each photo or each frame of video, and then passed through each photo. Or the rotation angle and speed recorded in the video header information of each frame to determine the frame interval. In the embodiment of the present invention, the larger the speed is, the smaller the frame interval should be; the larger the angle is, the smaller the frame interval should be; the larger the speed and the larger the angle, the smaller the frame interval should be. Meanwhile, in order to facilitate the user to view the panoramic stereoscopic image, when viewing the synthesized stereoscopic image, the stereoscopic image is displayed according to the viewing angle of the viewer viewing the screen or the stereoscopic image according to the rotation angle of the screen.

下面結合附圖對本發明實施例作進一步闡述。 The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

圖1為本發明實施例一種全景立體圖像的合成方法流程示意圖,如圖1所示,本發明實施例的方法,包括以下步驟:S11,獲取圖像資訊,所述圖像資訊為一組多幀連續圖像;在本發明實施例中,為了準確記錄多幀圖像資訊中每一 幀的移動情況,可以記錄每一幀圖像的移動角度和速度,因此,圖像資訊可以包含每一幀的移動角度和速度,所述角度和速度記錄在每一幀圖像的檔頭資訊中。 FIG. 1 is a schematic flowchart of a method for synthesizing a panoramic stereo image according to an embodiment of the present invention. As shown in FIG. 1 , the method of the embodiment of the present invention includes the following steps: S11: acquiring image information, where the image information is a group Multi-frame continuous image; in the embodiment of the present invention, in order to accurately record each of the multi-frame image information The movement of the frame can record the moving angle and speed of each frame of image. Therefore, the image information can include the moving angle and speed of each frame, and the angle and speed are recorded in the header information of each frame of the image. in.

在本發明實施例中,圖像資訊中包括的移動角度和速度可以均為勻速移動,也可以是變速移動,如圖2,圖3所示為俯視拍攝過程示意圖,其中,圖2為沿水平方向的移動拍攝俯視示意圖,在圖2中照相系統正對拍攝場景從左到右(水平方向)做勻速或變速平移運動,圖3為以拍攝物件為圓心的圓周移動拍攝俯視示意圖,在圖3中照相系統圍繞拍攝物體作勻速或變速圓周運動。在移動過程中,照相系統快速連拍或視頻錄影,從而獲得一組連續多幀圖像,比如,Img_1,Img_2,….Img_N1,Img_N1+1…。 In the embodiment of the present invention, the moving angle and the speed included in the image information may both move at a constant speed or may be a shifting movement. As shown in FIG. 2 and FIG. 3, a schematic diagram of a top view shooting process is shown, wherein FIG. 2 is horizontal. The moving direction of the direction is a schematic view of the top view. In Figure 2, the camera system is performing a uniform speed or variable-speed translation motion from left to right (horizontal direction) on the shooting scene. FIG. 3 is a schematic plan view of the circumferential movement of the object as a center, in FIG. The medium camera system makes a uniform or variable circular motion around the subject. During the movement, the camera system performs fast continuous shooting or video recording to obtain a set of consecutive multi-frame images, such as Img_1, Img_2, . . . Img_N1, Img_N1+1....

獲得一組多幀連續圖像,可以採用以下兩種方式:第一種方式:如果使用者開始獲取圖像幀後手持設備,比如手機,一直保持勻速勻角度的操作,可以採用延遲拍攝的方式,等使用者操作一定時間(1-2秒)後,取得手持設備的移動角度和加速度,並通過加速度可以得到速度,用這些參數決定單位時間內拍攝照片的數量或視頻圖像幀的數量:當手持設備角度和速度越大時,拍攝越多的圖像幀,以獲得更好的立體圖像;當手持設備角度和速度越小時,獲取較少的圖像幀,以減少重復資料處理量。 To obtain a set of multi-frame continuous images, you can use the following two methods: The first way: If the user starts to acquire the image frame and the handheld device, such as a mobile phone, keeps the uniform angle and the angle, you can use the delayed shooting method. After the user operates for a certain period of time (1-2 seconds), the moving angle and acceleration of the handheld device are obtained, and the speed can be obtained by the acceleration. These parameters are used to determine the number of photos taken per unit time or the number of video image frames: When the angle and speed of the handheld device are larger, more image frames are captured to obtain a better stereoscopic image; when the angle and speed of the handheld device are smaller, less image frames are acquired to reduce the amount of repeated data processing. .

在本發明實施例中,通過以下兩種實現情況判斷手持設備是否為勻速運動:第一種是利用加速度感測器,並結合重力感測器來判斷,如果加速度感測器顯示沒有加速度,但是重力感測器顯示有移動,說明就是勻速運動;第二種就是利用間隔一定時間獲得的圖像資料,比較相鄰兩個圖像資料的區別,然後根據區別的大小來判斷手機是否在做勻速運動。勻角度可以通過間隔一定時間得到陀螺儀的角度,如果 大部分角度一樣,就說明手機是勻角度的。 In the embodiment of the present invention, whether the handheld device is in uniform motion is determined by the following two implementation situations: the first is to use an acceleration sensor and is combined with a gravity sensor to determine if the acceleration sensor displays no acceleration, but The gravity sensor shows movement, which means uniform motion; the second is to use the image data obtained at a certain interval to compare the difference between the two adjacent image data, and then judge whether the mobile phone is doing uniform speed according to the difference size. motion. The angle can be obtained by spacing the gyroscope at a certain interval, if Most of the angles are the same, it means the phone is even angled.

第二種方式:在自動連拍或錄製視頻之前,除通過設置模組可以設置單位時間內連拍圖像的數量參數外,還可以同時設定手機角度和手機加速度等參數,從而實現單位場景角度內拍攝的圖像的數量或圖像幀的數量是相等的,保障後續合成立體照片後,查看立體照片時,立體照片畫面效果的一致性。此外,由於單位角度內的圖像數量或圖像幀的數量是相等的,可以避免在某一角度時,由於拍攝或錄製視頻時間過長,導致資料量較大,從而導致立體圖像合成處理時間過長的問題。 The second way: before the automatic continuous shooting or recording video, in addition to setting the module to set the number of continuous shooting images per unit time, you can also set the parameters such as the angle of the mobile phone and the acceleration of the mobile phone to achieve the unit scene angle. The number of images taken within the image or the number of image frames is equal, which ensures the consistency of the stereoscopic photo effect when viewing stereoscopic photos after subsequent synthesis of stereoscopic photos. In addition, since the number of images or the number of image frames in the unit angle is equal, it is possible to avoid a large amount of data due to the long time of shooting or recording video at a certain angle, thereby causing stereo image synthesis processing. The problem is too long.

S13,從所述多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像,並分別將所抽取的兩幀圖像進行立體圖像合成,以獲得一組不同視角的多幀立體圖像。 S13. Extracting, from the multi-frame continuous image, two frames of images satisfying a predetermined frame interval, and separately synthesizing the extracted two frames of images to obtain a plurality of multi-frame stereos with different viewing angles. image.

在本發明實施例中,為了合成立體圖像,可以從獲取的多幀連續圖像中抽取滿足預定幀間隔的兩幀圖像,比如,Img_i和Img_i+n作為有一定視差的左右圖,以用於合成立體圖像,其中,i從1開始,i為自然數,n為設定的間隔數,此間隔數可以根據下述兩種方式來確定:(1)如果使用者開始獲取圖像幀後手持設備一直保持勻速勻角度的操作(即第一種方式),就可以遵守公式:幀間隔=(總幀數/(角度*速度))/Ω決定抽取幀數的間隔,其中,所述總幀數為所述圖像資訊中多幀連續圖像的數量,所述Ω為測試所得的幀間隔經驗值。此間隔保持不變,即多幀連續圖像中只要滿足此幀間隔的兩幀圖像即可以兩兩組合,用來合成立體圖像。(2)如果是用上述第二種方式獲得的圖像幀,可以用第1圖像幀分別和第2圖像幀到第N圖像幀合成立體圖像,將合成圖像品質最優的組合作為抽取幀的間隔,此間隔保持不變,比如,假設第1圖像幀與第10圖像幀合成的立體圖像品 質最優,則幀間隔即為9,而且,多幀連續圖像中的其他幀之間也按此幀間隔確定用來合成立體圖像的兩幀圖像。 In the embodiment of the present invention, in order to synthesize a stereoscopic image, two frames of images satisfying a predetermined frame interval may be extracted from the acquired multi-frame continuous image, for example, Img_i and Img_i+n are used as left and right images with certain parallax, For synthesizing stereoscopic images, where i starts from 1, i is a natural number, and n is a set interval number, which can be determined according to the following two ways: (1) if the user starts to acquire an image frame After the rear handheld device maintains a uniform angular operation (ie, the first mode), the formula can be followed: frame interval = (total frame number / (angle * speed)) / Ω determines the interval of the number of extracted frames, wherein The total number of frames is the number of consecutive frames of the plurality of frames in the image information, and the Ω is an empirical value of the frame interval obtained by the test. This interval remains unchanged, that is, two frames of images in the multi-frame continuous image that satisfy the frame interval can be combined in two or two to synthesize a stereoscopic image. (2) If the image frame obtained by the second method described above is used, the first image frame and the second image frame to the Nth image frame can be combined to form a stereoscopic image, and the quality of the synthesized image is optimized. Combine as the interval of the extracted frame, the interval remains unchanged, for example, a stereoscopic image product synthesized by the first image frame and the tenth image frame If the quality is optimal, the frame interval is 9, and two frames of images used to synthesize the stereo image are also determined at the frame interval between other frames in the multi-frame continuous image.

在本發明實施例中,將多幀連續圖像中的滿足上述方式確定的幀間隔的兩幀圖像進行立體圖像合成,比如,將照片Img_i和Img_i+n合成立體圖,Img_i+1和Img_i+n+1合成立體圖像,依此類推,將所有滿足幀間隔的兩幀圖像合成立體圖像,這樣就可以獲得一組不同視角的多幀立體圖像比如,Img3d_1,Img3d_2…Img3d_N。 In the embodiment of the present invention, the two images of the frame interval determined by the above manner in the multi-frame continuous image are subjected to stereoscopic image synthesis, for example, the photos Img_i and Img_i+n are combined into a stereogram, Img_i+1 and Img_i. +n+1 synthesizes a stereoscopic image, and so on, and combines all the two frames of images satisfying the frame interval into a stereoscopic image, so that a plurality of multi-frame stereoscopic images of different viewing angles, such as Img3d_1, Img3d_2...Img3d_N, can be obtained.

可選地,在本發明實施例中,還包括:S15,對所述立體圖像在垂直方向上作運動補償。 Optionally, in the embodiment of the present invention, the method further includes: S15, performing motion compensation on the stereoscopic image in a vertical direction.

在本發明實施例中,獲取圖像過程中,由於存在垂直方向(Y軸方向)的不穩定,會使得所獲取的多幀圖像(Img_1,Img_2,....Img_N1,Img_N1+1...)產生上下移動,立體圖像合成過程中圖像拼接也會引入圖像的上下移動。因此,在顯示合成的立體圖像時,可以對上述多幀立體圖像垂直方向(Y軸方向)作運動補償,從而消除連續呈現多幀立體圖像時圖像抖動的問題,具體可以採用已有圖像拼接的演算法,比如,panaroma演算法來調節相鄰的立體圖像的圖元點的位置,從而消除圖像抖動的技術缺陷。 In the embodiment of the present invention, in the process of acquiring an image, due to the instability of the vertical direction (Y-axis direction), the acquired multi-frame image (Img_1, Img_2, . . . Img_N1, Img_N1+1. ..) Produce up and down movement, image stitching during stereo image synthesis will also introduce the image up and down movement. Therefore, when the synthesized stereoscopic image is displayed, motion compensation can be performed on the vertical direction (Y-axis direction) of the multi-frame stereo image, thereby eliminating the problem of image jitter when continuously displaying the multi-frame stereo image, and specifically There are algorithms for image stitching, such as the panaroma algorithm to adjust the position of the primitive points of adjacent stereo images, thereby eliminating the technical drawback of image jitter.

可選地,在本發明實施例中,還包括:S17,顯示所述多幀立體圖像。在本發明實施例中,可以分兩種方式來顯示多幀立體圖像,分別是根據人眼位置或旋轉角度資訊來確定多幀立體圖像中顯示哪幀立體圖像。 Optionally, in the embodiment of the present invention, the method further includes: S17, displaying the multi-frame stereo image. In the embodiment of the present invention, the multi-frame stereo image can be displayed in two ways, which is to determine which frame stereo image is displayed in the multi-frame stereo image according to the human eye position or the rotation angle information.

在本發明實施例中,圖4為根據人眼位置資訊顯示多幀立體圖像流程示意圖,如圖4所示,具體包括以下步驟:S41,獲取人眼相對於顯示所述多幀立體圖像的顯示螢幕的位置資訊。 In the embodiment of the present invention, FIG. 4 is a schematic diagram showing a flow of displaying a multi-frame stereoscopic image according to human eye position information. As shown in FIG. 4, the method further includes the following steps: S41: acquiring a human eye relative to displaying the multi-frame stereo image. Display the location information of the screen.

在本發明實施例中,獲取被跟蹤物件人眼位置資 訊,也是即是通過攝像頭拍攝被跟蹤人眼,獲取被跟蹤物件眼睛相對顯示所述多幀立體圖像的顯示螢幕的位置資訊。其中,當顯示螢幕前面存在多個用戶時,可以從攝像頭跟蹤的不同位置中選擇合適的人眼,比如,當顯示螢幕前面的左、中、右位置分別存在不同的人眼,則可以根據預先的設置規則,選擇顯示螢幕前面中間位置的人眼作為確定顯示多幀立體圖像的人眼位置。 In the embodiment of the present invention, obtaining the position of the human eye of the tracked object The information is also obtained by photographing the tracked human eye through the camera, and acquiring the position information of the display screen of the tracked object relative to the multi-frame stereo image. Wherein, when there are multiple users in front of the display screen, an appropriate human eye can be selected from different positions tracked by the camera. For example, when there are different human eyes in the left, middle, and right positions in front of the display screen, The setting rule selects the human eye that displays the middle position in front of the screen as the position of the human eye that determines the multi-frame stereo image.

S42,根據所述人眼位置資訊確定所述多幀立體圖像中顯示的特定立體圖像。在本發明實施例中,可以根據該人眼位置資訊計算出顯示哪張圖片;例如:如果觀察者兩眼連線的中間點位於顯示螢幕中間,那麼顯示的是多幀立體圖像中中間時刻拍攝的圖像所合成的立體圖像。當人眼位置位於顯示螢幕左邊的時侯,顯示出比中間時刻更早拍攝的圖片所合成的立體圖像,當人眼位置從中間位置轉向顯示螢幕右邊的時侯,顯示出中間時刻以後拍攝的圖片所合成的立體圖像。 S42. Determine a specific stereoscopic image displayed in the multi-frame stereoscopic image according to the human eye position information. In the embodiment of the present invention, which picture is displayed according to the position information of the human eye; for example, if the middle point of the line connecting the two eyes of the observer is located in the middle of the display screen, the intermediate time in the multi-frame stereo image is displayed. A stereoscopic image synthesized by the captured image. When the position of the human eye is on the left side of the display screen, a stereoscopic image synthesized by the picture taken earlier than the middle time is displayed, and when the position of the human eye is turned from the middle position to the right side of the display screen, the intermediate time is displayed. The stereo image of the picture.

在本發明實施例中,圖5為根據旋轉角度資訊顯示多幀立體圖像流程示意圖,如圖5所示,具體包括以下步驟:S51,獲取顯示所述多幀立體圖像的顯示螢幕的旋轉角度資訊。 In the embodiment of the present invention, FIG. 5 is a schematic diagram showing a flow of displaying a multi-frame stereoscopic image according to the rotation angle information. As shown in FIG. 5, the method further includes the following steps: S51: acquiring a rotation of a display screen displaying the multi-frame stereo image. Angle information.

在本發明實施例中,獲取顯示所述多幀立體圖像的顯示螢幕旋轉角度資訊,可以通過陀螺儀感測器X軸角速度及旋轉的時間來得到顯示螢幕旋轉的角度資訊,其中,可以以開始顯示立體圖像為初始狀態,再根據陀螺儀的角速度資訊計算出顯示螢幕繞X軸方向轉過的角度及方向。 In the embodiment of the present invention, the display screen rotation angle information of the multi-frame stereo image is obtained, and the angle information of the screen rotation can be obtained by using the gyro sensor X-axis angular velocity and the rotation time, wherein The stereoscopic image is initially displayed as an initial state, and the angle and direction of the display screen rotated about the X-axis direction are calculated according to the angular velocity information of the gyroscope.

S52,根據所述旋轉角度資訊確定所述多幀立體圖像中顯示的特定立體圖像。在本發明實施例中,可以根據旋轉角度資訊來讀取多幀圖像中的立體圖像並顯示。比如, 如果所獲取的圖像從左到右方向來拍攝的,所拍攝的圖像合成的立體圖像包手Img3D_1,Img3D_2,Img3D_3,Img3D_4...Img3D_N的N張圖片,Img3D_1為在最左邊位置所拍攝的圖像所合成的立體圖像,Img3D_N為在最右邊位置所拍攝的圖像所合成的立體圖像。在初始狀態時顯示為第N/2張,也即是拍攝過程的中間時刻圖像。如果順時針轉動,顯示出比中間時刻更早拍攝的圖片,轉過的角度越大,顯示拍攝的時間越早的圖片。如果逆時針轉動,顯示出比中間時刻更晚拍攝的圖片,轉過的角度越大,顯示拍攝的時間越晚的圖片。為了所顯的速度能適應人眼要求,不能刷屏太快,如果圖片的張數較多,可以作角速度分級的分析,如果角速度較慢為1級,每張都顯示。如果速度較快為2級,每隔1張顯示,如果角速度更快為3級,每隔2張顯示。 S52. Determine a specific stereoscopic image displayed in the multi-frame stereoscopic image according to the rotation angle information. In the embodiment of the present invention, the stereo image in the multi-frame image can be read and displayed according to the rotation angle information. such as, If the acquired image is taken from left to right, the stereo image of the captured image is composed of N pictures of Img3D_1, Img3D_2, Img3D_3, Img3D_4...Img3D_N, and Img3D_1 is at the leftmost position. The stereoscopic image synthesized by the captured image, Img3D_N is a stereoscopic image synthesized by the image taken at the rightmost position. In the initial state, it is displayed as the N/2th sheet, which is the intermediate moment image of the shooting process. If you turn clockwise, the picture taken earlier than the middle time is displayed. The larger the angle is, the earlier the picture is displayed. If it is rotated counterclockwise, a picture taken later than the middle time is displayed, and the larger the angle of the turn, the later the picture is displayed. In order to adapt to the requirements of the human eye, the screen can not be swiftly fast. If the number of pictures is large, the angular velocity grading analysis can be performed. If the angular velocity is slower, the display is displayed. If the speed is 2 levels, every 1 frame, if the angular speed is 3, it will be displayed every 2 frames.

需要說明的是,在本發明實施例中,在顯示多幀立體圖像時,如果同時存在人眼及旋轉角度資訊,可以設置優先順序以避免衝突,比如,旋轉角度資訊優先順序高於人眼位置資訊,則可以根據旋轉角度資訊來確定多幀立體圖像中顯示的特定立體圖像。 It should be noted that, in the embodiment of the present invention, when displaying a multi-frame stereo image, if both the human eye and the rotation angle information exist, the priority order may be set to avoid conflicts, for example, the rotation angle information is prioritized higher than the human eye. The position information can determine a specific stereoscopic image displayed in the multi-frame stereo image according to the rotation angle information.

需要說明的是,在本發明實施例中,利用兩幀圖像合成立體圖像、運動補償演算法以及人眼跟蹤演算法均可以採用現有技術中的演算法,本發明不作限定。 It should be noted that, in the embodiment of the present invention, the two-frame image synthesis stereo image, the motion compensation algorithm, and the human eye tracking algorithm may all adopt the algorithm in the prior art, which is not limited by the present invention.

在本發明實施例中,通過獲取一組多幀連續圖像;並從多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像進行立體圖像合成,以獲得一組不同視角的多幀立體圖像,解決了現有技術中沒有一種有效的圖像合成技術,填補了現有技術的空白。實現了全景立體圖像合成,滿足了用戶對全景立體圖像的觀看需求,而且,通過運動補償以消除呈現立體圖像時的圖像抖動,同時,可以根據人眼位置或旋轉 角度資訊進行全景立體圖像的顯示,進一步提升了使用者體驗。 In the embodiment of the present invention, a set of multi-frame continuous images is acquired; and from the multi-frame continuous image, two frames of images satisfying a predetermined frame interval are separately extracted for stereoscopic image synthesis to obtain a set of different perspectives. The multi-frame stereo image solves the gap in the prior art without solving an effective image synthesis technique in the prior art. The panoramic stereoscopic image synthesis is realized, which satisfies the user's viewing requirements for the panoramic stereoscopic image, and the motion compensation is used to eliminate the image shake when the stereoscopic image is presented, and at the same time, according to the position or rotation of the human eye. The angle information is used to display the panoramic stereo image, which further enhances the user experience.

圖6為本發明實施例一種全景立體圖像合成裝置示意圖,如圖6所示,包括:獲取單元61,用於獲取圖像資訊,所述圖像資訊為一組多幀連續圖像;抽取單元62,用於從所述多幀連續圖像中分別抽取滿足預定幀間隔的兩幀圖像,合成單元63,用於分別將抽取的所述兩幀圖像進行立體圖像合成,以獲得一組不同視角的多幀立體圖像。在本發明實施例中,所述圖像資訊包含每一幀圖像的移動角度和速度,所述角度和速度記錄在每一幀圖像的檔頭資訊中。 FIG. 6 is a schematic diagram of a panoramic stereoscopic image synthesizing apparatus according to an embodiment of the present invention. As shown in FIG. 6, the method includes: an acquiring unit 61, configured to acquire image information, where the image information is a set of multi-frame continuous images; The unit 62 is configured to separately extract two frames of images that satisfy a predetermined frame interval from the multi-frame continuous image, and the synthesizing unit 63 is configured to separately perform stereoscopic image synthesis on the extracted two frames of images. A set of multi-frame stereo images of different viewing angles. In the embodiment of the present invention, the image information includes a moving angle and a speed of each frame image, and the angle and speed are recorded in the header information of each frame image.

在本發明實施例中,抽取單元63,具體用於:從獲取的多幀連續圖像中抽取滿足預定幀間隔的兩幀圖像,其中,預定幀間隔根據以下兩種方式來確定:(1)如果使用者開始獲取圖像幀後手持設備一直保持勻速勻角度的操作(即第一種方式),就可以遵守公式:幀間隔=(總幀數/(角度*速度))/Ω決定抽取幀數的間隔,其中,所述總幀數為所述圖像資訊中多幀連續圖像的數量,所述Ω為測試所得的幀間隔經驗值。此間隔保持不變,即多幀連續圖像中只要滿足此幀間隔的兩幀圖像即可以兩兩組合,用來合成立體圖像。(2)可以用第1幀圖像分別和2到N幀合成立體圖像,將合成圖像品質最優組合作為抽取幀的間隔,此間隔保持不變,比如,假設第1幀圖像與第10幀圖像合成的立體圖像品質最優,則幀間隔即為9,而且,多幀連續圖像中的其他幀之間也按此幀間隔確定用來合成立體圖像的兩幀圖像。 In the embodiment of the present invention, the extracting unit 63 is configured to: extract two frames of images that satisfy a predetermined frame interval from the acquired multi-frame continuous image, wherein the predetermined frame interval is determined according to the following two manners: (1) If the user starts to acquire the image frame and the handheld device maintains a uniform angle (ie the first way), you can follow the formula: frame interval = (total number of frames / (angle * speed)) / Ω to determine the extraction The interval of the number of frames, wherein the total number of frames is the number of consecutive frames of the plurality of frames in the image information, and the Ω is an empirical value of the frame interval obtained by the test. This interval remains unchanged, that is, two frames of images in the multi-frame continuous image that satisfy the frame interval can be combined in two or two to synthesize a stereoscopic image. (2) The first frame image and the 2 to N frames can be combined to form a stereo image, and the composite image quality optimal combination is used as the interval of the extracted frame, and the interval remains unchanged, for example, assuming the first frame image and The image quality of the 10th frame image is optimal, and the frame interval is 9, and the other frames in the multi-frame continuous image are also determined according to the frame interval to form a two-frame image for synthesizing the stereo image. image.

在本發明實施例中,所述裝置,還包括:運動補償單元64,用於對所述立體圖像在垂直方向上作運動補償,以消除呈現所述立體圖像時的圖像抖動。在本發明實施例中,獲取多幀圖像過程中會產生上下移動,而且,在立體圖 像合成過程中,圖像拼接也會引入圖像的上下移動。因此,在顯示合成的立體圖像時,可以對上述多幀立體圖像垂直方向(Y軸方向)作運動補償,以消除連續呈現多幀立體圖像時圖像抖動。 In the embodiment of the present invention, the apparatus further includes: a motion compensation unit 64 configured to perform motion compensation on the stereoscopic image in a vertical direction to eliminate image jitter when the stereoscopic image is presented. In the embodiment of the present invention, an up and down movement occurs during the process of acquiring a multi-frame image, and, in the stereogram Like the compositing process, image stitching also introduces the image to move up and down. Therefore, when the synthesized stereoscopic image is displayed, the vertical direction (Y-axis direction) of the multi-frame stereoscopic image can be motion-compensated to eliminate image shake when the multi-frame stereoscopic image is continuously presented.

在本發明實施例中,所述裝置,還包括:顯示單元65,用於顯示所述多幀立體圖像。在本發明實施例中,為了方便使用者觀看合成的不同視角的多幀立體圖像,實現全景立體圖像觀看效果,可以通過顯示單元65呈現給使用者。 In the embodiment of the present invention, the device further includes: a display unit 65, configured to display the multi-frame stereo image. In the embodiment of the present invention, in order to facilitate the user to view the synthesized multi-frame stereo images of different viewing angles, the panoramic stereoscopic image viewing effect is realized, and may be presented to the user through the display unit 65.

為了方便給使用者呈現不同視角下的立體圖像,在本發明實施例中,所述裝置,還包括:人眼位置獲取單元66,用於獲取人眼相對於顯示多幀立體圖像的顯示螢幕的位置資訊,和/或,角度資訊獲取單元67,用於獲取顯示多幀立體圖像的顯示螢幕的旋轉角度資訊,顯示圖像確定單元68,用於根據人眼位置資訊或旋轉角度資訊,確定多幀立體圖像中顯示的特定立體圖像。 In the embodiment of the present invention, the apparatus further includes: a human eye position acquiring unit 66, configured to acquire a display of a human eye relative to displaying a multi-frame stereo image. The position information of the screen, and/or the angle information acquiring unit 67 is configured to acquire the rotation angle information of the display screen displaying the multi-frame stereo image, and the display image determining unit 68 is configured to use the position information of the human eye or the information of the rotation angle. A specific stereoscopic image displayed in the multi-frame stereoscopic image is determined.

在本發明實施例中,人眼位置獲取單元66可以通過攝像頭拍攝跟蹤人眼,同時,利用裸眼跟蹤演算法確定人眼相對於顯示螢幕的位置資訊。而且,當顯示螢幕前面存在多個用戶時,可以從攝像頭跟蹤的不同位置中選擇合適的人眼,比如,當顯示螢幕前面的左、中、右位置分別存在不同的人眼,則可以根據預先的設置規則,選擇顯示螢幕前面中間位置的人眼作為確定顯示多幀立體圖像的人眼位置。 In the embodiment of the present invention, the human eye position acquiring unit 66 can track and track the human eye through the camera, and at the same time, use the naked eye tracking algorithm to determine the position information of the human eye relative to the display screen. Moreover, when there are multiple users in front of the display screen, an appropriate human eye can be selected from different positions tracked by the camera. For example, when there are different human eyes in the left, middle, and right positions in front of the display screen, The setting rule selects the human eye that displays the middle position in front of the screen as the position of the human eye that determines the multi-frame stereo image.

在本發明實施例中,角度資訊獲取單元67可以通過顯示螢幕上的陀螺儀感應器獲取顯示螢幕的旋轉角度資訊。 In the embodiment of the present invention, the angle information acquiring unit 67 can obtain the rotation angle information of the display screen by displaying the gyro sensor on the screen.

在本發明實施例中,人眼位置獲取單元66及角度資訊獲取單元67可以同時存在,也可以只存在其中任意一個單元,當兩者同時存在時,為避免顯示圖像確定單元68在 確定顯示多幀立體圖像中的具體立體圖像時可能產生的衝突,可以設置優先順序,比如,優先根據人眼位置確定,當然,也可以優先根據旋轉角度來確定。 In the embodiment of the present invention, the human eye position obtaining unit 66 and the angle information acquiring unit 67 may exist at the same time, or only one of the units may exist. When both are present, the display image determining unit 68 is avoided. The conflict that may occur when displaying a specific stereoscopic image in the multi-frame stereoscopic image is determined, and the priority order may be set, for example, the priority is determined according to the position of the human eye, and of course, it may be determined preferentially according to the rotation angle.

需要說明的是,本發明實施例中的全景立體圖像合成裝置中各個單元之間的資訊交互、執行過程等內容,由於與本發明方法實施例基於同一構思,方法實施例中的具體內容同樣適用。本發明實施例中的各個單元可以作為單獨的硬體或軟體來實現,並且可以根據需要使用單獨的硬體或軟體來實現各個單元的功能的組合。 It should be noted that the content of the information interaction, the execution process, and the like between the units in the panoramic stereoscopic image synthesizing device in the embodiment of the present invention are the same as the specific concept in the method embodiment. Be applicable. The individual units in the embodiments of the present invention may be implemented as separate hardware or software, and a separate hardware or software may be used as needed to achieve a combination of functions of the respective units.

需要說明的是,在本發明實施例中,利用兩幀圖像合成立體圖像、運動補償演算法以及人眼跟蹤演算法均可以採用現有技術中的演算法,本發明不作限定。 It should be noted that, in the embodiment of the present invention, the two-frame image synthesis stereo image, the motion compensation algorithm, and the human eye tracking algorithm may all adopt the algorithm in the prior art, which is not limited by the present invention.

在本發明實施例中,通過獲取一組多幀連續圖像;並從多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像進行立體圖像合成,以獲得一組不同視角的多幀立體圖像,實現了全景立體圖像合成,滿足了用戶對全景立體圖像的觀看需求,而且,通過運動補償以消除呈現立體圖像時的圖像抖動,同時,可以根據人眼位置或旋轉角度資訊進行全景立體圖像的顯示,進一步提升了使用者體驗。 In the embodiment of the present invention, a set of multi-frame continuous images is acquired; and from the multi-frame continuous image, two frames of images satisfying a predetermined frame interval are separately extracted for stereoscopic image synthesis to obtain a set of different perspectives. The multi-frame stereo image realizes the panoramic stereo image synthesis, satisfies the user's viewing requirements for the panoramic stereo image, and also eliminates the image shake when the stereo image is presented by the motion compensation, and at the same time, according to the position of the human eye Or rotating the angle information to display the panoramic stereo image, further enhancing the user experience.

圖7為本發明實施例一種移動終端示意圖,包括:中央處理器71及顯示螢幕73,其中:中央處理器71,用於獲取圖像資訊,圖像資訊為一組多幀連續圖像;從多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像,以及分別將抽取的兩幀圖像進行立體圖像合成,以獲得一組不同視角的多幀立體圖像;顯示螢幕73,用於顯示多幀立體圖像。本發明實施例中,處理器所執行的上述操作可以以一段程式的形式存儲在記憶體中,當需要執行上述操作時,將該段程式調入處理器執行處理。 FIG. 7 is a schematic diagram of a mobile terminal according to an embodiment of the present invention, including: a central processing unit 71 and a display screen 73, wherein: a central processing unit 71 is configured to acquire image information, and the image information is a set of multiple frames of continuous images; In the multi-frame continuous image, two frames of images satisfying a predetermined frame interval are respectively extracted, and the extracted two frames are respectively subjected to stereoscopic image synthesis to obtain a plurality of multi-frame stereo images of different viewing angles; the display screen 73 For displaying multi-frame stereo images. In the embodiment of the present invention, the foregoing operations performed by the processor may be stored in the memory in the form of a program, and when the operation is required to be performed, the program is transferred to the processor to perform processing.

在本發明所有實施例中,移動終端可以包括諸如PDA、平板電腦、MP4、智慧手機,電紙書等可擕式移動電子設備。 In all embodiments of the present invention, the mobile terminal may include a portable mobile electronic device such as a PDA, a tablet, an MP4, a smart phone, an electronic paper book, or the like.

在本發明實施例中,通過獲取一組多幀連續圖像;並從多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像進行立體圖像合成,以獲得一組不同視角的多幀立體圖像,實現了全景立體圖像合成,滿足了用戶對全景立體圖像的觀看需求,而且,通過運動補償以消除呈現立體圖像時的圖像抖動,同時,可以根據人眼位置或旋轉角度資訊進行全景立體圖像的顯示,進一步提升了使用者體驗。 In the embodiment of the present invention, a set of multi-frame continuous images is acquired; and from the multi-frame continuous image, two frames of images satisfying a predetermined frame interval are separately extracted for stereoscopic image synthesis to obtain a set of different perspectives. The multi-frame stereo image realizes the panoramic stereo image synthesis, satisfies the user's viewing requirements for the panoramic stereo image, and also eliminates the image shake when the stereo image is presented by the motion compensation, and at the same time, according to the position of the human eye Or rotating the angle information to display the panoramic stereo image, further enhancing the user experience.

以上所述僅為本發明的實施方式,並非因此限制本發明的專利範圍,凡是利用本發明說明書及附圖內容所作的等效結構或等效流程變換,或直接或間接運用在其他相關的技術領域,均同理包括在本發明的專利保護範圍內。 The above is only the embodiment of the present invention, and is not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the invention and the drawings are directly or indirectly applied to other related technologies. The fields are all included in the scope of patent protection of the present invention.

S11、S13、S15、S17‧‧‧步驟 S11, S13, S15, S17‧‧‧ steps

Claims (15)

一種全景立體圖像的合成方法,其包括:獲取一圖像資訊,該圖像資訊為一第一多幀連續圖像,該圖像資訊包含每一該幀圖像的一移動角度和一速度,該移動角度和該速度記錄在每一該幀圖像的檔頭資訊中,該移動角度和該速度均為一勻速移動;從該第一多幀連續圖像中,分別抽取滿足一預定幀間隔的兩幀圖像,該預定幀間隔與該移動角度和該速度均成反比;分別將抽取的該兩幀圖像進行立體圖像合成,以獲得一組不同視角的一第二多幀立體圖像;其中,該預定幀間隔=(一總幀數/(該移動角度*該速度))/一幀間隔經驗值,其中,該總幀數為該圖像資訊中該第一多幀連續圖像的數量,該幀間隔經驗值為測試所得的經驗值。 A method for synthesizing a panoramic stereoscopic image, comprising: acquiring an image information, wherein the image information is a first multi-frame continuous image, the image information including a moving angle and a speed of each of the frame images The moving angle and the speed are recorded in the header information of each frame image, and the moving angle and the speed are both moving at a uniform speed; and from the first multi-frame continuous image, respectively, the predetermined frame is selected to be satisfied. Two frames of the image, the predetermined frame interval is inversely proportional to the moving angle and the speed; respectively, the extracted two frames of images are subjected to stereoscopic image synthesis to obtain a second multi-frame stereo with a different set of viewing angles. An image; wherein the predetermined frame interval = (a total number of frames / (the movement angle * the speed)) / a frame interval empirical value, wherein the total number of frames is the first multi-frame continuous in the image information The number of images, the empirical value of the frame interval is the empirical value obtained from the test. 一種全景立體圖像的合成方法,其包括:獲取一圖像資訊,該圖像資訊為一第一多幀連續圖像,該圖像資訊中包含一移動角度和一速度,該移動角度和該速度記錄在每一該幀圖像的檔頭資訊中,該移動角度和該速度為一變速移動;從該第一多幀連續圖像中,分別抽取滿足一預定幀間隔的兩幀圖像;分別將抽取的該兩幀圖像進行立體圖像合成,以獲得一組不同視角的一第二多幀立體圖像;其中,該預定幀間隔為該第一多幀連續圖像中一第一圖像幀分別與該第一多幀連續圖像中除該第一圖像幀之外的 其他圖像幀合成立體圖像後,選擇出品質最優的立體圖像對應的圖像幀與該第一圖像幀之間的幀間隔。 A method for synthesizing a panoramic stereoscopic image, comprising: acquiring an image information, wherein the image information is a first multi-frame continuous image, wherein the image information includes a moving angle and a speed, the moving angle and the The speed is recorded in the head information of each frame image, the moving angle and the speed are a shifting movement; and from the first multi-frame continuous image, two frames of images satisfying a predetermined frame interval are respectively extracted; Performing stereoscopic image synthesis on the extracted two frames of images to obtain a second multi-frame stereo image of different sets of perspectives; wherein the predetermined frame interval is a first one of the first multi-frame continuous images The image frame and the first multi-frame continuous image are respectively other than the first image frame After the other image frames are combined into the stereoscopic image, the frame interval between the image frame corresponding to the best-quality stereoscopic image and the first image frame is selected. 根據請求項2中所述的全景立體圖像的合成方法,其中,該變速移動為沿水平方向的移動或以拍攝物件為圓心的圓周移動。 A method of synthesizing a panoramic stereoscopic image according to claim 2, wherein the shifting movement is a movement in a horizontal direction or a circumference in which a subject is a center of a subject. 根據請求項3所述的全景立體圖像的合成方法,其中,該移動角度和該速度越大,獲取的該圖像資訊中的連續圖像數量越多,該移動角度和該速度越小,獲取的該圖像資訊中的連續圖像數量越少。 The method for synthesizing a panoramic stereoscopic image according to claim 3, wherein the larger the moving angle and the speed, the more the number of consecutive images in the acquired image information, the smaller the moving angle and the speed, The smaller the number of consecutive images in the acquired image information. 根據請求項2項所述的全景立體圖像的合成方法,其中,還包括:對該立體圖像在垂直方向上作運動補償,以消除呈現該立體圖像時的圖像抖動。 The method for synthesizing a panoramic stereoscopic image according to claim 2, further comprising: performing motion compensation on the stereoscopic image in a vertical direction to eliminate image shake when the stereoscopic image is presented. 根據請求項5項所述的全景立體圖像的合成方法,其中,還包括:顯示該第二多幀立體圖像。 The method for synthesizing a panoramic stereoscopic image according to claim 5, further comprising: displaying the second multi-frame stereoscopic image. 根據請求項6所述的全景立體圖像的合成方法,其中,顯示該第二多幀立體圖像,具體包括:獲取人眼相對於顯示該第一多幀立體圖像的顯示螢幕的一人眼位置資訊;根據該人眼位置資訊,確定該第二多幀立體圖像中顯示的特定立體圖像。 The method for synthesizing a panoramic stereoscopic image according to claim 6, wherein displaying the second multi-frame stereoscopic image comprises: acquiring a human eye with respect to a display screen displaying the first multi-frame stereoscopic image Position information; determining, according to the human eye position information, a specific stereoscopic image displayed in the second multi-frame stereoscopic image. 根據請求項6所述的全景立體圖像的合成方法,其中, 顯示該第二多幀立體圖像,具體包括:獲取顯示該第二多幀立體圖像的顯示螢幕的一旋轉角度資訊;根據該旋轉角度資訊,確定該第二多幀立體圖像中顯示的特定立體圖像。 a method for synthesizing a panoramic stereoscopic image according to claim 6, wherein Displaying the second multi-frame stereo image, specifically: acquiring a rotation angle information of the display screen displaying the second multi-frame stereo image; determining, according to the rotation angle information, the display in the second multi-frame stereo image Specific stereo image. 一種全景立體圖像合成裝置,其包括:一獲取單元,用於獲取一圖像資訊,該圖像資訊為一第一多幀連續圖像,該圖像資訊包含每一該幀圖像的一移動角度和一速度,該移動角度和該速度記錄在每一該幀圖像的一檔頭資訊中,該移動角度和該速度均為勻速移動;一抽取單元,用於從該第一多幀連續圖像中,分別抽取滿足一預定幀間隔的兩幀圖像,該預定幀間隔與該移動角度和速度均成反比;一合成單元,用於分別將抽取的該兩幀圖像進行立體圖像合成,以獲得一組不同視角的一第二多幀立體圖像;其中,該抽取單元中該預定幀間隔=(一總幀數/(該移動角度*該速度))/一幀間隔經驗值,其中,該總幀數為該圖像資訊中多幀連續圖像的數量,該幀間隔經驗值為測試所得的經驗值。 A panoramic stereoscopic image synthesizing device includes: an acquiring unit, configured to acquire an image information, wherein the image information is a first multi-frame continuous image, and the image information includes one of each of the frame images a moving angle and a speed, the moving angle and the speed are recorded in a header information of each frame image, the moving angle and the speed are moving at a constant speed; and an extracting unit is configured to use the first multiframe In the continuous image, two frames of images satisfying a predetermined frame interval are respectively extracted, and the predetermined frame interval is inversely proportional to the moving angle and the speed; and a synthesizing unit is configured to separately perform the extracted two frames of images. Forming a second multi-frame stereo image of a different set of angles; wherein the predetermined frame interval in the decimating unit = (a total number of frames / (the moving angle * the speed)) / a frame interval experience a value, wherein the total number of frames is the number of consecutive frames of the plurality of frames in the image information, and the empirical value of the frame interval is an empirical value obtained by the test. 一種全景立體圖像合成裝置,其包括:一獲取單元,用於獲取一圖像資訊,該圖像資訊為一第一多幀連續圖像,該圖像資訊包含每一該幀圖像的一移動角度和一速度,該移動角度和該速度為變速移動,該移動角度和該速度記錄在每一該幀圖像的一檔頭資訊中;一抽取單元,用於從該第一多幀連續圖像中,分別抽取滿足一預定幀間隔的兩幀圖像; 一合成單元,用於分別將抽取的該兩幀圖像進行立體圖像合成,以獲得一組不同視角的一第二多幀立體圖像;其中,該預定幀間隔為該第一多幀連續圖像中一第一圖像幀分別與該第一多幀連續圖像中除該第一圖像幀之外的其他圖像幀合成立體圖像後,選擇出品質最優的立體圖像對應的圖像幀與該第一圖像幀之間的幀間隔。 A panoramic stereoscopic image synthesizing device includes: an acquiring unit, configured to acquire an image information, wherein the image information is a first multi-frame continuous image, and the image information includes one of each of the frame images a moving angle and a speed, the moving angle and the speed being a shifting movement, the moving angle and the speed being recorded in a header information of each frame image; an extracting unit for continuously continuing from the first multiframe In the image, two frames of images satisfying a predetermined frame interval are respectively extracted; a synthesizing unit, configured to separately perform stereoscopic image synthesis on the extracted two frames of images to obtain a second multi-frame stereoscopic image of a different set of views; wherein the predetermined frame interval is the first multi-frame continuous A first image frame in the image is combined with another image frame other than the first image frame in the first multi-frame continuous image to select a stereo image corresponding to the best quality. The frame interval between the image frame and the first image frame. 根據請求項10所述的全景立體圖像合成裝置,其中包括:一運動補償單元,用於對該立體圖像在垂直方向上作運動補償,以消除呈現該立體圖像時的圖像抖動。 The panoramic stereoscopic image synthesizing apparatus according to claim 10, comprising: a motion compensating unit configured to perform motion compensation on the stereoscopic image in a vertical direction to eliminate image shake when the stereoscopic image is presented. 根據請求項11項所述的全景立體圖像合成裝置,其中包括:一顯示單元,用於顯示該第二多幀立體圖像。 The panoramic stereoscopic image synthesizing device of claim 11, comprising: a display unit for displaying the second multi-frame stereoscopic image. 根據請求項12所述的全景立體圖像合成裝置,其中,該全景立體圖像合成裝置還包括:一人眼位置獲取單元,用於獲取人眼相對於顯示該第二多幀立體圖像的顯示螢幕的一人眼位置資訊,一角度資訊獲取單元,用於獲取顯示該第二多幀立體圖像的顯示螢幕的一旋轉角度資訊,一顯示圖像確定單元,用於根據該人眼位置資訊或該旋轉角度資訊,確定該第二多幀立體圖像中顯示的特定立體圖像。 The panoramic stereoscopic image synthesizing device according to claim 12, wherein the panoramic stereoscopic image synthesizing device further comprises: a human eye position acquiring unit configured to acquire a display of the human eye relative to displaying the second multi-frame stereoscopic image a human eye position information of the screen, an angle information acquiring unit, configured to acquire a rotation angle information of the display screen displaying the second multi-frame stereo image, and a display image determining unit configured to use the human eye position information or The rotation angle information determines a specific stereoscopic image displayed in the second multi-frame stereoscopic image. 一種移動終端,其包括:一中央處理器,用於獲取圖像資訊,該圖像資訊為一第一 多幀連續圖像,該圖像資訊包含每一該幀圖像的一移動角度和一速度,該移動角度和該速度均為一勻速移動,該移動角度和該速度記錄在每一該幀圖像的檔頭資訊中;從該第一多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像,以及分別將抽取的該兩幀圖像進行立體圖像合成,以獲得一組不同視角的一第二多幀立體圖像,其中,該預定幀間隔=(一總幀數/(該移動角度*該速度))/一幀間隔經驗值,其中,該總幀數為該圖像資訊中該第一多幀連續圖像的數量,該幀間隔經驗值為測試所得的經驗值,該預定幀間隔與該移動角度和該速度均成反比;顯示螢幕,用於顯示該第二多幀立體圖像。 A mobile terminal includes: a central processing unit for acquiring image information, the image information being a first a multi-frame continuous image, the image information including a moving angle and a speed of each of the image of the frame, the moving angle and the speed are both moving at a constant speed, and the moving angle and the speed are recorded in each of the frame images In the header information of the image, from the first multi-frame continuous image, two frames of images satisfying a predetermined frame interval are respectively extracted, and the extracted two frames of images are respectively subjected to stereoscopic image synthesis to obtain a group a second multi-frame stereo image of different viewing angles, wherein the predetermined frame interval = (a total number of frames / (the moving angle * the speed)) / a frame interval empirical value, wherein the total number of frames is the figure For example, the number of consecutive images of the first plurality of frames in the information, the empirical value of the frame interval is an empirical value obtained by the test, the predetermined frame interval being inversely proportional to the moving angle and the speed; and a display screen for displaying the second Multi-frame stereo image. 一種移動終端,其包括:一中央處理器,用於獲取圖像資訊,該圖像資訊為一第一多幀連續圖像,該圖像資訊包含每一該幀圖像的一移動角度和一速度,該移動角度和該速度為一變速移動,該移動角度和該速度記錄在每一該幀圖像的檔頭資訊中;從該第一多幀連續圖像中,分別抽取滿足預定幀間隔的兩幀圖像,以及分別將抽取的該兩幀圖像進行立體圖像合成,以獲得一組不同視角的一第二多幀立體圖像,該預定幀間隔為該第一多幀連續圖像中一第一圖像幀分別與該第一多幀連續圖像中除該第一圖像幀之外的其他圖像幀合成立體圖像後,選擇出品質最優的立體圖像對應的圖像幀與該第一圖像幀之間的幀間隔;顯示螢幕,用於顯示該第二多幀立體圖像。 A mobile terminal includes: a central processing unit, configured to acquire image information, wherein the image information is a first multi-frame continuous image, the image information includes a moving angle of each of the frame images and a Speed, the moving angle and the speed are a shifting movement, the moving angle and the speed are recorded in the head information of each frame image; and from the first multi-frame continuous image, respectively, the predetermined frame interval is satisfied Two frames of images, and respectively extracting the two frames of images for stereoscopic image synthesis to obtain a second plurality of stereoscopic images of different sets of views, the predetermined frame interval being the first multi-frame continuous image Selecting a stereo image corresponding to the best quality after synthesizing the stereo image with the first image frame of the first one and the other image frames of the first multi-frame continuous image except the first image frame a frame interval between the image frame and the first image frame; a display screen for displaying the second multi-frame stereo image.
TW104137748A 2015-06-30 2015-11-16 Panoramic stereoscopic image synthesis method, apparatus and mobile terminal TWI584050B (en)

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510373146.5A CN106303495B (en) 2015-06-30 2015-06-30 Synthetic method, device and its mobile terminal of panoramic stereo image

Publications (2)

Publication Number Publication Date
TW201701051A TW201701051A (en) 2017-01-01
TWI584050B true TWI584050B (en) 2017-05-21

Family

ID=57651118

Family Applications (1)

Application Number Title Priority Date Filing Date
TW104137748A TWI584050B (en) 2015-06-30 2015-11-16 Panoramic stereoscopic image synthesis method, apparatus and mobile terminal

Country Status (2)

Country Link
CN (1) CN106303495B (en)
TW (1) TWI584050B (en)

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106851251A (en) * 2017-02-28 2017-06-13 宇龙计算机通信科技(深圳)有限公司 Three-D imaging method and three-dimensional imaging terminal
CN109672776B (en) 2017-10-16 2021-07-09 华为技术有限公司 Method and terminal for displaying dynamic image
CN109104576A (en) * 2018-10-29 2018-12-28 努比亚技术有限公司 A kind of panorama shooting method, wearable device and computer readable storage medium
TWI700000B (en) * 2019-01-29 2020-07-21 威盛電子股份有限公司 Image stabilization method and apparatus for panoramic video, and method for evaluating image stabilization algorithm
CN110769159B (en) * 2019-11-19 2021-07-09 海南车智易通信息技术有限公司 Photographing method, photographing device and panoramic photographing system
CN114697758B (en) * 2020-12-30 2023-03-31 北京金山云网络技术有限公司 Video processing method and device and electronic equipment

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012068380A (en) * 2010-09-22 2012-04-05 Sony Corp Image processor, imaging apparatus, image processing method, and program
US20130033566A1 (en) * 2011-08-02 2013-02-07 Sony Corporation Image processing device, and control method and computer readable medium

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10080006B2 (en) * 2009-12-11 2018-09-18 Fotonation Limited Stereoscopic (3D) panorama creation on handheld device
US20120019613A1 (en) * 2009-12-11 2012-01-26 Tessera Technologies Ireland Limited Dynamically Variable Stereo Base for (3D) Panorama Creation on Handheld Device
CN102905079B (en) * 2012-10-16 2015-08-19 小米科技有限责任公司 For the method for pan-shot, device and mobile terminal
CN104660897A (en) * 2013-11-20 2015-05-27 浪潮乐金数字移动通信有限公司 Acquisition method of 360-degree panoramic image based on mobile terminal

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012068380A (en) * 2010-09-22 2012-04-05 Sony Corp Image processor, imaging apparatus, image processing method, and program
US20130033566A1 (en) * 2011-08-02 2013-02-07 Sony Corporation Image processing device, and control method and computer readable medium

Also Published As

Publication number Publication date
TW201701051A (en) 2017-01-01
CN106303495A (en) 2017-01-04
CN106303495B (en) 2018-01-16

Similar Documents

Publication Publication Date Title
TWI584050B (en) Panoramic stereoscopic image synthesis method, apparatus and mobile terminal
CN109615703B (en) Augmented reality image display method, device and equipment
US20180309982A1 (en) System and method for creating a navigable, three-dimensional virtual reality environment having ultra-wide field of view
KR102214827B1 (en) Method and apparatus for providing augmented reality
CN106507094B (en) Correct the method and device of panoramic video display view angle
US10277890B2 (en) System and method for capturing and viewing panoramic images having motion parallax depth perception without image stitching
TWI547901B (en) Simulating stereoscopic image display method and display device
US20160065862A1 (en) Image Enhancement Based on Combining Images from a Single Camera
US20130335521A1 (en) Method of Synthesizing Stereo 3D Panorama Image and Related 3D Camera
CN106296589B (en) Panoramic image processing method and device
WO2017032336A1 (en) System and method for capturing and displaying images
CN108833877B (en) Image processing method and device, computer device and readable storage medium
JP2006287921A (en) Moving image generating apparatus, moving image generation method, and program
JP2016504828A (en) Method and system for capturing 3D images using a single camera
KR20130039522A (en) Apparatus and method of creating 3 dimension panorama image
EP3526639A1 (en) Display of visual data with a virtual reality headset
US9161018B2 (en) Methods and systems for synthesizing stereoscopic images
CN105898338A (en) Panorama video play method and device
US10244220B2 (en) Multi-camera time slice system and method of generating integrated subject, foreground and background time slice images
JP2006310936A (en) System for generating video image viewed at optional viewpoint
JP2018033107A (en) Video distribution device and distribution method
KR101212223B1 (en) Device taking a picture and method to generating the image with depth information
JP2010226390A (en) Imaging apparatus and imaging method
CN106680996A (en) Display method and display control system of head-mounted virtual reality display
US10587856B1 (en) Spherical visual content transitions

Legal Events

Date Code Title Description
MM4A Annulment or lapse of patent due to non-payment of fees