Summary of the invention
For this, the present invention provides a kind of multiple terminals associating display packing, Apparatus and system, to try hard to solve or at least alleviate Problems existing above.
According to an aspect of the present invention, it is provided that a kind of multiple terminals associating display packing, the method performs in the first mobile terminal, including step: input request in response to user, is arranged to combine the screen area quantity of display video;A playback terminal is associated for each screen area;Converting video data to the sub video data stream that quantity is corresponding with screen area quantity, wherein each sub video data stream is corresponding with each screen area respectively;And each sub video data stream is pushed to respectively the playback terminal being associated with each screen area to play.
Alternatively, combining in display packing in the multiple terminals according to the present invention, screen area quantity is at least 2.
Alternatively, combining in display packing in the multiple terminals according to the present invention, the step converting video data to the quantity sub video data stream corresponding with screen area quantity includes: to video data decoding to be played;Decoded partitioning video data is become the sub video data stream corresponding with each screen area;And respectively each sub video data stream is encoded, obtain the sub-video data stream of screen area quantity.
Alternatively, combine in the multiple terminals according to the present invention in display packing, decoded partitioning video data becomes the step of the sub video data stream corresponding with each screen area include: each image frame division of video data is become the sub image frame corresponding with each screen area;And splice each sub image frame that same screen area is corresponding, obtain sub video data stream.
Alternatively, combining in display packing in the multiple terminals according to the present invention, the step respectively each sub video data stream being encoded also includes: obtain the resolution of each playback terminal being associated;And respectively each sub video data stream is carried out interpolation process according to resolution.
Alternatively, combine in display packing in the multiple terminals according to the present invention, further comprise the steps of: and sequentially each screen area is numbered;And determine, according to number order, the physical location that the playback terminal of associated is put, to remind user to put playback terminal according to position.
Alternatively, combine in display packing in the multiple terminals according to the present invention, further comprise the steps of: and set up broadcast group with multiple playback terminals;Obtain the device identification of playback terminal;Wherein, playback terminal also includes described first mobile terminal.
Alternatively, combining in the multiple terminals according to the present invention in display packing, the step associating a playback terminal for each screen area includes: distributes one for each screen area and plays mark;And be that each mark of playing associates a device identification.
Alternatively, combine in display packing in the multiple terminals according to the present invention, further comprise the steps of: the multiple playback terminals in broadcast group and receive each sub video data stream, play each sub video data stream.
According to a further aspect in the invention, a kind of multiple terminals associating display device is provided, this device is suitable to reside in the first mobile terminal, device includes: input/output interface, be suitable to receive user and input request, it is further adapted for each sub video data stream is pushed to playback terminal respectively that be associated with each screen area, in order to play;Segmentation module, is suitable to input request according to user, is arranged to combine the screen area quantity of display video;Associated allocation module, is suitable for each screen area one playback terminal of association;And concatenation module, be suitable to convert video data to the sub video data stream that quantity is corresponding with screen area quantity, wherein each sub video data stream is corresponding with each screen area divided respectively.
Alternatively, combining in display device in the multiple terminals according to the present invention, screen area quantity is at least 2.
Alternatively, combine in the multiple terminals according to the present invention in display device, concatenation module includes: encoding and decoding subelement, is suitable to video data decoding to be played and respectively each sub video data stream is encoded, and obtains the sub-video data stream of screen area quantity;And splicing subelement, be suitable to become decoded partitioning video data the sub video data stream corresponding with each screen area.
Alternatively, combine in the multiple terminals according to the present invention in display device, splicing subelement is further adapted for each image frame division of video data being become the sub image frame corresponding with each screen area and splicing each sub image frame that same screen area is corresponding, obtains sub video data stream.
Alternatively, combining in display device in the multiple terminals according to the present invention, input/output interface is further adapted for obtaining the resolution of each playback terminal being associated;Encoding and decoding subelement is further adapted for respectively each sub video data stream being carried out interpolation process according to resolution.
Alternatively, combining in the multiple terminals according to the present invention in display device, segmentation module is further adapted for sequentially each screen area being numbered;And associated allocation module is further adapted for determining, according to number order, the physical location that the playback terminal of associated is put, to remind user to put playback terminal according to position.
Alternatively, combine in display device in the multiple terminals according to the present invention, also include: control module, be suitable to set up broadcast group with multiple playback terminals;And the device identification of acquisition playback terminal, wherein, playback terminal also includes the first mobile terminal.
Alternatively, combining in the multiple terminals according to the present invention in display device, associated allocation module is further adapted for obtaining the broadcasting mark of each screen area and playing mark one device identification of association for each.
Alternatively, combining in display device in the multiple terminals according to the present invention, be further adapted for residing in the multiple playback terminals in this broadcast group, input/output interface is further adapted for receiving by each sub video data stream;Also include playing module, be suitable to play each sub video data stream.
According to another aspect of the invention, it is provided that a kind of multiple terminals associating display system, including first mobile terminal and at least one other playback terminal, wherein all terminals in this system all have multiple terminals as above associating display device.
From technical scheme above, the present invention is different from the existing technology of tiled display picture material on the display screen of a mobile terminal, but set up broadcast group between the mobile terminal of multiple same sizes, using multiple mobile terminals as the virtual splicing of playback terminal, then partitioning video data is become multiple sub video data stream, sub video data stream is associated one by one with multiple playback terminals, master control equipment is done by choosing a mobile terminal, control the association output of each sub video data stream and arrive multiple playback terminals, realize multiple playback terminal and synchronize playing video data, to reach visually to expand the effect of display content, improve the viewing experience of user.
Detailed description of the invention
It is more fully described the exemplary embodiment of the disclosure below with reference to accompanying drawings.Although accompanying drawing showing the exemplary embodiment of the disclosure, it being understood, however, that may be realized in various forms the disclosure and should do not limited by embodiments set forth here.On the contrary, it is provided that these embodiments are able to be best understood from the disclosure, and complete for the scope of the present disclosure can be conveyed to those skilled in the art.
Fig. 1 is mobile terminal 100 organigram according to an embodiment of the invention.With reference to Fig. 1, mobile terminal 100 includes: memory interface 102, one or more data processor, image processor and/or CPU 104, and peripheral interface 106.Memory interface 102, one or more processor 104 and/or peripheral interface 106 both can be discrete components, it is also possible to be integrated in one or more integrated circuit.In the mobile terminal 100, various elements can be coupled by one or more communication bus or holding wire.Sensor, equipment and subsystem are alternatively coupled to peripheral interface 106, in order to help to realize several functions.Such as, motion sensor 110, optical sensor 112 and range sensor 114 are alternatively coupled to peripheral interface 106, to facilitate the functions such as orientation, illumination and range finding.Other sensors 116 are equally possible to be connected with peripheral interface 106, for instance alignment system (such as GPS), temperature sensor, biometric sensor or other sensor devices, thus can help to implement relevant function.
Camera sub-system 120 and optical pickocff 122 may be used for the realization of the camera function of convenient such as recording photograph and video clipping, and wherein said camera sub-system and optical pickocff can be such as charge-coupled image sensor (CCD) or complementary metal oxide semiconductors (CMOS) (CMOS) optical pickocff.Can helping to realize communication function by one or more radio communication subsystem 124, wherein radio communication subsystem can include radio-frequency transmitter and transmitter and/or light (such as infrared) Receiver And Transmitter.The particular design of radio communication subsystem 124 and embodiment can depend on one or more communication networks that mobile terminal 100 is supported.Such as, mobile terminal 100 can include being designed to the communication subsystem 124 of supporting GSM network, GPRS network, EDGE network, Wi-Fi or WiMax network and BlueboothTM network.Audio subsystem 126 can be coupled with speaker 128 and mike 130, in order to helps the function implementing to enable voice, for instance speech recognition, speech reproduction, digital record and telephony feature.
I/O subsystem 140 can include touch screen controller 142 and/or other input controllers 144 one or more.Touch screen controller 142 is alternatively coupled to touch screen 146.For example, what this touch screen 146 and touch screen controller 142 can use in multiple touch-sensing technology any detects the contact and movement or time-out that carry out with it, and wherein detection technology is including, but not limited to capacitive character, resistive, infrared and surface acoustic wave technique.Other input controllers 144 one or more are alternatively coupled to other input/control devicess 148, for instance the pointer device of one or more buttons, rocker switch, thumb wheel, infrared port, USB port and/or instruction pen etc.The one or more button (not shown) can include for controlling speaker 128 and/or the up/down button of mike 130 volume.
Memory interface 102 can be coupled with memorizer 150.This memorizer 150 can include high-speed random access memory and/or nonvolatile memory, for instance one or more disk storage equipment, one or more optical storage apparatus, and/or flash memories (such as NAND, NOR).Memorizer 150 can store operating system 152, for instance the operating system of Android, IOS or WindowsPhone etc.This operating system 152 can include for processing basic system services and execution depends on the instruction of task of hardware.Memorizer 150 can also store application 154.When these are applied in operation, can be loaded into processor 104 from memorizer 150, and run on the operating system run by processor 104, and utilize the interface that operating system and bottom hardware provide to realize the desired function of various user, such as instant messaging, web page browsing, pictures management etc..Application can be independently of what operating system provided, it is also possible to is that operating system carries.
According to one embodiment of present invention, it is provided that a kind of have the mobile terminal 100 combining display function, it is possible to by arranging that in multiple mobile terminals 100 corresponding multiple terminals associating display device 300 realizes above-mentioned functions.
Implementation according to the present invention, multiple terminals associating display system is made up of at least two mobile terminal 100.Further, for reaching the display effect of comfortable, the multiple mobile terminals 100 in this system are preferably provided with identical size.By one of them mobile terminal 100 of user setup as the master control equipment in this system, can communicate with other multiple mobile terminals 100 by setting up broadcast group.
Fig. 2 illustrates the flow chart of multiple terminals associating display packing 200 according to an embodiment of the invention.The method 200 starts from step S210, inputs request in response to user, is arranged to combine the screen area quantity of display video.Such as require that video content is being divided into the broadcasting of several regional combination as inputting on the first mobile terminal of master control equipment by user, input in response to user, first mobile terminal marks off respective numbers screen area, and usually, screen area all average divisions obtain.If wherein screen area quantity at least 2, it is possible to being include a master control equipment and other playback equipment, master control equipment can also make playback equipment simultaneously.
Subsequently in step S220, associate a mobile terminal 100 for above-mentioned each screen area, as playback equipment.
First, the first mobile terminal 100 sets up broadcast group with other mobile terminals 100 in this system.Such as, the forwarding of communication data, study etc. between each terminal are completed by RSTP agreement.Then, the device identification of all mobile terminals in the first acquisition for mobile terminal broadcast group.Meanwhile, the first mobile terminal, as master control equipment, distributes one for each screen area and plays mark, and as shown in Figure 4 A, the first mobile terminal marks off two screen areas in left and right, and correspondence plays mark 401 and 402 respectively.Finally, the first mobile terminal is respectively each plays mark one device identification of association, as shown in Figure 4 A, plays the corresponding mobile terminal 1001 of mark 401, plays the corresponding mobile terminal 1002 of mark 402.
In like manner, when video content to be divided into four screen areas by user, as shown in Figure 4 B, the broadcasting mark of 4 screen areas is 403,404,405 and 406, respectively corresponding following 4 mobile terminals 1003,1004,1005 and 1006.
It should be noted that, by described above it can be seen that as the first mobile terminal of master control equipment, it is also possible to play video content simultaneously as playback equipment.So, acquired device identification also includes the device identification of the first mobile terminal self, it to be associated with one of them broadcasting mark.
After screen area is associated with mobile terminal, subsequently it is necessary to consider the assignment problem of video content in step S230.According to one embodiment of present invention, converting video data to the sub video data stream that quantity is corresponding with screen area quantity, wherein each sub video data stream is corresponding with each screen area divided respectively.This step can be subdivided into again the following steps:
First, to video data decoding to be played.It is for instance possible to use ffmpeg decodes video data to be played, obtaining decoder by calling the ffmpeg API " avcodec_find_decoder " provided, circulation is read view data from video file and is decoded.In the present invention, video decoding algorithm not being limited, the decoding algorithm of any video image may be used to treat playing video data decoding, and combine use with embodiments of the invention.
Then, decoded partitioning video data is become the sub video data stream corresponding with each screen area.Such as, if having divided 2 screen areas, then just video data to be played to be divided into 2 sub-video data streams.Specific practice is: each image frame division of video data is become the sub image frame (such as, divide 2 sub image frame) corresponding with each screen area, then splices each sub image frame corresponding in same screen area, obtains sub video data stream.
Then, respectively ready-portioned each sub video data stream gravity is newly encoded, obtains the sub-video data stream of screen area quantity.
Consider to can be potentially encountered situations below: the resolution of video to be played is 1920*1080 originally, after being divided into 2 sub-videos, the resolution of each sub video data stream may be 960*1080, now, if the resolution of corresponding playback terminal is 1920*1080, for ensureing the viewing quality of video, in this step, it is necessary to sub video data flows to row interpolation operation.
Therefore in this step, following operation is also included: obtain the resolution of each playback terminal being associated, respectively each sub video data stream is carried out interpolation process according to the resolution of playback terminal.
Finally, each sub video data stream is pushed to respectively the playback terminal being associated with each screen area, in order to play.Similarly, the mobile terminal as playback equipment in broadcast group, namely playback terminal, receive each sub video data stream, and perform the dependent instruction each sub video data stream of broadcasting.
When pushing video data stream, need sub video data stream and playback terminal one_to_one corresponding on the one hand, on the other hand, in addition it is also necessary to note the order between each screen area.Therefore, in this step, also sequentially each screen area is numbered, determine, according to this number order, the physical location that the playback terminal of associated is put, to remind user to put mobile terminal according to position simultaneously.
According to certain embodiment, numbering herein can also the broadcasting mark of each screen area be general with step S220.Such as in Figure 4 A, or using 401,402 as the numbering of screen area, then, accordingly, the order of putting of two mobile terminals 1001,1002 of associated is necessary for being 1001 left sides being positioned at 1002.
According to above-mentioned flow process, when user wishes the viewing experience having large-size screen monitors or under some viewing scene needing amplified video content, it is possible to by this programme, by virtual for multiple mobile terminals splicing, and by suitable putting position, it is achieved expand display video content across screen.Meanwhile, by controlling master control equipment (namely the first mobile terminal), it is achieved the function such as the synchronization broadcasting of video content, time-out, rollback, to meet the viewing demand of user, Consumer's Experience is improved.
Fig. 3 illustrates the schematic diagram of multiple terminals associating display device 300 according to an embodiment of the invention.This device 300 includes: input/output interface 310, segmentation module 320, associated allocation module 330 and concatenation module 340.Wherein, segmentation module 320 couples with associated allocation module 330 and concatenation module 340 phase respectively.
Input/output interface 310 is suitable to receive user and inputs request, for instance user is determined by input numeral to divide several screen area, namely to mate several mobile terminal and realize combining broadcasting display video content.
According to another embodiment, input/output interface 310 is further adapted for receiving the selection input of user, and user chooses some mobile terminal by input and makes the first mobile terminal, as master control equipment.
Segmentation module 320 is suitable to input request according to user, is arranged to the screen area quantity that associating shows.Definition screen area quantity is at least 2.Meanwhile, segmentation module 320 is further adapted for sequentially each screen area being numbered, in order to follow-up multiple mobile terminals to playing video are ranked up.
Associated allocation module 330 is suitable for each screen area one mobile terminal of association as playback terminal.By above description it can be seen that associated allocation module 330 is further adapted for the physical location that the number order according to screen area determines that the playback terminal of associated is put, to remind user to put playback terminal according to position.
According to the embodiment of the present invention, this device 300 also includes controlling module, be suitable to set up broadcast group with the multiple mobile terminals in system, obtain the device identification of each playback terminal simultaneously, wherein, the first mobile terminal as master control equipment can also be used for playing video content as playback terminal, therefore the first mobile terminal is further adapted for obtaining the device identification of oneself.
Like this, associated allocation module 330 can also obtain the broadcasting mark of each screen area and play mark one device identification of association for each.Realizing method according to one, broadcasting mark herein can be exactly the numbering of each screen area.
Concatenation module 340 is suitable to video data is converted to sub video data stream that quantity is corresponding with screen area quantity accordingly, and wherein each sub video data stream is corresponding with each screen area divided respectively.
According to embodiments of the invention, concatenation module 340 includes: encoding and decoding subelement and splicing subelement.Wherein, encoding and decoding subelement is suitable to video data decoding to be played, splicing subelement is suitable to become decoded partitioning video data the sub video data stream corresponding with each screen area, then, encoding and decoding subelement is suitable to respectively each sub video data stream is encoded, and obtains the sub-video data stream of screen area quantity.Specifically, splicing subelement is further adapted for each image frame division in video data being become the sub image frame corresponding with each screen area and splicing each sub image frame that same screen area is corresponding, obtains sub video data stream.
Mentioned in based on the description of Fig. 2, when video to be played is recompiled, in addition it is also necessary to the problem considering video resolution.Now, input/output interface 310 is further adapted for obtaining the resolution of each playback terminal being associated, and encoding and decoding subelement is further adapted for respectively each sub video data stream being carried out interpolation process according to resolution.Being referred to Fig. 2 associated description, specific practice repeats no more herein.Encoding and decoding for video data process or interpolation processing, it is possible to realizing by calling ffmpeg, can also be realized by other algorithms, this programme is to not limiting herein.
Input/output interface 310 is further adapted for each sub video data stream is pushed to playback terminal respectively that be associated with each screen area, in order to play.Owing to each screen area, each sub video data stream and each playback terminal are associated by associated allocation module 330, so output interface is just passable by the mark correspondence pushing video data stream of association, user is well placed the position of playback terminal according to number order.
Fig. 4 A and Fig. 4 B respectively illustrates two examples being arranged to the screen area that associating shows, with reference to the description about Fig. 4 A and Fig. 4 B, describe in detail and how each screen area, each sub video data stream and each playback terminal are associated, repeat no more herein.
It is to be understood that, one or more in order to what simplify that the disclosure helping understands in each inventive aspect, herein above in the description of the exemplary embodiment of the present invention, each feature of the present invention is grouped together in single embodiment, figure or descriptions thereof sometimes.But, the method for the disclosure should be construed to and reflect an intention that namely the present invention for required protection requires than the feature more features being expressly recited in each claim.More precisely, as the following claims reflect, inventive aspect is in that all features less than single embodiment disclosed above.Therefore, it then follows claims of detailed description of the invention are thus expressly incorporated in this detailed description of the invention, wherein each claim itself as the independent embodiment of the present invention.
The module of the equipment that those skilled in the art are to be understood that in example disclosed herein or unit or assembly can be arranged in equipment as depicted in this embodiment, or alternatively can be positioned in one or more equipment different from the equipment in this example.Module in aforementioned exemplary can be combined as a module or be segmented into multiple submodule in addition.
Those skilled in the art are appreciated that, it is possible to carry out the module in the equipment in embodiment adaptively changing and they being arranged in one or more equipment different from this embodiment.Module in embodiment or unit or assembly can be combined into a module or unit or assembly, and multiple submodule or subelement or sub-component can be put them in addition.Except at least some in such feature and/or process or unit excludes each other, it is possible to adopt any combination that all processes or the unit of all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so disclosed any method or equipment are combined.Unless expressly stated otherwise, each feature disclosed in this specification (including adjoint claim, summary and accompanying drawing) can be replaced by the alternative features providing purpose identical, equivalent or similar.
A6, method as according to any one of A1-5, further comprise the steps of: and sequentially each screen area numbered;And determine, according to number order, the physical location that the playback terminal of associated is put, to remind user to put playback terminal according to position.A7, method as according to any one of A1-6, further comprise the steps of: and set up broadcast group with multiple playback terminals;Obtain the device identification of playback terminal;Wherein, playback terminal includes the first mobile terminal.A8, method as described in A7, wherein the step for each screen area one playback terminal of association includes: distributes one for each screen area and plays mark;And be that each mark of playing associates a device identification.A9, method as described in A7 or 8, further comprise the steps of: the multiple playback terminals in broadcast group and receive each sub video data stream, play each sub video data stream.
B14, device as described in B12 or 13, wherein, input/output interface is further adapted for obtaining the resolution of each playback terminal being associated;Encoding and decoding subelement is further adapted for respectively each sub video data stream being carried out interpolation process according to described resolution.B15, device as according to any one of B10-14, wherein, segmentation module is further adapted for sequentially described each screen area being numbered;And associated allocation module is further adapted for determining, according to described number order, the physical location that the playback terminal of associated is put, to remind user to put playback terminal according to position.B16, device as according to any one of B10-15, also include: controls module, is suitable to set up broadcast group with multiple playback terminals;And the device identification of acquisition playback terminal, wherein, playback terminal includes the first mobile terminal.B17, device as described in B16, wherein, associated allocation module is further adapted for obtaining the broadcasting mark of each screen area and playing mark one device identification of association for each.B18, device as described in B16 or 17, wherein, device is further adapted for residing in the multiple playback terminals in this broadcast group, and input/output interface is further adapted for receiving each sub video data stream;Device also includes playing module, is suitable to play each sub video data stream.
In addition, those skilled in the art it will be appreciated that, although embodiments more described herein include some feature included in other embodiments rather than further feature, but the combination of the feature of different embodiment means to be within the scope of the present invention and form different embodiments.Such as, in the following claims, the one of any of embodiment required for protection can mode use in any combination.
Additionally, some in described embodiment be described as at this can by the processor of computer system or the method implemented by other device performing described function or the combination of method element.Therefore, the processor with the necessary instruction for implementing described method or method element is formed for implementing the method or the device of method element.Additionally, the element described herein of device embodiment is the example of following device: this device is for implementing the function performed by the element of the purpose in order to implement this invention.
As used in this, unless specifically stated so, use ordinal number " first ", " second ", " the 3rd " etc. to describe plain objects and be merely representative of the different instances relating to similar object, and be not intended to imply the object being so described must have the time upper, spatially, sequence aspect or in any other manner to definite sequence.
Although the embodiment according to limited quantity describes the present invention, but benefits from above description, those skilled in the art understand, in the scope of the present invention thus described, it can be envisaged that other embodiments.Additionally, it should be noted that the language used in this specification is primarily to purpose that is readable and that instruct and selects, rather than select to explain or limit the theme of the present invention.Therefore, when not necessarily departing from the scope of the appended claims and spirit, many modifications and changes will be apparent from for those skilled in the art.For the scope of the present invention, disclosing that the present invention is done is illustrative and not restrictive, and it is intended that the scope of the present invention be defined by the claims appended hereto.