CN107205122A - The live camera system of multiresolution panoramic video and method - Google Patents
The live camera system of multiresolution panoramic video and method Download PDFInfo
- Publication number
- CN107205122A CN107205122A CN201710656265.0A CN201710656265A CN107205122A CN 107205122 A CN107205122 A CN 107205122A CN 201710656265 A CN201710656265 A CN 201710656265A CN 107205122 A CN107205122 A CN 107205122A
- Authority
- CN
- China
- Prior art keywords
- module
- video
- user
- sub
- streaming media
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 17
- 230000005540 biological transmission Effects 0.000 claims abstract description 11
- 238000005538 encapsulation Methods 0.000 claims abstract description 9
- 238000012545 processing Methods 0.000 claims description 30
- 230000003993 interaction Effects 0.000 claims description 24
- 238000013507 mapping Methods 0.000 claims description 20
- 230000000007 visual effect Effects 0.000 claims description 11
- 238000004891 communication Methods 0.000 claims description 9
- FGUUSXIOTUKUDN-IBGZPJMESA-N C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 Chemical compound C1(=CC=CC=C1)N1C2=C(NC([C@H](C1)NC=1OC(=NN=1)C1=CC=CC=C1)=O)C=CC=C2 FGUUSXIOTUKUDN-IBGZPJMESA-N 0.000 claims description 3
- 238000005516 engineering process Methods 0.000 abstract description 4
- 230000010365 information processing Effects 0.000 abstract description 2
- 238000007726 management method Methods 0.000 description 5
- 230000006835 compression Effects 0.000 description 3
- 238000007906 compression Methods 0.000 description 3
- 238000011161 development Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 210000005252 bulbus oculi Anatomy 0.000 description 1
- 238000013500 data storage Methods 0.000 description 1
- 210000001508 eye Anatomy 0.000 description 1
- 230000002045 lasting effect Effects 0.000 description 1
- 238000009331 sowing Methods 0.000 description 1
- 238000010025 steaming Methods 0.000 description 1
- 230000009897 systematic effect Effects 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/698—Control of cameras or camera modules for achieving an enlarged field of view, e.g. panoramic image capture
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L65/00—Network arrangements, protocols or services for supporting real-time applications in data packet communication
- H04L65/60—Network streaming of media packets
- H04L65/75—Media network packet handling
- H04L65/762—Media network packet handling at the source
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/21—Server components or server architectures
- H04N21/218—Source of audio or video content, e.g. local disk arrays
- H04N21/21805—Source of audio or video content, e.g. local disk arrays enabling multiple viewpoints, e.g. using a plurality of cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23424—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving splicing one content stream with another content stream, e.g. for inserting or substituting an advertisement
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/2343—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements
- H04N21/234363—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving reformatting operations of video signals for distribution or compliance with end-user requests or end-user device requirements by altering the spatial resolution, e.g. for clients with a lower screen resolution
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Databases & Information Systems (AREA)
- Business, Economics & Management (AREA)
- Marketing (AREA)
- Computer Networks & Wireless Communication (AREA)
- Studio Devices (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The live camera system of multiresolution panoramic video of the present invention is related to the information processing technology with method, the device includes image capture module, streaming media server module and user terminal module, this method includes gathering view data by image capture module, and is transferred to streaming media server module；The view data of streaming media server module collection is spliced in real time, and is stored, and is then compressed and is encoded into video flowing, and video flowing is entered into row format encapsulation according to transmission of video agreement, and user terminal module is transferred to after storage；Video is watched in user terminal module, photographing instruction is initiated, by streaming media server resume module, user's preview in display sub-module is checked, if user is satisfied with, pays to click on and downloads, preserve to the local photograph album of user equipment；If user is unsatisfied with, it need not pay, during return is live.Improve the sense of reality of user, realize build virtual reality required for image information and pass it to client and show function.
Description
Technical field
The live camera system of multiresolution panoramic video of the present invention belongs to field of information processing with method.
Background technology
In this day and age, net cast is increasingly welcome by masses, makes a general survey of the development history of net cast, is by video point
Developed on the basis of broadcasting, carried out by internet and stream media technology it is live, so as to it is real-time, comprehensively propagate video
Content.Net cast is built strong presence, attracted eyeball by true, lively propagation, reaches with deep impression, note
Recall lasting communication effect, can truly, intuitively, comprehensively publicize, oneself complete steaming media platform be shown, in early stage
Video playback when, the viewing area of video content is because the reason such as shooting, such as：The angle coverage of camera lens, scope, cause
Its fixation is non-adjustable, and user can only watch current region, and video pictures can only follow camera lens to move and change, user's viewing
Without third dimension, there can not be impression on the spot in person.
With the development of panoramic video technology, occur in that 360 degree of panoramic videos, can allow user comprehensive 360 degree from
, can front and rear up and down, even far and near distance viewing by watching.Typically by multiple video cameras simultaneously to same space
Under object multiple angle shots are carried out to realize, allow user in the case where video playback need not be suspended playing end, from
By switching viewing angle；Panoramic video has third dimension and the sense of reality, and visual experience can be provided well；It can realize that body faces
The experience in its border and the surprised effect more made one see with different eyes.
With the continuous improvement of people's demand, image is recorded as recording for a kind of new souvenir of people, so it is desirable to
True, comprehensive, intuitively video is not only can see in watching video live broadcast, can also obtain in real time high quality graphic with
Retain or share with friend.So how fast and effeciently to build the image information required for virtual reality and pass it to visitor
Family end is simultaneously shown, improves the sense of reality and comfort of Consumer's Experience, will turn into a urgent problem to be solved.
The content of the invention
The invention provides a kind of live camera system of multiresolution panoramic video and method, virtual reality is effectively built
Required image information simultaneously passes it to client and shown.
The object of the present invention is achieved like this：
A kind of live camera system of multiresolution panoramic video, including image capture module, streaming media server module and
User terminal module, described image acquisition module is wirelessly transferred submodule including image capture device and first；The Streaming Media
Server module includes processing submodule, sub-module stored and second and is wirelessly transferred submodule；Described processing submodule includes
Real-time concatenation unit, video encoding unit and core processing unit；The sub-module stored includes storing sub-units a, storage
Unit b and storing sub-units c；User terminal module includes at least one user equipment, and each user equipment includes display submodule
Block, user interaction sub module and the 3rd are wirelessly transferred submodule.
The described live camera system of multiresolution panoramic video, it is described：
Image capture device：For gathering view data；
First is wirelessly transferred submodule：For the view data of collection to be transferred into streaming media server module；
Real-time concatenation unit：View data for splicing collection in real time；
Video encoding unit：Video flowing is encoded into for spliced panoramic image data to be compressed, and by video
Stream enters row format encapsulation according to transmission of video agreement and distributed；
Core processing unit：For handling the instruction that user terminal module request is taken pictures；
Storing sub-units：For storing panoramic image data；
Storing sub-units：For storing index file and video flowing；
Storing sub-units：For storing the image that user terminal module request is taken pictures；
Second is wirelessly transferred submodule：For streaming media server module and image capture module, user terminal module it
Between communication；
Display sub-module：For showing panoramic picture or video flowing；
User interaction sub module：For send viewing video, take pictures, preview, paying and download request；
3rd is wirelessly transferred submodule：For the communication between user terminal module and streaming media server module.
The described live camera system of multiresolution panoramic video, described streaming media server module also includes user and managed
Module is managed, is used for receiving the ID authentication request from each user equipment, and based on the ID authentication request corresponding
Family equipment carries out authentication, sends authentication success message to set with the user to the corresponding user equipment after the authentication has been successful
It is standby to set up data cube computation.
The described live camera system of multiresolution panoramic video, described user equipment also includes User Access Module,
For to streaming media server module send ID authentication request, and receive the certification from streaming media server module into
After work(information data cube computation is set up with streaming media server module.
A kind of multiresolution panoramic video realized on the live camera system of multiresolution panoramic video is live to be clapped
According to method, comprise the following steps：
Step a, by image capture device gather view data, first is wirelessly transferred submodule by the view data of collection
It is transferred to streaming media server module；
Step b, second are wirelessly transferred the view data that submodule receives collection, and real-time concatenation unit is by the image of collection
Data are spliced in real time, and spliced panoramic image data is stored to storing sub-units a；
Step c, core processing unit can generate panoramic video data directory file according to timestamp information, and set up index
The mapping relations of file and original panoramic images data, and index file is preserved to storing sub-units b, video according to mapping relations
Spliced panoramic image data is compressed and is encoded into video flowing by coding unit, and by video flowing according to transmission of video agreement
Enter row format encapsulation, store to storing sub-units b, while being wirelessly transferred submodule by second is transferred to user terminal module；
Step d, the 3rd are wirelessly transferred submodule and receive video flowing, and viewing instruction is initiated in user interaction sub module,
Play, initiated in user interaction sub module in request photographing instruction, request photographing instruction comprising real-time in display sub-module
The video requency frame data and the attitude information of user equipment taken pictures, video requency frame data include the mark of image sequence and the position of timestamp
Put, the 3rd, which is wirelessly transferred submodule, is transferred to second request photographing instruction and is wirelessly transferred submodule；
Step e, second are wirelessly transferred submodule and receive request photographing instruction, and core processing unit first can be according to taking pictures
Original panoramic images in the video requency frame data and existing mapping relations index search storing sub-units a that are sent in instruction, so
Intercept the high-resolution of correspondence attitude information with a kind of method in original panoramic images according to the attitude information of user equipment afterwards
Picture, sectional drawing is preserved to storing sub-units c；
Step f, the user preview in display sub-module are checked, if user is satisfied with, and are clicked on and downloaded by paying, preserved extremely
The local photograph album of user equipment；If user is unsatisfied with, it need not pay, during return is live.
A kind of described live photographic method of multiresolution panoramic video, the height of interception correspondence attitude information described in step e
Resolution chart, comprises the following steps：
Step e1, by original panoramic images it is both that planar rectangular picture is remapped on a sphere, with planar rectangular
The side of picture generates corresponding panorama as sphere semi arch and maps sphere, and planar rectangular picture is mapped into panorama mapping
On sphere；
Step e2, the angular range for determining according to the attitude information of user equipment the user visual field；According to the angle in the user visual field
Degree scope searches spherical figure to be cut on panorama mapping sphere；
Step e3, spherical figure will have been cut it is mapped to target rectangle picture；The size of target rectangle picture is preset value,
The preset value is set based on experience value, or the screen size parameter included in the photographing instruction sent according to user equipment come
Setting.
Beneficial effect：
The live camera system of multiresolution panoramic video of the present invention, view data, transmission are obtained by image capture module
Streaming media server module is given, streaming media server module is spliced view data, is then compressed coding, is compressed into
The video flowing of different resolution, and the video flowing of compression is entered into row format according to transmission of video agreement encapsulate and distribute and deposit
Storage, when the user equipment of user terminal module be sent to streaming media server module take pictures request when, streaming media server module
Handled, then the image transmitting that request is taken pictures is shown to user equipment in user equipment.User can use the system real
When watch different resolution panoramic video, with meet different user and meanwhile watch panoramic video fluency；User is using
When the systematic perspective sees low resolution panoramic video, realized by sending the photographing instruction comprising video requency frame data and attitude information
High-definition picture is obtained in real time, the sense of reality of user VR experience is improved, and realizes the image built required for virtual reality
Information simultaneously passes it to client and the function of showing.
Brief description of the drawings
Fig. 1 is the live camera system structure chart of multiresolution panoramic video.
In figure：1 image capture module, 11 image capture devices, 12 first are wirelessly transferred submodule, 2 streaming media servers
Module, 21 processing submodules, 211 real-time concatenation units, 212 video encoding units, 213 core processing units, 22 storage submodules
Block, 221 storing sub-units a, 222 storing sub-units b, 223 storing sub-units c, 23 second are wirelessly transferred submodule, 3 users end
End module, 31 user equipmenies, 311 display sub-modules, 312 user interaction sub modules, 313 the 3rd are wirelessly transferred submodule.
Embodiment
The specific embodiment of the invention is described in further detail below in conjunction with the accompanying drawings.
Specific embodiment one
A kind of live camera system of multiresolution panoramic video, as shown in figure 1, being taken including image capture module 1, Streaming Media
Business device module 2 and user terminal module 3, described image acquisition module 1 are wirelessly transferred son including image capture device 11 and first
Module 12；The streaming media server module 2 includes processing submodule 21, sub-module stored 22 and second and is wirelessly transferred submodule
23；Described processing submodule 21 includes real-time concatenation unit 211, video encoding unit 212 and core processing unit 213；Institute
Stating sub-module stored 22 includes storing sub-units a221, storing sub-units b222 and storing sub-units c223；User terminal module 3
Including at least one user equipment, each user equipment includes display sub-module 311, the nothing of user interaction sub module 312 and the 3rd
Line transmits submodule 313.
The described live camera system of multiresolution panoramic video, it is described：
Image capture device 11：For gathering view data；
First is wirelessly transferred submodule 12：For the view data of collection to be transferred into streaming media server module 2；
Real-time concatenation unit 211：View data for splicing collection in real time；
Video encoding unit 212：Video flowing is encoded into for spliced panoramic image data to be compressed, and will be regarded
Frequency stream enters row format encapsulation according to transmission of video agreement and distributed；
Core processing unit 213：The instruction taken pictures for handling user terminal module 3 to ask；
Storing sub-units a221：For storing panoramic image data；
Storing sub-units b222：For storing index file and video flowing；
Storing sub-units c223：The image taken pictures for storing user terminal module 3 to ask；
Second is wirelessly transferred submodule 23：For streaming media server module 2 and image capture module 1, user terminal mould
Communication between block 3；
Display sub-module 311：For showing panoramic picture or video flowing；
User interaction sub module 312：For send viewing video, take pictures, preview, paying and download request；
3rd is wirelessly transferred submodule 313：For the communication between user terminal module 3 and streaming media server module 2.
The described live camera system of multiresolution panoramic video, described streaming media server module 2 also includes user
Management module, for receiving the ID authentication request from each user equipment, and based on the ID authentication request to corresponding
User equipment carries out authentication, after the authentication has been successful to the corresponding user equipment send authentication success message with the user
Equipment sets up data cube computation.
The described live camera system of multiresolution panoramic video, described user equipment also includes User Access Module,
For sending ID authentication request to streaming media server module 2, and receiving the certification from streaming media server module 2
After successful information data cube computation is set up with streaming media server module 2.
The course of work：Image capture module 1 is wirelessly transferred submodule 12 comprising image capture device 11 and first, for adopting
Collect panoramic image data in its visual field, and the panoramic image data collected is wirelessly transferred submodule 12 by first and be transferred to
Streaming media server module 2.
Streaming media server module 2 is wirelessly transferred submodule 23 comprising processing submodule 21, sub-module stored 22 and second；
Wherein processing submodule 21 includes real-time concatenation unit 211, video encoding unit 212 and core processing unit 213；Store submodule
Block 22 includes storing sub-units a221, storing sub-units b222 and storing sub-units c223.Second, which is wirelessly transferred submodule 23, uses
In the communication of streaming media server module 2, image capture module 1 and user terminal module 3.
Wherein, real-time concatenation unit 211 is used to splice the panoramic image data that collects in real time, and by spliced panorama
View data is stored in storing sub-units a221；Video encoding unit 212 is used to carry out spliced panoramic image data
Compressed encoding, is compressed into smoothness, SD, high definition, the video flowing of super clear different resolution, and by the video flowing of compression according to video
Host-host protocol enters row format and encapsulates and distribute；Core processing unit 213 is used to handle the finger that user terminal module 3 asks to take pictures
Order；Storing sub-units b222 is used to store index file and panoramic video data；Storing sub-units c223 is used to store user's end
End module 3 asks the image taken pictures.
User terminal module 3 includes one or more user equipmenies, and each user equipment includes display sub-module 311, use
Family interaction submodule 312 and the 3rd is wirelessly transferred submodule 313.Display sub-module 311 is used to show panoramic video/image；With
Family interaction submodule 312 be used to sending viewing video, take pictures, preview, paying, downloads etc. are asked；3rd is wirelessly transferred submodule
313 are used for the communication of user terminal module 3 and streaming media server module 2.Wherein, user equipment can be smart mobile phone, put down
The terminal device such as plate computer or wear-type VR systems.
Streaming media server module 2 also includes user management module, and each user in one or more user equipmenies
Equipment also includes User Access Module.Wherein, user management module please for receiving the authentication from each user equipment
Ask, and authentication is carried out to corresponding user equipment based on the ID authentication request, after the authentication has been successful to the corresponding use
Family equipment sends authentication success message to set up data cube computation with the user equipment；The User Access Module of each user equipment is used
In to streaming media server module 2 send ID authentication request, and receive the certification from streaming media server module 2 into
After work(information data cube computation is set up with streaming media server module 2.
The panoramic image data that streaming media server module 2 is received initially enters real-time concatenation unit 211 and carries out panorama
Splicing, and spliced panoramic image data is stored in storing sub-units a221, while will by video encoding unit 212
Spliced panoramic image data is compressed coding, is compressed into smoothness, SD, high definition, the video flowing of super clear different resolution,
And the video flowing of compression is entered into row format encapsulation according to transmission of video agreement and distributed；User can be by right on distinct device
In the video data that the player real-time decoding streaming media server module distribution answered comes, selection user interaction sub module 312
The live key of viewing carry out real-time panoramic video viewing, user can select smoothness, SD, high definition, super clear different resolution
Watching mode.
User can select the camera button in user interaction sub module 312 to be clapped in real time when viewing panoramic video is live
According to, to obtain high-definition picture, the display in display sub-module 311.
Specific embodiment two
A kind of multiresolution panoramic video realized on the live camera system of multiresolution panoramic video is live to be clapped
According to method, it is characterised in that comprise the following steps：
Step a, by image capture device 11 gather view data, first is wirelessly transferred submodule 12 by the image of collection
Data are transferred to streaming media server module 2；
Step b, second are wirelessly transferred the view data that submodule 23 receives collection, and real-time concatenation unit 211 will be gathered
View data splice in real time, and spliced panoramic image data is stored to storing sub-units a221；
Step c, core processing unit 213 can generate panoramic video data directory file according to timestamp information, and set up
The mapping relations of index file and original panoramic images data, and index file is preserved to storing sub-units according to mapping relations
Spliced panoramic image data is compressed and is encoded into video flowing by b222, video encoding unit 212, and by video flowing according to
Transmission of video agreement enters row format encapsulation, stores to storing sub-units b222, is passed while being wirelessly transferred submodule 23 by second
It is defeated by user terminal module 3；
Step d, the 3rd are wirelessly transferred submodule 313 and receive video flowing, and viewing is initiated in user interaction sub module 312
Instruction, is played in display sub-module 311, and request photographing instruction is initiated in user interaction sub module 312, photographing instruction is asked
In the attitude information comprising the video requency frame data taken pictures in real time and user equipment, video requency frame data include image sequence mark and
The position of timestamp, the 3rd, which is wirelessly transferred submodule 313, is transferred to second request photographing instruction and is wirelessly transferred submodule 23；
Step e, second are wirelessly transferred submodule 23 and receive request photographing instruction, and core processing unit 213 first can root
According to original complete in the video requency frame data and existing mapping relations index search storing sub-units a221 sent in photographing instruction
Scape image, then intercepts correspondence attitude information in original panoramic images according to the attitude information of user equipment with a kind of method
High-resolution pictures, sectional drawing is preserved to storing sub-units c223；
Step f, the user preview in display sub-module 311 are checked, if user is satisfied with, and are clicked on and downloaded by paying, protected
Deposit to the local photograph album of user equipment；If user is unsatisfied with, it need not pay, during return is live.
A kind of described live photographic method of multiresolution panoramic video, the height of interception correspondence attitude information described in step e
Resolution chart, comprises the following steps：
Step e1, by original panoramic images it is both that planar rectangular picture is remapped on a sphere, with planar rectangular
The side of picture generates corresponding panorama as sphere semi arch and maps sphere, and planar rectangular picture is mapped into panorama mapping
On sphere；
Step e2, the angular range for determining according to the attitude information of user equipment the user visual field；According to the angle in the user visual field
Degree scope searches spherical figure to be cut on panorama mapping sphere；
Step e3, spherical figure will have been cut it is mapped to target rectangle picture；The size of target rectangle picture is preset value,
The preset value is set based on experience value, or the screen size parameter included in the photographing instruction sent according to user equipment come
Setting.
Image capture module 1 is wirelessly transferred submodule 12 comprising image capture device 11 and first, is set by IMAQ
Standby 11 gather panoramic image data in its visual field, and first is wirelessly transferred submodule 12 is transferred to stream by the panoramic image data of collection
Media server module 2；
Streaming media server module 2 is wirelessly transferred submodule 23 comprising processing submodule 21, sub-module stored 22 and second；
Wherein processing submodule 21 includes real-time concatenation unit 211, video encoding unit 212 and core processing unit 213；Store submodule
Block 22 includes storing sub-units a221, storing sub-units b222 and storing sub-units c223.Second, which is wirelessly transferred submodule 23, connects
The view data of collection is received, real-time concatenation unit 211 splices the view data of collection in real time, and by spliced panorama sketch
As data storage to storing sub-units a221；Core processing unit 213 can generate panoramic video data rope according to timestamp information
Quotation part, and the mapping relations of index file and original panoramic images data are set up, and preserve index file according to mapping relations
To storing sub-units b222, spliced panoramic image data is compressed and is encoded into video flowing by video encoding unit 212, and
Video flowing is entered into row format encapsulation according to transmission of video agreement, stored to storing sub-units b222, while passing through the second wireless biography
Defeated submodule 23 is transferred to user terminal module 3；
Streaming media server module 2 also includes user management module, and each user in one or more user equipmenies
Equipment also includes User Access Module.Wherein, user management module please for receiving the authentication from each user equipment
Ask, and authentication is carried out to corresponding user equipment based on the ID authentication request, after the authentication has been successful to the corresponding use
Family equipment sends authentication success message to set up data cube computation with the user equipment；The User Access Module of each user equipment is used
In to streaming media server module 2 send ID authentication request, and receive the certification from streaming media server module 2 into
After work(information data cube computation is set up with streaming media server module 2.
User terminal module 3 includes one or more user equipmenies, and each user equipment includes display sub-module 311, use
Family interaction submodule 312 and the 3rd is wirelessly transferred submodule 313.Display sub-module 311 is used to show panoramic video/image；With
Family interaction submodule 312 be used to sending viewing video, take pictures, preview, paying, downloads etc. are asked；3rd is wirelessly transferred submodule
313 are used for the communication of user terminal module 3 and streaming media server module 2.Wherein, user equipment can be smart mobile phone, put down
The terminal device such as plate computer or wear-type VR systems.
3rd, which is wirelessly transferred submodule 313, receives video flowing, and viewing instruction is initiated in user interaction sub module 312,
Played in display sub-module 311, user can pass through corresponding player real-time decoding streaming media server on distinct device
The live key of viewing in the video data that module distribution comes, selection user interaction sub module 312 carries out real-time panoramic video
Viewing, user can select smoothness, SD, high definition, the watching mode of super clear different resolution.User is straight in viewing panoramic video
Sowing time can select the camera button in user interaction sub module 312 to be taken pictures in real time, to obtain high-definition picture.User is whole
The video requency frame data taken pictures in real time and the attitude information of user equipment are included in the photographing instruction that end module 3 is sent；Video frame number
According to the mark including image sequence and the position of timestamp；Attitude information can be set by user by way of contact action
Put, the work(of user interaction sub module 312 and display sub-module 311 can be realized simultaneously by touch screen module in this case
Energy；The posture such as integrated gyroscope submodule can also be sensed in user interaction sub module 312, user equipment is obtained with this
Current pose information.3rd, which is wirelessly transferred submodule 313, is transferred to second request photographing instruction and is wirelessly transferred submodule 23.
Second, which is wirelessly transferred submodule 23, receives request photographing instruction, and core processing unit 213 first can be according to taking pictures
Original panoramic images in the video requency frame data and existing mapping relations index search storing sub-units a221 that are sent in instruction,
Then the high-resolution of correspondence attitude information is intercepted with a kind of method in original panoramic images according to the attitude information of user equipment
Rate picture, sectional drawing is preserved to storing sub-units c223；User's preview in display sub-module 311 is checked, if user is satisfied with,
Clicked on and downloaded by paying, preserved to the local photograph album of user equipment；It if user is unsatisfied with, need not pay, return straight
In broadcasting.
The method that the high-resolution pictures of correspondence attitude information are intercepted described in core processing unit 213 is as follows：
The original panoramic images (planar rectangular picture) are remapped on a sphere, i.e., with planar rectangular picture
Side generates corresponding panorama as sphere semi arch and maps sphere, and planar rectangular picture is mapped into panorama mapping sphere
On；The angular range in the user visual field is determined according to the attitude information of user equipment；According to the angular range in the user visual field in panorama
Spherical figure to be cut is searched on mapping sphere；This has been cut into spherical figure and has been mapped to target rectangle picture；The target rectangle
The size of picture is preset value, and the preset value can set based on experience value, or taking pictures of being sent according to user equipment
The screen size parameter that is included in instruction is set.The high-resolution target rectangle picture is finally saved in storing sub-units
Under the corresponding account of the user of this in c223, for the preview and download that user is final.
Claims (6)
1. a kind of live camera system of multiresolution panoramic video, it is characterised in that including image capture module (1), Streaming Media
Server module (2) and user terminal module (3), described image acquisition module (1) include image capture device (11) and first
It is wirelessly transferred submodule (12)；The streaming media server module (2) include processing submodule (21), sub-module stored (22) and
Second is wirelessly transferred submodule (23)；Described processing submodule (21) includes real-time concatenation unit (211), video encoding unit
And core processing unit (213) (212)；The sub-module stored (22) includes storing sub-units a (221), storing sub-units b
And storing sub-units c (223) (222)；User terminal module (3) includes at least one user equipment, and each user equipment includes
Display sub-module (311), user interaction sub module (312) and the 3rd are wirelessly transferred submodule (313).
2. the live camera system of multiresolution panoramic video according to claim 1, it is characterised in that described：
Image capture device (11)：For gathering view data；
First is wirelessly transferred submodule (12)：For the view data of collection to be transferred into streaming media server module (2)；
Real-time concatenation unit (211)：View data for splicing collection in real time；
Video encoding unit (212)：Video flowing is encoded into for spliced panoramic image data to be compressed, and by video
Stream enters row format encapsulation according to transmission of video agreement and distributed；
Core processing unit (213)：For handling the instruction that user terminal module (3) request is taken pictures；
Storing sub-units a (221)：For storing panoramic image data；
Storing sub-units b (222)：For storing index file and video flowing；
Storing sub-units c (223)：For storing the image that user terminal module (3) request is taken pictures；
Second is wirelessly transferred submodule (23)：For streaming media server module (2) and image capture module (1), user terminal
Communication between module (3)；
Display sub-module (311)：For showing panoramic picture or video flowing；
User interaction sub module (312)：For send viewing video, take pictures, preview, paying and download request；
3rd is wirelessly transferred submodule (313)：For logical between user terminal module (3) and streaming media server module (2)
Letter.
3. the live camera system of multiresolution panoramic video according to claim 1, it is characterised in that described Streaming Media
Server module (2) also includes user management module, for receiving the ID authentication request from each user equipment, and is based on
The ID authentication request carries out authentication to corresponding user equipment, is sent after the authentication has been successful to the corresponding user equipment
Authentication success message with the user equipment to set up data cube computation.
4. the live camera system of multiresolution panoramic video according to claim 1, it is characterised in that described user sets
It is standby also to include User Access Module, for sending ID authentication request to streaming media server module (2), and come from receiving
After the authentication success message of streaming media server module (2) data cube computation is set up with streaming media server module (2).
5. it is a kind of in the multiresolution realized described in claim 1,2,3 or 4 on the live camera system of multiresolution panoramic video
The live photographic method of panoramic video, it is characterised in that comprise the following steps：
Step a, by image capture device (11) gather view data, first is wirelessly transferred submodule (12) by the image of collection
Data are transferred to streaming media server module (2)；
Step b, second are wirelessly transferred the view data that submodule (23) receives collection, and real-time concatenation unit (211) will gather
View data splice in real time, and spliced panoramic image data is stored to storing sub-units a (221)；
Step c, core processing unit (213) can generate panoramic video data directory file according to timestamp information, and set up rope
The mapping relations of quotation part and original panoramic images data, and index file is preserved to storing sub-units b according to mapping relations
(222), spliced panoramic image data is compressed by video encoding unit (212) is encoded into video flowing, and by video flowing
Enter row format encapsulation according to transmission of video agreement, store to storing sub-units b (222), while being wirelessly transferred submodule by second
Block (23) is transferred to user terminal module (3)；
Step d, the 3rd are wirelessly transferred submodule (313) and receive video flowing, and viewing is initiated in user interaction sub module (312)
Instruction, is played in display sub-module (311), and request photographing instruction is initiated in user interaction sub module (312), and request is taken pictures
Comprising the video requency frame data taken pictures in real time and the attitude information of user equipment in instruction, video requency frame data includes the mark of image sequence
Know and timestamp position, the 3rd, which is wirelessly transferred submodule (313), is transferred to second request photographing instruction and is wirelessly transferred submodule
Block (23)；
Step e, second are wirelessly transferred submodule (23) and receive request photographing instruction, and core processing unit (213) first can root
According to original in the video requency frame data and existing mapping relations index search storing sub-units a (221) sent in photographing instruction
Panoramic picture, then intercepts correspondence attitude information in original panoramic images according to the attitude information of user equipment with a kind of method
High-resolution pictures, sectional drawing is preserved to storing sub-units c (223)；
Step f, the user preview in display sub-module (311) are checked, if user is satisfied with, and are clicked on and downloaded by paying, preserved
To the local photograph album of user equipment；If user is unsatisfied with, it need not pay, during return is live.
6. a kind of live photographic method of multiresolution panoramic video according to claim 5, it is characterised in that step e institutes
The high-resolution pictures of interception correspondence attitude information are stated, are comprised the following steps：
Step e1, by original panoramic images it is both that planar rectangular picture is remapped on a sphere, with planar rectangular picture
Side generate corresponding panorama as sphere semi arch and map sphere, and planar rectangular picture is mapped to panorama mapping sphere
On；
Step e2, the angular range for determining according to the attitude information of user equipment the user visual field；According to the angle model in the user visual field
It is trapped among on panorama mapping sphere and searches spherical figure to be cut；
Step e3, spherical figure will have been cut it is mapped to target rectangle picture；The size of target rectangle picture is preset value, described
Preset value is set based on experience value, or is set according to the screen size parameter included in the photographing instruction that user equipment is sent
It is fixed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710656265.0A CN107205122A (en) | 2017-08-03 | 2017-08-03 | The live camera system of multiresolution panoramic video and method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710656265.0A CN107205122A (en) | 2017-08-03 | 2017-08-03 | The live camera system of multiresolution panoramic video and method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107205122A true CN107205122A (en) | 2017-09-26 |
Family
ID=59911679
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710656265.0A Pending CN107205122A (en) | 2017-08-03 | 2017-08-03 | The live camera system of multiresolution panoramic video and method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107205122A (en) |
Cited By (18)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107592549A (en) * | 2017-10-27 | 2018-01-16 | 哈尔滨市舍科技有限公司 | Panoramic video based on two-way communication plays camera system |
| CN107592548A (en) * | 2017-09-29 | 2018-01-16 | 哈尔滨市舍科技有限公司 | A kind of live image collecting device and image transfer method taken pictures of real time panoramic |
| CN107659824A (en) * | 2017-10-20 | 2018-02-02 | 哈尔滨市舍科技有限公司 | Virtual player based on user perspective |
| CN107707830A (en) * | 2017-10-27 | 2018-02-16 | 哈尔滨市舍科技有限公司 | Panoramic video based on one-way communication plays camera system |
| CN108495144A (en) * | 2018-03-30 | 2018-09-04 | 哈尔滨市舍科技有限公司 | A method of obtaining picture |
| CN108769739A (en) * | 2018-06-27 | 2018-11-06 | 哈尔滨市舍科技有限公司 | The method that picture and short-sighted frequency are obtained from panoramic video |
| CN108924590A (en) * | 2018-06-27 | 2018-11-30 | 哈尔滨市舍科技有限公司 | Video playing and camera system |
| CN110213541A (en) * | 2019-06-05 | 2019-09-06 | 福建网能科技开发有限责任公司 | Ammeter, mutual inductor image collecting method and system based on Streaming Media |
| CN110267026A (en) * | 2019-07-05 | 2019-09-20 | 浙江开奇科技有限公司 | Data processing method, mobile terminal and system based on 5G network |
| CN110278410A (en) * | 2019-05-20 | 2019-09-24 | 上海澳马信息技术服务有限公司 | A kind of more mesh panoramic video joining methods and system |
| CN110602523A (en) * | 2019-10-12 | 2019-12-20 | 重庆灏漫科技有限公司 | VR panoramic live multimedia processing and synthesizing system and method |
| WO2020001610A1 (en) * | 2018-06-29 | 2020-01-02 | 华为技术有限公司 | Method and device for joining video |
| CN111083368A (en) * | 2019-12-26 | 2020-04-28 | 南京感动科技有限公司 | Simulation physics cloud platform panoramic video display system based on high in clouds |
| CN112351307A (en) * | 2020-10-26 | 2021-02-09 | 咪咕视讯科技有限公司 | Screenshot method, server, terminal equipment and computer readable storage medium |
| CN112565606A (en) * | 2020-12-02 | 2021-03-26 | 鹏城实验室 | Panoramic video intelligent transmission method and equipment and computer storage medium |
| CN113014888A (en) * | 2021-03-17 | 2021-06-22 | 广州穿越千机创新科技有限公司 | Unmanned aerial vehicle performance viewing method, device and system based on VR technology |
| CN113810755A (en) * | 2021-09-15 | 2021-12-17 | 北京百度网讯科技有限公司 | Panoramic video preview method and device, electronic equipment and storage medium |
| CN115118883A (en) * | 2022-06-28 | 2022-09-27 | 润博全景文旅科技有限公司 | Image preview method, device and equipment |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060215930A1 (en) * | 2005-03-25 | 2006-09-28 | Fujitsu Limited | Panorama image generation program, panorama image generation apparatus, and panorama image generation method |
| CN201440655U (en) * | 2009-07-14 | 2010-04-21 | 索尼株式会社 | Multimedia information downloader employing UWB wireless transmission technology |
| CN103763479A (en) * | 2013-12-31 | 2014-04-30 | 深圳英飞拓科技股份有限公司 | Splicing device for real-time high speed high definition panoramic video and method thereof |
| CN103973944A (en) * | 2013-02-06 | 2014-08-06 | 深圳市振华微电子有限公司 | Hemispherical omnidirectional imaging device and hemispherical omnidirectional imaging method |
| CN106231317A (en) * | 2016-09-29 | 2016-12-14 | 三星电子（中国）研发中心 | Video processing, coding/decoding method and device, VR terminal, audio/video player system |
| CN106331764A (en) * | 2016-09-14 | 2017-01-11 | 乐视控股（北京）有限公司 | Panoramic video sharing method and panoramic video sharing device |
| CN106341667A (en) * | 2016-11-10 | 2017-01-18 | 广西师范大学 | UAV (Unmanned Aerial Vehicle) based 3D panorama video remote monitoring system and image acquisition control method thereof |
| CN106550182A (en) * | 2017-01-10 | 2017-03-29 | 哈尔滨市舍科技有限公司 | Shared unmanned plane viewing system |
| CN106791623A (en) * | 2016-12-09 | 2017-05-31 | 深圳市云宙多媒体技术有限公司 | A kind of panoramic video joining method and device |
| CN106803874A (en) * | 2017-03-13 | 2017-06-06 | 杭州昇星网络科技有限公司 | A kind of full-view camera |
| CN106888351A (en) * | 2015-12-16 | 2017-06-23 | 上海酷景信息技术有限公司 | A kind of panoramic video recording system |
| CN106992959A (en) * | 2016-11-01 | 2017-07-28 | 深圳市圆周率软件科技有限责任公司 | A kind of 3D panoramas audio frequency and video live broadcast system and audio/video acquisition method |
-
2017
- 2017-08-03 CN CN201710656265.0A patent/CN107205122A/en active Pending
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20060215930A1 (en) * | 2005-03-25 | 2006-09-28 | Fujitsu Limited | Panorama image generation program, panorama image generation apparatus, and panorama image generation method |
| CN201440655U (en) * | 2009-07-14 | 2010-04-21 | 索尼株式会社 | Multimedia information downloader employing UWB wireless transmission technology |
| CN103973944A (en) * | 2013-02-06 | 2014-08-06 | 深圳市振华微电子有限公司 | Hemispherical omnidirectional imaging device and hemispherical omnidirectional imaging method |
| CN103763479A (en) * | 2013-12-31 | 2014-04-30 | 深圳英飞拓科技股份有限公司 | Splicing device for real-time high speed high definition panoramic video and method thereof |
| CN106888351A (en) * | 2015-12-16 | 2017-06-23 | 上海酷景信息技术有限公司 | A kind of panoramic video recording system |
| CN106331764A (en) * | 2016-09-14 | 2017-01-11 | 乐视控股（北京）有限公司 | Panoramic video sharing method and panoramic video sharing device |
| CN106231317A (en) * | 2016-09-29 | 2016-12-14 | 三星电子（中国）研发中心 | Video processing, coding/decoding method and device, VR terminal, audio/video player system |
| CN106992959A (en) * | 2016-11-01 | 2017-07-28 | 深圳市圆周率软件科技有限责任公司 | A kind of 3D panoramas audio frequency and video live broadcast system and audio/video acquisition method |
| CN106341667A (en) * | 2016-11-10 | 2017-01-18 | 广西师范大学 | UAV (Unmanned Aerial Vehicle) based 3D panorama video remote monitoring system and image acquisition control method thereof |
| CN106791623A (en) * | 2016-12-09 | 2017-05-31 | 深圳市云宙多媒体技术有限公司 | A kind of panoramic video joining method and device |
| CN106550182A (en) * | 2017-01-10 | 2017-03-29 | 哈尔滨市舍科技有限公司 | Shared unmanned plane viewing system |
| CN106803874A (en) * | 2017-03-13 | 2017-06-06 | 杭州昇星网络科技有限公司 | A kind of full-view camera |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107592548B (en) * | 2017-09-29 | 2020-05-12 | 哈尔滨市一舍科技有限公司 | Image acquisition device and image transmission method for real-time panoramic live broadcast shooting |
| CN107592548A (en) * | 2017-09-29 | 2018-01-16 | 哈尔滨市舍科技有限公司 | A kind of live image collecting device and image transfer method taken pictures of real time panoramic |
| CN107659824A (en) * | 2017-10-20 | 2018-02-02 | 哈尔滨市舍科技有限公司 | Virtual player based on user perspective |
| CN107707830A (en) * | 2017-10-27 | 2018-02-16 | 哈尔滨市舍科技有限公司 | Panoramic video based on one-way communication plays camera system |
| CN107592549A (en) * | 2017-10-27 | 2018-01-16 | 哈尔滨市舍科技有限公司 | Panoramic video based on two-way communication plays camera system |
| CN107707830B (en) * | 2017-10-27 | 2020-06-16 | 青岛一舍科技有限公司 | Panoramic video playing and photographing system based on one-way communication |
| CN107592549B (en) * | 2017-10-27 | 2020-06-09 | 哈尔滨市一舍科技有限公司 | Panoramic video playing and photographing system based on two-way communication |
| CN108495144A (en) * | 2018-03-30 | 2018-09-04 | 哈尔滨市舍科技有限公司 | A method of obtaining picture |
| CN108769739A (en) * | 2018-06-27 | 2018-11-06 | 哈尔滨市舍科技有限公司 | The method that picture and short-sighted frequency are obtained from panoramic video |
| CN108924590B (en) * | 2018-06-27 | 2021-09-03 | 青岛一舍科技有限公司 | Video playing and photographing system |
| CN108924590A (en) * | 2018-06-27 | 2018-11-30 | 哈尔滨市舍科技有限公司 | Video playing and camera system |
| WO2020001610A1 (en) * | 2018-06-29 | 2020-01-02 | 华为技术有限公司 | Method and device for joining video |
| CN110278410A (en) * | 2019-05-20 | 2019-09-24 | 上海澳马信息技术服务有限公司 | A kind of more mesh panoramic video joining methods and system |
| CN110213541A (en) * | 2019-06-05 | 2019-09-06 | 福建网能科技开发有限责任公司 | Ammeter, mutual inductor image collecting method and system based on Streaming Media |
| CN110267026A (en) * | 2019-07-05 | 2019-09-20 | 浙江开奇科技有限公司 | Data processing method, mobile terminal and system based on 5G network |
| CN110602523A (en) * | 2019-10-12 | 2019-12-20 | 重庆灏漫科技有限公司 | VR panoramic live multimedia processing and synthesizing system and method |
| CN111083368A (en) * | 2019-12-26 | 2020-04-28 | 南京感动科技有限公司 | Simulation physics cloud platform panoramic video display system based on high in clouds |
| CN112351307A (en) * | 2020-10-26 | 2021-02-09 | 咪咕视讯科技有限公司 | Screenshot method, server, terminal equipment and computer readable storage medium |
| CN112565606A (en) * | 2020-12-02 | 2021-03-26 | 鹏城实验室 | Panoramic video intelligent transmission method and equipment and computer storage medium |
| CN113014888A (en) * | 2021-03-17 | 2021-06-22 | 广州穿越千机创新科技有限公司 | Unmanned aerial vehicle performance viewing method, device and system based on VR technology |
| CN113810755A (en) * | 2021-09-15 | 2021-12-17 | 北京百度网讯科技有限公司 | Panoramic video preview method and device, electronic equipment and storage medium |
| CN115118883A (en) * | 2022-06-28 | 2022-09-27 | 润博全景文旅科技有限公司 | Image preview method, device and equipment |
| CN115118883B (en) * | 2022-06-28 | 2024-02-02 | 润博全景文旅科技有限公司 | Image preview method, device and equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107205122A (en) | The live camera system of multiresolution panoramic video and method | |
| Fan et al. | A survey on 360 video streaming: Acquisition, transmission, and display | |
| CN107040794A (en) | Video broadcasting method, server, virtual reality device and panoramic virtual reality play system | |
| US20210097667A1 (en) | Video quality measurement for virtual cameras in volumetric immersive media | |
| US20060244831A1 (en) | System and method for supplying and receiving a custom image | |
| CN106792228A (en) | A kind of living broadcast interactive method and system | |
| US20190335166A1 (en) | Deriving 3d volumetric level of interest data for 3d scenes from viewer consumption data | |
| CN105657438A (en) | Method and apparatus for processing panoramic live video resource | |
| JP2019159950A (en) | Information processing device and information processing method | |
| CN108769755A (en) | High-resolution full view frequency live streaming camera system and method | |
| US11055917B2 (en) | Methods and systems for generating a customized view of a real-world scene | |
| WO2018196658A1 (en) | Virtual reality media file generation method and device, storage medium, and electronic device | |
| CN107370993B (en) | A kind of virtual shooting system and method based on two-dimentional code watermark | |
| CN104335243A (en) | Processing panoramic pictures | |
| CN104065951A (en) | Video shooting method, video playing method and intelligent glasses | |
| CN107341754A (en) | Unmanned plane view scenic spot management system | |
| CN107592549A (en) | Panoramic video based on two-way communication plays camera system | |
| US20240080429A1 (en) | Video data processing method and apparatus, computer device, computer readable storage medium, and computer program product | |
| CN114004927A (en) | 3D video model reconstruction method and device, electronic equipment and storage medium | |
| Lee et al. | Realistic visual environment for immersive projection display system | |
| CN109104632A (en) | A kind of realization method and system of television terminal AR scene | |
| CN112492289A (en) | Immersion media data processing method and device, storage medium and electronic device | |
| CN106713890A (en) | Image processing method and device | |
| KR102105510B1 (en) | Server, method and user device for providing time slice video | |
| CN113473165A (en) | Live broadcast control system, live broadcast control method, device, medium and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 150016 Heilongjiang Province, Harbin Economic Development Zone haping Road District Dalian road and Xingkai road junction Applicant after: HARBIN YISHE TECHNOLOGY CO., LTD. Address before: 150016 Heilongjiang City, Harbin province Daoli District, quiet street, unit 54, unit 2, layer 4, No. 3 Applicant before: HARBIN YISHE TECHNOLOGY CO., LTD. |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170926 |