CN115361596A - Panoramic video data processing method and device, electronic device and storage medium - Google Patents
Panoramic video data processing method and device, electronic device and storage medium Download PDFInfo
- Publication number
- CN115361596A CN115361596A CN202210779732.XA CN202210779732A CN115361596A CN 115361596 A CN115361596 A CN 115361596A CN 202210779732 A CN202210779732 A CN 202210779732A CN 115361596 A CN115361596 A CN 115361596A
- Authority
- CN
- China
- Prior art keywords
- tag
- panoramic video
- coordinate
- label
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 28
- 238000003860 storage Methods 0.000 title claims abstract description 10
- 238000000034 method Methods 0.000 claims abstract description 34
- 238000012545 processing Methods 0.000 claims abstract description 32
- 230000004048 modification Effects 0.000 claims description 19
- 238000012986 modification Methods 0.000 claims description 19
- 238000006243 chemical reaction Methods 0.000 claims description 16
- 238000004590 computer program Methods 0.000 claims description 15
- 238000012217 deletion Methods 0.000 claims description 4
- 230000037430 deletion Effects 0.000 claims description 4
- 239000003550 marker Substances 0.000 claims description 2
- 230000000694 effects Effects 0.000 abstract description 7
- 238000005516 engineering process Methods 0.000 abstract description 5
- 230000008569 process Effects 0.000 description 16
- 238000010586 diagram Methods 0.000 description 11
- 230000005540 biological transmission Effects 0.000 description 7
- 230000006870 function Effects 0.000 description 6
- 239000011159 matrix material Substances 0.000 description 6
- 230000000007 visual effect Effects 0.000 description 6
- 230000009466 transformation Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 238000010606 normalization Methods 0.000 description 4
- 230000009471 action Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000000265 homogenisation Methods 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 230000007480 spreading Effects 0.000 description 2
- 238000003892 spreading Methods 0.000 description 2
- 102100029469 WD repeat and HMG-box DNA-binding protein 1 Human genes 0.000 description 1
- 101710097421 WD repeat and HMG-box DNA-binding protein 1 Proteins 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 230000005484 gravity Effects 0.000 description 1
- 230000006698 induction Effects 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 238000013178 mathematical model Methods 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000001360 synchronised effect Effects 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 239000013598 vector Substances 0.000 description 1
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/80—Generation or processing of content or additional data by content creator independently of the distribution process; Content per se
- H04N21/83—Generation or processing of protective or descriptive data associated with content; Content structuring
- H04N21/84—Generation or processing of descriptive data, e.g. content descriptors
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/20—Servers specifically adapted for the distribution of content, e.g. VOD servers; Operations thereof
- H04N21/23—Processing of content or additional data; Elementary server operations; Server middleware
- H04N21/234—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs
- H04N21/23418—Processing of video elementary streams, e.g. splicing of video streams or manipulating encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream or rendering scenes according to encoded video stream scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/60—Network structure or processes for video distribution between server and client or between remote clients; Control signalling between clients, server and network components; Transmission of management data between server and client, e.g. sending from server to client commands for recording incoming content stream; Communication details between server and client
- H04N21/63—Control signaling related to video distribution between client, server and network components; Network processes for video distribution between server and clients or between remote clients, e.g. transmitting basic layer and enhancement layers over different transmission paths, setting up a peer-to-peer communication via Internet between remote STB's; Communication protocols; Addressing
- H04N21/643—Communication protocols
- H04N21/6437—Real-time Transport Protocol [RTP]
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Two-Way Televisions, Distribution Of Moving Picture Or The Like (AREA)
Abstract
The application relates to a panoramic video data processing method, a panoramic video data processing device, an electronic device and a storage medium, wherein the panoramic video data processing method comprises the following steps: acquiring data stream and tag information of a panoramic video image from a data system; carrying out panoramic decoding on the data stream, and obtaining the panoramic video image according to the decoded data stream; determining the marking position of a label in the panoramic video image according to the label information, and drawing the label at the marking position; receiving a tag operation request input by a user, and sending the tag operation request to the data system, wherein the tag operation request is used for updating tag information in the data system. By the method and the device, the problem that the existing panoramic video playing technology does not support operations of adding, deleting and modifying the tag in the panoramic video by the user is solved, and the effect of updating the tag on the client according to the user operation is realized.
Description
Technical Field
The present application relates to the field of data processing, and in particular, to a method and an apparatus for processing panoramic video data, an electronic apparatus, and a storage medium.
Background
Video security has become the inseparable part of security protection trade, and in most control use scenes today, all need carry out 360 degrees control to the environment, and traditional single-lens control is difficult to satisfy. With the development of science and technology, video data can be collected in a panoramic manner by a multi-lens camera, the decoding mode of the video is also changed into panoramic decoding, and a new playing mode, namely panoramic video playing, can be derived. The user can move the video through the mouse to realize omnibearing watching; even can replace the mouse to drag through gravity induction system, the rotation of response people's head, let the person watch the video in his circumstances.
In an actual application scenario, a user often needs to mark content on a panoramic video according to own preference on a video client. However, the existing panoramic video playing technology does not support the operations of adding, deleting and modifying tags in the panoramic video by the user, and further cannot meet the requirement of marking content on the panoramic video by the user at the video client according to the preference of the user.
Aiming at the problem that the existing panoramic video playing technology does not support the operations of adding, deleting, modifying labels and the like in the panoramic video by a user, no effective solution is provided at present.
Disclosure of Invention
The embodiment provides a panoramic video data processing method, a panoramic video data processing device, an electronic device and a storage medium, so as to solve the problem that operations of adding, deleting and modifying tags and the like in a panoramic video by a user are not supported in the related art.
In a first aspect, a method for processing panoramic video data is provided in this embodiment, the method including:
acquiring data stream and tag information of a panoramic video image from a data system;
carrying out panoramic decoding on the data stream, and obtaining the panoramic video image according to the decoded data stream;
determining the marking position of a label in the panoramic video image according to the label information, and drawing the label at the marking position;
receiving a tag operation request input by a user, and sending the tag operation request to the data system, wherein the tag operation request is used for updating tag information in the data system.
In some embodiments, said panorama decoding said data stream and obtaining said panorama video image from said decoded data stream comprises:
establishing a panoramic coordinate model, and determining an initial view angle of the panoramic video image;
and carrying out panoramic decoding on the data stream according to the panoramic coordinate model, and obtaining a panoramic video image with the initial view angle according to the decoded data stream.
In some of these embodiments, the tag information includes tag location information, and the data system includes a head end device;
the acquiring the data stream and the tag information of the panoramic video image from the data system comprises:
acquiring a data stream of the panoramic video image from the front-end equipment, wherein the data stream carries the tag position information;
extracting the tag position information from the data stream, wherein the tag position information comprises a first coordinate, and the first coordinate is a coordinate of the tag in the panoramic plane image;
the determining the marking position of the tag in the panoramic video image according to the tag information comprises:
converting the first coordinate into a second coordinate according to a preset first conversion rule, and determining the marking position of the label in the panoramic video image according to the second coordinate; and the second coordinate is the coordinate of the label in the panoramic video image, and the first conversion rule is determined according to the panoramic coordinate model.
In some of these embodiments, said determining a tag location in said panoramic video image from said second coordinates comprises:
determining a current panoramic video image displayed in a playing window, and determining a second coordinate in the current panoramic video image;
converting a second coordinate in the current panoramic video image into a third coordinate according to a preset second conversion rule, wherein the third coordinate is the coordinate of the label in the playing window;
and determining the mark position according to the third coordinate.
In some of these embodiments, the initial perspective of the panoramic video image comprises an initial angle and an initial height, the panoramic coordinate model comprises a cylindrical coordinate model and a spherical coordinate model;
the converting the first coordinate into the second coordinate according to a preset first conversion rule includes:
converting the first coordinates to cylindrical coordinates under the cylindrical coordinate model based on the initial angle and initial height;
converting the cylindrical coordinates into spherical coordinates under the spherical coordinate model;
and converting the spherical coordinate into the second coordinate through a preset coordinate processing rule.
In some of these embodiments, the tag information further comprises tag content information, the data system further comprises a server;
the acquiring the data stream and the tag information of the panoramic video image from the data system further comprises:
and acquiring corresponding label content information from a server according to the label position information, wherein the label content information comprises label name information and label picture information.
In some of these embodiments, the tag operation request includes at least one of: a tag addition request, a tag deletion request and a tag modification request;
the receiving a tag operation request input by a user and sending the tag operation request to the data system comprises:
receiving a tag adding request input by a user, wherein the tag adding request comprises a tag adding instruction and newly added tag information, and sending the tag adding instruction and the newly added tag information to the data system;
and/or receiving a tag deleting request input by a user, wherein the tag deleting request comprises a tag deleting instruction, and sending the tag deleting instruction to the data system;
and/or receiving a tag modification request input by a user, wherein the tag modification request comprises a tag modification instruction and modified tag information, and sending the tag modification instruction and the modified tag information to the data system.
In a second aspect, there is provided in this embodiment a panoramic video data processing apparatus, the apparatus comprising:
the data receiving module is used for acquiring data streams and tag information of the panoramic video images from the data system;
the data decoding module is used for carrying out panoramic decoding on the data stream and obtaining the panoramic video image according to the decoded data stream;
the tag acquisition module is used for determining the marking position of a tag in the panoramic video image according to the tag information and drawing the tag at the marking position;
the request receiving module is used for receiving a tag operation request input by a user and sending the tag operation request to the data system, wherein the tag operation request is used for updating tag information in the data system.
In a third aspect, in this embodiment, there is provided an electronic apparatus, including a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the panoramic video data processing method according to the first aspect when executing the computer program.
In a fourth aspect, in the present embodiment, there is provided a storage medium having stored thereon a computer program which, when executed by a processor, implements the panoramic video data processing method of the first aspect described above.
Compared with the related art, according to the panoramic video data processing method provided by the embodiment, the client can draw the tag in the panoramic video image according to the existing tag information, and meanwhile, after the user inputs the tag operation request, the tag information can be updated according to the tag operation request. The problem that the existing panoramic video playing technology does not support the operations of adding, deleting and modifying the tags in the panoramic video by the user is solved, and the effect of updating the tags on the client according to the user operation is realized.
The details of one or more embodiments of the application are set forth in the accompanying drawings and the description below to provide a more thorough understanding of the application.
Drawings
The accompanying drawings, which are included to provide a further understanding of the application and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the application and together with the description serve to explain the application and not to limit the application. In the drawings:
fig. 1 is a block diagram of a hardware configuration of a terminal of a panoramic video data processing method of the present embodiment.
Fig. 2 is a block diagram of a panoramic video data processing system according to the present embodiment.
Fig. 3 is a flowchart of a panoramic video data processing method according to the present embodiment.
Fig. 4 is a panoramic video image using a panoramic decoding method in the present embodiment.
Fig. 5 is a panoramic plane image in the original decoding method in the present embodiment.
Fig. 6 is a system block diagram of a panoramic video data processing system according to the preferred embodiment.
Fig. 7 is a flowchart of a panoramic video data processing method according to the preferred embodiment.
Fig. 8 is a block diagram of the structure of a panoramic video data processing apparatus of the present embodiment.
Detailed Description
For a clearer understanding of the objects, aspects and advantages of the present application, reference is made to the following description and accompanying drawings.
Unless defined otherwise, technical or scientific terms referred to herein shall have the same general meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The use of the terms "a" and "an" and "the" and similar referents in the context of this application do not denote a limitation of quantity, either in the singular or the plural. The terms "comprises," "comprising," "has," "having," and any variations thereof, as referred to in this application, are intended to cover non-exclusive inclusions; for example, a process, method, and system, article, or apparatus that comprises a list of steps or modules (elements) is not limited to the listed steps or modules, but may include other steps or modules (elements) not listed or inherent to such process, method, article, or apparatus. Reference in this application to "connected," "coupled," and the like is not intended to be limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. Reference to "a plurality" in this application means two or more. "and/or" describes the association relationship of the associated object, indicating that there may be three relationships, for example, "a and/or B" may indicate: a exists alone, A and B exist simultaneously, and B exists alone. In general, the character "/" indicates a relationship in which the objects associated before and after are an "or". The terms "first," "second," "third," and the like in this application are used for distinguishing between similar items and not necessarily for describing a particular sequential or chronological order.
The method embodiments provided in the present embodiment may be executed in a terminal, a computer, or a similar computing device. For example, the method is executed on a terminal, and fig. 1 is a block diagram of a hardware structure of the terminal according to the panoramic video data processing method of the present embodiment. As shown in fig. 1, the terminal may include one or more processors 102 (only one shown in fig. 1) and a memory 104 for storing data, wherein the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA. The terminal may also include a transmission device 106 for communication functions and an input-output device 108. It will be understood by those of ordinary skill in the art that the structure shown in fig. 1 is merely an illustration and is not intended to limit the structure of the terminal described above. For example, the terminal may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.
The memory 104 may be used to store a computer program, for example, a software program and a module of application software, such as a computer program corresponding to a panoramic video data processing method in the present embodiment, and the processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, so as to implement the above-mentioned method. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the terminal over a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The transmission device 106 is used to receive or transmit data via a network. The network described above includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a Network adapter (NIC) that can be connected to other Network devices via a base station to communicate with the internet. In one example, the transmission device 106 may be a Radio Frequency (RF) module, which is used to communicate with the internet in a wireless manner.
A panoramic video data processing system is provided in the present embodiment, and fig. 2 is a block diagram of a panoramic video data processing system of the present embodiment. As shown in fig. 2, the panoramic video data processing system includes: a client 210, a server 220, and a front-end device 230, wherein the client 210 may be multiple. The client 210 mainly plays the panoramic video and provides a tag operation entry, specifically acquires a data stream of the panoramic video from the front-end device 230, thereby playing the corresponding panoramic video, and simultaneously acquires tag content information from the server 220, and the data stream includes tag position information, thereby drawing a tag in a panoramic video picture, and receiving a tag operation instruction input by a user, and sending the instruction to the server 220. The server 220 is mainly used for storing the tag information and transmitting the tag operation instruction, the specific tag position information and the tag content information are stored in the server 220, and meanwhile, the tag position information is synchronized to the front-end device 230, and after receiving the tag operation instruction, corresponding operation actions are executed, such as adding, deleting, and modifying the tag information, and the tag operation instruction is sent to the front-end device 230. The front-end device 230 is mainly used for storing the tag location information, and carrying the tag location information in the data stream of the panoramic video, and after receiving the tag operation instruction, performing a corresponding operation action, such as adding, deleting, and modifying the tag location information in the data stream.
In the present embodiment, a panoramic video data processing method is provided, and the panoramic video data processing method can be applied to the panoramic video data processing system provided in the present embodiment. Fig. 3 is a flowchart of a panoramic video data processing method according to the present embodiment, and as shown in fig. 3, the flowchart includes the following steps:
step S310, acquiring data stream and tag information of the panoramic video image from a data system.
Specifically, the data system may be a system composed of a server and a front-end device, and mainly provides video data for the client. When a client receives a request for opening a panoramic video input by a user, a data stream and label information are obtained from a data system, wherein the data stream is used for playing a panoramic video image, and the label information is used for drawing a label in the panoramic video image.
And step S320, carrying out panoramic decoding on the data stream, and obtaining a panoramic video image according to the decoded data stream.
Specifically, the client decodes the data stream in a panoramic decoding manner, so as to obtain frame pictures of the panoramic video, that is, obtain panoramic video images, and continuously play the panoramic video images. It should be further noted that there may be two decoding manners for the data stream. Firstly, the panoramic decoding mode adopted in this step, fig. 4 is a panoramic video image adopting the panoramic decoding mode in this embodiment, and as shown in fig. 4, a frame picture obtained after decoding can simulate a spherical visual effect of an actual scene; secondly, an original decoding mode of a data stream is adopted, fig. 5 is a panoramic plane image adopting the original decoding mode in this embodiment, as shown in fig. 5, a decoded frame picture shows a visual effect of spreading a spherical surface into a plane, and the picture has a certain distortion. In this step, the client decodes in a panoramic decoding manner, so that the panoramic video image is played with the visual effect shown in fig. 4.
And step S330, determining the marking position of the label in the panoramic video image according to the label information, and drawing the label at the marking position.
Specifically, after the client starts playing the panoramic video image, the client draws a corresponding label on the panoramic video image according to the label information. The specific label information comprises label position information and label content information, then the marking position of the label in the panoramic video image is determined according to the label position information, and the label is drawn at the corresponding position according to the label content information. It should be further noted that in this step, the position of the tag in the panoramic video image is determined, not in the panoramic plane image. Therefore, in one specific embodiment, when the tag position information is the coordinates of the tag in the image, the coordinates of the tag in the panoramic plane image need to be converted into the coordinates of the tag in the panoramic video image.
Step S340, receiving a tag operation request input by a user, and sending the tag operation request to the data system, where the tag operation request is used to update tag information in the data system.
Specifically, the client can also receive a tag operation request input by a user in the process of playing the panoramic video, and send the tag operation request to the data system, and the data system updates the tag information according to the tag operation request, so that the client can obtain the updated tag information later and draw a new tag according to the new tag information, thereby updating the tag.
It should be further noted that in one preferred embodiment, the client communicates with the data system via a real-time transport protocol, so that the data stream and the tag information obtained by the client from the data system are updated in real time. And in a process of playing the panoramic video completely once, the client can execute the steps repeatedly in a circulating way. For example, after the client acquires the data stream and the tag information at the current moment, the client plays the current panoramic video frame, and draws a corresponding tag in the current panoramic video frame (steps S310 to S330), and if a tag operation request input by the user is received at this time, the tag operation request is sent to the data system to update the tag information (step S340), and then steps S310 to S330 are executed again, that is, the video stream and the tag information at the next moment are acquired, and the tag information at the next moment is the updated tag information, so that the tag is drawn again according to the updated tag information. The above process is a complete tag real-time refreshing process.
Through the steps, the client can not only draw the label in the panoramic video image according to the existing label information, but also modify the label in the panoramic video image. And when the client and the data system adopt a real-time transmission protocol, after a user inputs a label operation request, the label information can be updated in time according to the label operation request, and then the client draws a new label according to the updated label information at the next moment, so that the effect of updating the label in real time according to the user operation on the client is realized, such as adding, deleting and modifying the label. And the client side does not directly execute corresponding operation according to the label operation request, but sends the label operation request to the data system, updates the label information in the data system, and then acquires the updated label information to redraw the label. When the tag operation request is received by one of the clients, the updated tag information can be synchronously received by other clients.
In some embodiments, the performing panorama decoding on the data stream, and obtaining a panorama video image according to the decoded data stream specifically includes:
establishing a panoramic coordinate model, and determining an initial view angle of a panoramic video;
and carrying out panoramic decoding on the data stream according to the panoramic coordinate model, and obtaining a panoramic video image with an initial view angle according to the decoded data stream.
In a specific embodiment, the panorama decoding needs to be performed in a panorama coordinate model, where the panorama coordinate model at least includes a spherical coordinate model and a cylindrical coordinate model. Therefore, a client needs to establish a panoramic coordinate model before performing panoramic decoding on a data stream, and then complete panoramic decoding according to the panoramic coordinate model; meanwhile, an initial view angle of the panoramic video image needs to be determined, and the panoramic video image is played at the initial view angle.
In some embodiments, the tag information includes tag location information, and the data system includes a head-end device;
the step of acquiring the data stream and the tag information of the panoramic video image from the data system specifically includes:
acquiring a data stream of a panoramic video image from front-end equipment, wherein the data stream carries tag position information;
extracting tag position information from the data stream, wherein the tag position information comprises a first coordinate, and the first coordinate is a coordinate of a tag in the panoramic plane image;
the step of determining the marking position of the tag in the panoramic video image according to the tag information specifically comprises the following steps:
and converting the first coordinate into a second coordinate according to a preset first conversion rule, and determining the mark position in the panoramic video image according to the second coordinate, wherein the second coordinate is the coordinate of the label in the panoramic video image.
Specifically, in this embodiment, the client sends a request for acquiring a corresponding URL (uniform resource locator) to the front-end device, after receiving the URL returned by the front-end device, the client sends a request for acquiring a data stream to the front-end device based on the URL, and after receiving the request, the front-end device sends a data stream of a corresponding panoramic video image to the client. The label information at least comprises label position information, the label position information is stored in a server and front-end equipment in the data system, and the label position information is carried in a data stream sent by the front-end equipment, so that the client can directly extract the label position information from the obtained data stream. Preferably, the client obtains the data stream from the front-end device through a real-time transport protocol, such as an RTSP protocol. The tag position information includes coordinates of the tag in the panoramic video image, and the panoramic video image is obtained by decoding the data stream in an original manner, and the visual effect of the panoramic video image is inferior to that of the panoramic video image obtained by the panoramic decoding manner.
Further, in one particular embodiment, the initial perspective of the panoramic video includes an initial angle and an initial height; the panoramic coordinate model comprises a cylindrical coordinate model and a spherical coordinate model;
the step of converting the first coordinate into the second coordinate according to a preset first conversion rule specifically includes:
converting the first coordinates into cylindrical coordinates under a cylindrical coordinate model based on the initial angle and the initial height;
converting the cylindrical coordinates into spherical coordinates under a spherical coordinate model;
and converting the spherical coordinates into second coordinates through a preset coordinate processing rule.
In a specific embodiment, a specific process of converting the coordinates of the tag in the panoramic planar image into the coordinates of the tag in the panoramic video image is as follows: converting the first coordinate by combining the initial angle and the initial height under the cylindrical coordinate model, thereby obtaining a corresponding cylindrical coordinate; then, converting the corresponding cylindrical coordinates under a spherical coordinate model to obtain corresponding spherical coordinates; and finally, processing the corresponding spherical coordinate according to a preset processing rule to finally obtain a converted second coordinate. Specifically, the preset processing rule includes: and (3) carrying out coordinate homogenization processing on the spherical coordinates, then realizing coordinate transformation by multiplying the spherical coordinates by the transformation matrix, and finally carrying out coordinate normalization processing to obtain the coordinates of the label in the panoramic video image.
It is further explained that the description of the coordinate transformation function in the present embodiment as a mathematical model is as follows: given that an arbitrary point a = (X, Y) in the panoramic plane coordinate system, the point a corresponds to the coordinate value (X, Y) of a' in the VR graph (panoramic video image) coordinate system after VR conversion. The VR effect in this example is a 360 x 90 lower hemisphere model.
First, the image coordinate system uses the upper left corner as the origin, the right horizontal direction as the positive x-axis direction, and the downward vertical direction as the positive y-axis direction.
The coordinates (x, y) in the panoramic planar coordinate system are then converted to cylindrical coordinates, similar to rolling a complete planar image into a reel. In the cylindrical coordinate system, the y-direction quantity is equal to the image coordinate system in size, the direction is opposite, the x direction becomes a cylindrical circumference, and finally, one-time normalization is performed. Thus, the relationship between the cylindrical coordinates and the image coordinates is obtained as follows:
wherein x and y are coordinates in a panoramic plane coordinate system, W is the image width, and H is the image height.
Secondly, the cylindrical surface coordinate is converted into a lower hemisphere coordinate, and the lower half part of the cylindrical surface is contracted and converted into a hemisphere. First, y' in the hemisphere coordinate is obtained, and specific calculation is carried outThe formula is as follows:
knowing the coordinates of y ", the concentric radius of the point where y" is located can be obtained from the equation of a circle
When x = x '× R', z = z '× R', the relationship between the hemispherical coordinates and the cylindrical coordinates is obtained by the following equation:
through the above steps, the coordinate points in the panoramic plane coordinate system have been converted into coordinate points in the hemispherical coordinate system.
And finally, combining the coordinates in the hemispherical coordinate system with the transformation matrix. Convert (x ", y", z ") into homogeneous vectors:
and then multiplying the space coordinate with a projection matrix (P), a view matrix (V) and a model matrix (M) to obtain a four-dimensional space coordinate (X ', Y', Z ', W'), wherein the formula is as follows:
normalization, the final result is:
the above process is a complete process for converting the coordinates in the panoramic plane coordinate system into the coordinates in the panoramic video image coordinate system.
Further, in one embodiment, the step of determining the position of the marker in the panoramic video image according to the second coordinate specifically includes:
determining a current panoramic video image displayed in a playing window, and determining a second coordinate in the current panoramic image;
converting the second coordinate in the current panoramic image into a third coordinate according to a preset second conversion rule, wherein the third coordinate is the coordinate of the label in the playing window;
and determining the position of the mark according to the third coordinate.
In a specific embodiment, the panoramic video image is a spherical image of 360 degrees, and the playing window can only display a part of the panoramic video image at the same time, and the part of the panoramic video image is the current panoramic video image, so that the client only needs to draw a label in the current panoramic video image displayed by the playing window at the same time. The client needs to determine the current panoramic video image displayed in the playing window first, and then determine the second coordinate in the current panoramic video image, so as to determine the tag in the playing window. Further, in order to determine the relative position of the tag in the playing window, a new coordinate system needs to be established on the current panoramic video image, and then the tag in the panoramic video image is converted into a third coordinate, where the third coordinate is the coordinate of the tag in the new coordinate system, and the relative position of the tag in the playing window can be determined according to the coordinate. The initial play window is taken as an example for explanation. The center point of the general initial playing window corresponds to the origin of coordinates of the panoramic video image, i.e. the center point coordinates of the initial panoramic video image displayed in the initial playing window are (0, 0). At this time, one corner vertex of the initial playing window can be used as an origin point, so that a new coordinate system is established, and the coordinate value of the label in the panoramic video image is compared with the coordinate value of one corner vertex of the playing window in the panoramic video image, so that two ratios X are obtained n And Y n If the two ratios are both in the interval (-1, 1), it indicates that the corresponding tag is in the initial playing window, and further determines that the tag is the tag to be drawn, and then converts the coordinate of the tag in the panoramic image into the coordinate of the tag in the new coordinate system.
The following examples all refer to coordinate systems with x values increasing to the left on the abscissa and y values increasing downward on the ordinate.
Illustratively, a new coordinate system for the current panoramic video image is established with the top left corner vertex of the initial playing window as the origin, and the coordinate values of the tag in the panoramic video image are compared with the coordinate values of the bottom right corner vertex of the initial playing window in the panoramic video image, so as to determine two ratios Xn and Yn, then the coordinates (X, Y) of the tag in the new coordinate system satisfy: x = (1 + Xn) × W/2 (W is the width of the initial playing window), Y = (1 + Yn) × H/2 (H is the height of the initial playing window). Specifically, in this example, the width of the initial playing window may be set to 100, and the height of the initial playing window may be set to 200, that is, the coordinate of the vertex of the lower right corner of the initial playing window in the new coordinate system is (100, 200), and since the center point of the initial playing window corresponds to the origin of coordinates of the panoramic video image, the coordinate of the vertex of the lower right corner of the initial playing window in the panoramic video image is (50, 100). For example, when the coordinates of the tag in the panoramic video image are (50, 100), the ratios Xn and Yn are 1 and 1, respectively, the calculation formula is substituted to obtain X and Y which are 100 and 200, respectively, and the coordinates of the tag in the new coordinate system are (100 and 200), that is, the vertex position of the lower right corner of the initial playing window. For another example, when the coordinates of the tag in the panoramic video image are (0, 0), the ratios Xn and Yn are 0 and 0, respectively, the calculation formula is substituted to obtain X and Y which are 50 and 100, respectively, and the coordinates of the tag in the new coordinate system are (50 and 100), that is, the position of the center point of the initial playing window. For another example, when the coordinates of the tag in the panoramic video image are (50, 100), the ratios Xn and Yn are-1 and-1, respectively, the calculation formula is substituted to find that X and Y are 0 and 0, respectively, and the coordinates of the tag in the new coordinate system are (0 and 0), that is, the vertex position of the upper left corner of the initial playing window.
In some of these embodiments, the tag information further includes tag content information, and the data system further includes a server;
the step of obtaining the data stream and the tag information of the panoramic video image from the data system further specifically comprises:
and acquiring corresponding label content information from the server according to the label position information, wherein the label content information comprises label name information and label picture information.
In a specific embodiment, the data system further includes a server, the tag information further includes tag content information, and the tag content information is mainly stored in the server, so that the client needs to obtain the tag content information from the server. After the client acquires the label position information, the client acquires the label content information associated with the label position information from the server, and the label content information and the label position information can be corresponded through the unique identification of the label. Further, the tag content information includes tag name information and tag picture information, the name information is an identification name of the tag, and the picture information may be a linkage picture corresponding to the tag or a URL of the linkage picture.
In some of these embodiments, the tag operation request includes at least one of: a tag addition request, a tag deletion request and a tag modification request;
receiving a tag operation request input by a user, and sending the tag operation request to a data system comprises the following steps:
receiving a tag adding request input by a user, wherein the tag adding request comprises a tag adding instruction and newly added tag information, and sending the tag adding instruction and the newly added tag information to a data system;
and/or receiving a tag deleting request input by a user, wherein the tag deleting request comprises a tag deleting instruction, and sending the tag deleting instruction to the data system;
and/or receiving a label modification request input by a user, wherein the label modification request comprises a label modification instruction and modified label information, and sending the label modification instruction and the modified label information to the data system.
Specifically, in this embodiment, the tag operation request includes at least three types. Specifically, the client may send the tag operation request to a server in the data system, the server updates tag location information and tag content information stored therein according to the tag operation request, and meanwhile, forwards the tag operation request to the front-end device, and the front-end device updates the tag location information stored therein according to the tag operation request. One of the requests is a tag addition request, that is, a new tag is added to the current panoramic video image. When the user uses the label, the user can mark the image position of the label needing to be added, then the name of the label and the associated picture (label content information) are input, the client takes the coordinate of the marked position of the user as the coordinate of a new label (label position information), and then the client is bound with the name of the label and the associated picture, so that the information forms the newly added label information, and the newly added label information and the label adding instruction are sent to the server. And the server executes the action of adding the label according to the label adding instruction so as to store the newly added label information, and simultaneously, the server synchronizes the coordinates of the label to the front-end equipment which also stores the coordinates of the label. It should be further noted that the tag location information and the tag content information are bound by an identification ID, that is, each tag has a uniquely identified ID, and both the tag location information and the tag content information belonging to the same tag are in a mapping relationship with the corresponding ID. Another request is a label deleting request, when a user uses the label deleting system, the user can delete the existing label, the client sends a label deleting instruction and the ID of the label to be deleted to the server, the server deletes the label position information and the label content information associated with the corresponding ID through the label deleting instruction, and transmits the received data instruction to the front-end equipment, and the front-end equipment does not carry the deleted coordinate position information in the data stream at the next moment according to the label position information associated with the flaw of the label deleting instruction and the related ID, so that the client does not draw the deleted label in the next frame of picture. The server sends the modified label information and the label modification instruction to the server, and the server replaces the original label information with the modified label information according to the label modification instruction.
The present embodiment is described and illustrated below by means of preferred embodiments.
Fig. 6 is a system block diagram of a panoramic video data processing system according to the preferred embodiment. As shown in fig. 6, the panoramic video data processing system includes: client 210, server 220, and front-end 230.
Among the main functions of the client 210 are:
processing operations of opening a video by a user, adding, deleting, modifying and the like of a label;
carrying out panoramic decoding on the video;
and converting the panoramic plane coordinates of the tags in the code stream into panoramic video coordinates, calculating and superposing the panoramic video coordinates in the panoramic video.
The main functions of the server 220 are:
obtaining RTSP information from the equipment and returning the RTSP information to the client;
storing detailed information (name, linkage information, etc.) of the tag;
and sending the label adding and deleting operations to the equipment.
The main functions of the front-end device 230 are:
storing the label position, carried in the code stream (not directly superposed on the video picture);
returning RTSP information to the service;
and processing the RTSP pull stream of the client.
When in use, the user performs operations to the client 210, including video opening, tag operation (tag adding/deleting/changing); the client 210 is responsible for providing an operation entry for a user, pulling a stream, extracting a tag coordinate in a code stream, converting a panoramic plane coordinate into a panoramic video coordinate, acquiring detailed information (name, linkage picture and the like) of each tag from the server 220, and finally drawing and superposing the tags on a video picture; the server 220 stores the tag detail information and is responsible for transmitting the client's request; the front-end device 230 is responsible for collecting pictures, storing tag positions, and supporting RTSP streaming of the client.
Fig. 7 is a flowchart of a panoramic video data processing method according to the preferred embodiment. As shown in fig. 7, the panoramic video data processing method includes the steps of:
and step S700, acquiring a stream pulling URL from the front-end equipment, and performing RTSP stream pulling after the URL is acquired. Specifically, after receiving a request of a user to open the panoramic video, the client executes the step and judges whether the stream pulling is successful. If the pull flow fails, the flow is terminated, and if the pull flow succeeds, the following step S710 is executed.
Step S710, a cylindrical coordinate model and a spherical coordinate model are established, and an initial angle and an initial height are selected. The specific client executes the step after the pull stream is successful.
And S720, decoding the panoramic video and starting to play the panoramic video. And the specific client carries out panoramic decoding on the equipment code stream and returns the video frames at the initial angle and height for playing.
And step S730, analyzing the coordinate information of all the labels in the code stream on the panoramic plane.
Step S740, acquiring detailed information of each tag. The specific client acquires detailed information of the label from the server, wherein the detailed information comprises a label name and label linkage information.
Step S750, traversing the panoramic plane coordinates of all the tags, and converting the panoramic plane coordinates into coordinates on the panoramic image; the specific conversion means is as follows: firstly, coordinate conversion is carried out on panoramic plane coordinates under a cylindrical coordinate model by combining an initial angle and height, then coordinate conversion is carried out under a spherical coordinate model, and finally coordinate homogenization, matrix multiplication and coordinate normalization processing are carried out to obtain coordinates of the label on the panoramic image.
Step S760, outputting the ratio X of the label coordinates on the panoramic image to the video window n And Y n . Specifically, the client outputs a ratio X of the conversion coordinate value of the label n to the coordinate value of the video window n And Y n (n is 0, a), and a is the number of tags).
Step S770, determining X n 、Y n Are all in the interval (-1,1)]The label coordinates within. Specifically, the client traverses each output ratio X n ，Y n The screened ratios are all (-1, 1)]Coordinate points within the interval.
Step S780, calculating coordinate transformation, and transforming the coordinates of the tag on the panoramic video into a coordinate point (x, y) on the current panoramic image (the panoramic image in the video playing window), wherein: x = (1+X) n ) W/2 (W is broadcast)Window width), y = (1 + Y) n ) H/2 (H is play window high).
Step S790, selecting the tab style picture, drawing the tab at the screen (play window) (x, y), and carrying the tab information.
In the above flow, steps S750 to S790 may be referred to as a tag refresh, and after the panoramic video is played, the tag needs to be refreshed in real time after the tag is changed (tag addition/deletion/modification). And after step S790 is finished, it is determined whether to traverse the panoramic plane coordinates of all the tags, if so, the process is finished, otherwise, step S750 is continuously executed.
According to the method, the label is overlaid and drawn by the client, and compared with direct overlaying on equipment, more flexible interaction can be performed, such as label display/hiding, double-click on the label to view linkage information and the like.
The tag refresh process in the above process is further explained below, and the tag refresh process includes the following steps:
step a, a client acquires a code stream of the panoramic video, wherein the code stream contains the existing label information, the client plays the panoramic video according to the analyzed code stream, and draws the existing label in the panoramic video according to the existing label information. Illustratively, a client gets a stream through a video URL, and front-end equipment returns a data stream carrying a tag ID and a tag coordinate to the client; then the client side obtains the detailed label information from the server, and the server returns all label IDs, label coordinates and detailed label information (name and linkage information) to the client side; further, when the client needs to draw a tag with the ID of a, the client queries detailed tag information corresponding to the tag with the ID of a from all the tag information sent by the server; and finally, drawing a label with the ID of a by combining the name, the coordinate and the linkage information.
And b, receiving label operation input by a user, wherein the label operation comprises adding, deleting and modifying, the client converts the operation point into a coordinate on the panoramic image, and sends the label operation of the user to the front-end equipment, the front-end equipment correspondingly modifies a database for storing label information and returns a code stream, and the modified label information is attached to the code stream. Illustratively, a user adds a label at a playing window (x 1, y 1), and inputs a label name and a picture URL, the client generates a unique ID for the label, associates and binds the coordinate information, the name information and the picture information through the ID, and sends the label information and a label adding request to the server, and the server stores the label information and synchronizes the label ID and the label coordinate in the front-end device, thereby completing a label adding operation.
And step c, the client updates the label state and redraws the label according to the modified label information.
The steps are the process when the user adds/deletes/modifies the label on the platform (client and server), the device will continuously return the code stream with the label information after starting playing, when the user performs the operation of adding, deleting and modifying the label, the operation will be issued to the device, the device modifies the database storing the label information and updates the label information attached in the code stream, and the client redraws the label after the platform detects the change of the label information.
It should be further noted that the various terms in the preferred embodiment include:
platform: a medium for providing device-related services to a user, comprising a client and a server;
panoramic plane: decoding the code stream of the panoramic equipment in an original mode to obtain a panoramic plane, presenting the effect of spreading the spherical surface into a plane, wherein the graph has distortion, and the visual effect can refer to fig. 5.
Panoramic video: the effect of panoramic decoding of the panoramic equipment code stream by establishing a spherical model is basically distortion-free, the panoramic equipment code stream can be dragged and watched by a mouse at 360 degrees, and the visual effect can be shown in figure 4.
The panoramic video data processing method in the present embodiment is further described by the above preferred embodiments. The embodiment provides a method for enabling a user to add a label to a panoramic video, so as to improve interaction between the user and the video.
It should be noted that the steps illustrated in the above-described flow diagrams or in the flow diagrams of the figures may be performed in a computer system such as a set of computer-executable instructions, and that while a logical order is illustrated in the flow diagrams, in some cases, the steps illustrated or described may be performed out of order from that shown herein. For example, steps S310 to S340 are executed circularly, and in the circular execution process, step S340 is executed only when the user has an operation, and is not involved in the circular execution process at other times.
In this embodiment, a panoramic video data processing apparatus is further provided, and the apparatus is used to implement the foregoing embodiments and preferred embodiments, and the description already made is omitted. The terms "module," "unit," "subunit," and the like as used below may implement a combination of software and/or hardware for a predetermined function. Although the means described in the embodiments below are preferably implemented in software, an implementation in hardware, or a combination of software and hardware is also possible and contemplated.
Fig. 8 is a block diagram showing a configuration of a panoramic video data processing apparatus of the present embodiment, which, as shown in fig. 8, includes:
a data receiving module 810, configured to obtain a data stream and tag information of a panoramic video image from a data system;
a data decoding module 820, configured to perform panoramic decoding on the data stream, and obtain a panoramic video image according to the decoded data stream;
the tag obtaining module 830 is configured to determine a tag position of the tag in the panoramic video image according to the tag information, and draw the tag at the tag position;
the request receiving module 840 is configured to receive a tag operation request input by a user, and send the tag operation request to the data system, where the tag operation request is used to update tag information in the data system.
Through the modules, the client can draw the label in the panoramic video image according to the existing label information, meanwhile, after a user inputs a label operation request, the label information can be updated in time according to the label operation request, and then the client draws a new label according to the updated label information at the next moment, so that the effect of updating the label according to the user operation on the client is achieved, such as adding, deleting and modifying the label. And the client side does not directly execute corresponding operation according to the label operation request, but sends the label operation request to the server, updates the label information in the server and the front-end equipment, and then acquires the updated label information to redraw the label. In actual use, a plurality of clients share the server and the front-end device, so that when one client receives a tag operation request input by a user, the other clients can synchronously receive updated tag information.
Further specific working principles of the above modules are specifically described in the method embodiment, and thus are not described herein again.
It should be noted that the above modules may be functional modules or program modules, and may be implemented by software or hardware. For a module implemented by hardware, the modules may be located in the same processor; or the modules can be respectively positioned in different processors in any combination.
There is also provided in this embodiment an electronic device comprising a memory having a computer program stored therein and a processor configured to execute the computer program to perform the steps of any of the method embodiments described above.
Optionally, the electronic apparatus may further include a transmission device and an input/output device, wherein the transmission device is connected to the processor, and the input/output device is connected to the processor.
Optionally, in this embodiment, the processor may be configured to execute the following steps by a computer program:
step S310, acquiring data stream and tag information of the panoramic video image from a data system.
And step S320, carrying out panoramic decoding on the data stream, and obtaining a panoramic video image according to the decoded data stream.
And step S330, determining the marking position of the label in the panoramic video image according to the label information, and drawing the label at the marking position.
Step S340, receiving a tag operation request input by a user, and sending the tag operation request to the data system, where the tag operation request is used to update tag information in the data system.
It should be noted that, for specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiment and optional implementation manners, and details are not described in this embodiment again.
In addition, in combination with the panoramic video data processing method provided in the foregoing embodiment, a storage medium may also be provided to implement this embodiment. The storage medium has a computer program stored thereon; the computer program, when executed by a processor, implements any of the panoramic video data processing methods of the above embodiments.
It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to be limiting. All other embodiments, which can be derived by a person skilled in the art from the examples provided herein without any inventive step, shall fall within the scope of protection of the present application.
It is obvious that the drawings are only examples or embodiments of the present application, and it is obvious to those skilled in the art that the present application can be applied to other similar cases according to the drawings without creative efforts. Moreover, it should be appreciated that such a development effort might be complex and lengthy, but would nevertheless be a routine undertaking of design, fabrication, and manufacture for those of ordinary skill having the benefit of this disclosure, and is not intended to limit the present disclosure to the particular forms disclosed herein.
The term "embodiment" is used herein to mean that a particular feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of the present application. The appearances of the phrase in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. It is to be expressly and implicitly understood by one of ordinary skill in the art that the embodiments described in this application may be combined with other embodiments without conflict.
The above examples only express several embodiments of the present application, and the description thereof is more specific and detailed, but not construed as limiting the scope of the patent protection. It should be noted that, for a person skilled in the art, several variations and modifications can be made without departing from the concept of the present application, which falls within the scope of protection of the present application. Therefore, the protection scope of the present application should be subject to the appended claims.
Claims (10)
1. A method of panoramic video data processing, the method comprising:
acquiring data stream and tag information of a panoramic video image from a data system;
carrying out panoramic decoding on the data stream, and obtaining the panoramic video image according to the decoded data stream;
determining the marking position of a label in the panoramic video image according to the label information, and drawing the label at the marking position;
receiving a tag operation request input by a user, and sending the tag operation request to the data system, wherein the tag operation request is used for updating tag information in the data system.
2. The method of claim 1, wherein the performing panorama decoding on the data stream and obtaining the panorama video image according to the decoded data stream comprises:
establishing a panoramic coordinate model, and determining an initial view angle of the panoramic video image;
and carrying out panoramic decoding on the data stream according to the panoramic coordinate model, and obtaining a panoramic video image with the initial view angle according to the decoded data stream.
3. The panoramic video data processing method of claim 2, wherein the tag information includes tag location information, and the data system includes a front-end device;
the acquiring the data stream and the tag information of the panoramic video image from the data system comprises:
acquiring a data stream of the panoramic video image from the front-end equipment, wherein the data stream carries the tag position information;
extracting the tag position information from the data stream, wherein the tag position information comprises a first coordinate, and the first coordinate is a coordinate of the tag in the panoramic plane image;
the determining the marking position of the tag in the panoramic video image according to the tag information comprises:
converting the first coordinate into a second coordinate according to a preset first conversion rule, and determining the marking position of the label in the panoramic video image according to the second coordinate;
and the second coordinate is the coordinate of the label in the panoramic video image, and the first conversion rule is determined according to the panoramic coordinate model.
4. The panoramic video data processing method of claim 3, wherein the determining the marker position of the tag in the panoramic video image according to the second coordinate comprises:
determining a current panoramic video image displayed in a playing window, and determining a second coordinate in the current panoramic video image;
converting a second coordinate in the current panoramic video image into a third coordinate according to a preset second conversion rule, wherein the third coordinate is the coordinate of the label in the playing window;
and determining the mark position according to the third coordinate.
5. The panoramic video data processing method according to claim 3 or claim 4, wherein the initial view angle of the panoramic video image includes an initial angle and an initial height, and the panoramic coordinate model includes a cylindrical coordinate model and a spherical coordinate model;
the converting the first coordinate into the second coordinate according to a preset first conversion rule includes:
converting the first coordinates to cylindrical coordinates under the cylindrical coordinate model based on the initial angle and initial height;
converting the cylindrical coordinates into spherical coordinates under the spherical coordinate model;
and converting the spherical coordinate into the second coordinate through a preset coordinate processing rule.
6. The panoramic video data processing method according to claim 3 or claim 4, wherein the tag information further includes tag content information, the data system further includes a server;
the acquiring the data stream and the tag information of the panoramic video image from the data system further comprises:
and acquiring corresponding label content information from a server according to the label position information, wherein the label content information comprises label name information and label picture information.
7. The panoramic video data processing method of any one of claims 1 to 4, wherein the tag operation request includes at least one of: a tag addition request, a tag deletion request and a tag modification request;
the receiving a tag operation request input by a user and sending the tag operation request to the data system comprises:
receiving a tag adding request input by a user, wherein the tag adding request comprises a tag adding instruction and newly added tag information, and sending the tag adding instruction and the newly added tag information to the data system;
and/or receiving a label deleting request input by a user, wherein the label deleting request comprises a label deleting instruction, and sending the label deleting instruction to the data system;
and/or receiving a tag modification request input by a user, wherein the tag modification request comprises a tag modification instruction and modified tag information, and sending the tag modification instruction and the modified tag information to the data system.
8. A panoramic video data processing apparatus, characterized in that the apparatus comprises:
the data receiving module is used for acquiring data streams and tag information of the panoramic video images from the data system;
the data decoding module is used for carrying out panoramic decoding on the data stream and obtaining the panoramic video image according to the decoded data stream;
the tag obtaining module is used for determining the marking position of a tag in the panoramic video image according to the tag information and drawing the tag at the marking position;
the request receiving module is used for receiving a tag operation request input by a user and sending the tag operation request to the data system, wherein the tag operation request is used for updating tag information in the data system.
9. An electronic device comprising a memory and a processor, wherein the memory has stored therein a computer program, and the processor is configured to execute the computer program to perform the panoramic video data processing method of any one of claims 1 to 7.
10. A computer-readable storage medium on which a computer program is stored, which computer program, when being executed by a processor, carries out the steps of the panoramic video data processing method of any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210779732.XA CN115361596A (en) | 2022-07-04 | 2022-07-04 | Panoramic video data processing method and device, electronic device and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210779732.XA CN115361596A (en) | 2022-07-04 | 2022-07-04 | Panoramic video data processing method and device, electronic device and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115361596A true CN115361596A (en) | 2022-11-18 |
Family
ID=84030828
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210779732.XA Pending CN115361596A (en) | 2022-07-04 | 2022-07-04 | Panoramic video data processing method and device, electronic device and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115361596A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117544805A (en) * | 2023-11-16 | 2024-02-09 | 广电运通集团股份有限公司 | Video playing method and system, video tag generating method and client |
Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101877139A (en) * | 2009-04-30 | 2010-11-03 | 爱国者全景(北京)网络科技发展有限公司 | Method and system for realizing spacial hot spots in three-dimensional video panorama |
| CN103327238A (en) * | 2013-05-29 | 2013-09-25 | 山西绿色光电产业科学技术研究院（有限公司） | Space registering method based on panoramic all-in-one speed dome camera |
| CN106357976A (en) * | 2016-08-30 | 2017-01-25 | 深圳市保千里电子有限公司 | Omni-directional panoramic image generating method and device |
| CN109063039A (en) * | 2018-07-17 | 2018-12-21 | 高新兴科技集团股份有限公司 | A kind of video map dynamic labels display methods and system based on mobile terminal |
| CN109063123A (en) * | 2018-08-01 | 2018-12-21 | 深圳市城市公共安全技术研究院有限公司 | Method and system for adding annotations to panoramic video |
| CN109166178A (en) * | 2018-07-23 | 2019-01-08 | 中国科学院信息工程研究所 | A kind of significant drawing generating method of panoramic picture that visual characteristic is merged with behavioral trait and system |
| CN109274926A (en) * | 2017-07-18 | 2019-01-25 | 杭州海康威视系统技术有限公司 | A kind of image processing method, equipment and system |
| CN110413839A (en) * | 2019-07-05 | 2019-11-05 | 高新兴科技集团股份有限公司 | A kind of label data sharing method, device and the equipment of image capture device |
| CN111107419A (en) * | 2019-12-31 | 2020-05-05 | 福州大学 | Panoramic video-based playing and label multi-point instant adding method |
| CN113906727A (en) * | 2020-08-13 | 2022-01-07 | 深圳市大疆创新科技有限公司 | Panoramic playback method, device and system, shooting equipment and movable platform |
| CN114155292A (en) * | 2021-11-29 | 2022-03-08 | 中国人民解放军火箭军工程大学 | Indoor positioning and attitude determining method and device based on panoramic video |
| CN114549289A (en) * | 2022-02-17 | 2022-05-27 | 深圳市商汤科技有限公司 | Image processing method, image processing device, electronic equipment and computer storage medium |
| CN114676284A (en) * | 2022-03-09 | 2022-06-28 | 无锡市公安局 | Management method, management server and management system for labels in video |
-
2022
- 2022-07-04 CN CN202210779732.XA patent/CN115361596A/en active Pending
Patent Citations (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101877139A (en) * | 2009-04-30 | 2010-11-03 | 爱国者全景(北京)网络科技发展有限公司 | Method and system for realizing spacial hot spots in three-dimensional video panorama |
| CN103327238A (en) * | 2013-05-29 | 2013-09-25 | 山西绿色光电产业科学技术研究院（有限公司） | Space registering method based on panoramic all-in-one speed dome camera |
| CN106357976A (en) * | 2016-08-30 | 2017-01-25 | 深圳市保千里电子有限公司 | Omni-directional panoramic image generating method and device |
| CN109274926A (en) * | 2017-07-18 | 2019-01-25 | 杭州海康威视系统技术有限公司 | A kind of image processing method, equipment and system |
| CN109063039A (en) * | 2018-07-17 | 2018-12-21 | 高新兴科技集团股份有限公司 | A kind of video map dynamic labels display methods and system based on mobile terminal |
| CN109166178A (en) * | 2018-07-23 | 2019-01-08 | 中国科学院信息工程研究所 | A kind of significant drawing generating method of panoramic picture that visual characteristic is merged with behavioral trait and system |
| CN109063123A (en) * | 2018-08-01 | 2018-12-21 | 深圳市城市公共安全技术研究院有限公司 | Method and system for adding annotations to panoramic video |
| CN110413839A (en) * | 2019-07-05 | 2019-11-05 | 高新兴科技集团股份有限公司 | A kind of label data sharing method, device and the equipment of image capture device |
| CN111107419A (en) * | 2019-12-31 | 2020-05-05 | 福州大学 | Panoramic video-based playing and label multi-point instant adding method |
| CN113906727A (en) * | 2020-08-13 | 2022-01-07 | 深圳市大疆创新科技有限公司 | Panoramic playback method, device and system, shooting equipment and movable platform |
| CN114155292A (en) * | 2021-11-29 | 2022-03-08 | 中国人民解放军火箭军工程大学 | Indoor positioning and attitude determining method and device based on panoramic video |
| CN114549289A (en) * | 2022-02-17 | 2022-05-27 | 深圳市商汤科技有限公司 | Image processing method, image processing device, electronic equipment and computer storage medium |
| CN114676284A (en) * | 2022-03-09 | 2022-06-28 | 无锡市公安局 | Management method, management server and management system for labels in video |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117544805A (en) * | 2023-11-16 | 2024-02-09 | 广电运通集团股份有限公司 | Video playing method and system, video tag generating method and client |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| KR102627363B1 (en) | Localization determination for mixed reality systems | |
| CN110602554B (en) | Cover image determining method, device and equipment | |
| EP2652638B1 (en) | System and method for recognition of items in media data and delivery of information related thereto | |
| US11748955B2 (en) | Network-based spatial computing for extended reality (XR) applications | |
| JP6669403B2 (en) | Communication device, communication control method, and communication system | |
| CN104205157B (en) | Cooperate media collection system and method | |
| US20230091710A1 (en) | Image processing method and apparatus, electronic device, and storage medium | |
| JP2004064784A (en) | Method for providing multi-resolution video to plural users, computer program product, and apparatus | |
| CN101872243B (en) | System and method for realizing 360-degree panoramic play following real space direction | |
| CN113891105A (en) | Picture display method and device, storage medium and electronic equipment | |
| CN103959220B (en) | Method for achieving augmented reality, and user equipment | |
| CN112492231B (en) | Remote interaction method, device, electronic equipment and computer readable storage medium | |
| CN109328462A (en) | A kind of method and device for stream video content | |
| CN106780310B (en) | Projection graph construction method and device | |
| US20170269895A1 (en) | Efficient communication interface for casting interactively controlled visual content | |
| JP2020527883A5 (en) | ||
| CN106162357A (en) | Obtain the method and device of video content | |
| CN115361596A (en) | Panoramic video data processing method and device, electronic device and storage medium | |
| CN109104632A (en) | A kind of realization method and system of television terminal AR scene | |
| CN108933954A (en) | Method of video image processing, set-top box and computer readable storage medium | |
| CN108769755A (en) | High-resolution full view frequency live streaming camera system and method | |
| CN111147883A (en) | Live broadcast method and device, head-mounted display equipment and readable storage medium | |
| CN104010204B (en) | Image information processing method and device | |
| CN114359051A (en) | Image processing method, image processing device, image processing system, and storage medium | |
| CN110661880A (en) | Remote assistance method, system and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |