CN111859159B - Information pushing and video processing method and equipment - Google Patents

Information pushing and video processing method and equipment Download PDF

Info

Publication number
CN111859159B
CN111859159B CN202010777528.5A CN202010777528A CN111859159B CN 111859159 B CN111859159 B CN 111859159B CN 202010777528 A CN202010777528 A CN 202010777528A CN 111859159 B CN111859159 B CN 111859159B
Authority
CN
China
Prior art keywords
article
information
searched
appearance
positioning information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010777528.5A
Other languages
Chinese (zh)
Other versions
CN111859159A (en
Inventor
崔英林
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Lianshang Network Technology Co Ltd
Original Assignee
Shanghai Lianshang Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Lianshang Network Technology Co Ltd filed Critical Shanghai Lianshang Network Technology Co Ltd
Priority to CN202010777528.5A priority Critical patent/CN111859159B/en
Publication of CN111859159A publication Critical patent/CN111859159A/en
Application granted granted Critical
Publication of CN111859159B publication Critical patent/CN111859159B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • G06F16/9535Search customisation based on user profiles and personalisation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/73Querying
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computational Linguistics (AREA)
  • Library & Information Science (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

The embodiment of the application discloses information pushing and video processing methods and equipment. One embodiment of the information pushing method comprises the following steps: performing code stream conversion on the video data to obtain a video stream and positioning information of the appearance article of the video stream; determining positioning information of the searched object in response to receiving an object searching instruction of the user; determining a play time point based on the positioning information of the searched object; jumping to a video frame corresponding to the playing time point to start playing; querying push information of the search object and presenting the push information. According to the embodiment, the positioning information of the article is pre-buried in the video stream, when the article searching instruction of the user is identified, the user quickly jumps to the image of the article to be searched, and the pushing information of the article to be searched is automatically presented, so that the user can review the scene of the article to be searched, the requirement that the user knows the information of the article interested in the user in detail is met, the article interested in the user can be conveniently purchased, and the operation cost of the user is saved.

Description

Information pushing and video processing method and equipment
Technical Field
The embodiment of the application relates to the technical field of computers, in particular to information pushing and video processing methods and equipment.
Background
With the rapid development of the internet, the functions supported by video applications are more and more varied, such as a live broadcast function, an on-demand function, and the like. Thereby attracting more and more users to watch video and longer. Various items such as clothing, decorations, food, etc. often appear in video. If a user wants to review items of interest that were previously present in the video, the video needs to be played from the head until the items of interest are seen. If the user wants to acquire the detailed information of the object of interest, the user needs to run the video application in the background, then open the search application or shopping application, and input the object name for searching.
Disclosure of Invention
The embodiment of the application provides information pushing and video processing methods and equipment.
In a first aspect, an embodiment of the present application provides an information pushing method, including: performing code stream conversion on the video data to obtain a video stream and positioning information of the appearance article of the video stream; determining positioning information of the searched object in response to receiving an object searching instruction of the user; determining a play time point based on the positioning information of the searched object; jumping to a video frame corresponding to the playing time point to start playing; querying push information of the search object and presenting the push information.
In some embodiments, receiving an item search instruction of a user includes: collecting voice information of a user; identifying the voice information and determining the names of the articles contained in the voice information; and if the object name is matched with the appearance object of the video stream, determining the matched appearance object as a search object.
In some embodiments, determining a play time point based on the location information of the search item includes: extracting an appearance video frame and an appearance time point of the searched article from the positioning information of the searched article; determining a slice starting time point at which a video frame appears as a playing time point; and jumping to a video frame corresponding to the playing time point to start playing, including: and jumping to a playing time point to quickly play the slice to an appearance time point.
In some embodiments, extracting the occurrence video frame and the occurrence time point of the search item from the positioning information of the search item includes: inquiring an article identifier corresponding to the article name of the searched article in an article name table, wherein the article name table is used for storing the corresponding relation between the article identifier and the article name; and inquiring the appearance video frame and the appearance time point corresponding to the article identification of the searched article in the positioning information table, wherein the positioning information table is used for storing the corresponding relation between the article identification and the positioning information.
In some embodiments, the method further comprises: if the search object appears in multiple slices of the video stream, jumping to the slice with the earliest start time, or the list presenting multiple slices, and jumping to the slice selected by the user.
In some embodiments, the positioning information includes coordinate information; and presenting the push information, including: based on the coordinate information of the search object, a display area is determined in the appearance video frame, and push information of the search object is displayed in the display area.
In some embodiments, the coordinate information is a percentage coordinate; and determining a display area in the presence video frame based on the coordinate information of the search item, comprising: calculating dot matrix coordinates of the searched objects based on the resolution of the playing device and the percentage coordinates of the searched objects; and setting the area corresponding to the lattice coordinates as a display area.
In some embodiments, calculating lattice coordinates of the search item based on the resolution of the playback device and the percentage coordinates of the search item includes: if the coordinate system of the percentage coordinates is the same as the screen coordinate system of the playing device, correspondingly multiplying the horizontal pixel value and the vertical pixel value of the resolution of the playing device with the horizontal coordinate value and the vertical coordinate value of the percentage coordinates of the searched object to obtain the lattice coordinates of the searched object.
In some embodiments, calculating lattice coordinates of the search object based on the resolution of the playback device and the percentage coordinates of the search object further comprises: if the coordinate system of the percentage coordinates is different from the screen coordinate system of the playing device, converting the coordinate system of the percentage coordinates to obtain converted percentage coordinates under the screen coordinate system; and correspondingly multiplying the horizontal pixel value and the vertical pixel value of the resolution of the playing equipment with the horizontal coordinate value and the vertical coordinate value of the conversion percentage coordinate of the searched article to obtain the dot matrix coordinate of the searched article.
In a second aspect, an embodiment of the present application provides a video processing method, including: carrying out object identification on the video stream, and determining the appearance object of the video stream; acquiring positioning information of the appearance article; and adding the positioning information of the appearance article into a corresponding video frame protocol to generate video data.
In some embodiments, obtaining location information for the presence of an item includes: position identification is carried out on the video stream, and coordinate information of the article is determined; the coordinate information of the present article is added to the positioning information of the present article.
In some embodiments, the position identification of the video stream, the determination of the coordinate information of the presence of the item, comprises: simulating a pilot video stream on pilot equipment; carrying out position identification on the video stream to obtain dot matrix coordinates of the appearing articles; and determining coordinate information of the appearing article based on the lattice coordinates of the appearing article.
In some embodiments, determining the coordinate information of the presence item based on the lattice coordinates of the presence item includes: and correspondingly dividing the horizontal coordinate value and the vertical coordinate value of the dot matrix coordinates of the appearing articles with the horizontal pixel value and the vertical pixel value of the resolution of the test seeding equipment to obtain the percentage coordinates of the appearing articles.
In some embodiments, obtaining location information for the presence of an item includes: determining a slice start time point of an appearance video frame of an appearance object; the slice start time point of the appearance article is added to the positioning information of the appearance article.
In some embodiments, adding the location information of the presence item to the corresponding video frame protocol includes: and expanding network abstraction layer information of a corresponding video frame protocol based on the positioning information of the appearance article.
In a third aspect, an embodiment of the present application provides an information pushing apparatus, including: the code stream conversion unit is configured to perform code stream conversion on the video data to obtain the video stream and the positioning information of the appearance article of the video stream; a slice determination unit configured to determine positioning information of a searched item in response to receiving an item search instruction of a user; a time determination unit configured to determine a play time point based on the positioning information of the search object; the playing jump unit is configured to jump to a video frame corresponding to a playing time point to start playing; and an information presentation unit configured to inquire push information of the search object and present the push information.
In some embodiments, the slice determination unit is further configured to: collecting voice information of a user; identifying the voice information and determining the names of the articles contained in the voice information; and if the object name is matched with the appearance object of the video stream, determining the matched appearance object as a search object.
In some embodiments, the time determination unit comprises: an extraction subunit configured to extract an appearance video frame and an appearance time point of the search item from the positioning information of the search item; a determination subunit configured to determine a slice start time point at which the video frame appears as a play time point; and the skip play unit is further configured to: and jumping to a playing time point to quickly play the slice to an appearance time point.
In some embodiments, the extraction subunit is further configured to: inquiring an article identifier corresponding to the article name of the searched article in an article name table, wherein the article name table is used for storing the corresponding relation between the article identifier and the article name; and inquiring the appearance video frame and the appearance time point corresponding to the article identification of the searched article in the positioning information table, wherein the positioning information table is used for storing the corresponding relation between the article identification and the positioning information.
In some embodiments, the apparatus further comprises: and a slice skipping unit configured to skip to a slice with the earliest start time or to list the plurality of slices and skip to a slice selected by the user if the search object appears in the plurality of slices of the video stream.
In some embodiments, the positioning information includes coordinate information; the information presentation unit includes: an information presentation subunit configured to determine a display area in the presence video frame based on the coordinate information of the search item, and to display push information of the search item in the display area.
In some embodiments, the coordinate information is a percentage coordinate; the information presentation subunit includes: a calculation module configured to calculate lattice coordinates of the search object based on the resolution of the playback device and the percentage coordinates of the search object; and the setting module is configured to set the area corresponding to the lattice coordinates as a display area.
In some embodiments, the computing module is further configured to: if the coordinate system of the percentage coordinates is the same as the screen coordinate system of the playing device, correspondingly multiplying the horizontal pixel value and the vertical pixel value of the resolution of the playing device with the horizontal coordinate value and the vertical coordinate value of the percentage coordinates of the searched object to obtain the lattice coordinates of the searched object.
In some embodiments, the computing module is further configured to: if the coordinate system of the percentage coordinates is different from the screen coordinate system of the playing device, converting the coordinate system of the percentage coordinates to obtain converted percentage coordinates under the screen coordinate system; and correspondingly multiplying the horizontal pixel value and the vertical pixel value of the resolution of the playing equipment with the horizontal coordinate value and the vertical coordinate value of the conversion percentage coordinate of the searched article to obtain the dot matrix coordinate of the searched article.
In a fourth aspect, an embodiment of the present application provides a video processing apparatus, including: the determining unit is configured to identify the articles of the video stream and determine the articles of the video stream; an acquisition unit configured to acquire positioning information of an appearance article; and the adding unit is configured to add the positioning information of the appearance article to the corresponding video frame protocol and generate video data.
In some embodiments, the acquisition unit comprises: a determining subunit configured to perform position identification on the video stream, and determine coordinate information of the appearing article; an adding subunit configured to add the coordinate information of the presence item to the positioning information of the presence item.
In some embodiments, determining the subunit comprises: a pilot-on module configured to simulate a pilot-on video stream on a pilot-on device; the identification module is configured to identify the position of the video stream to obtain dot matrix coordinates of the appearing article; and a determining module configured to determine coordinate information of the presence item based on the lattice coordinates of the presence item.
In some embodiments, the determination module is further configured to: and correspondingly dividing the horizontal coordinate value and the vertical coordinate value of the dot matrix coordinates of the appearing articles with the horizontal pixel value and the vertical pixel value of the resolution of the test seeding equipment to obtain the percentage coordinates of the appearing articles.
In some embodiments, the acquisition unit is further configured to: determining a slice start time point of an appearance video frame of an appearance object; the slice start time point of the appearance article is added to the positioning information of the appearance article.
In some embodiments, the adding unit is further configured to: and expanding network abstraction layer information of a corresponding video frame protocol based on the positioning information of the appearance article.
In a fifth aspect, embodiments of the present application provide a computer apparatus, comprising: one or more processors; a storage device having one or more programs stored thereon; the one or more programs, when executed by the one or more processors, cause the one or more processors to implement the method as described in any of the implementations of the first aspect or to implement the method as described in any of the implementations of the second aspect.
In a sixth aspect, embodiments of the present application provide a computer readable medium having stored thereon a computer program which, when executed by a processor, implements a method as described in any of the implementations of the first aspect or implements a method as described in any of the implementations of the second aspect.
The information pushing and video processing method and device provided by the embodiment of the application firstly perform code stream conversion on video data to obtain video streams and positioning information of the appearing articles of the video streams; then, in response to receiving an article searching instruction of a user, determining positioning information of the searched article; then determining a playing time point based on the positioning information of the searched object; and finally, jumping to a video frame corresponding to the playing time point to start playing, inquiring pushing information of the searched object, and presenting the pushing information. Through pre-burying the positioning information of the article in the video stream, when the article searching instruction of the user is identified, the user quickly jumps to the image of the article to be searched, and the pushing information of the article to be searched is automatically presented, so that the user can review the appearance scene of the article to be searched, the requirement that the user knows the information of the article interested in the user in detail is met, the article interested in the user can be conveniently purchased, and the operation cost of the user is saved.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments, made with reference to the accompanying drawings in which:
FIG. 1 is an exemplary system architecture in which the present application may be applied;
FIG. 2 is a flow chart of one embodiment of an information push method according to the present application;
FIG. 3 is a flow chart of yet another embodiment of an information push method according to the present application;
FIG. 4 is a flow chart of one embodiment of a video processing method according to the present application;
FIG. 5 is a schematic diagram of a computer system suitable for use in implementing embodiments of the present application.
Detailed Description
The application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the application and are not limiting of the application. It should be noted that, for convenience of description, only the portions related to the present application are shown in the drawings.
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. The application will be described in detail below with reference to the drawings in connection with embodiments.
Fig. 1 illustrates an exemplary system architecture 100 to which an embodiment of an information push, video processing method of the present application may be applied.
As shown in fig. 1, devices 101, 102 and network 103 may be included in system architecture 100. The network 103 is used to provide a medium for communication links between the devices 101, 102. The network 103 may include various connection types, such as wired, wireless communication links, or fiber optic cables, among others.
The devices 101, 102 may be hardware devices or software that support network connections to provide various network services. When the device is hardware, it may be a variety of electronic devices including, but not limited to, smartphones, tablets, laptop portable computers, desktop computers, servers, and the like. In this case, the hardware device may be realized as a distributed device group composed of a plurality of devices, or may be realized as a single device. When the device is software, it can be installed in the above-listed electronic device. In this case, as software, it may be implemented as a plurality of software or software modules for providing distributed services, for example, or as a single software or software module. The present invention is not particularly limited herein.
In practice, a device may provide a corresponding network service by installing a corresponding client application or server application. After the device has installed the client application, it may be embodied as a client in network communication. Accordingly, after the server application is installed, it may be embodied as a server in network communications.
By way of example, in fig. 1, device 101 is embodied as a client and device 102 is embodied as a server. For example, device 101 may be a client of a video class application and device 102 may be a server of the video class application.
It should be noted that, the information pushing method and the video processing method provided by the embodiments of the present application may be performed by the device 101. When the device 101 performs the information push method, it may be a playback device. When the device 102 performs the video processing method, it may be a trial-and-error device.
It should be understood that the number of networks and devices in fig. 1 is merely illustrative. There may be any number of networks and devices as desired for an implementation.
With continued reference to fig. 2, a flow 200 of one embodiment of an information push method according to the present application is shown. The information pushing method comprises the following steps:
Step 201, performing code stream conversion on the video data to obtain the video stream and the positioning information of the appearance article of the video stream.
In this embodiment, the execution body of the information pushing method (for example, the device 101 shown in fig. 1) may acquire video data from a background server (for example, the device 102 shown in fig. 1) of the video application, and perform code stream conversion on the video data to obtain a video stream and positioning information of the appearance article of the video stream.
The video data may include, among other things, a video stream and location information of the presence item of the video stream. Video streams are playable data including, but not limited to, television shows, movies, live broadcasts, short videos, and the like. The positioning information of the appearing items of the video stream is non-playable data for positioning the items appearing in the video stream, including but not limited to item names, item identifications, slice identifications to which the video frames of the appearing items belong, item appearance time points, slice start time points, and the like. The presence item may be an item that appears in the video stream, such as clothing, decorations, food, and the like. The slice to which the video frame in which the item appears belongs may comprise a series of consecutive video frames in the video stream, with the item appearing in at least one of the video frames.
For video frames in the video stream, not all the articles appear in each frame of video frame, but also not all the articles appear, so that the code stream coding processing is only carried out on the video frames where the articles with the positioning information are positioned, the video frame protocol is modified, and the positioning information is added in the original video frame protocol. The video data is added with the unplayable data and cannot be directly played, so that the video data is required to be subjected to code stream conversion, and the playable video stream and the unplayable positioning information are separated. The code stream conversion can adopt a static transcoding mode or a dynamic transcoding mode.
When the video frame protocol is modified, the modification modes are different for different protocol formats. Taking h.264 as an example, adding positioning information is supported by extending NAL (Network Abstraction Layer ) information of the video frame protocol. NAL may include NAL HEADER, NAL Extension, and NAL payload, among others. NAL HEADER may be used to store basic information of the video frames. NAL payload may be used to store a binary stream of video frames. NAL Extension may be used to store positioning information. NAL Extension is inserted into a video frame while not changing the data stored by the NAL payload to form a new video frame.
It should be noted that, the playback device typically plays the video stream while performing the transcoding. Therefore, in the video stream playing process, the positioning information of the appearing articles of the video stream can be obtained successively. Also, in order to facilitate searching, the obtained positioning information may be stored in a positioning information table.
Step 202, determining positioning information of the searched object in response to receiving an object searching instruction of the user.
In this embodiment, the user may issue an article search instruction to the execution body described above. In the case of receiving an item search instruction from a user, the execution body may determine positioning information for searching for an item.
In general, the user may issue an item search instruction during playback of the video stream, or may issue an item search instruction during playback of the video stream for the first time.
And when the video stream is played for the first time, the positioning information table can store the positioning information of all the articles in the video stream. Therefore, in the process of replaying the video stream, the item searched by the user may be the appearance item of the replayed video frame or the appearance item of the video frame which is not replayed yet.
For a user to issue an item search instruction in the process of playing a video stream for the first time, positioning information of an item appearing in a video frame that has been played in the process of playing the video stream for the first time may be stored in a positioning information table. Thus, the item searched by the user may only be the presence item of the video frame that has been played.
Generally, a user may issue an item search instruction in a variety of ways. Wherein, the more convenient way is to send out the article searching instruction by speaking the name of the searched article. At this time, the executing body may collect voice information of the user, identify the voice information, and determine the article name contained in the voice information. And if the object name is matched with the appearance object of the video stream, determining the matched appearance object as a search object. If the object name is not matched with the object appearing in the video stream, continuing to collect the voice information of the user. Since the positioning information of the appearance objects of the played video frames are stored in the positioning information table, the positioning information table generally stores the positioning information of a plurality of appearance objects. The presence item that matches the item name contained in the user's voice information is the user's search item. For example, the user speaks "watch", and the presence items corresponding to the location information table include a watch of brand a, clothing of brand B, and shoes of brand C, only the watch of brand a matches "watch", which is the user's search item.
Step 203, determining a play time point based on the positioning information of the searched object.
In this embodiment, the executing body may determine the play time point based on the positioning information of the search object.
In general, in the case where each frame of video frame is a complete map, the execution subject may take the occurrence time point of the search object contained in the positioning information as a play time point. However, due to coding factors, most video frames are not necessarily complete maps. In this case, the start time point of the slice to which the video frame of the search object included in the positioning information belongs needs to be set as the play time point. Wherein the first frame video frame in the slice is encoded based on the full map, and the subsequent video frames are encoded based on the change information of the earlier video frame. For example, if the second frame of video frame is added by one item from the first frame of video frame, only the added item information is encoded.
Step 204, jumping to the video frame corresponding to the playing time point to start playing.
In this embodiment, the executing body may jump to the video frame corresponding to the playing time point to start playing.
In general, in the case where the play time point is the occurrence time point of the search item, the first frame video frame skipped to may present the search item. When the playing time point is the starting time point of the slice to which the video frame of the searched article belongs, the first frame of the video frame jumped to does not present the searched article, and the searched article can be presented when the playing time point of the searched article is reached.
Step 205, query the push information of the search object and present the push information.
In this embodiment, the executing body may query the pushing information of the search object and present the pushing information.
In general, the executing body may query the push information of the search object through various approaches. For example, in the case of locally storing push information of a large number of items, the push information of the search item is locally searched. For another example, when the video application integrates the search function or the shopping function, a push information acquisition request for searching for an item is sent to a background server of the video application, and push information for searching for an item returned from the background server of the video application is received. For another example, a push information acquisition request of searching for an item is sent to a background server of the search application or shopping application, and push information of the search item returned by the background server of the search application or shopping application is received.
The pushing information may be a link for the user to browse the detailed information of the search item or a link for purchasing the search item. In general, push information may be presented on a currently playing video frame, particularly in the vicinity of a search item. The user may then perform a corresponding operation based on the push information to view the details of the search item or purchase the search item.
In some embodiments, in the case where the positioning information includes coordinate information, the executing body may determine a display area in the presence video frame based on the coordinate information of the search object, and display push information of the search object in the display area. The region corresponding to the coordinate information may be a display region.
According to the information pushing method provided by the embodiment of the application, firstly, code stream conversion is carried out on video data to obtain video streams and positioning information of the video streams on the appearance objects; then, in response to receiving an article searching instruction of a user, determining positioning information of the searched article; then determining a playing time point based on the positioning information of the searched object; and finally, jumping to a video frame corresponding to the playing time point to start playing, inquiring pushing information of the searched object, and presenting the pushing information. Through pre-burying the positioning information of the article in the video stream, when the article searching instruction of the user is identified, the user quickly jumps to the image of the article to be searched, and the pushing information of the article to be searched is automatically presented, so that the user can review the appearance scene of the article to be searched, the requirement that the user knows the information of the article interested in the user in detail is met, the article interested in the user can be conveniently purchased, and the operation cost of the user is saved.
With further reference to fig. 3, a flow 300 is shown that is yet another embodiment of an information push method according to the present application. The information pushing method comprises the following steps:
Step 301, performing code stream conversion on the video data to obtain the video stream and the positioning information of the appearance article of the video stream.
In response to receiving the item search instruction from the user, location information for searching for the item is determined 302.
In this embodiment, the specific operations of steps 301 to 302 are described in detail in steps 201 to 202 in the embodiment shown in fig. 2, and are not described herein.
Step 303, extracting the appearance video frame and appearance time point of the searched article from the positioning information of the searched article.
In this embodiment, the execution subject of the information push method (for example, the apparatus 101 shown in fig. 1) may extract the appearance video frame and the appearance time point of the search object from the positioning information of the search object.
In some embodiments, in the case that the positioning information includes an item name, an appearance video frame of the item, and an appearance time point, the execution entity may query the positioning information table for the appearance video frame and the appearance time point of the search item directly based on the name of the search item. The positioning information table may be used to store positioning information of the appearance article of the video stream obtained by performing code stream conversion on the video data.
In some embodiments, where the location information does not include the name of the item, the user cannot directly query the location information table based on the name of the searched item. To facilitate voice queries by the user, an item name table is established. Wherein the item name table may be used to store correspondence between item identifications and item names. The location information table may be used to store correspondence between item identifications and location information. The two are associated by the item identification. Specifically, the execution body may first query, in the item name table, an item identifier corresponding to an item name of the searched item; and then searching the appearance video frame and the appearance time point corresponding to the object identification of the searched object in the positioning information table. For ease of retrieval, the same items appearing in different slices may be pooled for storage, as shown in tables 1 and 2. Wherein table 1 shows a positioning information table and table 2 shows an article name table.
TABLE 1
TABLE 2
In step 304, the slice start time point at which the video frame appears is determined as the play time point.
In this embodiment, the execution body may determine a slice start time point at which a video frame appears as the play time point.
Step 305, jump to the play time point to fast play the slice to the appearance time point.
In this embodiment, the execution body may jump to the play time point to play the slice fast to the appearance time point. Generally, the playback speed from the point in time when the slice begins to the point in time when the slice occurs is relatively fast in order to quickly present the search item. And, starting playing from the slice start time point, the picture of the search object presented to the user can be ensured to be complete.
In some embodiments, where the search object appears in multiple slices of the video stream, the execution entity may prefer to jump to the slice with the earliest start time, or may list the multiple slices, and jump to the slice selected by the user.
Step 306, query the pushing information of the search object.
In this embodiment, the specific operation of step 306 is described in detail in step 205 in the embodiment shown in fig. 2, and will not be described herein.
Step 307, calculating the lattice coordinates of the searched objects based on the resolution of the playing device and the percentage coordinates of the searched objects.
In the present embodiment, in the case where the coordinate information is a percentage coordinate, the above-described execution subject may calculate the lattice coordinate of the search object based on the resolution of the playback apparatus and the percentage coordinate of the search object.
Since different playback devices have different screen resolutions, the coordinate information in the positioning information is a percentage coordinate in order to accommodate the different screen resolutions. And lattice coordinates are required for determining the display area, so that the percentage coordinates need to be converted into corresponding lattice coordinates. Specifically, the execution body may multiply the horizontal pixel value and the vertical pixel value of the resolution of the playback apparatus with the horizontal coordinate value and the vertical coordinate value of the percentage coordinate of the search object, to obtain the dot matrix coordinate of the search object.
For example, a video stream is played using a playback device having a resolution of a×b. If the percentage coordinates of the searched object are (x/a, y/B), the lattice coordinates of the searched object are (x a, y x B/B). Wherein a, B, A and B are positive integers, x is a positive integer not greater than a, y is a positive integer not greater than B, x/a and y/B are positive integers not greater than 1, x is a/a and y is B/B is a positive integer.
In general, the coordinate system of the percentage coordinates is the same as the screen coordinate system of the playback device, and is positive with the upper left corner as the origin, the right as the horizontal axis, and the downward as the vertical axis. At this time, the horizontal pixel value and the vertical pixel value of the resolution of the playing device are directly multiplied by the horizontal coordinate value and the vertical coordinate value of the percentage coordinate of the searched object, so that the lattice coordinate of the searched object can be obtained. Under special conditions, if the coordinate system of the percentage coordinates is different from the screen coordinate system of the playing device, the coordinate system of the percentage coordinates needs to be converted first to obtain converted percentage coordinates under the screen coordinate system; and correspondingly multiplying the horizontal pixel value and the vertical pixel value of the resolution of the playing equipment with the horizontal coordinate value and the vertical coordinate value of the conversion percentage coordinate of the searched article to obtain the dot matrix coordinate of the searched article.
Step 308, setting the area corresponding to the lattice coordinates as a display area, and displaying push information of the searched objects in the display area.
In this embodiment, the execution body may set an area corresponding to the lattice coordinates as a display area, and display push information of the search object in the display area. In this way, the push information is presented in the vicinity of the search object.
As can be seen from fig. 3, compared with the embodiment corresponding to fig. 2, the procedure 300 of the information push method in this embodiment highlights the step of determining the play time point and the step of determining the display area. Therefore, the scheme described in the embodiment starts playing from the slice starting time point, and the picture of the searched object presented to the user can be ensured to be complete. And the coordinate information in the positioning information is a percentage coordinate, and the corresponding lattice coordinate is obtained through coordinate conversion, so that the method is suitable for different screen resolutions of different playing devices.
With continued reference to fig. 4, a flow 400 of one embodiment of a video processing method according to the present application is shown. The video processing method comprises the following steps:
Step 401, identifying objects in the video stream, and determining the appearance objects in the video stream.
In this embodiment, the execution subject of the video processing method (e.g., the apparatus 101 shown in fig. 1) may perform item identification on the video stream to determine the presence item of the video stream.
In general, the executing entity may determine the presence item of the video stream in a variety of ways. In some embodiments, a person skilled in the art may perform item identification on the video stream, and input the identification result to the execution body. In some embodiments, the executing entity may split the video stream into a series of video frames and perform item identification on each frame of video frame to determine the presence item of the video stream.
Step 402, obtaining location information of the presence item.
In this embodiment, the executing body may acquire the positioning information of the present article. Wherein the positioning information of the appearing article is non-playable data for positioning the appearing article in the video stream.
In some embodiments, the positioning information may include coordinate information. Specifically, the executing body can perform position identification on the video stream to determine coordinate information of the appearing object; the coordinate information of the present article is added to the positioning information of the present article. Wherein the coordinate information may be determined by simulating the pilot video stream on the pilot device. Specifically, a pilot video stream is simulated on a first pilot device; then, carrying out position identification on the video stream to obtain dot matrix coordinates of the appearing article; and finally, determining the coordinate information of the appearing article based on the dot matrix coordinates of the appearing article.
In general, in the case where screen resolutions of most playback devices and trial playback devices are unified, the coordinate information may be lattice coordinates. However, in practical applications, different playback devices have different screen resolutions, and in order to accommodate the different screen resolutions, the coordinate information in the positioning information is a percentage coordinate. Specifically, the horizontal coordinate value and the vertical coordinate value of the dot matrix coordinates of the appearing object are correspondingly divided with the horizontal pixel value and the vertical pixel value of the resolution of the test seeding device, and the percentage coordinates of the appearing object can be obtained.
For example, a video stream is on-demand using a standard device with resolution a x b, and if the lattice coordinates of the emerging items captured on the on-demand device are (x, y), then the percentage coordinates of the emerging items are (x/a, y/b). Wherein a and b are positive integers, x is a positive integer not greater than a, y is a positive integer not greater than b, and x/a and y/b are positive integers not greater than 1.
It should be noted that the selection of the resolution of the trial playing device needs to be matched with the video resolution, for example, the selection above 720p is 16:9, and the following selection is 4:3. In this way, errors can be reduced as much as possible.
In some embodiments, the executing entity may further determine a slice start time point of the appearance video frame of the appearance article, and add the slice start time point of the appearance article to the positioning information of the appearance article.
And step 403, adding the positioning information of the appearance article to a corresponding video frame protocol to generate video data.
In this embodiment, the executing body may add the positioning information of the appearance article to the corresponding video frame protocol to generate the video data.
In general, the video frame protocol is modified by performing code stream encoding processing on the video frame where the article with the positioning information is located, so that the positioning information can be added in the original video frame protocol. When the video frame protocol is modified, the modification modes are different for different protocol formats. Taking h.264 as an example, adding positioning information is supported by expanding NAL information of a corresponding video frame protocol based on positioning information of an appearance article. NAL may include NAL HEADER, NAL Extension, and NAL payload, among others. NAL HEADER may be used to store basic information of the video frames. NAL payload may be used to store a binary stream of video frames. NAL Extension may be used to store positioning information. NAL Extension is inserted into a video frame while not changing the data stored by the NAL payload to form a new video frame.
The video processing method provided by the embodiment of the application comprises the steps of firstly, carrying out object identification on a video stream and determining the appearance object of the video stream; then obtaining the positioning information of the appearing article; and finally, adding the positioning information of the appearing article to a corresponding video frame protocol to generate video data, thereby realizing the addition of non-playable data in the video stream.
Referring now to FIG. 5, there is illustrated a schematic diagram of a computer system 500 suitable for use in a computer device (e.g., device 101 of FIG. 1) for implementing embodiments of the present application. The computer device shown in fig. 5 is only an example and should not be construed as limiting the functionality and scope of use of embodiments of the application.
As shown in fig. 5, the computer system 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage section 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, ROM 502, and RAM 503 are connected to each other through a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
The following components are connected to the I/O interface 505: an input section 506 including a keyboard, a mouse, and the like; an output portion 507 including a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker, and the like; a storage portion 508 including a hard disk and the like; and a communication section 509 including a network interface card such as a LAN card, a modem, or the like. The communication section 509 performs communication processing via a network such as the internet. The drive 510 is also connected to the I/O interface 505 as needed. A removable medium 511 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 510 as needed so that a computer program read therefrom is mounted into the storage section 508 as needed.
In particular, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flowcharts. In such an embodiment, the computer program may be downloaded and installed from a network via the communication portion 509, and/or installed from the removable media 511. The above-described functions defined in the method of the present application are performed when the computer program is executed by a Central Processing Unit (CPU) 501.
The computer readable medium according to the present application may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the context of this document, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present application, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.
Computer program code for carrying out operations of the present application may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or electronic device. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented in software or in hardware. The described units may also be provided in a processor, for example, described as: a processor includes a code stream conversion unit, a slice determination unit, a time determination unit, a play jump unit, and an information presentation unit. The names of these units do not in each case limit the unit itself, for example, the code stream conversion unit may also be described as "a unit for code stream converting video data, obtaining a video stream and positioning information of the appearance of the video stream". As another example, it can be described as: a processor includes a determination unit, an acquisition unit, and an addition unit. The names of these units do not in each case constitute a limitation of the unit itself, for example the determination unit may also be described as "unit for item identification of a video stream, determining the presence of items of the video stream".
As another aspect, the present application also provides a computer-readable medium that may be contained in the computer device described in the above embodiment; or may exist alone without being assembled into the computer device. The computer readable medium carries one or more programs which, when executed by the computer device, cause the computer device to: performing code stream conversion on the video data to obtain a video stream and positioning information of the appearance article of the video stream; determining positioning information of the searched object in response to receiving an object searching instruction of the user; determining a play time point based on the positioning information of the searched object; jumping to a video frame corresponding to the playing time point to start playing; querying push information of the search object and presenting the push information. Or cause the computer device to: carrying out object identification on the video stream, and determining the appearance object of the video stream; acquiring positioning information of the appearance article; and adding the positioning information of the appearance article into a corresponding video frame protocol to generate video data.
The above description is only illustrative of the preferred embodiments of the present application and of the principles of the technology employed. It will be appreciated by persons skilled in the art that the scope of the application referred to in the present application is not limited to the specific combinations of the technical features described above, but also covers other technical features formed by any combination of the technical features described above or their equivalents without departing from the inventive concept described above. Such as the above-mentioned features and the technical features disclosed in the present application (but not limited to) having similar functions are replaced with each other.

Claims (12)

1. An information pushing method, comprising:
Performing code stream conversion on video data to obtain a video stream and positioning information of the appearance article of the video stream;
Determining positioning information of the searched object in response to receiving an object searching instruction of the user;
determining a playing time point based on the positioning information of the searched object;
Jumping to the video frame corresponding to the playing time point to start playing;
Inquiring push information of the searched article and presenting the push information;
wherein, the determining a playing time point based on the positioning information of the searched article includes:
Inquiring an article identifier corresponding to the article name of the searched article in an article name table, wherein the article name table is used for storing the corresponding relation between the article identifier and the article name;
Inquiring an appearance video frame and an appearance time point corresponding to the article identification of the searched article in a positioning information table, wherein the positioning information table is used for storing the corresponding relation between the article identification and the positioning information;
And determining the slice starting time point of the video frame as the playing time point.
2. The method of claim 1, wherein the receiving the item search instruction of the user comprises:
collecting voice information of the user;
identifying the voice information and determining the names of the articles contained in the voice information;
and if the object name is matched with the appearance object of the video stream, determining the matched appearance object as the searching object.
3. The method of claim 1, wherein the jumping to the video frame corresponding to the play time point starts playing, comprising:
and jumping to the playing time point to quickly play the slice to the occurrence time point.
4. A method according to claim 3, wherein the method further comprises:
if the search object appears in a plurality of slices of the video stream, jumping to the slice with the earliest starting time, or presenting the plurality of slices in a list, and jumping to the slice selected by the user.
5. The method of claim 1, wherein the positioning information comprises coordinate information; and
The presenting the push information includes:
And determining a display area in the appearance video frame based on the coordinate information of the searched object, and displaying pushing information of the searched object in the display area.
6. The method of claim 5, wherein the coordinate information is a percentage coordinate; and
The determining a display area in the appearance video frame based on the coordinate information of the searched object comprises the following steps:
calculating dot matrix coordinates of the searched objects based on the resolution of the playing device and the percentage coordinates of the searched objects;
and setting the area corresponding to the lattice coordinates as the display area.
7. The method of claim 6, wherein the calculating lattice coordinates of the search item based on the resolution of the playback device and the percentage coordinates of the search item comprises:
And if the coordinate system of the percentage coordinates is the same as the screen coordinate system of the playing equipment, correspondingly multiplying the horizontal pixel value and the vertical pixel value of the resolution of the playing equipment with the horizontal coordinate value and the vertical coordinate value of the percentage coordinates of the searched article to obtain the lattice coordinates of the searched article.
8. The method of claim 7, wherein the calculating lattice coordinates of the search item based on the resolution of the playback device and the percentage coordinates of the search item further comprises:
If the coordinate system of the percentage coordinates is different from the screen coordinate system of the playing device, converting the coordinate system of the percentage coordinates to obtain converted percentage coordinates under the screen coordinate system;
And correspondingly multiplying the horizontal pixel value and the vertical pixel value of the resolution of the playing equipment with the horizontal coordinate value and the vertical coordinate value of the conversion percentage coordinate of the searched article to obtain the dot matrix coordinate of the searched article.
9. A video processing method, comprising:
carrying out object identification on a video stream, and determining the appearance object of the video stream;
acquiring positioning information of the appearing article;
adding the positioning information of the appearing article to a corresponding video frame protocol to generate video data;
Wherein, the obtaining the positioning information of the appearance article includes:
Simulating the video stream on a trial broadcasting device; performing position identification on the video stream to obtain dot matrix coordinates of the appearing object; correspondingly dividing the horizontal coordinate value and the vertical coordinate value of the dot matrix coordinate of the appearing article by the horizontal pixel value and the vertical pixel value of the resolution of the test seeding equipment to obtain the percentage coordinate of the appearing article; adding the coordinate information of the appearance article to the positioning information of the appearance article; or alternatively
Determining a slice start time point of an appearance video frame of the appearance object; and adding the slice starting time point of the appearance article into the positioning information of the appearance article.
10. The method of claim 9, wherein the adding the location information of the presence item to a corresponding video frame protocol comprises:
And expanding network abstraction layer information of a corresponding video frame protocol based on the positioning information of the appearance article.
11. A computer device, comprising:
one or more processors;
A storage device on which one or more programs are stored;
When executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-8 or the method of any of claims 9-10.
12. A computer readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of any of claims 1-8 or implements the method of any of claims 9-10.
CN202010777528.5A 2020-08-05 2020-08-05 Information pushing and video processing method and equipment Active CN111859159B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010777528.5A CN111859159B (en) 2020-08-05 2020-08-05 Information pushing and video processing method and equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010777528.5A CN111859159B (en) 2020-08-05 2020-08-05 Information pushing and video processing method and equipment

Publications (2)

Publication Number Publication Date
CN111859159A CN111859159A (en) 2020-10-30
CN111859159B true CN111859159B (en) 2024-07-05

Family

ID=72971300

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010777528.5A Active CN111859159B (en) 2020-08-05 2020-08-05 Information pushing and video processing method and equipment

Country Status (1)

Country Link
CN (1) CN111859159B (en)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103081488A (en) * 2010-06-29 2013-05-01 高通股份有限公司 Signaling video samples for trick mode video representations
CN105100911A (en) * 2014-05-06 2015-11-25 夏普株式会社 Intelligent multimedia system and method

Family Cites Families (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20080154889A1 (en) * 2006-12-22 2008-06-26 Pfeiffer Silvia Video searching engine and methods
CN107124659A (en) * 2014-04-30 2017-09-01 广州市动景计算机科技有限公司 The output intent and device of a kind of Item Information
CN105681918A (en) * 2015-09-16 2016-06-15 乐视致新电子科技(天津)有限公司 Method and system for presenting article relevant information in video stream
CN105898362A (en) * 2015-11-25 2016-08-24 乐视网信息技术(北京)股份有限公司 Video content retrieval method and device
CN107277558B (en) * 2017-06-19 2020-03-31 网宿科技股份有限公司 Player client, system and method for realizing synchronization of live video
CN110149558B (en) * 2018-08-02 2023-03-28 腾讯科技(深圳)有限公司 Video playing real-time recommendation method and system based on content identification
CN109348275B (en) * 2018-10-30 2021-07-30 百度在线网络技术(北京)有限公司 Video processing method and device
CN109697245A (en) * 2018-12-05 2019-04-30 百度在线网络技术(北京)有限公司 Voice search method and device based on video web page
CN111163367B (en) * 2020-01-08 2023-08-29 百度在线网络技术(北京)有限公司 Information searching method, device, equipment and medium based on playing video

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103081488A (en) * 2010-06-29 2013-05-01 高通股份有限公司 Signaling video samples for trick mode video representations
CN105100911A (en) * 2014-05-06 2015-11-25 夏普株式会社 Intelligent multimedia system and method

Also Published As

Publication number Publication date
CN111859159A (en) 2020-10-30

Similar Documents

Publication Publication Date Title
CN109640129B (en) Video recommendation method and device, client device, server and storage medium
CN105721620B (en) Video information method for pushing and device and video information exhibit method and apparatus
US20150234814A1 (en) Method and apparatus for identifying media content
CN111523566A (en) Target video clip positioning method and device
CN104065979A (en) Method for dynamically displaying information related with video content and system thereof
CN109120949B (en) Video message pushing method, device, equipment and storage medium for video set
WO2017080173A1 (en) Nature information recognition-based push system and method and client
WO2022028177A1 (en) Information pushing method, video processing method, and device
US20230093621A1 (en) Search result display method, readable medium, and terminal device
US20170132267A1 (en) Pushing system and method based on natural information recognition, and a client end
US20130074139A1 (en) Distributed system for linking content of video signals to information sources
US8798437B2 (en) Moving image processing apparatus, computer-readable medium storing thumbnail image generation program, and thumbnail image generation method
JP2001157192A (en) Method and system for providing object information
CN110958470A (en) Multimedia content processing method, device, medium and electronic equipment
CN104918136A (en) Video positioning method and device
CN110930220A (en) Display method, display device, terminal equipment and medium
WO2020042375A1 (en) Method and apparatus for outputting information
CN105828103A (en) Video processing method and player
CN108600780B (en) Method for pushing information, electronic device and computer readable medium
JP2010268103A (en) Client terminal and computer program for moving picture distribution service
CN112784103A (en) Information pushing method and device
CN109241344B (en) Method and apparatus for processing information
CN111859159B (en) Information pushing and video processing method and equipment
CN113298589A (en) Commodity information processing method and device, and information acquisition method and device
CN111784478A (en) Method and apparatus for price comparison of items

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant