WO2021073328A1 - 视频搜索的方法及装置、终端和存储介质 - Google Patents
视频搜索的方法及装置、终端和存储介质 Download PDFInfo
- Publication number
- WO2021073328A1 WO2021073328A1 PCT/CN2020/114799 CN2020114799W WO2021073328A1 WO 2021073328 A1 WO2021073328 A1 WO 2021073328A1 CN 2020114799 W CN2020114799 W CN 2020114799W WO 2021073328 A1 WO2021073328 A1 WO 2021073328A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- target
- searched
- video image
- result
- page
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/78—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
- G06F16/783—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/73—Querying
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/70—Information retrieval; Database structures therefor; File system structures therefor of video data
- G06F16/73—Querying
- G06F16/738—Presentation of query results
Definitions
- the present disclosure relates to the field of computer technology, and in particular to a method and device, terminal and storage medium for video search.
- the current image recognition technology is to transfer the photo to the server after taking a picture, and then the server will identify and search the objects or people in the picture, and then return the search results to the sender. For the time being, there is no implementation scheme for searching based on video.
- the present disclosure provides a video search method and device, terminal and storage medium.
- the present disclosure adopts the following technical solutions.
- the present disclosure provides a video search method, including:
- the first recommendation result corresponding to the first target to be searched is acquired, and the first recommendation result is displayed on a search result page.
- the present disclosure provides a video search device, including:
- the receiving module is used to receive the first event generated when the first control is triggered;
- the acquiring module is used to acquire the current video image frame played in the video playback page when the first event is triggered, the first target to be searched located by the second control in the current video image frame, and the first target to be searched.
- the display module is used to display the first recommendation result on the search result page.
- the present disclosure provides a terminal, including: at least one memory and at least one processor;
- the memory is used to store program code
- the processor is used to call the program code stored in the memory to execute the above method.
- the present disclosure provides a storage medium, the storage medium is used to store program code, and the program code is used to execute the above-mentioned method.
- the video search method provided by the present disclosure can classify and recognize video image frames according to the selected current video content.
- the recognized content includes people and objects displayed in all video images, and can also be people and objects within a selected area.
- the embodiments of the present disclosure can facilitate the user to obtain the commodity and character information in the video.
- Fig. 1 is a flowchart of a video search method according to an embodiment of the present disclosure.
- Fig. 2 is a schematic diagram of a video image frame being played on a video playback page of an embodiment of the present disclosure.
- Fig. 3 is a schematic diagram of a first control transformed into a second control in an embodiment of the present disclosure.
- Fig. 4 is a schematic diagram of a search result page of an embodiment of the present disclosure.
- Fig. 5 is a schematic diagram of a search result page according to another embodiment of the present disclosure.
- Fig. 6 is a schematic diagram of a search result page according to still another embodiment of the present disclosure.
- FIG. 7 is a schematic diagram of replacing the target to be searched in an embodiment of the present disclosure.
- Fig. 8 is a schematic structural diagram of a video search device according to an embodiment of the present disclosure.
- FIG. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present disclosure.
- FIG. 1 is a flowchart of a video search method according to an embodiment of the present disclosure, which includes the following steps.
- S100 In the video playback page, receive a first event generated when a first control is triggered.
- FIG. 2 is a schematic diagram of a video image frame being played on a video playback page of an embodiment of the present disclosure.
- the embodiment of the present disclosure may include the first control as the startup path, for example, it may be a frame selection button or a picture recognition button on the running terminal of the program.
- the difference between the frame selection button and the image recognition button is that clicking the image recognition button can send the automatic recognition instruction information, that is, automatically select the information that is considered valid in a frame; and the frame selection button can be a frame of the image selected by the user Part of the information.
- the acquisition instruction may be issued by clicking the first control as shown in FIG.
- the first control shown is presented in the form of a circular icon.
- the first control may also appear in other forms, and is not limited to the form of a circle or an icon.
- the first event may include that the first control is activated to display in a manner such as a shape change; it may also include, for example, lowering the brightness of the display area other than the first control for display; of course, the above example is only used as
- the embodiments of the present disclosure may also include other methods for activating the display, and are not limited to those described in the previous example.
- the playback is stopped, the playback of the video file can be stopped, and the corresponding acquired video image frame is determined.
- the time stamp of the video playback at the moment of the trigger point is recorded, and the video image frame in the corresponding video file is determined according to the time stamp.
- the second control may include at least one of a floating frame or a floating point.
- the second control includes a floating box and a floating point;
- the first target to be searched may include a first target and a second target.
- the first target positioned by the floating frame and the second target positioned by the floating point in the current video image frame can be respectively obtained;
- the display position of the first target in the current video image frame and the position of the second target The display position in the current video image frame; and the floating frame is displayed at the display position of the first object in the current video image frame, and the floating point is displayed at the display position of the second object in the current video image frame.
- FIG. 3 is a schematic diagram of the first control transformed into the second control in the embodiment of the present disclosure.
- box A and point B are the first target to be searched.
- the first control can become, for example, a rectangle with rounded corners, and at the same time become larger and move in the direction of A, finally forming a rounded corner for positioning the A target Rectangular floating frame.
- the video image frame can also be divided into a background part and a main part, and then the display parameters corresponding to the floating frame can be obtained, which can include the position information and size information of the floating frame, and specifically can include the upper left corner of the floating frame.
- the first pixel is in the X coordinate of the video image frame
- the first pixel in the upper left corner of the floating frame is in the Y coordinate of the video image frame
- the length and width of the floating frame etc.
- the floating frame can be positioned according to the position information and size determined on the video image frame by the above four display parameters.
- the floating frame can be represented by the display start coordinates and the frame length and width, and the limited area of the floating frame can be set.
- the floating frame of this embodiment may also be an irregular shape identified by the boundary, or determine the initial position and stroke path of the floating frame, and obtain the change relative to the change of the abscissa x Function f(x) and so on.
- the embodiment of the present disclosure may not only include one target to be searched, but the first target to be searched may include A and B at the same time; further, the embodiment of the present disclosure does not limit the number of the first target to be searched, more or less
- the first target to be searched for should also be included in the protection scope of the embodiments of the present disclosure; and the second control may not be limited to the form of a floating box or a floating point, and other reasonable forms of the second control can be considered as a simple replacement of the above-mentioned embodiment.
- S400 Acquire a first recommendation result corresponding to the first target to be searched, and display the first recommendation result on a search result page.
- the embodiment of the present disclosure may receive a second event generated by the second control being triggered; and in response to the second event, evoke a search result page; wherein the search result page at least partially covers the current video image frame and obscures the second event.
- Two controls The first result to be recommended corresponding to the first target and the second result to be recommended corresponding to the second target are respectively obtained; the first tag used to identify the first target and the first recommended result are correspondingly displayed in the first search result page Within a region; and correspondingly displaying the second label for identifying the second target and the second recommendation result in the second region in the search result page.
- the embodiment of the present disclosure may include the step of searching the first recommendation result corresponding to the second control when the second control is triggered, and calling out the search result page for displaying the first recommendation result.
- the search result page can be partially or completely covered on the video image frame.
- the related search results of people and objects can be displayed separately, or displayed in tab pages.
- FIG. 4 is a schematic diagram of a search result page of an embodiment of the present disclosure.
- the search result page shown in FIG. 4 includes the first result to be recommended corresponding to the first target A and the second result to be recommended corresponding to the second target B displayed in columns.
- the first target A is a product, and the upper part of the search result page is the same or similar results corresponding to the product A; the second target B is a person, and the lower part of the search result page is a related video result corresponding to the person B.
- the embodiment of the present disclosure may further include a first tag a for identifying the first target A and a second tag b for identifying the second target B.
- the search result page may include, for example, one horizontal column and multiple vertical columns. Wherein, the horizontal column can display tags corresponding to different targets in the video, that is, corresponding to the commodities/persons in sequence.
- the first label a when the first result to be recommended corresponding to the first target A is displayed, the first label a can be slightly enlarged as an identification; and when the second result to be recommended corresponding to the second target B is switched to display, the first result can be restored.
- the vertical columns may respectively correspond to the horizontal columns, and each vertical column may respectively display other videos a related to the character A, and the same and similar items b of the product B in a vertical direction.
- the horizontal column can be a fixed column or a scrolling column.
- each label When there are not many recognition targets, each label can be arranged horizontally; when there are many recognition targets, it can be selectively displayed in a horizontal scroll mode.
- vertical products/videos can also be arranged in a vertical scroll bar.
- the character videos can be sorted, for example, by the click-through rate; and the product category can be arranged according to the similarity with the target product as described above, for example.
- other permutation logic may also be used, as long as a reasonable result is obtained.
- the first results to be recommended may be presented in two columns as shown in the figure, or arranged in other arrangements. The embodiment of the present disclosure does not limit the arrangement of the results.
- the order of arrangement for example, can be based on the similarity score, as a basis for sorting multiple search results, and results with high similarity can be ranked first.
- Other factors that affect rankings may also include, for example, video playback volume, etc., which are not limited here.
- the first target to be searched may also include valid targets and invalid targets.
- all the video image frames in the video played on the video playback page can also be obtained; and the effective target in each video image frame can be obtained to obtain a global result; among them, the effective target includes characters, commodities, and items in the At least one; an invalid target is a target result other than a valid target.
- the embodiments of the present disclosure may classify the first target to be searched by sieving targets by models such as multi-path detection models, which may specifically be commodity detection models, face detection models, and article detection models. Among them, the model may include, for example, facial feature information, product feature information, and so on. Then divide the first target to be searched into at least one category.
- the first target to be searched is a valid target, obtain the first recommended result corresponding to the valid target, and display the first recommended result on the search result page; if the first target to be searched is an invalid target, display None on the search results page Results; or if the first target to be searched is a valid target, obtain the first recommended result corresponding to the valid target, and sequentially display the first recommended result and the global results other than the first recommended result in the search results page; such as the first The target to be searched is an invalid target, and the global result is displayed on the search result page.
- the acquisition of the first recommendation result in the embodiment of the present disclosure may include online acquisition and offline acquisition.
- the first recommendation result may be data stored on the server or locally, such as item introduction, purchase link, or person-related video, or other related information.
- the returned result can be the recognized area coordinates and the recognized effective target. If the server does not have the relevant cache of the video file, the server can perform online recognition in real time, compare features with the products in the server, for example, the products in the product library, and then return the same or similar product results; if the relevant content is identified as a person, it can be extracted For example, the face information is used to search for other videos containing the image or name of the person by video feature.
- the relevant results can be transmitted through the client interface.
- the client for example, a program installed on a mobile phone
- it can also directly call its own product library according to the recognition result; further, an external link can also be set in the own product library, which can be accessed through the external link.
- the link jumps to, for example, applets, other apps, or web pages.
- the offline acquisition method includes acquiring the result of the current video image frame and returning the global search result at the same time. Among them, the online acquisition method may not be able to obtain effective results, that is, the product or character information cannot be recognized. Therefore, the user interface displays the prompt information as shown in FIG. 5.
- FIG. 6 is a schematic diagram of a search result page according to still another embodiment of the present disclosure. In Figure 6, after the current results a and b, one of the global results c is also provided.
- this embodiment can also set tags, which can not only help classification, but also serve as a basis for de-duplication, such as comparing the results of the same tag in the global results to determine whether they belong to the same item/person, or similar items / Similar people.
- the former can return only one result as the result of deduplication; the latter can be classified into one category if it is similar.
- the present disclosure may also include the step of re-determining the target to be searched.
- the search result page is moved to the edge of the current video image frame and part of the search result page is hidden to display the second control. Detect whether the first display position of the second control is changed, if it changes, obtain the second display position of the second control; determine the corresponding second target to be searched according to the second display position; obtain the second target to be searched Corresponding second recommendation result; and then evoke the search result page, and replace the first recommendation result in the search result page with the second recommendation result.
- FIG. 7 is a schematic diagram of changing the target to be searched in an embodiment of the present disclosure.
- the search result page is partially hidden under the current video image frame.
- the search result page can also be folded in other ways, which is not limited here. Move the position of the floating frame from A to G, and/or move the position of the floating point from B to H, thereby obtaining the second target to be searched.
- the step of obtaining and displaying the second recommendation result from the second target to be searched may refer to the step of obtaining and displaying the first recommendation result from the first target to be searched, which will not be repeated here.
- the embodiments of the present disclosure may also include a risk control model, which may specifically be a shielding mechanism provided with filtering features.
- a risk control model which may specifically be a shielding mechanism provided with filtering features.
- the filtering criteria can be violent shots, exposed images, or masking when the recognized facial features involve privacy. More specifically, it can be that the face detection model only includes the celebrity atlas, and when the detection result is related to the face When the detection model does not match, the filter feature is added, and the result is neither detection nor search, so personal privacy can be protected.
- the filtering criteria of the embodiments of the present disclosure are not limited to the above, and any reasonable filtering evaluation criteria can be included in the embodiments of the present disclosure.
- the video search method provided by the embodiments of the present disclosure can be used for online detection or offline detection.
- the detection and search results can be linked with the commodity library, which can help merchants better arrange consumption stock and consumption increase, and optimize product structure.
- the video server it can also play a role as a hot guide and a flow guide, so that content providers can arrange content services more reasonably.
- a content ranking strategy can be established based on the returned results, and a rule generator can be automatically established based on, for example, the amount of likes and content quality.
- an embodiment of the present disclosure also provides a video search device 10, which may include a receiving module 30, an acquiring module 50, and a display module 70.
- the receiving module 30 can be used to receive the first event generated when the first control is triggered.
- the acquiring module 50 can be used to acquire the current video image frame played in the video playback page when the first event is triggered, the first target to be searched located by the second control in the current video image frame, and the first target to be searched in the current video image.
- the display module 70 can be used to display the first recommendation result on the search result page.
- the relevant part can refer to the part of the description of the method embodiment.
- the device embodiments described above are merely illustrative, and the modules described as separate modules may or may not be separate. Some or all of the modules can be selected according to actual needs to achieve the objectives of the solutions of the embodiments. Those of ordinary skill in the art can understand and implement it without creative work.
- the visual detection and search method and device of the present disclosure have been described based on the embodiments and application examples.
- the present disclosure also provides a terminal and storage medium, which are described below.
- FIG. 9 shows a schematic structural diagram of an electronic device (such as a terminal device or a server) 800 suitable for implementing the embodiments of the present disclosure.
- Terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablets), PMPs (portable multimedia players), vehicle-mounted terminals (e.g. Mobile terminals such as car navigation terminals) and fixed terminals such as digital TVs, desktop computers, etc.
- the electronic device shown in FIG. 9 is only an example, and should not bring any limitation to the function and scope of use of the embodiments of the present disclosure.
- the electronic device 800 may include a processing device (such as a central processing unit, a graphics processor, etc.) 801, which can be loaded into a random access device according to a program stored in a read-only memory (ROM) 802 or loaded from a storage device 808.
- the program in the memory (RAM) 803 executes various appropriate actions and processing.
- various programs and data required for the operation of the electronic device 800 are also stored.
- the processing device 801, the ROM 802, and the RAM 803 are connected to each other through a bus 804.
- An input/output (I/O) interface 805 is also connected to the bus 804.
- the following devices can be connected to the I/O interface 805: including input devices 806 such as touch screen, touch pad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, liquid crystal display (LCD), speakers, vibration An output device 807 such as a device; a storage device 808 such as a magnetic tape, a hard disk, etc.; and a communication device 809.
- the communication device 809 may allow the electronic device 800 to perform wireless or wired communication with other devices to exchange data.
- FIG. 3 shows an electronic device 800 having various devices, it should be understood that it is not required to implement or have all of the illustrated devices. It may alternatively be implemented or provided with more or fewer devices.
- an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
- the computer program may be downloaded and installed from the network through the communication device 809, or installed from the storage device 808, or installed from the ROM 802.
- the processing device 801 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
- the aforementioned computer-readable medium in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two.
- the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
- a computer-readable storage medium may be any tangible medium that contains or stores a program, and the program may be used by or in combination with an instruction execution system, apparatus, or device.
- a computer-readable signal medium may include a data signal propagated in a baseband or as a part of a carrier wave, and a computer-readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing.
- the computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium.
- the computer-readable signal medium may send, propagate, or transmit the program for use by or in combination with the instruction execution system, apparatus, or device .
- the program code contained on the computer-readable medium can be transmitted by any suitable medium, including but not limited to: wire, optical cable, RF (Radio Frequency), etc., or any suitable combination of the above.
- the client and server can communicate with any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium.
- Communication e.g., communication network
- Examples of communication networks include local area networks (“LAN”), wide area networks (“WAN”), the Internet (for example, the Internet), and end-to-end networks (for example, ad hoc end-to-end networks), as well as any currently known or future research and development network of.
- the above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist alone without being assembled into the electronic device.
- the aforementioned computer-readable medium carries one or more programs, and when the aforementioned one or more programs are executed by the electronic device, the electronic device is caused to execute the aforementioned method of the present disclosure.
- the computer program code used to perform the operations of the present disclosure may be written in one or more programming languages or a combination thereof.
- the above-mentioned programming languages include object-oriented programming languages—such as Java, Smalltalk, C++, and also conventional Procedural programming language-such as "C" language or similar programming language.
- the program code can be executed entirely on the user's computer, partly on the user's computer, executed as an independent software package, partly on the user's computer and partly executed on a remote computer, or entirely executed on the remote computer or server.
- the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (for example, using an Internet service provider to pass Internet connection).
- LAN local area network
- WAN wide area network
- each block in the flowchart or block diagram may represent a module, program segment, or part of code, and the module, program segment, or part of code contains one or more for realizing the specified logical function Executable instructions.
- the functions marked in the block may also occur in a different order from the order marked in the drawings. For example, two blocks shown in succession can actually be executed substantially in parallel, and they can sometimes be executed in the reverse order, depending on the functions involved.
- each block in the block diagram and/or flowchart, and the combination of the blocks in the block diagram and/or flowchart can be implemented by a dedicated hardware-based system that performs the specified functions or operations Or it can be realized by a combination of dedicated hardware and computer instructions.
- the units involved in the embodiments described in the present disclosure can be implemented in software or hardware. Among them, the name of the unit does not constitute a limitation on the unit itself under certain circumstances.
- exemplary types of hardware logic components include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Product (ASSP), System on Chip (SOC), Complex Programmable Logical device (CPLD) and so on.
- FPGA Field Programmable Gate Array
- ASIC Application Specific Integrated Circuit
- ASSP Application Specific Standard Product
- SOC System on Chip
- CPLD Complex Programmable Logical device
- a machine-readable medium may be a tangible medium, which may contain or store a program for use by the instruction execution system, apparatus, or device or in combination with the instruction execution system, apparatus, or device.
- the machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
- the machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing.
- machine-readable storage media would include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.
- RAM random access memory
- ROM read-only memory
- EPROM or flash memory erasable programmable read-only memory
- CD-ROM compact disk read-only memory
- magnetic storage device or any suitable combination of the above.
- a video search method including:
- the first recommendation result corresponding to the first target to be searched is acquired, and the first recommendation result is displayed on a search result page.
- the second control includes at least one of a floating frame or a floating point.
- the first target to be searched includes a first target and a second target
- the second control includes a floating frame and a floating point
- the first to-be-searched target positioned by the second control in the current video image frame and the first display position of the first to-be-searched target in the current video image frame are acquired, in the first display position
- the step of displaying the second control includes:
- the floating frame is displayed at the display position of the first target in the current video image frame, and the floating point is displayed at the display position of the second target in the current video image frame.
- a method characterized in that the first recommendation result corresponding to the first target to be searched is obtained, and the first recommendation result is displayed in the search result.
- the steps on the page include:
- a second label for identifying the second target and the second recommendation result are correspondingly displayed in the second area of the search result page.
- the first target to be searched includes a valid target and an invalid target
- the method further includes:
- the effective target is at least one of a character, a commodity, and an article; the invalid target is a target result other than the effective target.
- a method characterized in that the first recommendation result corresponding to the first target to be searched is obtained, and the first recommendation result is displayed in the search result.
- the steps on the page also include:
- the first target to be searched is a valid target
- the first recommended result corresponding to the valid target is acquired, and the first recommended result is displayed on the search result page; as in the first to be searched
- the target is an invalid target, and no results are displayed on the search results page; or
- the first recommendation result corresponding to the valid target is acquired, and the first recommendation result and the global result other than the first recommendation result are displayed in sequence In the search result page; if the first target to be searched is an invalid target, the global result is displayed in the search result page.
- a method characterized in that the first recommendation result corresponding to the first target to be searched is acquired, and the first recommendation result is displayed in the search Before the results page, it also includes:
- the search result page at least partially covers the current video image frame and obscures the second control.
- a method characterized in that the first recommendation result corresponding to the first target to be searched is acquired, and the first recommendation result is displayed in the search After the steps in the result page, it also includes:
- a method characterized in that the method further includes:
- search result page is aroused, and the first recommendation result in the search result page is replaced with the second recommendation result.
- a video search device including:
- the receiving module is used to receive the first event generated when the first control is triggered;
- the acquiring module is used to acquire the current video image frame played in the video playback page when the first event is triggered, the first target to be searched located by the second control in the current video image frame, and the first target to be searched.
- the display module is used to display the first recommendation result on the search result page.
- a terminal including: at least one memory and at least one processor;
- the at least one memory is used to store program code
- the at least one processor is used to call the program code stored in the at least one memory to execute the method described in any one of the above items.
- a storage medium is provided, the storage medium is used to store program code, and the program code is used to execute the above-mentioned method.
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Data Mining & Analysis (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Computational Linguistics (AREA)
- Library & Information Science (AREA)
- User Interface Of Digital Computer (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
Description
Claims (12)
- 一种视频搜索的方法,包括:在视频播放页面中,接收第一控件被触发产生的第一事件;响应所述第一事件,获取所述第一事件被触发时所述视频播放页面中所播放的当前视频图像帧;获取所述当前视频图像帧中由第二控件定位的第一待搜索目标及所述第一待搜索目标在所述当前视频图像帧中的第一显示位置,在所述第一显示位置显示所述第二控件;及获取所述第一待搜索目标对应的第一推荐结果,并将所述第一推荐结果显示在搜索结果页面中。
- 根据权利要求1所述的方法,其特征在于,所述第二控件包括浮动框或浮动点中的至少一个。
- 根据权利要求2所述的方法,其特征在于,所述第一待搜索目标包括第一目标和第二目标,所述第二控件包括浮动框和浮动点;其中,获取所述当前视频图像帧中由第二控件定位的第一待搜索目标及所述第一待搜索目标在所述当前视频图像帧中的第一显示位置,在所述第一显示位置显示所述第二控件的步骤包括:分别获取所述当前视频图像帧中的由所述浮动框定位的所述第一目标和由所述浮动点定位的所述第二目标;获取所述第一目标在所述当前视频图像帧中的显示位置,以及所述第二目标在所述当前视频图像帧中的显示位置;及在所述第一目标在所述当前视频图像帧中的显示位置显示所述浮动框,在所述第二目标在所述当前视频图像帧中的显示位置显示所述浮动点。
- 根据权利要求3所述的方法,其特征在于,所述获取所述第一待搜索目标对应的第一推荐结果,并将所述第一推荐结果显示在搜索结果页面中的步骤包括:分别获取所述第一目标对应的第一待推荐结果和所述第二目标对应的第二待推荐结果;将用于标识所述第一目标的第一标签和所述第一推荐结果对应地显示在所述搜索结果页面中的第一区域内;及将用于标识所述第二目标的第二标签和所述第二推荐结果对应地显示在所述搜索结果页面中的第二区域内。
- 根据权利要求1所述的方法,其特征在于,所述第一待搜索目标包括有效目标和无效目标;其中,所述方法还包括:获取所述视频播放页面所播放的视频中的全部视频图像帧;及获取每一所述视频图像帧中的所述有效目标,得到全局结果;其中,所述有效目标为人物、商品和物品中的至少一种;所述无效目标为除所述有效目标以外的目标结果。
- 根据权利要求5所述的方法,其特征在于,所述获取所述第一待搜索目标对应的第一推荐结果,并将所述第一推荐结果显示在搜索结果页面中的步骤还包括:如所述第一待搜索目标为有效目标,获取所述有效目标对应的所述第一推荐结果,并将所述第一推荐结果显示在所述搜索结果页面中;如所述第一待搜索目标为无效目标,在所述搜索结果页面中显示无结果;或如所述第一待搜索目标为有效目标,获取所述有效目标对应的所述第一推荐结果,依次将所述第一推荐结果和除所述第一推荐结果之外的所述全局结果显示在所述搜索结果页面中;如所述第一待搜索目标为无效目标,将所述全局结果显示在所述搜索结果页面中。
- 根据权利要求1所述的方法,其特征在于,在所述获取所述第一待搜索目标对应的第一推荐结果,并将所述第一推荐结果显示在搜索结果页面中之前,还包括:接收所述第二控件被触发产生的第二事件;及响应于所述第二事件,唤起所述搜索结果页面;其中,所述搜索结果页面至少部分地覆盖于所述当前视频图像帧上并遮挡所述第二控件。
- 根据权利要求7所述的方法,其特征在于,在所述获取所述第 一待搜索目标对应的第一推荐结果,并将所述第一推荐结果显示在搜索结果页面中的步骤之后,还包括:移动所述搜索结果页面至所述当前视频图像帧的边缘并隐藏部分所述搜索结果页面,以显示所述第二控件。
- 根据权利要求8所述的方法,其特征在于,所述方法还包括:检测所述第二控件所在的所述第一显示位置是否发生变化,如果发生变化,获取所述第二控件的第二显示位置;根据所述第二显示位置确定对应的第二待搜索目标;获取所述第二待搜索目标对应的第二推荐结果;及再唤起所述搜索结果页面,将所述搜索结果页面中的所述第一推荐结果更换为所述第二推荐结果。
- 一种视频搜索的装置,包括:接收模块,用于接收第一控件被触发产生的第一事件;获取模块,用于获取所述第一事件被触发时视频播放页面中所播放的当前视频图像帧、所述当前视频图像帧中由第二控件定位的第一待搜索目标及所述第一待搜索目标在所述当前视频图像帧中的第一显示位置、和所述第一待搜索目标对应的第一推荐结果;及显示模块,用于将所述第一推荐结果显示在搜索结果页面中。
- 一种终端,包括:至少一个存储器和至少一个处理器;其中,所述至少一个存储器用于存储程序代码,所述至少一个处理器用于调用所述至少一个存储器所存储的程序代码执行权利要求1至9中任一项所述的方法。
- 一种存储介质,所述存储介质用于存储程序代码,所述程序代码用于执行权利要求1至9中任一项所述的方法。
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2022522608A JP7488333B2 (ja) | 2019-10-17 | 2020-09-11 | ビデオ検索方法、装置、端末、及び記憶媒体 |
US17/722,334 US11630861B2 (en) | 2019-10-17 | 2022-04-16 | Method and apparatus for video searching, terminal and storage medium |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201910990463.X | 2019-10-17 | ||
CN201910990463.XA CN110704684B (zh) | 2019-10-17 | 2019-10-17 | 视频搜索的方法及装置、终端和存储介质 |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US17/722,334 Continuation US11630861B2 (en) | 2019-10-17 | 2022-04-16 | Method and apparatus for video searching, terminal and storage medium |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2021073328A1 true WO2021073328A1 (zh) | 2021-04-22 |
Family
ID=69200515
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/CN2020/114799 WO2021073328A1 (zh) | 2019-10-17 | 2020-09-11 | 视频搜索的方法及装置、终端和存储介质 |
Country Status (4)
Country | Link |
---|---|
US (1) | US11630861B2 (zh) |
JP (1) | JP7488333B2 (zh) |
CN (1) | CN110704684B (zh) |
WO (1) | WO2021073328A1 (zh) |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023134407A1 (zh) * | 2022-01-17 | 2023-07-20 | 北京字跳网络技术有限公司 | 内容搜索方法、装置、设备及介质 |
Families Citing this family (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN110704684B (zh) * | 2019-10-17 | 2022-08-09 | 北京字节跳动网络技术有限公司 | 视频搜索的方法及装置、终端和存储介质 |
CN112199522B (zh) * | 2020-08-27 | 2023-07-25 | 深圳一块互动网络技术有限公司 | 互动实现方法、终端、服务端、计算机设备及存储介质 |
CN115225945A (zh) * | 2021-04-20 | 2022-10-21 | 北京字节跳动网络技术有限公司 | 对象展示方法、装置、电子设备及计算机可读存储介质 |
CN113343005A (zh) * | 2021-05-17 | 2021-09-03 | 北京百度网讯科技有限公司 | 搜索方法、装置、电子设备以及可读存储介质 |
CN113535031A (zh) * | 2021-08-03 | 2021-10-22 | 北京字跳网络技术有限公司 | 页面显示方法、装置、设备及介质 |
CN113589991A (zh) * | 2021-08-13 | 2021-11-02 | 北京字跳网络技术有限公司 | 一种文本输入方法、装置、电子设备和存储介质 |
CN115878844A (zh) * | 2021-09-27 | 2023-03-31 | 北京有竹居网络技术有限公司 | 基于视频的信息展示方法及装置、电子设备和存储介质 |
CN115878838A (zh) * | 2021-09-27 | 2023-03-31 | 北京有竹居网络技术有限公司 | 基于视频的信息展示方法、装置、电子设备及存储介质 |
CN114443897A (zh) * | 2022-02-10 | 2022-05-06 | 北京字跳网络技术有限公司 | 一种视频推荐方法、装置、电子设备和存储介质 |
CN114786062A (zh) * | 2022-03-07 | 2022-07-22 | 维沃移动通信有限公司 | 信息推荐方法、装置和电子设备 |
CN114780790A (zh) * | 2022-05-05 | 2022-07-22 | 北京字节跳动网络技术有限公司 | 内容搜索方法、装置、设备和存储介质 |
CN114969524A (zh) * | 2022-05-24 | 2022-08-30 | 北京字节跳动网络技术有限公司 | 信息搜索方法、装置、设备及介质 |
CN115665498A (zh) * | 2022-10-27 | 2023-01-31 | 北京字跳网络技术有限公司 | 一种视频处理方法、装置、设备及存储介质 |
CN116471429B (zh) * | 2023-06-20 | 2023-08-25 | 上海云梯信息科技有限公司 | 基于行为反馈的图像信息推送方法及实时视频传输系统 |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104112129A (zh) * | 2014-06-25 | 2014-10-22 | 小米科技有限责任公司 | 图像识别方法及装置 |
CN105095483A (zh) * | 2015-08-14 | 2015-11-25 | 北京铭嘉实咨询有限公司 | 图片码识别方法和系统 |
CN109388725A (zh) * | 2018-10-30 | 2019-02-26 | 百度在线网络技术(北京)有限公司 | 通过视频内容进行搜索的方法及装置 |
CN110704684A (zh) * | 2019-10-17 | 2020-01-17 | 北京字节跳动网络技术有限公司 | 视频搜索的方法及装置、终端和存储介质 |
Family Cites Families (14)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2756073B2 (ja) * | 1993-04-06 | 1998-05-25 | 株式会社富士通ソーシアルサイエンスラボラトリ | データ検索方法 |
JP3093987B2 (ja) * | 1997-03-07 | 2000-10-03 | 日立ソフトウエアエンジニアリング株式会社 | 部分動画像関連情報検索方法及びシステム |
JP2004054435A (ja) | 2002-07-17 | 2004-02-19 | Toshiba Corp | ハイパーメディア情報提示方法、ハイパーメディア情報提示プログラムおよびハイパーメディア情報提示装置 |
US20070250775A1 (en) * | 2006-04-19 | 2007-10-25 | Peter Joseph Marsico | Methods, systems, and computer program products for providing hyperlinked video |
US8239359B2 (en) | 2008-09-23 | 2012-08-07 | Disney Enterprises, Inc. | System and method for visual search in a video media player |
KR101727316B1 (ko) * | 2010-10-08 | 2017-04-17 | 삼성전자 주식회사 | 비디오 재생장치 및 그 위치탐색방법 |
US8854474B2 (en) * | 2011-03-08 | 2014-10-07 | Nice Systems Ltd. | System and method for quick object verification |
US20140178041A1 (en) * | 2012-12-26 | 2014-06-26 | Balakesan P. Thevar | Content-sensitive media playback |
CN106776957A (zh) * | 2016-12-05 | 2017-05-31 | 乐视控股(北京)有限公司 | 内容搜索方法、装置及电子设备 |
KR20180091285A (ko) * | 2017-02-06 | 2018-08-16 | 삼성전자주식회사 | 동영상 관련 서비스 표시 방법, 저장 매체 및 이를 위한 전자 장치 |
CN107657011A (zh) * | 2017-09-25 | 2018-02-02 | 小草数语(北京)科技有限公司 | 视频内容搜索方法、装置及其设备 |
CN110121093A (zh) * | 2018-02-06 | 2019-08-13 | 优酷网络技术(北京)有限公司 | 视频中目标对象的搜索方法及装置 |
WO2019182583A1 (en) * | 2018-03-21 | 2019-09-26 | Rovi Guides, Inc. | Systems and methods for presenting auxiliary video relating to an object a user is interested in when the user returns to a frame of a video in which the object is depicted |
CN109218750B (zh) * | 2018-10-30 | 2022-01-04 | 百度在线网络技术(北京)有限公司 | 视频内容检索的方法、装置、存储介质和终端设备 |
-
2019
- 2019-10-17 CN CN201910990463.XA patent/CN110704684B/zh active Active
-
2020
- 2020-09-11 JP JP2022522608A patent/JP7488333B2/ja active Active
- 2020-09-11 WO PCT/CN2020/114799 patent/WO2021073328A1/zh active Application Filing
-
2022
- 2022-04-16 US US17/722,334 patent/US11630861B2/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN104112129A (zh) * | 2014-06-25 | 2014-10-22 | 小米科技有限责任公司 | 图像识别方法及装置 |
CN105095483A (zh) * | 2015-08-14 | 2015-11-25 | 北京铭嘉实咨询有限公司 | 图片码识别方法和系统 |
CN109388725A (zh) * | 2018-10-30 | 2019-02-26 | 百度在线网络技术(北京)有限公司 | 通过视频内容进行搜索的方法及装置 |
CN110704684A (zh) * | 2019-10-17 | 2020-01-17 | 北京字节跳动网络技术有限公司 | 视频搜索的方法及装置、终端和存储介质 |
Cited By (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
WO2023134407A1 (zh) * | 2022-01-17 | 2023-07-20 | 北京字跳网络技术有限公司 | 内容搜索方法、装置、设备及介质 |
Also Published As
Publication number | Publication date |
---|---|
CN110704684A (zh) | 2020-01-17 |
US20220237227A1 (en) | 2022-07-28 |
JP7488333B2 (ja) | 2024-05-21 |
JP2022553174A (ja) | 2022-12-22 |
CN110704684B (zh) | 2022-08-09 |
US11630861B2 (en) | 2023-04-18 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
WO2021073328A1 (zh) | 视频搜索的方法及装置、终端和存储介质 | |
US10474233B2 (en) | Enabling augmented reality using eye gaze tracking | |
EP2983077B1 (en) | Display control device, display control method, and display control program | |
KR102173123B1 (ko) | 전자장치에서 이미지 내의 특정 객체를 인식하기 위한 방법 및 장치 | |
CN109740085B (zh) | 一种页面内容的展示方法、装置、设备及存储介质 | |
CN105320403B (zh) | 用于提供内容的方法和装置 | |
CN108920515B (zh) | 网页显示过程的信息推荐方法、装置、设备及存储介质 | |
CN110020140A (zh) | 推荐内容显示方法、装置及系统 | |
CA3102222C (en) | Method, device, terminal equipment and storage medium of sharing personal information | |
EP2596421A2 (en) | Fisheye-based presentation of information for mobile devices | |
WO2020200146A1 (zh) | 页面信息处理方法、装置及电子设备 | |
US20150186491A1 (en) | Personalized electronic magazine | |
WO2020259522A1 (zh) | 一种内容查找方法、相关设备及计算机可读存储介质 | |
WO2018004200A1 (en) | Electronic device and information providing method thereof | |
WO2023071491A1 (zh) | 百科信息确定方法、显示方法、装置、设备和介质 | |
US9619519B1 (en) | Determining user interest from non-explicit cues | |
JP2008146492A (ja) | 情報提供装置、情報提供方法、及びコンピュータプログラム | |
CN106507177A (zh) | 用于生成弹幕的方法和装置 | |
WO2024001578A1 (zh) | 书籍信息处理方法、装置、设备和存储介质 | |
CN111581554A (zh) | 一种信息推荐方法及装置 | |
KR20130128800A (ko) | 미디어 기기에서의 콘텐츠 정렬 방법 및 장치와 그 방법에 대한 프로그램 소스를 저장한 기록 매체 | |
CN112202958B (zh) | 截图方法、装置及电子设备 | |
CN107145314B (zh) | 一种显示处理方法和装置、一种用于显示处理的装置 | |
CN110825909A (zh) | 视频图像的识别方法、装置、服务器、终端和存储介质 | |
CN115086774B (zh) | 资源显示方法、装置、电子设备及存储介质 |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20876926 Country of ref document: EP Kind code of ref document: A1 |
|
ENP | Entry into the national phase |
Ref document number: 2022522608 Country of ref document: JP Kind code of ref document: A |
|
NENP | Non-entry into the national phase |
Ref country code: DE |
|
32PN | Ep: public notification in the ep bulletin as address of the adressee cannot be established |
Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 05.08.2022) |
|
122 | Ep: pct application non-entry in european phase |
Ref document number: 20876926 Country of ref document: EP Kind code of ref document: A1 |