CN113672769A - Multi-window automatic recommendation method, device, terminal and medium based on image recognition - Google Patents

Multi-window automatic recommendation method, device, terminal and medium based on image recognition Download PDF

Info

Publication number
CN113672769A
CN113672769A CN202110884205.0A CN202110884205A CN113672769A CN 113672769 A CN113672769 A CN 113672769A CN 202110884205 A CN202110884205 A CN 202110884205A CN 113672769 A CN113672769 A CN 113672769A
Authority
CN
China
Prior art keywords
app
video
structured array
window
automatic recommendation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110884205.0A
Other languages
Chinese (zh)
Inventor
邓薇
周胜杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Konka Group Co Ltd
Original Assignee
Konka Group Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Konka Group Co Ltd filed Critical Konka Group Co Ltd
Priority to CN202110884205.0A priority Critical patent/CN113672769A/en
Publication of CN113672769A publication Critical patent/CN113672769A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/70Information retrieval; Database structures therefor; File system structures therefor of video data
    • G06F16/78Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/783Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
    • G06F16/7837Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content using objects detected or recognised in the video content
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/90Details of database functions independent of the retrieved data types
    • G06F16/95Retrieval from the web
    • G06F16/953Querying, e.g. by the use of web search engines
    • G06F16/9535Search customisation based on user profiles and personalisation

Abstract

The invention discloses a multi-window automatic recommendation method, a device, a terminal and a medium based on image recognition, wherein the method comprises the following steps: identifying predetermined elements in the video through an image identification technology to obtain a structured array in the video; carrying out butt joint processing on each element data in the obtained structured array and the APP, and distributing to corresponding APP to search corresponding recommended resources; and obtaining each APP resource in real time, controlling each APP application to be opened through multiple windows, and displaying the searched detailed interface of the recommended resource in the window. The method aims to solve the problem that the existing large-screen terminal is too single in life and entertainment scenes and cannot automatically associate related applications to construct more life and entertainment scenes. The terminal intelligently identifies the objects in the playing picture and automatically associates the corresponding APP for searching and multi-window display, and a richer and comprehensive daily life entertainment scene is established for the user.

Description

Multi-window automatic recommendation method, device, terminal and medium based on image recognition
Technical Field
The invention relates to the technical field of smart televisions, in particular to a multi-window automatic recommendation method, a multi-window automatic recommendation device, a multi-window automatic recommendation terminal and a multi-window automatic recommendation medium based on image recognition.
Background
With the development of science and technology and the continuous improvement of the living standard of people, the use of various intelligent terminals such as an intelligent television is more and more popularized, and the intelligent television becomes an indispensable leisure tool in the life of people.
With the rapid development of the information age, the application of the big data analysis automatic recommendation technology is more and more extensive. The television is used as a terminal large screen, and the smart television in the prior art has relatively single functions about life entertainment scenes, cannot automatically associate related applications to construct more life entertainment scenes, and is sometimes inconvenient for users to use.
Thus, there is still a need for improvement and development of the prior art.
Disclosure of Invention
The invention mainly aims to provide a multi-window automatic recommendation method and device based on image recognition, an intelligent terminal and a storage medium. The method aims to solve the problems that the existing large-screen television is only used for watching videos under most conditions, more related information cannot be recommended according to video playing content and automatic association related application, and more life and entertainment scenes are constructed.
In order to achieve the above object, a first aspect of the present invention provides a multi-window automatic recommendation method based on image recognition, wherein the method includes:
identifying predetermined elements in the video through an image identification technology to obtain a structured array in the video;
carrying out butt joint processing on each element data in the obtained structured array and the APP, and distributing to corresponding APP to search corresponding recommended resources;
and obtaining each APP resource in real time, controlling each APP application to be opened through multiple windows, and displaying the searched detailed interface of the recommended resource in the window.
Optionally, the step of obtaining each APP resource in real time, controlling to open each APP application through multiple windows, and displaying the searched detailed interface of the recommended resource in a window includes:
and controlling the APP opened in the multiple windows and the corresponding detailed interface to be dynamically adjusted according to the elements identified by the video playing progress.
Optionally, the step of identifying the predetermined element in the video by using an image identification technology to obtain the structured array in the video includes:
presetting predetermined elements needing to be identified in a video to determine the dimension of a structured array;
and/or dynamically adjusting the preset elements to be identified in the video according to the behavior habits of the user, and determining the dimensionality of the structured array;
the predetermined elements include persons and/or items and/or buildings.
Optionally, the step of identifying the predetermined element in the video by using an image identification technology to obtain the structured array in the video includes:
when a building is identified from the video through an image identification technology, building information is obtained, and the building information is associated with tourist attractions;
when an article is identified from the video through an image identification technology, article information is obtained, and the article information is related to a commodity brand and a commodity type;
when people are identified from the video through an image identification technology, people information is obtained, and when the people are identified as stars, the people are associated with the participating movie and video, live broadcast delivery, introduction products and microblog articles.
Optionally, the step of performing docking processing on each element data in the obtained structured array and the APP, and distributing to the corresponding APP to search for the corresponding recommended resource includes:
carrying out butt joint processing on each element data in the structured array through a person butt joint system, and distributing to corresponding APP to search corresponding recommended resources;
wherein the character docking system docks a plurality of everyday entertainment platforms or APPs.
Optionally, the step of performing docking processing on each element data in the obtained structured array and the APP, and distributing to the corresponding APP to search for the corresponding recommended resource further includes:
when each element data in the structured array comprises the building information and the associated tourist attractions, distributing the element data to the corresponding tourist APP to search for the corresponding recommended resource;
when each element data in the structured array comprises article information, associated commodity brands and commodity types, distributing the data to corresponding shopping APPs to search corresponding recommended resources;
when each element data in the structured array comprises the star character information, the star character information is distributed to the movie and video, live broadcast delivery, dialect products and microblog articles which are related to the star character and participate correspondingly.
Optionally, the step of obtaining each APP resource in real time, controlling to open each APP application through multiple windows, and displaying the searched detailed interface of the recommended resource in a window further includes:
receiving a user operation instruction to check the relevant recommended resources and carrying out further operation;
and when the commodity information is selected to be opened, automatically adjusting the commodity information to be in a price comparison mode, and displaying the sorted commodity price comparison.
The second aspect of the present invention provides an image recognition-based multi-window automatic recommendation apparatus, wherein the apparatus includes:
the video identification module is used for identifying preset elements in the video through an image identification technology to obtain a structured array in the video;
the search distribution module is used for carrying out butt joint processing on each element data in the obtained structured array and the APP and distributing the element data to recommended resources corresponding to the APP search;
the search opening module is used for acquiring each APP resource in real time, controlling each APP application to be opened through multiple windows and displaying a detail interface of the searched recommended resource in the window;
and the searching dynamic updating module is used for controlling the elements identified according to the video playing progress to dynamically adjust the opened APP in the multiple windows and the corresponding detailed interface.
A third aspect of the present invention provides an intelligent terminal, where the intelligent terminal includes a memory, a processor, and an image recognition-based multi-window automatic recommendation program stored in the memory and executable on the processor, and when the image recognition-based multi-window automatic recommendation program is executed by the processor, the method implements any one of the steps of the image recognition-based multi-window automatic recommendation method.
A fourth aspect of the present invention provides a computer-readable storage medium, where an image-recognition-based multi-window automatic recommendation program is stored in the computer-readable storage medium, and when executed by a processor, the image-recognition-based multi-window automatic recommendation program implements any one of the steps of the image-recognition-based multi-window automatic recommendation method.
From the above, the invention adopts: identifying predetermined elements in the video through an image identification technology to obtain a structured array in the video; carrying out butt joint processing on each element data in the obtained structured array and the APP, and distributing to corresponding APP to search corresponding recommended resources; and obtaining each APP resource in real time, controlling each APP application to be opened through multiple windows, and displaying the searched detailed interface of the recommended resource in the window. The invention adds new functions to the smart television: the method comprises the steps of automatically identifying preset elements in a video according to the played video, controlling an application APP to correspondingly search corresponding recommended resources according to the preset elements, and opening the APP and the corresponding recommended resources through multiple windows. The method provides more comprehensive and rapid daily life entertainment scenes for the user using the large-screen terminal, and effectively improves the user viscosity.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.
FIG. 1 is a schematic flowchart of a multi-window automatic recommendation method based on image recognition according to an embodiment of the present invention;
FIG. 2 is a flowchart illustrating the step S100 in FIG. 1 according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating a detailed process of step S200 in FIG. 1 according to an embodiment of the present invention;
fig. 4 is a schematic flowchart of a specific process for image-based recognition multi-window recommendation of a television according to an embodiment of the present invention;
FIG. 5 is a diagram illustrating detailed information corresponding to television multi-window display data according to an embodiment of the present invention;
FIG. 6 is a schematic structural diagram of an automatic multi-window recommendation device based on image recognition according to an embodiment of the present invention;
fig. 7 is a schematic block diagram of an internal structure of an intelligent terminal according to an embodiment of the present invention.
Detailed Description
In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the invention. It will be apparent, however, to one skilled in the art that the present invention may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present invention with unnecessary detail.
It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.
It is also to be understood that the terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used in the specification of the present invention and the appended claims, the singular forms "a," "an," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise.
It should be further understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.
As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when …" or "upon" or "in response to a determination" or "in response to a detection". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted depending on the context to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".
The technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the drawings of the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention, but the present invention may be practiced in other ways than those specifically described and will be readily apparent to those of ordinary skill in the art without departing from the spirit of the present invention, and therefore the present invention is not limited to the specific embodiments disclosed below.
With the development of science and technology and the continuous improvement of the living standard of people, the use of various intelligent terminals such as an intelligent television is more and more popularized, and the intelligent television becomes an indispensable leisure tool in the life of people.
However, currently, the use method of people for large-screen terminals such as smart televisions still remains in a stage of only playing videos or screen-casting applications, and when a user wants to watch videos related to food, the user needs to search program names or names of food and then open related videos; when a user sees a beautiful scene in a video and wants to further know more information of the scene, the user cannot know the scene name; when a user sees a favorite commodity while watching a television and wants to buy the same commodity, the user needs to pause and start a mobile phone to learn the same commodity. The smart television in the prior art cannot automatically identify details of characters, objects and scenes appearing in a television picture, so that when a user cannot further acquire information of interested objects appearing in a playing video, more daily life entertainment scenes cannot be provided for a large-screen user.
In order to solve the problems of the prior art, the invention adopts the following steps: identifying predetermined elements in the video through an image identification technology to obtain a structured array in the video; recombining the acquired data of the structured array, carrying out butt joint processing on each element data in the structured array, and distributing to corresponding APP to search for corresponding recommended resources; and acquiring corresponding APP resources in real time, controlling to open the corresponding APP through multiple windows to search the corresponding recommended resources, and skipping to a detail interface of the corresponding search when the APP is opened. The invention adds new functions to the smart television: the method comprises the steps of automatically identifying preset elements in a video according to the played video, controlling an application APP to correspondingly search corresponding recommended resources according to the preset elements, and opening the APP and the corresponding recommended resources through multiple windows. The method provides more comprehensive and rapid daily life entertainment scenes for the user using the large-screen terminal, and effectively improves the user viscosity.
Exemplary method
As shown in fig. 1, an embodiment of the present invention provides a multi-window automatic recommendation method based on image recognition, and specifically, the method includes the following steps:
s100, identifying predetermined elements in the video through an image identification technology to obtain a structured array in the video;
in this embodiment, the terminal device performs image recognition on a predetermined element in a video being played through an image recognition technology, obtains related information of the element in the video, and combines the related information into a structured array. The predetermined elements include things such as people, objects, buildings, landscapes, and the like. The structured array is obtained by performing integrated analysis on predetermined elements appearing in the video, for example, links such as item types and commodity brands of items appearing in the video, tourist attractions corresponding to appearing scenes, introduction products related to appearing characters, movie works, microblog articles, and star chat, and an information group consisting of the characters, the items, the scenery and corresponding related information in the elements is the structured array.
S200, carrying out butt joint processing on each element data in the acquired structured array and the APP, and distributing to corresponding APP to search for corresponding recommended resources;
in this embodiment, the terminal device performs docking processing on the acquired data of each element in the structured array and the APP with the matched element type, and controls the corresponding APP to search for the recommended resource related to the information of each element. For example when element in the structured array that terminal equipment obtained includes mug, microphone and certain singer, then handles through APP butt joint, the mug corresponds the panning APP, many APPs are pieced together to the microphone correspondence, singer corresponds browser APP, and is further terminal equipment control the mug that appears in panning APP direct search or the recognition search video, many APP search microphones of piecing together, browser APP searches for the personal information that corresponds the singer who discerns.
And S300, acquiring each APP resource in real time, controlling each APP application to be opened through multiple windows, and displaying the searched detailed interface of the recommended resource in the window.
In this embodiment, the terminal device obtains each downloaded APP resource in the terminal device in real time, opens each APP in a multi-window mode, and displays a detail page of recommended content searched from each APP in a window of the multi-window mode. For example, a series of commodity pages of the same type of mug are displayed in the Tanbao APP, a plurality of commodity pages of the microphone with high sales are displayed by the multi-piece APP, and the browser APP searches the name of the singer and opens the page of personal detail information of the singer for display.
The steps of obtaining all APP resources in real time, controlling all APP applications to be opened through multiple windows, and displaying the searched detailed interfaces of the recommended resources in the windows comprise the following steps:
and controlling the APP opened in the multiple windows and the corresponding detailed interface to be dynamically adjusted according to the elements identified by the video playing progress.
In this embodiment, the terminal device identifies elements in the current picture according to the video playing progress, that is, the video picture, and dynamically switches and calls the corresponding APP to open the corresponding detailed interface. The frequency of the identification video pictures may be set to 10s, half a minute, or longer or shorter intervals. In addition, the user is also set to recognize when the picture is paused, the situation that when the user needs to recognize characters or objects in the video picture, the APP and the corresponding detailed page interface are not refreshed is avoided, a manual refreshing mode is provided for the user, and the user experience is improved.
As can be seen from the above, the multi-window automatic recommendation method based on image recognition provided by the embodiment of the present invention includes: identifying predetermined elements in the video through an image identification technology to obtain a structured array in the video; recombining the acquired data of the structured array, carrying out butt joint processing on each element data in the structured array, and distributing to corresponding APP to search for corresponding recommended resources; and acquiring corresponding APP resources in real time, controlling to open the corresponding APP through multiple windows to search the corresponding recommended resources, and skipping to a detail interface of the corresponding search when the APP is opened. The invention adds new functions to the smart television: the method comprises the steps of automatically identifying preset elements in a video according to the played video, controlling an application APP to correspondingly search corresponding recommended resources according to the preset elements, and opening the APP and the corresponding recommended resources through multiple windows. The method provides more comprehensive and rapid daily life entertainment scenes for the user using the large-screen terminal, and effectively improves the user viscosity.
The terminal device includes a smart television, a tablet, a notebook computer, and the like, in this embodiment, a smart television commonly used by a user is used as the terminal device, and when the terminal device is another device, the specific scheme in this embodiment may be referred to.
Specifically, in an application scenario, as shown in fig. 2, the step S100 includes:
s101, when a building is identified from a video through an image identification technology, building information is obtained, and the building information is associated with tourist attractions;
s102, when an article is identified from the video through an image identification technology, acquiring article information, and associating the article information with a commodity brand and a commodity type;
and S103, identifying people from the video through an image identification technology, acquiring the information of the people, and associating the people with the video, live broadcast delivery, introduction products and microblog articles involved when the people are identified as stars.
Wherein, the step of identifying the predetermined element in the video by the image identification technology and obtaining the structured array in the video comprises:
presetting predetermined elements needing to be identified in a video to determine the dimension of a structured array;
and/or dynamically adjusting the preset elements to be identified in the video according to the behavior habits of the user, and determining the dimensionality of the structured array;
the predetermined elements include persons and/or items and/or buildings.
For example, before using the image recognition-based multi-window automatic recommendation function in the smart television, the user a needs to set a predetermined element to be recognized in the smart television to determine the structured array dimension. The predetermined elements include persons and/or items and/or buildings. For example, an object and a person are used as predetermined elements to be identified in a manner preset by a manufacturer or set by a user, the determined structured array dimension only includes the object and the person, and when the multi-window automatic recommendation function based on image identification is started, the smart television acquires and identifies the object and the person appearing in the played video in real time. Besides the preset elements with larger categories such as people, articles, landscape buildings and the like, the preset elements with small categories such as men, women, old, few, living goods, electronic goods, Chinese buildings, ancient buildings and the like can be accurately set. By the method, the user can self-define and set to identify and recommend the favorite content.
According to another method, the smart television dynamically adjusts predetermined elements which need to be acquired and identified from a video according to user behavior habits, for example, the predetermined elements preset by the user A are objects and characters, but the user A never opens an identified detailed page related to the characters in the process of using the smart television, and the user A also frequently opens detailed pages of furniture identified by the smart television, so that the user has little interest in the characters appearing in the video through judgment of the smart television, and has great interest in the furniture and other articles, the smart television controls and adjusts the predetermined elements, the specific gravity of the identified characters is reduced, and the identification acquisition specific gravity of the furniture in the objects is improved. The preference of the user can be obtained through detailed analysis by the method for intelligently analyzing the behavior habits of the user and further adjusting the preset elements, the identification and the acquisition are carried out according to the favorite content of the user, the calculation amount of the intelligent television can be greatly reduced, the viscosity of the user is improved, and the trouble caused by the fact that the user is difficult to select due to too many options or inaccurate screening due to too few options when manually setting the preset elements is reduced.
For example, user a sets up sights, items, and people in the smart tv. When a user is playing a picture of a quebeck game in a harry baud movie, the picture contains actors daniel redkriff decorating the harry baud, a golden bougie, and the venue anikaiburg of the movie. The intelligent television identifies the building Annikeburg through image recognition and associates the Annikeburg with the tourist attractions related to the Annikeburg; identifying a gold probe, and associating the gold probe as a role playing prop with an unknown brand; identifying Harry potter, associating the Harry potter with actor Dener Leidekriff, and associating with other video movies, dialect products, microblog articles and live delivery information related to the actor Dener Leidekriff, so that the structured array acquired by the intelligent television identification comprises the buildings, articles, persons and associated information. The smart television analyzes and acquires people, objects, buildings and landscapes appearing in the video picture in real time and associates related information of the people, the objects, the buildings and the landscapes through preset elements preset by a user, so that the user can conveniently find and browse detailed information and related information of things appearing in the video picture.
Specifically, in an application scenario, as shown in fig. 3, the step S200 includes:
step S201, carrying out butt joint processing on each element data in the structured array through a person butt joint system, and distributing to corresponding APP to search for corresponding recommended resources;
and S202, the character docking system docks various daily life entertainment platforms or APPs.
Wherein, the step of carrying out butt joint processing on each element data in the obtained structured array and the APP and distributing to the corresponding APP to search the corresponding recommended resources further comprises:
when each element data in the structured array comprises the building information and the associated tourist attractions, distributing the element data to the corresponding tourist APP to search for the corresponding recommended resource;
when each element data in the structured array comprises article information, associated commodity brands and commodity types, distributing the data to corresponding shopping APPs to search corresponding recommended resources;
when each element data in the structured array comprises the star character information, the star character information is distributed to the movie and video, live broadcast delivery, dialect products and microblog articles which are related to the star character and participate correspondingly.
For example, the smart television interfaces the element data anigebauer, the gold probe, the halibaud and the associated related information thereof contained in the structured array with the corresponding APP through the character interfacing system, allocates the elements and the related information thereof to the corresponding APP for searching, and searches the recommended resource of the element. The recommended resources comprise the same type of commodity links, tourist attraction related information, character details and the like. The character docking system is a docking system for people and things and is used for correspondingly docking the related information of each element contained in the structured array into a corresponding application APP. For example, relevant tourist spot information of Annikeburg is docked to a browser APP, the APP is controlled to inquire the tourist spot information of Annikeburg, if domestic tourist spots such as Jiuzhaigou are inquired, the APP is docked to a travel tourist APP to search the Jiuzhaigou, and multiple travel schemes are displayed; docking a gold explorer item into a shopping APP, such as a Taobao APP, and searching the Taobao APP for a role-playing property gold explorer, and displaying a series of commodity links selling the commodity; the method comprises the steps of docking related information of an actor daniel Redchrift of a Harry Baud into a browser APP for searching, docking related microblog articles to a microblog APP for related searching, docking related movie works to the movie APP for related searching, listing a series of movie works with the actor daniel Redchrift, associating a dialect product to a shopping APP for searching, and docking a name of the actor daniel Redchrift to a live-broadcasting and goods-carrying APP for searching. By the method, people and things appearing in the smart television playing picture can be automatically associated to the related application APP for searching, user operation is reduced, convenience is brought to the user, and user stickiness is increased.
Specifically, in an application scenario, after the steps of obtaining each APP resource in real time, controlling each APP application to be opened through multiple windows, and displaying the searched detailed interface of the recommended resource in a window, the method further includes:
receiving a user operation instruction to check the relevant recommended resources and carrying out further operation;
and when the commodity information is selected to be opened, automatically adjusting the commodity information to be in a price comparison mode, and displaying the sorted commodity price comparison.
For example, after the smart television interfaces elements in a video picture to corresponding APPs and performs searching and display, when the smart television receives an operation instruction of the user a, the user a positions a television focus on a detailed page of the browser APP corresponding to, for example, annikiberg through a remote controller and clicks a confirmation key, the smart television controls to open the page or to amplify and display the content of the page; when a user A selects to open a Taobao APP series commodity page corresponding to a search gold seeker, the intelligent television controls to automatically select a browsing mode that the commodity sorting condition corresponding to the Taobao APP is the money amount ascending trend, and the commodities are sorted according to the price and then are amplified or displayed in the screen of the intelligent television in a full screen mode. The sorting conditions can be adjusted according to user habits, for example, when the user habits use the sorting conditions with the nearest distance, the preset sorting conditions are automatically switched when the user controls to open the shopping APP, and more convenient use experience is brought to the user.
In the embodiment of the present invention, the method for automatically recommending multiple windows based on image recognition is further specifically described based on an application scenario, fig. 4 is a specific flow diagram of television recommendation based on image recognition multiple windows provided in the embodiment of the present invention, and as shown in fig. 4, the steps include:
step S10, start, proceed to step S11;
step S11, starting the television, and entering step S12;
step S12, starting the video application, and entering step S13;
step S13, entering a recommendation mode, and entering step S14;
s14, the television acquires a structured array in a video picture in real time through an image recognition technology, returns structured data in the structured array in real time, and enters S15;
step S15, sending the returned structured data to a task docking system for further docking processing, and entering step S16;
step S16, processing the structured data through a human docking system, including correspondingly distributing the data to relevant APP to search for corresponding recommended resources, and entering step S17;
s17, after acquiring recommended resources corresponding to APP search in real time, the television displays the detail page of the acquired recommended resources through a multi-window mode, and the step S18 is entered;
s18, controlling the APP and the detail page opened by the multiple windows to be adjusted in real time according to the current video playing picture, and entering the step S20;
and step S20, end.
Therefore, the user B starts the television and starts the video application with the recommendation mode, the user B selects one network video to play, the television acquires the structured array in the video in real time through an image recognition technology, and returns the structured data in the structured array in real time. The structured array includes people, items, buildings, and further analyzes features of the people, items, and buildings, including tourist attractions associated with the buildings, merchandise brands and merchandise types associated with the items, people's stars associated with the people, including live products, video movies, products for pronouncing, microblog articles, or star talks, etc. associated with the stars. Further, the real-time returned structured data is sent to a person docking system for further processing, wherein the person docking system is a docking system for persons, things or objects, and is used for correspondingly docking corresponding APP on the data in the structured array. The television processes the data through the person docking system, and specifically includes that the data are distributed to corresponding APP, and the APP corresponding to the data searches for corresponding recommended resources. The recommended resources comprise detail pages or a series of commodity links and other data-related pages after search of the APPs. The corresponding APP interfaces the item to a shopping mall APP, for example, through a people interface system; data related to videos, such as live tape goods, video films and the like, are connected to a video APP, a short video APP and a live APP in an interfacing manner; the tourist attraction data is connected to travel APPs such as a travel APP and a go APP; and data of microblog articles of people, stars and super phone calls are connected to APPs such as microblogs and WeChats. Further, after the television acquires the recommended resource corresponding to the APP search in real time, the recommended resource detail page is displayed in a small window through a multi-window mode as shown in fig. 5, so that a user can browse in real time. And the television controls to dynamically adjust the APP and the detail page opened by the multiple windows according to the content identified in the video playing picture in real time, and if the user switches the playing content or the picture jumps to other scenes, the structured array data in the new picture is acquired in real time, and the detail page displayed by the small window in the figure 5 is updated. By the method, the user can acquire the detailed information of things appearing in the video in real time while watching the video, and a rich entertainment scene is built for the user.
Further, in an embodiment, in order to avoid that the processing capacity of the terminal device is deficient due to the excessively fast refresh of the video picture or the detailed content in the multiple windows is excessively fast, when the user does not switch the television channel or play the video, data that has appeared in the same video is cached, and is sorted according to the number of times that the data appears, and the data with a large number of times that appears is sorted into the window with a more prominent or front position; or arrange the data types preferred by the user, such as the data of characters, the data of architectural scenes or the data of articles, in a more prominent or front window according to the preference of the user. The favorite mode of the user is judged by acquiring the data types frequently watched by the user through the data types preset by the user or the data types intelligently acquired by the television, and analyzing and setting the data types. And the user experience of the user when using the recommendation function is further improved.
Exemplary device
As shown in fig. 6, corresponding to the multi-window automatic recommendation method based on image recognition, an embodiment of the present invention further provides a multi-window automatic recommendation device based on image recognition, where the multi-window automatic recommendation device based on image recognition includes:
the video identification module 610 is configured to identify a predetermined element in a video through an image identification technology, and obtain a structured array in the video;
in this embodiment, the terminal device performs image recognition on a predetermined element in a video being played through an image recognition technology, obtains related information of the element in the video, and combines the related information into a structured array. The predetermined elements include things such as people, objects, buildings, landscapes, and the like. The structured array is obtained by performing integrated analysis on predetermined elements appearing in the video, for example, links such as item types and commodity brands of items appearing in the video, tourist attractions corresponding to appearing scenes, introduction products related to appearing characters, movie works, microblog articles, and star chat, and an information group consisting of the characters, the items, the scenery and corresponding related information in the elements is the structured array.
The search distribution module 620 is configured to perform docking processing on each element data in the obtained structured array and the APP, and distribute the element data to recommended resources corresponding to search of the APP;
in this embodiment, the terminal device performs docking processing on the acquired data of each element in the structured array and the APP with the matched element type, and controls the corresponding APP to search for the recommended resource related to the information of each element. For example when element in the structured array that terminal equipment obtained includes mug, microphone and certain singer, then handles through APP butt joint, the mug corresponds the panning APP, many APPs are pieced together to the microphone correspondence, singer corresponds browser APP, and is further terminal equipment control the mug that appears in panning APP direct search or the recognition search video, many APP search microphones of piecing together, browser APP searches for the personal information that corresponds the singer who discerns.
A search opening module 630, configured to obtain each APP resource in real time, control to open each APP application through multiple windows, and display a detailed interface of the searched recommended resource in a window;
in this embodiment, the terminal device obtains each downloaded APP resource in the terminal device in real time, opens each APP in a multi-window mode, and displays a detail page of recommended content searched from each APP in a window of the multi-window mode. For example, a series of commodity pages of the same type of mug are displayed in the Tanbao APP, a plurality of commodity pages of the microphone with high sales are displayed by the multi-piece APP, and the browser APP searches the name of the singer and opens the page of personal detail information of the singer for display.
And the searching dynamic updating module 640 is used for controlling the opened APP in the multiple windows and the corresponding detailed interface to be dynamically adjusted according to the elements identified by the video playing progress.
In this embodiment, the terminal device identifies elements in the current picture according to the video playing progress, that is, the video picture, and dynamically switches and calls the corresponding APP to open the corresponding detailed interface. The frequency of the identification video pictures may be set to 10s, half a minute, or longer or shorter intervals. In addition, the user is also set to recognize when the picture is paused, the situation that when the user needs to recognize characters or objects in the video picture, the APP and the corresponding detailed page interface are not refreshed is avoided, a manual refreshing mode is provided for the user, and the user experience is improved.
As can be seen from the above, the multi-window automatic recommendation method based on image recognition provided by the embodiment of the present invention includes: through the video identification module 610, identifying a predetermined element in a video through an image identification technology to obtain a structured array in the video; through the search distribution module 620, the acquired data of the structured array are recombined, and the data of each element in the structured array are subjected to butt joint processing and distributed to the recommended resource corresponding to the search of the corresponding APP; the search opening module 630 acquires the corresponding APP resource in real time, controls the corresponding APP to search the corresponding recommended resource through multiple windows, and skips to the detail interface of the corresponding search while opening the APP; through the dynamic update module 640, the opened APP in the multiple windows and the corresponding detailed interface are dynamically adjusted according to the elements identified by the video playing progress. The invention adds new functions to the smart television: the method comprises the steps of automatically identifying preset elements in a video according to the played video, controlling an application APP to correspondingly search corresponding recommended resources according to the preset elements, and opening the APP and the corresponding recommended resources through multiple windows. The method provides more comprehensive and rapid daily life entertainment scenes for the user using the large-screen terminal, and effectively improves the user viscosity.
Specifically, in this embodiment, a smart television commonly used by a user is taken as an example for explanation, but the display terminal may also be other display devices, which is not specifically limited herein, and when the display terminal is other devices, the specific scheme in this embodiment may be referred to.
Specifically, in this embodiment, the specific functions of each module of the multi-window automatic recommendation device based on image recognition may refer to the corresponding descriptions in the multi-window automatic recommendation method based on image recognition, and are not described herein again.
Based on the above embodiment, the present invention further provides an intelligent terminal, and a schematic block diagram thereof may be as shown in fig. 7. The intelligent terminal comprises a processor, a memory, a network interface and a display screen which are connected through a system bus. Wherein, the processor of the intelligent terminal is used for providing calculation and control capability. The memory of the intelligent terminal comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a multi-window auto-recommendation program based on image recognition. The internal memory provides an environment for an operating system in the non-volatile storage medium and the running of the multi-window automatic recommendation program based on the image recognition. The network interface of the intelligent terminal is used for being connected and communicated with an external terminal through a network. When being executed by a processor, the multi-window automatic recommendation program based on image recognition realizes the steps of any one of the multi-window automatic recommendation methods based on image recognition. The display screen of the intelligent terminal can be a liquid crystal display screen or an electronic ink display screen.
It will be understood by those skilled in the art that the block diagram of fig. 7 is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation to the intelligent terminal to which the solution of the present invention is applied, and a specific intelligent terminal may include more or less components than those shown in the figure, or combine some components, or have different arrangements of components.
In one embodiment, an intelligent terminal is provided, where the intelligent terminal includes a memory, a processor, and an image recognition-based multi-window automatic recommendation program stored in the memory and executable on the processor, and the image recognition-based multi-window automatic recommendation program performs the following operation instructions when executed by the processor:
identifying predetermined elements in the video through an image identification technology to obtain a structured array in the video;
carrying out butt joint processing on each element data in the obtained structured array and the APP, and distributing to corresponding APP to search corresponding recommended resources;
and obtaining each APP resource in real time, controlling each APP application to be opened through multiple windows, and displaying the searched detailed interface of the recommended resource in the window.
The embodiment of the invention also provides a computer-readable storage medium, wherein the computer-readable storage medium is stored with a multi-window automatic recommendation program based on image recognition, and when being executed by a processor, the multi-window automatic recommendation program based on image recognition realizes the steps of any one of the multi-window automatic recommendation methods based on image recognition provided by the embodiment of the invention.
It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present invention.
It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned functions may be distributed as different functional units and modules according to needs, that is, the internal structure of the apparatus may be divided into different functional units or modules to implement all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.
In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.
Those of ordinary skill in the art would appreciate that the elements and algorithm steps of the examples described in connection with the embodiments disclosed herein may be implemented as electronic hardware, or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present invention.
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus/terminal device and method may be implemented in other ways. For example, the above-described embodiments of the apparatus/terminal device are merely illustrative, and for example, the division of the above modules or units is only one logical division, and the actual implementation may be implemented by another division, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not executed.
The integrated modules/units described above, if implemented in the form of software functional units and sold or used as separate products, may be stored in a computer readable storage medium. Based on such understanding, all or part of the flow of the method according to the embodiments of the present invention may also be implemented by a computer program, which may be stored in a computer-readable storage medium and can implement the steps of the embodiments of the method when the computer program is executed by a processor. The computer program includes computer program code, and the computer program code may be in a source code form, an object code form, an executable file or some intermediate form. The computer readable medium may include: any entity or device capable of carrying the above-mentioned computer program code, recording medium, usb disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), electrical carrier wave signal, telecommunication signal, software distribution medium, etc. It should be noted that the contents contained in the computer-readable storage medium can be increased or decreased as required by legislation and patent practice in the jurisdiction.
The above-mentioned embodiments are only used for illustrating the technical solutions of the present invention, and not for limiting the same; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those skilled in the art; the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not depart from the spirit and scope of the embodiments of the present invention, and they should be construed as being included therein.

Claims (10)

1. A multi-window automatic recommendation method based on image recognition is characterized by comprising the following steps:
identifying predetermined elements in the video through an image identification technology to obtain a structured array in the video;
carrying out butt joint processing on each element data in the obtained structured array and the APP, and distributing to corresponding APP to search corresponding recommended resources;
and obtaining each APP resource in real time, controlling each APP application to be opened through multiple windows, and displaying the searched detailed interface of the recommended resource in the window.
2. The image-recognition-based multi-window automatic recommendation method according to claim 1, wherein the step of obtaining each APP resource in real time, controlling each APP application to be opened through multiple windows, and displaying the searched detailed interface of the recommended resource in a window comprises the following steps:
and controlling the APP opened in the multiple windows and the corresponding detailed interface to be dynamically adjusted according to the elements identified by the video playing progress.
3. The image-recognition-based multi-window automatic recommendation method according to claim 1, wherein the step of recognizing the predetermined elements in the video by image recognition technology to obtain the structured array in the video comprises:
presetting predetermined elements needing to be identified in a video to determine the dimension of a structured array;
and/or dynamically adjusting the preset elements to be identified in the video according to the behavior habits of the user, and determining the dimensionality of the structured array;
the predetermined elements include persons and/or items and/or buildings.
4. The image-recognition-based multi-window automatic recommendation method according to claim 1, wherein the step of recognizing the predetermined elements in the video by image recognition technology to obtain the structured array in the video comprises:
when a building is identified from the video through an image identification technology, building information is obtained, and the building information is associated with tourist attractions;
when an article is identified from the video through an image identification technology, article information is obtained, and the article information is related to a commodity brand and a commodity type;
when people are identified from the video through an image identification technology, people information is obtained, and when the people are identified as stars, the people are associated with the participating movie and video, live broadcast delivery, introduction products and microblog articles.
5. The image-recognition-based multi-window automatic recommendation method according to claim 1, wherein the step of performing docking processing on each element data in the obtained structured array and the APP and distributing the element data to the corresponding APP to search for the corresponding recommended resource comprises:
carrying out butt joint processing on each element data in the structured array through a person butt joint system, and distributing to corresponding APP to search corresponding recommended resources;
wherein the character docking system docks a plurality of everyday entertainment platforms or APPs.
6. The image-recognition-based multi-window automatic recommendation method according to claim 1, wherein the step of performing docking processing on each element data in the obtained structured array and the APP and distributing the element data to the corresponding APP to search for the corresponding recommended resource further comprises:
when each element data in the structured array comprises the building information and the associated tourist attractions, distributing the element data to the corresponding tourist APP to search for the corresponding recommended resource;
when each element data in the structured array comprises article information, associated commodity brands and commodity types, distributing the data to corresponding shopping APPs to search corresponding recommended resources;
when each element data in the structured array comprises the star character information, the star character information is distributed to the movie and video, live broadcast delivery, dialect products and microblog articles which are related to the star character and participate correspondingly.
7. The image-recognition-based multi-window automatic recommendation method according to claim 1, wherein the step of obtaining APP resources in real time, controlling opening APP applications through multiple windows, and displaying the searched detailed interface of the recommended resources in a window further comprises:
receiving a user operation instruction to check the relevant recommended resources and carrying out further operation;
and when the commodity information is selected to be opened, automatically adjusting the commodity information to be in a price comparison mode, and displaying the sorted commodity price comparison.
8. An apparatus for multi-window automatic recommendation based on image recognition, the apparatus comprising:
the video identification module is used for identifying preset elements in the video through an image identification technology to obtain a structured array in the video;
the search distribution module is used for carrying out butt joint processing on each element data in the obtained structured array and the APP and distributing the element data to recommended resources corresponding to the APP search;
the search opening module is used for acquiring each APP resource in real time, controlling each APP application to be opened through multiple windows and displaying a detail interface of the searched recommended resource in the window;
and the searching dynamic updating module is used for controlling the elements identified according to the video playing progress to dynamically adjust the opened APP in the multiple windows and the corresponding detailed interface.
9. An intelligent terminal, characterized in that the intelligent terminal comprises a memory, a processor and a multi-window automatic recommendation program based on image recognition, which is stored on the memory and can run on the processor, and when the multi-window automatic recommendation program based on image recognition is executed by the processor, the steps of the multi-window automatic recommendation method based on image recognition according to any one of claims 1-7 are implemented.
10. A computer-readable storage medium, wherein a multi-window automatic recommendation program based on image recognition is stored on the computer-readable storage medium, and when being executed by a processor, the multi-window automatic recommendation program based on image recognition realizes the steps of the multi-window automatic recommendation method based on image recognition according to any one of claims 1 to 7.
CN202110884205.0A 2021-08-03 2021-08-03 Multi-window automatic recommendation method, device, terminal and medium based on image recognition Pending CN113672769A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110884205.0A CN113672769A (en) 2021-08-03 2021-08-03 Multi-window automatic recommendation method, device, terminal and medium based on image recognition

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110884205.0A CN113672769A (en) 2021-08-03 2021-08-03 Multi-window automatic recommendation method, device, terminal and medium based on image recognition

Publications (1)

Publication Number Publication Date
CN113672769A true CN113672769A (en) 2021-11-19

Family

ID=78541500

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110884205.0A Pending CN113672769A (en) 2021-08-03 2021-08-03 Multi-window automatic recommendation method, device, terminal and medium based on image recognition

Country Status (1)

Country Link
CN (1) CN113672769A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114995711A (en) * 2022-05-23 2022-09-02 深圳康佳电子科技有限公司 Multi-window display method, terminal equipment and storage medium for hot search information

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104754413A (en) * 2013-12-30 2015-07-01 北京三星通信技术研究有限公司 Image search based television signal identification and information recommendation method and device
CN108055590A (en) * 2017-12-20 2018-05-18 聚好看科技股份有限公司 The display methods of the graphic user interface of television image sectional drawing
CN108111898A (en) * 2017-12-20 2018-06-01 聚好看科技股份有限公司 The display methods and smart television of the graphic user interface of television image sectional drawing
CN108322806A (en) * 2017-12-20 2018-07-24 青岛海信电器股份有限公司 The display methods of the graphic user interface of smart television and television image sectional drawing
CN109598541A (en) * 2018-11-19 2019-04-09 中信国安广视网络有限公司 Large-size screen monitors advertisement recommended method based on video identification

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104754413A (en) * 2013-12-30 2015-07-01 北京三星通信技术研究有限公司 Image search based television signal identification and information recommendation method and device
CN108055590A (en) * 2017-12-20 2018-05-18 聚好看科技股份有限公司 The display methods of the graphic user interface of television image sectional drawing
CN108111898A (en) * 2017-12-20 2018-06-01 聚好看科技股份有限公司 The display methods and smart television of the graphic user interface of television image sectional drawing
CN108322806A (en) * 2017-12-20 2018-07-24 青岛海信电器股份有限公司 The display methods of the graphic user interface of smart television and television image sectional drawing
CN110944133A (en) * 2017-12-20 2020-03-31 海信视像科技股份有限公司 Smart television and display method of graphical user interface of television picture screenshot
CN111031271A (en) * 2017-12-20 2020-04-17 海信视像科技股份有限公司 Smart television and display method of graphical user interface of television picture screenshot
CN109598541A (en) * 2018-11-19 2019-04-09 中信国安广视网络有限公司 Large-size screen monitors advertisement recommended method based on video identification

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114995711A (en) * 2022-05-23 2022-09-02 深圳康佳电子科技有限公司 Multi-window display method, terminal equipment and storage medium for hot search information

Similar Documents

Publication Publication Date Title
CN110519621B (en) Video recommendation method and device, electronic equipment and computer readable medium
US9877074B2 (en) Information processing apparatus program to recommend content to a user
CN108471542B (en) Video resource playing method based on intelligent sound box, intelligent sound box and storage medium
WO2023016409A1 (en) Data interaction method and apparatus, device, and storage medium
US8386942B2 (en) System and method for providing digital multimedia presentations
US20080201287A1 (en) Dissimilar item recommendation method, device, and program thereof
US20170105040A1 (en) Display method, apparatus and related display panel
KR20160011613A (en) Method and device for information acquisition
CN109754316B (en) Product recommendation method, product recommendation system and storage medium
CN103686344A (en) Enhanced video system and method
CN101668139A (en) Video display device, video display method and system
JP2012015917A (en) Content viewing system, content recommendation method and content display apparatus
CN105808182A (en) Display control method and system, advertisement breach judging device and video and audio processing device
CN107682717A (en) Service recommendation method, device, equipment and storage medium
WO2022247220A1 (en) Interface processing method and apparatus
US10616634B2 (en) Display device and operating method of a display device
CN113254779A (en) Content search method, device, equipment and medium
CN105898592A (en) Client play data allocation method and client
CN113672769A (en) Multi-window automatic recommendation method, device, terminal and medium based on image recognition
CN114513687A (en) Server and media asset recommendation method
CN114095793A (en) Video playing method and device, computer equipment and storage medium
WO2023169361A1 (en) Information recommendation method and apparatus and electronic device
WO2016206035A1 (en) Information recommendation method and user terminal
CN115460452A (en) Display device and channel playing method
CN109618231A (en) A kind of movie and television contents recommended method and system based on cold start-up

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination