CN115587202A - Resource processing method and device based on intelligent photo album housekeeper system - Google Patents

Resource processing method and device based on intelligent photo album housekeeper system Download PDF

Info

Publication number
CN115587202A
CN115587202A CN202110756191.4A CN202110756191A CN115587202A CN 115587202 A CN115587202 A CN 115587202A CN 202110756191 A CN202110756191 A CN 202110756191A CN 115587202 A CN115587202 A CN 115587202A
Authority
CN
China
Prior art keywords
multimedia resource
key
resource
picture
determining
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202110756191.4A
Other languages
Chinese (zh)
Inventor
李卫军
覃鸿
李智伟
卢宝莉
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Institute of Semiconductors of CAS
Original Assignee
Institute of Semiconductors of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Institute of Semiconductors of CAS filed Critical Institute of Semiconductors of CAS
Priority to CN202110756191.4A priority Critical patent/CN115587202A/en
Publication of CN115587202A publication Critical patent/CN115587202A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43—Querying
    • G06F16/432—Query formulation
    • G06F16/434—Query formulation using image data, e.g. images, photos, pictures taken by a user
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43—Querying
    • G06F16/435—Filtering based on additional data, e.g. user or group profiles
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/43—Querying
    • G06F16/438—Presentation of query results
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/487—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using geographical or spatial information, e.g. location
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/40—Information retrieval; Database structures therefor; File system structures therefor of multimedia data, e.g. slideshows comprising image and additional audio data
    • G06F16/48—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/489—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using time information

Abstract

The invention provides a resource processing method and a device based on an intelligent photo album housekeeper system, wherein the method comprises the following steps: determining a first multimedia resource selected by a user in an intelligent photo album housekeeper system; determining a key object contained in the first multimedia resource according to the first multimedia resource; searching a second multimedia resource containing a key object in the intelligent photo album manager system; and recommending the second multimedia resource. According to the method and the device, the second multimedia resource with the same key object as the first multimedia resource is determined based on the first multimedia resource selected by the user, and the second multimedia resource is automatically recommended to the user, so that the second multimedia resource can be recommended according to the interest of the user, manual reference of each multimedia data is not needed, the efficiency is high, and meanwhile, the problem of missed search caused by user errors can be avoided.

Description

Resource processing method and device based on intelligent photo album housekeeper system
Technical Field
The invention relates to the technical field of computers, in particular to a resource processing method and device based on an intelligent photo album housekeeper system.
Background
With the development of computer technology and multimedia technology, electronic photo albums are widely used. The electronic photo album is a special electronic product different from a traditional photo album, and can be used for storing and displaying multimedia data in intelligent equipment (such as a mobile phone, a computer and the like).
When a user needs to query a plurality of multimedia data with commonalities from an electronic album, if all photos with backgrounds of grasslands need to be queried, all multimedia data are mostly manually checked in the prior art, and photos with backgrounds of grasslands are manually screened out from all multimedia data.
Disclosure of Invention
The invention provides a resource processing method and device based on an intelligent photo album housekeeping system, which are used for solving the defect of low efficiency of inquiring common multimedia data in an electronic photo album in the prior art.
The invention provides a resource processing method based on an intelligent photo album housekeeper system, which comprises the following steps:
determining a first multimedia resource selected by a user in an intelligent photo album housekeeper system;
determining a key object contained in the first multimedia resource according to the first multimedia resource;
searching a second multimedia resource containing the key object in the intelligent photo album housekeeping system;
and recommending the second multimedia resource.
According to the resource processing method based on the intelligent photo album housekeeping system, provided by the invention, the key object contained in the first multimedia resource is determined according to the first multimedia resource, and the method specifically comprises the following steps:
if the first multimedia resource is a picture, determining all objects contained in the picture, and determining a key object contained in the first multimedia resource according to the position of each object in the picture, the proportion of each object in the area of the picture, and the definition of each object on the picture;
if the first multimedia resource is a video, cutting the video into a plurality of pictures, determining all objects contained in the corresponding pictures aiming at each picture obtained by cutting, and determining key objects contained in the corresponding pictures according to the positions of the objects in the corresponding pictures, the proportion of the objects in the corresponding picture areas and the definition of the objects on the corresponding pictures; and determining the key objects contained in the first multimedia resources according to the key objects contained in the pictures.
According to the resource processing method based on the intelligent album housekeeping system, provided by the invention, the key object contained in the first multimedia resource is determined according to the key object contained in each picture, and the method specifically comprises any one of the following steps:
determining a key object with the frequency higher than a preset value according to the key objects contained in each picture, and taking the key object with the frequency higher than the preset value as the key object contained in the first multimedia resource;
and determining key objects with the occurrence frequency higher than a preset value according to the key objects contained in each picture, judging the continuous occurrence condition of the key objects with the occurrence frequency higher than the preset value in each picture for the key objects with the occurrence frequency higher than the preset value, and taking the key objects with the continuous occurrence conditions meeting preset conditions as the key objects contained in the first multimedia resource.
According to the resource processing method based on the intelligent photo album housekeeping system, provided by the invention, the second multimedia resource containing the key object is searched in the intelligent photo album housekeeping system, and the method specifically comprises the following steps:
inquiring the object name matched with the key object according to the feature data of the key object and a database table; the database table stores the corresponding relation between the feature data of each key object and the object name of the key object;
determining a second multimedia resource containing the key object according to the object name and the object content marking result of each multimedia resource in the intelligent album housekeeping system; when each multimedia resource is added into the intelligent photo album housekeeping system, an object content marking result is generated, and the object content marking result is marked with an object name contained in the multimedia resource.
According to the resource processing method based on the intelligent photo album housekeeping system, provided by the invention, the second multimedia resource is recommended, and the method specifically comprises the following steps:
displaying a first multimedia resource selected by a user in a first display frame;
displaying the second multimedia resource in a second display frame; wherein the second display frame is arranged adjacent to the first display frame.
According to the resource processing method based on the intelligent album manager system, provided by the invention, the second multimedia resource containing the key object is searched in the intelligent album manager system, and the method specifically comprises the following steps:
and searching a second multimedia resource containing the key object with the time within a preset time range in the intelligent photo album manager system.
According to the resource processing method based on the intelligent album housekeeping system, provided by the invention, the second multimedia resource is recommended, and the method specifically comprises the following steps:
and classifying and displaying the second multimedia resource according to the generation place of the second multimedia resource.
According to the resource processing method based on the intelligent album housekeeping system, provided by the invention, the key object contained in the first multimedia resource is determined according to the first multimedia resource, and the method specifically comprises the following steps:
if the first multimedia resource is a picture, performing target segmentation on the picture to obtain a segmented target; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library to determine a key object contained in the first media resource; the preset image feature library stores image features of all scenic spots, buildings or meeting places;
if the first multimedia resource is a video, cutting the video into a plurality of pictures, and performing target segmentation on each picture obtained by cutting to obtain a segmented target; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library, and determining a key object contained in the first media resource; and the preset image feature library stores the image features of all scenic spots, buildings or meeting places.
The invention also provides a resource processing device based on the intelligent photo album housekeeper system, which comprises:
the system comprises a first determining module, a first judging module and a second determining module, wherein the first determining module is used for determining a first multimedia resource selected by a user in an intelligent photo album housekeeping system;
a second determining module, configured to determine, according to the first multimedia resource, a key object included in the first multimedia resource;
the searching module is used for searching a second multimedia resource containing the key object in the intelligent photo album housekeeping system;
and the recommending module is used for recommending the second multimedia resource.
The invention also provides an electronic device, which comprises a memory, a processor and a computer program which is stored on the memory and can run on the processor, wherein the processor realizes the steps of the resource processing method based on the intelligent photo album housekeeping system when executing the computer program.
The present invention also provides a non-transitory computer readable storage medium, on which a computer program is stored, the computer program, when being executed by a processor, implementing the steps of the resource processing method based on the intelligent photo album manager system as described in any one of the above.
According to the resource processing method and device based on the intelligent photo album housekeeping system, the second multimedia resource with the same key object as the first multimedia resource is determined based on the first multimedia resource selected by the user, and the second multimedia resource is automatically recommended to the user, so that the second multimedia resource can be recommended according to the interest of the user, manual reference for each multimedia data is not needed, the efficiency is high, and meanwhile the problem of missed search caused by user errors can be avoided.
Drawings
In order to more clearly illustrate the technical solutions of the present invention or the prior art, the drawings needed for the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
FIG. 1 is a schematic flow chart of a resource processing method based on an intelligent photo album housekeeping system according to the present invention;
FIG. 2 is a schematic structural diagram of a resource processing device based on an intelligent photo album housekeeping system according to the present invention;
fig. 3 is a schematic structural diagram of an electronic device provided in the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without inventive step based on the embodiments of the present invention, are within the scope of protection of the present invention.
The electronic photo album is a special electronic product different from a traditional photo album, and can be used for storing and displaying multimedia data in intelligent equipment (such as a mobile phone, a computer and the like). When a user needs to query a plurality of multimedia data with commonalities from an electronic album, if all photos with backgrounds of grasslands need to be queried, all multimedia data are mostly manually checked in the prior art, and photos with backgrounds of grasslands are manually screened out from all multimedia data.
Therefore, the invention provides a resource processing method based on the intelligent photo album housekeeping system. Fig. 1 is a schematic flowchart of a resource processing method based on an intelligent photo album manager system according to the present invention, and as shown in fig. 1, the method includes the following steps:
and step 110, determining a first multimedia resource selected by the user in the intelligent photo album housekeeping system.
In this step, the first multimedia resource may include one or more of an album picture, an album video, a picture of a third-party application program, and a video of the third-party application program in the intelligent album housekeeping system. When the user selects the first multimedia resource in the intelligent photo album housekeeping system, the first multimedia resource can be considered as the resource which is interested by the user.
The present embodiment may determine, based on operation information performed by a user on any multimedia resource, whether the multimedia resource is a resource of interest of the user, that is, whether the multimedia resource is a first multimedia resource selected by the user.
For example, when the number of times that a user clicks any multimedia resource in the intelligent album housekeeping system is greater than a preset number of clicks, the multimedia resource may be considered as a resource which is interested by the user, that is, the multimedia resource is determined to be a first multimedia resource selected by the user.
When a user opens any multimedia resource in the intelligent photo album housekeeping system, the multimedia resource can be regarded as a resource which is interested by the user, namely, the multimedia resource is determined to be the first multimedia resource selected by the user.
When a user performs a search operation on any multimedia resource in the intelligent photo album housekeeping system, the multimedia resource can be considered as a resource which is interested by the user, namely, the multimedia resource is determined as a first multimedia resource selected by the user.
Step 120, determining a key object included in the first multimedia resource according to the first multimedia resource.
In this step, since the first multimedia asset can be considered as a resource of interest to the user, the key object contained in the first multimedia asset can be considered as an object of interest to the user. The key objects may include characters, animals, objects, scenes, backgrounds, etc. in the first multimedia resource. The determination of the key object included in the first multimedia resource may be performed in the first multimedia resource according to a preset rule, or may also be performed in the first multimedia resource by setting a corresponding rule according to different requirements of a user, which is not specifically limited in this embodiment.
For example, when the area ratio of the person a in the picture 1 is greater than the preset ratio, the person a may be considered as a key object in the picture 1. When the time length of the scene a in the video 1 is longer than the preset time length, the scene a may be considered as a key object of the video 1.
Step 130, searching a second multimedia resource containing a key object in the intelligent photo album housekeeping system;
and step 140, recommending the second multimedia resource.
In this embodiment, it should be noted that, since the key object may be regarded as an object of interest to the user, the second multimedia resource including the key object may also be regarded as a resource of interest to the user. The second multimedia resource may include one or more of an album picture, an album video, a picture of a third-party application program, and a video of the third-party application program in the intelligent album housekeeping system.
After determining a key object contained in the first multimedia resource, searching in the intelligent photo album manager system based on the key object, and determining a second multimedia resource containing the key object. If the number of the searched second multimedia resources containing the key objects is 1, the second multimedia resources can be directly recommended to the user, that is, the second multimedia resources are considered to be the resources in which the user is interested. If a plurality of second multimedia resources containing the key object are obtained through searching, the plurality of second multimedia resources can be directly recommended to the user, and can also be recommended to the user after being optimized according to preset screening conditions.
For example, when a plurality of second multimedia resources are obtained, one or more of duplicate removal, similarity removal, bad quality removal, network picture removal and/or network video removal may be performed on the second multimedia resources. When the second multimedia resource is optimized (such as duplicate removal, similarity removal, poor quality removal, network picture and/or network video removal), a corresponding intelligent model can be adopted to judge whether the optimization is needed. For example, for the optimization process of the similarity removal, a plurality of photos may be input to the similarity recognition model, which are determined by the similarity recognition model, and the similar photos may be labeled. In addition, after the similar photos are selected, optimization can be performed according to comparison of pixel values, for example, the photo with the highest pixel value in the similar photos is reserved, and other similar photos are deleted.
In addition, the similar removal and the poor removal can also be optimized according to the definition, the shooting parameters, the proportion of the core objects and the like of the photos, for example, the photos with the shooting parameters in a preset range are used as the similar photos, one photo is randomly reserved to delete other photos, or the best quality photo (such as the photo with the highest definition) is screened from the similar photos and the other photos are deleted; and the photos with the definition smaller than the preset value can be used as inferior photos and deleted. The network picture and the network video can be judged according to the source of the data, the content of the data and the like, for example, if the source of the data is an untrusted network, the data can be considered as the network picture or the network video; if the content of the data has the forbidden content, the data can be considered as a network picture or a network video.
Therefore, the embodiment optimizes the plurality of second multimedia resources and recommends the optimized second multimedia resources to the user, so that the second multimedia resources with higher quality can be recommended to the user, the time for the user to screen the second multimedia resources can be saved, and the resource processing efficiency of the user can be improved.
In addition, in this embodiment, the second multimedia resource may be displayed on the electronic screen of the intelligent album caretaker system body to perform resource recommendation, the second multimedia resource may also be sent to a user intelligent device (such as a mobile phone, a computer, etc.) to be displayed, and the recommendation may also be displayed on the electronic screen of the intelligent album caretaker system body and the user intelligent device synchronously, which is not limited in this embodiment.
Compared with the traditional method that a user manually checks all multimedia data and manually screens interested resources from all multimedia data, the method can determine the second multimedia resource with the same key object as the first multimedia resource based on the first multimedia resource selected by the user and automatically recommend the second multimedia resource to the user.
According to the resource processing method based on the intelligent photo album housekeeping system, provided by the embodiment of the invention, based on the first multimedia resource selected by the user, the second multimedia resource with the same key object as the first multimedia resource is determined, and the second multimedia resource is automatically recommended to the user, so that the second multimedia resource can be recommended according to the interest of the user, manual reference for each multimedia data is not needed, the efficiency is higher, and meanwhile, the problem of missed search caused by user error can be avoided.
In addition, the intelligent photo album housekeeping system can also be used for automatically reading multimedia incremental data in the intelligent equipment after the intelligent equipment establishes connection; the multimedia incremental data refers to multimedia data newly added on the intelligent device from the last time of reading from the intelligent device to the current time. For example, at time a, when the intelligent device is connected with the intelligent photo album housekeeping system, the intelligent photo album housekeeping system reads a video 1 and a video 2 in the intelligent device; when moment b smart machine and intelligent album butler system are connected, in the smart machine this moment except containing video 1 and video 2, still include video 3, the smart machine has increased video 3 (can regard video 3 as multimedia increment data) under moment b promptly, because video 1 and video 2 have been read last time, thereby under the second mode of operation, intelligent album butler system only can read video 3 this time, and can not read video 1 and video 2 again.
Therefore, compared with the traditional electronic photo album, the multimedia data in the intelligent device needs to be copied to the electronic photo album manually, and the intelligent photo album housekeeper system in the embodiment can automatically read and backup the multimedia data in the intelligent device when the connection is established with the intelligent device, so that the system is simpler, more convenient and faster.
Based on the foregoing embodiment, determining a key object included in a first multimedia resource according to the first multimedia resource specifically includes:
if the first multimedia resource is a picture, determining all objects contained in the picture, and determining key objects contained in the first multimedia resource according to the position of each object in the picture, the proportion of each object occupying the area of the picture and the definition of each object on the picture;
if the first multimedia resource is a video, the video is cut into a plurality of pictures, all objects contained in the corresponding picture are determined aiming at each picture obtained by cutting, and key objects contained in the corresponding picture are determined according to the position of each object in the corresponding picture, the proportion of each object in the area of the corresponding picture and the definition degree of each object on the corresponding picture; and determining the key objects contained in the first multimedia resource according to the key objects contained in the pictures.
In this embodiment, if the first multimedia resource is a picture, all objects included in the picture are determined, and a key object included in the first multimedia resource is determined according to a position of each object in the picture, a ratio of each object occupying an area of the picture, and a degree of clarity of each object on the picture. It can be understood that the closer the position of each object in the picture is to the center, the greater the proportion of the area of each object in the picture, and the higher the definition of each object on the picture, the greater the probability that the object is a key object.
It should be noted that the positions of the objects in the picture, the proportion of the objects in the picture area, and the weight of the definition of the objects in the picture may be set respectively, the key values of the objects are determined comprehensively, and the objects with the key values larger than the preset value are used as the key objects.
For example, when the position of the object in the picture is a preset position, the occupied weight coefficient is a, when the proportion of the object occupying the picture area is greater than the preset proportion, the occupied weight coefficient is b, when the definition of the object on the picture is greater than the preset definition, the occupied weight coefficient is c, finally, the weight coefficients are integrated to obtain the key value g = a + b + c of the object, and if g is greater than the preset value, the corresponding object is determined to be the key object.
If the first multimedia resource is a video, the video is cut into a plurality of pictures, all objects contained in the corresponding pictures are determined aiming at each picture obtained by cutting, and the key objects contained in the corresponding pictures are determined according to the positions of the objects in the corresponding pictures, the proportion of the objects in the corresponding pictures and the definition of the objects on the corresponding pictures. The method for determining the key object included in the corresponding picture may adopt the method for determining the key object included in the first multimedia resource in the foregoing embodiment, which is not described in detail in this embodiment.
After determining the key objects included in each picture, the sum of the key objects included in each picture may be used as the key objects included in the first multimedia resource, or after screening the key objects included in each picture according to a preset rule, the screened key objects are used as the key objects included in the first multimedia resource. For example, an object whose occurrence frequency of a key object included in each picture is higher than a preset value may be used as a key object included in the first multimedia resource.
Based on any of the embodiments described above, the determining a key object included in the first multimedia resource according to a key object included in each picture specifically includes any one of the following:
determining key objects with the occurrence frequency higher than a preset value according to the key objects contained in each picture, and taking the key objects with the occurrence frequency higher than the preset value as the key objects contained in the first multimedia resource;
and determining the key objects with the appearance frequency higher than the preset value according to the key objects contained in each picture, judging the continuous appearance condition of the key objects with the appearance frequency higher than the preset value in each picture for the key objects with the appearance frequency higher than the preset value, and taking the key objects with the continuous appearance condition meeting the preset condition as the key objects contained in the first multimedia resource.
In this embodiment, after determining the key objects included in each picture, the key objects with the occurrence frequency higher than the preset value are determined therefrom, and the key objects with the occurrence frequency higher than the preset value are taken as the key objects included in the first multimedia resource. For example, if the first multimedia resource is a video, and the video is divided into 10 pictures, the key object 1 appears in 5 pictures (i.e. the appearance frequency is 5/10= 0.5), the key object 2 appears in 3 pictures (i.e. the appearance frequency is 3/10= -0.3), the key object 3 appears in 8 pictures (i.e. the appearance frequency is 8/10= -0.8), and if the preset value is 0.4, the key object 1 and the key object 3 may be determined as key objects included in the first multimedia resource.
In addition, in this embodiment, a key object whose occurrence frequency is higher than a preset value may be determined according to the key objects included in each picture, and for the key object whose occurrence frequency is higher than the preset value, a continuous occurrence condition of the key object whose occurrence frequency is higher than the preset value in each picture is determined, and the key object whose continuous occurrence condition meets a preset condition is used as the key object included in the first multimedia resource. For example, if the first multimedia resource is a video and the video is cut into 10 pictures, the key object 1 appears in 5 pictures (i.e., the appearance frequency is 5/10= 0.5), the key object 2 appears in 3 pictures (i.e., the appearance frequency is 3/10= 0.3), the key object 3 appears in 8 pictures (i.e., the appearance frequency is 8/10= 0.8), and if the preset value is 0.4, the key object 1 and the key object 3 may be determined to be the pending key object. If the key object 1 appears in 2 consecutive pictures and the key object 3 appears in 5 consecutive pictures (the preset condition is satisfied, that the key object appears in at least 3 consecutive pictures), the key object 3 may be the key object included in the first multimedia resource.
Based on any of the above embodiments, searching for the second multimedia resource containing the key object in the intelligent album housekeeping system specifically includes:
inquiring the object name matched with the key object according to the feature data of the key object and the database table; the database table stores the corresponding relation between the feature data of each key object and the object name of the key object;
determining a second multimedia resource containing a key object according to the object name and the object content marking result of each multimedia resource in the intelligent album housekeeping system; when each multimedia resource is added into the intelligent album housekeeping system, an object content marking result is generated, and the object content marking result is marked with an object name contained in the multimedia resource.
In this embodiment, feature extraction is performed on the key object, so that feature data of the key object can be obtained, and an object name matching the key object can be determined by querying in the database table based on the feature data of the key object. The database table may be configured to establish and store a corresponding relationship between the feature data of each key object and the object name of the key object by acquiring the feature data of a large number of key objects in advance, labeling the object names corresponding to the feature data, and thus, based on the feature data of the key object, the object name matched with the key object can be quickly obtained by querying in the database table.
After the object name is determined, since each multimedia resource in the intelligent album housekeeping system carries the object content marking result, if the object name carried in the object content marking result is consistent with the object name matched with the key object, it can be determined that the corresponding multimedia resource contains the key object, that is, the multimedia resource is the second multimedia resource.
When each multimedia resource is added into the intelligent photo album housekeeper system, the object content mark can be marked manually, and an object content mark result is generated according to the manual marking; the multimedia resource can also be input into the object recognition model, and if the multimedia resource obtained by the object recognition model contains the preset object, a corresponding object content marking result is generated.
Therefore, according to the feature data and the data table of the key object, the object name matched with the key object can be quickly and accurately inquired in the database table, so that the second multimedia resource containing the key object can be accurately searched in the intelligent album manager system based on the object name, and further intelligent recommendation for the user is realized.
Based on any of the above embodiments, recommending the second multimedia resource specifically includes:
displaying a first multimedia resource selected by a user in a first display frame;
displaying the second multimedia resource in the second display frame; the second display frame is arranged adjacent to the first display frame.
In this embodiment, an electronic screen may be set on the smart album body, and the electronic screen is divided into at least 2 display areas, that is, a first display frame and a second display frame. The first display frame is used for displaying the first multimedia resource selected by the user, the second display frame is used for displaying the second multimedia resource, and the second display frame and the first display frame are arranged adjacently, so that the user can visually and simultaneously view the first multimedia resource and the second multimedia resource and can compare the first multimedia resource with the second multimedia resource. It can be understood that if the user thinks that the second multimedia resource is not the resource in which the user is interested, the user can select to delete the corresponding second multimedia resource, and at this time, a collected information bar appears in the electronic screen area, so that the user inputs the key object in the first multimedia resource under the corresponding account, and the intelligent album housekeeping system can memorize and store the content of interest of the account corresponding to the user, so as to accurately recommend the resource next time.
Based on any of the above embodiments, searching for the second multimedia resource containing the key object in the intelligent album housekeeping system specifically includes:
and searching a second multimedia resource containing the key object with the time within a preset time range in the intelligent photo album housekeeping system.
In this embodiment, after determining the key object of the first multimedia resource, the second multimedia resource including the key object whose time is within the preset time range is searched in the intelligent album housekeeping system, so that the resource which is interested by the user within the preset time range can be recommended to the user. The preset time range may be a time range input by a user according to actual requirements, or a time range preset based on the generation time of the first multimedia resource, for example, if the generation time of the first multimedia resource is 1 month and 15 days, the preset time range may be set to be 1 month and 1 day to 1 month and 31 days, so that the second multimedia resource generated in the month by the first multimedia resource may be searched.
Based on any of the above embodiments, recommending the second multimedia resource specifically includes:
and classifying and displaying the second multimedia resource according to the generation place of the second multimedia resource.
In this embodiment, after the second multimedia resource is determined, the second multimedia resource is classified and displayed according to the generation location of the second multimedia resource. For example, several folders may be created according to the generation location of the second multimedia resource, wherein the second multimedia resource corresponding to the generation location of the same province may be placed in the same folder, or the generation location of the first multimedia resource may be the central point, and the area may be divided according to the distance from the central point, such as creating folder 1 at the generation location within 1km from the central point, creating folder 2 at the generation location between 1km and 5km from the central point, and creating folder 3 at the generation location between 5km and 10km from the central point.
It should be noted that, for each multimedia resource newly added to the intelligent album caretaker system, a generation location mark may be performed on the multimedia resource, where the mark may be a geographic location coordinate of the generation location, so that the distance between the generation location of the second multimedia resource and the generation location of the first multimedia resource may be conveniently calculated.
Therefore, the second multimedia resources are classified and displayed according to the generation places of the second multimedia resources, so that the user can quickly and accurately select the second multimedia resources of corresponding categories to view according to actual requirements, and time of the user is saved.
Based on any of the above embodiments, determining a key object included in a first multimedia resource according to the first multimedia resource specifically includes:
if the first multimedia resource is a picture, performing target segmentation on the picture to obtain a segmented target; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library to determine a key object contained in the first media resource; the preset image feature library stores image features of all scenic spots, buildings or meeting places;
if the first multimedia resource is a video, cutting the video into a plurality of pictures, and performing target segmentation on the pictures aiming at each picture obtained by cutting to obtain segmented targets; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library to determine a key object contained in the first media resource; the preset image feature library stores image features of all scenic spots or buildings or meeting places.
In this embodiment, if the first multimedia resource is a picture and the picture includes a large amount of information about the scenery spot, the building or the meeting place, the scenery spot, the building or the meeting place may be a key object of the first multimedia resource, and the key object cannot be accurately extracted based on the position in the picture, the ratio of the scenery spot to the building area, and the definition of the scenery spot on the picture. Therefore, the present embodiment performs object segmentation on the picture to obtain the segmented object, for example, the background portion in the picture may be segmented to obtain the segmented object.
After the segmented target is obtained, performing feature extraction on the segmented target to obtain the image feature of the segmented target; and matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library, and if the image characteristics are consistent with the image characteristics of the scenic spots, buildings or meeting places stored in the preset image characteristic library, taking the corresponding target as a key object contained in the first media resource. The preset image feature library can be established by collecting a large number of pictures of scenic spots or buildings or meeting places and extracting image features corresponding to the pictures.
If the first multimedia resource is a video, cutting the video into a plurality of pictures, and performing target segmentation on the pictures aiming at each picture obtained by cutting to obtain segmented targets; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library, determining key objects in all pictures, and determining key objects contained in the first media resource based on the key objects in all the pictures; the preset image feature library stores image features of all scenic spots or buildings or meeting places. The method for determining the image characteristics of the segmented target may adopt the method for determining the image characteristics of the segmented target when the first multimedia resource is a picture in the above embodiment, which is not described in detail in this embodiment.
It should be noted that, if the first multimedia resource is a video, after determining the key objects in each picture, the sum of the key objects included in each picture may be used as the key objects included in the first multimedia resource, or after filtering the key objects included in each picture according to a preset rule, the filtered key objects are used as the key objects included in the first multimedia resource. For example, an object whose occurrence frequency of a key object included in each picture is higher than a preset value may be used as a key object included in the first multimedia resource, or a key object whose occurrence frequency is higher than a preset value may be determined according to a key object included in each picture, and for the key object whose occurrence frequency is higher than the preset value, a continuous occurrence condition of the key object whose occurrence frequency is higher than the preset value in each picture is determined, and the key object whose continuous occurrence condition satisfies a preset condition is used as the key object included in the first multimedia resource.
The following describes the resource processing device based on the intelligent album housekeeping system according to the present invention, and the following described resource processing device based on the intelligent album housekeeping system and the above described resource processing method based on the intelligent album housekeeping system may be referred to each other.
Based on any embodiment, the present invention provides a resource processing apparatus based on an intelligent album housekeeping system, as shown in fig. 2, the apparatus includes:
a first determining module 210, configured to determine a first multimedia resource selected by a user in an intelligent photo album housekeeping system;
a second determining module 220, configured to determine, according to the first multimedia resource, a key object included in the first multimedia resource;
the searching module 230 is used for searching a second multimedia resource containing a key object in the intelligent photo album housekeeping system;
and a recommending module 240, configured to recommend the second multimedia resource.
Based on any of the above embodiments, the second determining module 220 includes:
the first object determining module is used for determining all objects contained in the picture if the first multimedia resource is the picture, and determining key objects contained in the first multimedia resource according to the position of each object in the picture, the proportion of each object occupying the area of the picture and the definition of each object on the picture;
the second object determining module is used for cutting the video into a plurality of pictures if the first multimedia resource is the video, determining all objects contained in the corresponding picture aiming at each picture obtained by cutting, and determining key objects contained in the corresponding picture according to the position of each object in the corresponding picture, the proportion of each object in the area of the corresponding picture and the definition of each object on the corresponding picture; and determining the key objects contained in the first multimedia resource according to the key objects contained in the pictures.
Based on any of the embodiments, the second object determining module is specifically configured to:
determining a key object with the frequency higher than a preset value according to the key objects contained in each picture, and taking the key object with the frequency higher than the preset value as the key object contained in the first multimedia resource;
or, determining a key object with an appearance frequency higher than a preset value according to the key objects contained in each picture, judging the continuous appearance condition of the key object with the appearance frequency higher than the preset value in each picture for the key object with the appearance frequency higher than the preset value, and taking the key object with the continuous appearance condition meeting the preset condition as the key object contained in the first multimedia resource.
Based on any of the embodiments above, the searching module 230 is specifically configured to:
inquiring the object name matched with the key object according to the feature data of the key object and a database table; the database table stores the corresponding relation between the feature data of each key object and the object name of the key object;
determining a second multimedia resource containing the key object according to the object name and the object content marking result of each multimedia resource in the intelligent photo album housekeeping system; when each multimedia resource is added into the intelligent photo album housekeeping system, an object content marking result is generated, and the object content marking result is marked with an object name contained in the multimedia resource.
Based on any of the above embodiments, the recommending module 240 is specifically configured to:
displaying a first multimedia resource selected by a user in a first display frame;
displaying the second multimedia resource in a second display frame; wherein the second display frame is arranged adjacent to the first display frame.
Based on any of the embodiments above, the searching module 230 is specifically configured to:
and searching a second multimedia resource containing the key object with the time within a preset time range in the intelligent photo album manager system.
Based on any of the above embodiments, the recommending module 240 is specifically configured to:
and classifying and displaying the second multimedia resource according to the generation place of the second multimedia resource.
Based on any of the above embodiments, the second determining module 220 is specifically configured to:
if the first multimedia resource is a picture, performing target segmentation on the picture to obtain a segmented target; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library to determine a key object contained in the first media resource; the preset image feature library stores image features of all scenic spots, buildings or meeting places;
if the first multimedia resource is a video, cutting the video into a plurality of pictures, and performing target segmentation on each picture obtained by cutting to obtain a segmented target; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library, and determining a key object contained in the first media resource; and the preset image feature library stores the image features of all scenic spots, buildings or meeting places.
Fig. 3 is a schematic structural diagram of an electronic device provided in the present invention, and as shown in fig. 3, the electronic device may include: a processor (processor) 310, a memory (memory) 320, a communication interface (communications interface) 330 and a communication bus 340, wherein the processor 310, the memory 320 and the communication interface 330 communicate with each other via the communication bus 340. The processor 310 may call logic instructions in the memory 320 to perform a resource handling method based on the intelligent photo album caretaker system, the method comprising: determining a first multimedia resource selected by a user in an intelligent photo album housekeeping system; determining a key object contained in the first multimedia resource according to the first multimedia resource; searching a second multimedia resource containing the key object in the intelligent photo album manager system; and recommending the second multimedia resource.
In addition, the logic instructions in the memory 320 may be implemented in the form of software functional units and stored in a computer readable storage medium when the logic instructions are sold or used as independent products. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a removable hard disk, a Read-only memory (ROM), a Random Access Memory (RAM), a magnetic disk, an optical disk, or other various media capable of storing program codes.
In another aspect, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, the computer program including program instructions, when the program instructions are executed by a computer, the computer being capable of executing the resource processing method based on the intelligent photo album manager system provided by the above methods, the method including: determining a first multimedia resource selected by a user in an intelligent photo album housekeeping system; determining a key object contained in the first multimedia resource according to the first multimedia resource; searching a second multimedia resource containing the key object in the intelligent photo album housekeeping system; and recommending the second multimedia resource.
In yet another aspect, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, the computer program being implemented by a processor to perform the resource processing method based on the intelligent photo album manager system provided in the above aspects, the method including: determining a first multimedia resource selected by a user in an intelligent photo album housekeeper system; determining a key object contained in the first multimedia resource according to the first multimedia resource; searching a second multimedia resource containing the key object in the intelligent photo album housekeeping system; and recommending the second multimedia resource.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of this embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment may be implemented by software plus a necessary general hardware platform, and may also be implemented by hardware. With this understanding in mind, the above-described technical solutions may be embodied in the form of a software product, which can be stored in a computer-readable storage medium such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the methods described in the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A resource processing method based on an intelligent photo album housekeeping system is characterized by comprising the following steps:
determining a first multimedia resource selected by a user in an intelligent photo album housekeeping system;
determining a key object contained in the first multimedia resource according to the first multimedia resource;
searching a second multimedia resource containing the key object in the intelligent photo album housekeeping system;
and recommending the second multimedia resource.
2. The resource processing method based on the intelligent album manager system according to claim 1, wherein determining a key object included in the first multimedia resource according to the first multimedia resource specifically includes:
if the first multimedia resource is a picture, determining all objects contained in the picture, and determining key objects contained in the first multimedia resource according to the position of each object in the picture, the proportion of each object in the area of the picture and the definition of each object on the picture;
if the first multimedia resource is a video, cutting the video into a plurality of pictures, determining all objects contained in the corresponding pictures aiming at each picture obtained by cutting, and determining key objects contained in the corresponding pictures according to the positions of the objects in the corresponding pictures, the proportion of the objects in the corresponding picture areas and the definition of the objects on the corresponding pictures; and determining the key objects contained in the first multimedia resource according to the key objects contained in the pictures.
3. The resource processing method based on the intelligent album manager system according to claim 2, wherein the determining the key object included in the first multimedia resource according to the key object included in each picture specifically includes any one of the following:
determining a key object with the frequency higher than a preset value according to the key objects contained in each picture, and taking the key object with the frequency higher than the preset value as the key object contained in the first multimedia resource;
and determining key objects with the occurrence frequency higher than a preset value according to the key objects contained in each picture, judging the continuous occurrence condition of the key objects with the occurrence frequency higher than the preset value in each picture for the key objects with the occurrence frequency higher than the preset value, and taking the key objects with the continuous occurrence condition meeting the preset condition as the key objects contained in the first multimedia resource.
4. The resource processing method based on the intelligent album housekeeping system according to claim 1, wherein searching for the second multimedia resource containing the key object in the intelligent album housekeeping system specifically comprises:
inquiring the object name matched with the key object according to the feature data of the key object and a database table; the database table stores the corresponding relation between the feature data of each key object and the object name of the key object;
determining a second multimedia resource containing the key object according to the object name and the object content marking result of each multimedia resource in the intelligent photo album housekeeping system; when each multimedia resource is added into the intelligent album housekeeping system, an object content marking result is generated, and the object content marking result is marked with an object name contained in the multimedia resource.
5. The resource processing method based on the intelligent photo album housekeeping system according to claim 1, wherein recommending the second multimedia resource specifically comprises:
displaying a first multimedia resource selected by a user in a first display frame;
displaying the second multimedia resource in a second display frame; wherein the second display frame is arranged adjacent to the first display frame.
6. The resource processing method based on the intelligent album housekeeping system according to claim 1, wherein searching for the second multimedia resource containing the key object in the intelligent album housekeeping system specifically comprises:
and searching a second multimedia resource containing the key object with the time within a preset time range in the intelligent photo album housekeeping system.
7. The resource processing method based on the intelligent photo album housekeeping system according to claim 1, wherein recommending the second multimedia resource specifically comprises:
and according to the generation place of the second multimedia resource, classifying and displaying the second multimedia resource according to the generation place.
8. The resource processing method based on the intelligent photo album housekeeping system according to claim 1, wherein determining the key object included in the first multimedia resource according to the first multimedia resource specifically comprises:
if the first multimedia resource is a picture, performing target segmentation on the picture to obtain a segmented target; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library, and determining a key object contained in the first media resource; the preset image feature library stores image features of all scenic spots, buildings or meeting places;
if the first multimedia resource is a video, cutting the video into a plurality of pictures, and performing target segmentation on each picture obtained by cutting to obtain a segmented target; performing feature extraction on the segmented target to obtain the image features of the segmented target; matching the image characteristics of the segmented target with the image characteristics stored in a preset edge profile library to determine a key object contained in the first media resource; and the preset image feature library stores the image features of all scenic spots, buildings or meeting places.
9. A resource processing device based on an intelligent photo album housekeeping system is characterized by comprising:
the system comprises a first determining module, a first judging module and a second determining module, wherein the first determining module is used for determining a first multimedia resource selected by a user in an intelligent photo album housekeeping system;
the second determining module is used for determining a key object contained in the first multimedia resource according to the first multimedia resource;
the searching module is used for searching a second multimedia resource containing the key object in the intelligent photo album housekeeping system;
and the recommending module is used for recommending the second multimedia resource.
10. An electronic device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the processor when executing the program implements the steps of the intelligent photo album manager system based resource handling method according to any one of claims 1 to 9.
CN202110756191.4A 2021-07-05 2021-07-05 Resource processing method and device based on intelligent photo album housekeeper system Pending CN115587202A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110756191.4A CN115587202A (en) 2021-07-05 2021-07-05 Resource processing method and device based on intelligent photo album housekeeper system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110756191.4A CN115587202A (en) 2021-07-05 2021-07-05 Resource processing method and device based on intelligent photo album housekeeper system

Publications (1)

Publication Number Publication Date
CN115587202A true CN115587202A (en) 2023-01-10

Family

ID=84771003

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110756191.4A Pending CN115587202A (en) 2021-07-05 2021-07-05 Resource processing method and device based on intelligent photo album housekeeper system

Country Status (1)

Country Link
CN (1) CN115587202A (en)

Similar Documents

Publication Publication Date Title
JP4305079B2 (en) Image management device
US10762126B2 (en) System and method for reducing similar photos for display and product design
US20160321831A1 (en) Computer-readable recording medium having stored therein album producing program, album producing method, and album producing device
CN104657423A (en) Method and device thereof for sharing contents of applications
US20100054601A1 (en) Image Tagging User Interface
US10276213B2 (en) Automatic and intelligent video sorting
KR101832680B1 (en) Searching for events by attendants
CN110232131B (en) Creative material searching method and device based on creative tag
CN107590267A (en) Information-pushing method and device, terminal and readable storage medium storing program for executing based on picture
CN111932394A (en) Bad root cause path analysis method and system based on association rule mining
CN109116718B (en) Method and device for setting alarm clock
CN108921193B (en) Picture input method, server and computer storage medium
CN110781195A (en) System, method and device for updating point of interest information
CN112417195A (en) Trademark inquiry system and method based on mobile terminal and storage medium
EP3564833B1 (en) Method and device for identifying main picture in web page
CN115587202A (en) Resource processing method and device based on intelligent photo album housekeeper system
CN113837181B (en) Screening method, screening device, computer equipment and computer readable storage medium
CN113657370B (en) Character recognition method and related equipment thereof
CN114187180A (en) Picture splicing method and device
CN114780780A (en) Image retrieval method, image retrieval device, electronic equipment and storage medium
CN104978623A (en) Personnel information management service system
US20200380748A1 (en) System and method for reducing similar photos for display and product design
CN111163170A (en) Photo sharing method, system and server
CN113553980A (en) Method, system and device for generating trademark graphic element codes of pictures
US20220027609A1 (en) Webpage illustration processing method, system, device and storage medium

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination