CN114693535A

CN114693535A - Image processing method, image processing device, storage medium and computer equipment

Info

Publication number: CN114693535A
Application number: CN202011624292.8A
Authority: CN
Inventors: 高占宁; 任沛然; 王攀; 谢宣松
Original assignee: Alibaba Group Holding Ltd
Current assignee: Alibaba Group Holding Ltd
Priority date: 2020-12-30
Filing date: 2020-12-30
Publication date: 2022-07-01

Abstract

The invention discloses an image processing method, an image processing device, a storage medium and computer equipment. Wherein, the method comprises the following steps: acquiring an image; extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; determining a trimming frame of the image according to the visual characteristics; and cutting the image according to the cutting frame to obtain the cut image. The invention solves the technical problem of inaccurate cutting when the image is cut in the related technology.

Description

Image processing method, image processing device, storage medium and computer equipment

Technical Field

The present invention relates to the field of image processing, and in particular, to an image processing method, an image processing apparatus, a storage medium, and a computer device.

Background

When processing images, it is often necessary to adjust the frame. For example, the original image is subjected to a scaling process of stretching and compressing, or the original image is cropped in a center-on-center cropping manner, and a cropped image is obtained. However, the visual content in the image is very complicated and varied, and in the related art, a high-quality image is often not obtained after the frame is adjusted, and the content and composition of the image frame have problems of distortion, deficiency and the like.

In view of the above problems, no effective solution has been proposed.

Disclosure of Invention

The embodiment of the invention provides an image processing method, an image processing device, a storage medium and computer equipment, which are used for at least solving the technical problem of inaccurate cutting when an image is cut in the related art.

According to an aspect of an embodiment of the present invention, there is provided an image processing method including: acquiring an image; extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; determining a trimming frame of the image according to the visual characteristics; and cutting the image according to the cutting frame to obtain the cut image.

Optionally, determining a trimming frame of the image according to the visual feature includes: under the condition that the visual features comprise a plurality of feature regions, distributing different weight values for the feature regions, wherein the weight values represent the importance of the feature regions; determining a cropping frame of the image according to the plurality of feature areas after the weight is distributed, wherein the cropping frame comprises the feature area with the maximum weight in the plurality of feature areas.

Optionally, determining a trimming frame of the image according to the plurality of feature areas to which the weight values are assigned includes: fusing the plurality of feature areas with different weights to form a thermodynamic diagram, wherein different colors in the thermodynamic diagram represent feature areas with different weights; and determining the size and the position of the cutting frame according to the thermodynamic diagram.

Optionally, cutting the image according to the cutting frame to obtain a cut image, including: determining that the trimming frame exceeds an excess portion of the image in a case where the trimming frame exceeds the image itself; and after the image is cut according to the cutting frame, filling the excess part to obtain the cut image.

Optionally, in a case that the image includes a plurality of shot images, and the plurality of shot images are obtained by video cutting, the method further includes: and splicing the cut multiple lens images to obtain a cut video.

Optionally, before stitching the cut multiple lens images to obtain a cut video, the method further includes: and smoothing the cutting frame in each lens image in the plurality of lens images by adopting a spline interpolation method to obtain the smoothed cutting frame, wherein the smoothed cutting frame is used for cutting to obtain a plurality of cut lens images.

Optionally, extracting visual features of the image comprises: inputting the image into an image feature model to obtain visual features of the image, wherein the image feature model is obtained by performing machine learning by adopting a first data set, and training data in the first data set comprises: images and visual features of images.

Optionally, the image includes a plurality of material images of a predetermined object, and after the image is cropped according to the cropping frame to obtain a cropped image, the method further includes: inputting each material image in the plurality of cut material images into an image special effect model to obtain a special effect image of each cut material image, wherein the image special effect model is obtained by adopting a second data set to perform machine learning, and training data in the second data set comprises: a material image and a special effect image of the material image; and splicing the special effect images corresponding to the plurality of material images to obtain the video of the preset object.

Optionally, the stitching the special effect images corresponding to the plurality of material images to obtain the video of the predetermined object includes: displaying the special effect images corresponding to the plurality of material images on a display interface; receiving adjustment operation for adjusting the special effect images corresponding to the plurality of material images; responding to the adjustment operation, and adjusting the special effect images corresponding to the plurality of material images to obtain a plurality of adjusted special effect images; and splicing the adjusted plurality of special effect images to obtain the video of the preset object.

According to another aspect of the embodiments of the present invention, there is also provided an image processing method, including: displaying an image on the interactive interface; displaying visual features of the image on the interactive interface, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; displaying a cropping frame of the image on the interactive interface, wherein the cropping frame is determined according to the visual features; and displaying a cutting result on the interactive interface, wherein the cutting result is obtained by cutting the image according to the cutting frame.

According to still another aspect of an embodiment of the present invention, there is also provided an image processing apparatus including: the first acquisition module is used for acquiring an image; the extraction module is used for extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; a determining module, configured to determine a trimming frame of the image according to the visual feature; and the cutting module is used for cutting the image according to the cutting frame to obtain the cut image.

According to still another aspect of an embodiment of the present invention, there is also provided an image processing apparatus including: the first display module is used for displaying images on the interactive interface; the second display module is used for displaying visual features of the image on the interactive interface, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; a third display module, configured to display a trimming frame of the image on the interactive interface, where the trimming frame is determined according to the visual feature; and the fourth display module is used for displaying a cutting result on the interactive interface, wherein the cutting result is obtained by cutting the image according to the cutting frame.

According to still another aspect of the embodiments of the present invention, there is provided a storage medium including a stored program, wherein when the program runs, a device on which the storage medium is located is controlled to execute any one of the image processing methods described above.

According to yet another aspect of the embodiments of the present invention, there is also provided a computer apparatus including a memory storing a computer program and a processor; the processor is configured to execute the computer program stored in the memory, and when the computer program runs, the processor is enabled to execute any one of the image processing methods.

According to still another aspect of the embodiments of the present invention, there is also provided an image processing method including: acquiring a material of an object; identifying the information of the material by adopting an artificial intelligence identification algorithm; and performing visual processing on the material by adopting an artificial intelligence visual algorithm according to the information of the material to obtain the video of the object.

Optionally, in a case that the material is an image material, identifying information of the material by using the artificial intelligence identification algorithm includes at least one of: identifying the category of the image material by adopting a category algorithm in the artificial intelligence identification algorithm; identifying the position of a target object in the image material by adopting a position algorithm in the artificial intelligence identification algorithm; identifying a salient region in the image material by adopting a characteristic algorithm in the artificial intelligence identification algorithm; and identifying the aesthetic score of the image material by adopting a scoring algorithm in the artificial intelligent identification algorithm.

Optionally, the material is visually processed by using an artificial intelligence visual algorithm, including at least one of: and screening the materials by adopting an artificial intelligent vision algorithm, cutting a target object in the image materials, arranging various types of materials, and rendering the image materials.

Optionally, outputting, by an interaction device, an intermediate result, wherein the intermediate result includes at least one of: and adopting the artificial intelligence vision algorithm to perform vision processing on the material according to the result of the information of the material identified by the artificial intelligence identification algorithm.

According to still another aspect of an embodiment of the present invention, there is also provided an image processing apparatus including: the second acquisition module is used for acquiring the material of the object; the identification module is used for identifying the information of the material by adopting an artificial intelligence identification algorithm; and the processing module is used for carrying out visual processing on the material by adopting an artificial intelligence visual algorithm according to the information of the material to obtain the video of the object.

In the embodiment of the invention, the mode of obtaining the image and extracting the visual characteristics of the image is adopted, the purpose of obtaining the image after the image is cut according to the requirement is achieved by determining the cutting frame of the image according to the visual characteristics and cutting the image according to the cutting frame, so that the technical effect of reasonably cutting the image according to the visual content of the image is realized, and the technical problem of inaccurate cutting when the image is cut in the related technology is solved.

Drawings

The accompanying drawings, which are included to provide a further understanding of the invention and are incorporated in and constitute a part of this application, illustrate embodiment(s) of the invention and together with the description serve to explain the invention without limiting the invention. In the drawings:

fig. 1 is a block diagram of a hardware configuration of a computer terminal for implementing an image processing method according to an embodiment of the present invention;

FIG. 2 is a flowchart of a first image processing method according to embodiment 1 of the present invention;

FIG. 3 is a flowchart of a second image processing method according to embodiment 1 of the present invention;

FIG. 4 is a schematic diagram of a video generating method using image processing according to an embodiment of the present invention;

FIG. 5 is a flowchart of an image processing method three according to embodiment 1 of the present invention;

FIG. 6 is a schematic flow diagram of an image processing method according to an alternative embodiment of the invention;

fig. 7 is a block diagram of a first configuration of an image processing apparatus according to embodiment 2 of the present invention;

FIG. 8 is a block diagram showing a second configuration of an image processing apparatus according to embodiment 3 of the present invention;

FIG. 9 is a block diagram showing a third configuration of an image processing apparatus according to embodiment 4 of the present invention;

fig. 10 is a block diagram of a computer terminal according to an embodiment of the present invention.

Detailed Description

In order to make the technical solutions of the present invention better understood, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

It should be noted that the terms "first," "second," and the like in the description and claims of the present invention and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used is interchangeable under appropriate circumstances such that the embodiments of the invention described herein are capable of operation in sequences other than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

Example 1

There is also provided, in accordance with an embodiment of the present invention, an image processing method embodiment, it should be noted that the steps illustrated in the flowchart of the accompanying drawings may be performed in a computer system such as a set of computer-executable instructions, and that, although a logical order is illustrated in the flowchart, in some cases, the steps illustrated or described may be performed in an order different than here.

The method provided by the embodiment 1 of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Fig. 1 shows a hardware configuration block diagram of a computer terminal (or mobile device) for implementing an image processing method. As shown in fig. 1, the computer terminal 10 (or mobile device) may include one or more (shown as 102a, 102b, … …, 102 n) processors 102 (the processors 102 may include, but are not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA, etc.), memories 104 for storing data, and a transmission device for communication functions. Besides, the method can also comprise the following steps: a display, an input/output interface (I/O interface), a Universal Serial BUS (USB) port (which may be included as one of the ports of the BUS), a network interface, a power source, and/or a camera. It will be understood by those skilled in the art that the structure shown in fig. 1 is only an illustration and is not intended to limit the structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components than shown in FIG. 1, or have a different configuration than shown in FIG. 1.

It should be noted that the one or more processors 102 and/or other data processing circuitry described above may be referred to generally herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part in software, hardware, firmware, or any combination thereof. Further, the data processing circuit may be a single stand-alone processing module, or incorporated in whole or in part into any of the other elements in the computer terminal 10 (or mobile device). As referred to in the embodiments of the application, the data processing circuit acts as a processor control (e.g. selection of a variable resistance termination path connected to the interface).

The memory 104 may be used to store software programs and modules of application software, such as program instructions/data storage devices corresponding to the image processing method in the embodiment of the present invention, and the processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the image processing method of the application program. The memory 104 may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include memory located remotely from the processor 102, which may be connected to the computer terminal 10 via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The transmission device is used for receiving or transmitting data via a network. Specific examples of the network described above may include a wireless network provided by a communication provider of the computer terminal 10. In one example, the transmission device includes a Network adapter (NIC) that can be connected to other Network devices through a base station so as to communicate with the internet. In one example, the transmission device may be a Radio Frequency (RF) module, which is used for communicating with the internet in a wireless manner.

The display may be, for example, a touch screen type Liquid Crystal Display (LCD) that may enable a user to interact with a user interface of the computer terminal 10 (or mobile device).

Under the above operating environment, the present application provides an image processing method as shown in fig. 2. Fig. 2 is a flowchart of a first image processing method according to embodiment 1 of the present invention. As shown in fig. 2, the method comprises the steps of:

step S202, acquiring an image;

step S204, extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image;

step S206, determining a trimming frame of the image according to the visual characteristics;

and S208, cutting the image according to the cutting frame to obtain the cut image.

In the above steps, by adopting a mode of obtaining the image and extracting the visual characteristics of the image, determining the cutting frame of the image according to the visual characteristics and cutting the image according to the cutting frame, the purpose that the obtained image is the image cut according to the information content of the image is achieved, so that the technical effect of reasonably cutting the image according to the visual content of the image is achieved, and the technical problem of inaccurate cutting when the image is cut in the related technology is solved.

As an alternative embodiment, the image may be input into an image feature model to obtain visual features of the image, where the image feature model is obtained by performing machine learning using a first data set, and training data in the first data set includes: images and visual features of images. The image feature model can analyze the visual content in the image and extract the visual features of the image from the visual content. The image feature model is obtained by performing machine learning training by adopting training data comprising images and visual features of the images, so that the visual features of various types of images can be extracted. For example, visual features such as faces, text, subjects, etc. may be extracted from the image. By adopting the image characteristic model, the efficiency and the accuracy of extracting the visual characteristics from the image can be improved, and the subsequent adjustment of the image frame is convenient.

As an alternative embodiment, the trimming frame of the image may be determined as follows: under the condition that the visual features comprise a plurality of feature regions, distributing different weights for the plurality of feature regions, wherein the weights represent the importance of the feature regions; and determining a cropping frame of the image according to the plurality of feature areas after the weight is distributed, wherein the cropping frame comprises the feature area with the maximum weight in the plurality of feature areas. When an image includes a plurality of feature areas, cropping the image using the cropping frame sometimes faces a need to accept or reject the plurality of feature areas, because the cropping frame meeting the frame size ratio requirement sometimes cannot select all the feature area frames. At this time, according to the present embodiment, the above problem can be solved, and the present embodiment may allocate weights representing the importance of the area to a plurality of feature areas, and then ensure that the most important feature area is included in the cropping frame, thereby implementing retention of the most important information of the image in the process of cropping the image.

As an alternative embodiment of the present invention, the weight may be assigned to the feature region in a variety of ways, for example, information such as the image type, area, and relationship with the peripheral feature region of the feature region may be first identified, and then the weight of each feature region may be determined according to the information. For example, when the image type of the feature region is recognized as different types such as a face region, a text region, a body region, and the like, different weights are given according to the type; when the area of the characteristic region is remarkably large, a higher weight is distributed to the characteristic region; or when most of the feature regions are distributed around a particular feature region, a higher weight is assigned to the feature region at the center. By assigning weights to a plurality of feature regions, the importance of the plurality of feature regions can be well distinguished.

As an alternative embodiment, according to a plurality of feature areas after the weight values are assigned, the trimming frame of the image may be determined as follows: fusing the plurality of characteristic regions with different weights to form a thermodynamic diagram, wherein different colors in the thermodynamic diagram represent the characteristic regions with different weights; and determining the size and the position of the cutting frame according to the thermodynamic diagram. When the number of the feature areas is large, weights are given to a plurality of isolated feature areas, and overall analysis is complex, so that a good cutting frame demarcation effect is difficult to obtain. According to the respective weights of the plurality of characteristic areas, the weight distribution of the characteristic areas of the whole image is drawn into a thermodynamic diagram, so that the important area in the whole image can be simply and clearly determined, and the size and the position of the cutting frame can be conveniently set. Different colors in the thermodynamic diagram represent the weight of the region, namely the importance of the region, and the discrete feature regions in the image can be fused into a continuous image weight distribution function by the method, so that the basis for selecting the cropping frame is optimized.

As an alternative embodiment, in the case where the trimming frame exceeds the image itself, the excess portion of the trimming frame exceeding the image is determined; and after the image is cut according to the cutting frame, filling the excess part to obtain the cut image. In this embodiment, the excess image portion is filled, so that the excess portion is not too abrupt, and the overall style of each image area in the picture is consistent and the visual effect is similar.

As an optional embodiment, in a case that the image includes a plurality of lens images, and the plurality of lens images are obtained by video cutting, the embodiment may further stitch the plurality of cut lens images to obtain a cut video. The cut lens images belonging to the same lens are spliced, so that the technical effect of cutting the lens video can be achieved. Since the visual characteristics in the video change rapidly, it is difficult to directly crop the video, and this embodiment provides a method for cropping the video according to the importance of the visual characteristics in a plurality of shot images, so that the process of cropping the video is simpler.

As an optional embodiment, before the plurality of cut lens images are spliced to obtain the cut video, a cut frame in each of the plurality of lens images may be smoothed by a spline interpolation method to obtain a smoothed cut frame, where the smoothed cut frame is used for cutting to obtain the plurality of cut lens images. Since the positions of the cropping frames respectively selected from the multiple shot images in the respective images may not be completely consistent, when the images in the two cropping frames are processed into a continuous video, jitter may occur in the video. For example, the face of a person is selected by using a cropping frame in the two images, but the respective cropping frames cannot be completely overlapped due to the influence of other areas in the images when the cropping frame is created. In the embodiment, the trimming frame is smoothed by a spline interpolation method, so that the positions of the trimming frames in a plurality of images are approximately equivalent, and when the images are processed into videos, the camera shake of the video images can be avoided, and a better video playing effect can be obtained.

As an alternative embodiment, in a case that an image includes a plurality of material images of a predetermined object, after the image is cut according to a cutting frame and a cut image is obtained, each of the plurality of cut material images may be input into an image special effect model to obtain a special effect image of each cut material image, and then the special effect images corresponding to the plurality of material images are spliced to obtain a video of the predetermined object. The image special effect model is obtained by machine learning through a second data set, and training data in the second data set comprise material images and special effect images of the material images. The special effect is added to the material image through the image special effect model to generate a special effect image, and richer effects can be added to the cut image. In addition, after a plurality of material images of the preset object are cut, the input image special effect model is used for carrying out special effect addition, so that the special effect styles of the material images related to the preset object can be kept consistent, the video styles of the preset object obtained through splicing are unified, and the characteristics are distinct. The image special effect model obtained by machine learning training can reduce the workload of adding special effects to the image, and the material image obtains more stable and more targeted special effect effects.

As an optional embodiment, the special effect images corresponding to the plurality of material images may be stitched in the following manner: displaying the special effect images corresponding to the plurality of material images on a display interface, receiving adjustment operation for adjusting the special effect images corresponding to the plurality of material images, responding to the adjustment operation, adjusting the special effect images corresponding to the plurality of material images to obtain a plurality of adjusted special effect images, and splicing the plurality of adjusted special effect images to obtain a video of a preset object. Through the embodiment, the user can be allowed to adjust the added special effect type in real time or afterwards when adding the special effect to the material image, so that the special effect of the video obtained after splicing can be adjusted according to the feedback of the user, and the autonomy of the user and the freedom degree of the video are increased.

Fig. 3 is a flowchart of an image processing method two according to embodiment 1 of the present invention. As shown in fig. 3, the method comprises the steps of:

step S302, obtaining the material of the object;

step S304, identifying information of the material by adopting an artificial intelligence identification algorithm;

and S306, performing visual processing on the material by adopting an artificial intelligence visual algorithm according to the information of the material to obtain the video of the object.

Through the steps, the information of the material is identified by adopting the artificial intelligence identification algorithm, and the material is subjected to visual processing by adopting the artificial intelligence visual algorithm, so that the aim of manufacturing the video by adopting the artificial intelligence in the whole process is fulfilled, and the technical effects of efficiently and accurately manufacturing the video are achieved.

As an alternative embodiment, the object may be a target object for producing a video, may be a concrete real object, and may also be an abstract virtual object. For example, when the object is a commodity, the object may be a concrete real object of the commodity or a virtual model of the commodity.

As an alternative embodiment, when the material of the object is obtained, the type of the material may be various, for example, the material may be an image, a text, a video, an audio, an animation, or the like. The above-mentioned multiple types of materials are ways of describing the object in multiple ways. For various materials, the artificial intelligence algorithm used in identifying the information of the materials may be different.

As an alternative embodiment, in the case that the material is an image material, the information of the material is identified by using an artificial intelligence identification algorithm, and the artificial intelligence identification algorithm includes at least one of the following: identifying the category of the pixel material by adopting a category algorithm in an artificial intelligence identification algorithm; identifying the position of a target object in the image material by adopting a position algorithm in an artificial intelligence identification algorithm; identifying a salient region in the image material by adopting a characteristic algorithm in an artificial intelligence identification algorithm; and identifying the aesthetic scores of the image materials by adopting a scoring algorithm in the artificial intelligence identification algorithm. The type of the image may be a category into which the object is divided, for example, when the object is a commodity, the category may be a category such as clothing, articles for daily use, food, and the like. The position of the target object in the image material may refer to the position of a person, the position of an animal, the position of some markers, and the like in the image material. The salient region in the image material may be a region in the image material for explaining or reminding a user of attention, for example, when the object is a commodity (bowl), whether the bowl can be identified by using a microwave oven, or the like. The aesthetic score for the image material referred to above may be whether the image material can be used directly for subsequent video splicing or use.

As an alternative embodiment, the visual processing of the material using artificial intelligence visual algorithms can include a variety of techniques, for example, at least one of: and screening the materials by adopting an artificial intelligent visual algorithm, cutting a target object in the image materials, arranging various types of materials, and rendering the image materials. For example, the quality of the acquired material is poor, and although the effect is better after the subsequent processing, the acquired material still has the possibility of being unusable. Therefore, unnecessary processing resources are avoided being consumed, the collected materials can be directly screened, and the materials which do not meet the requirements are directly discarded. When processing image material, sometimes the image material includes some unnecessary objects, for example, some more prominent background or reference objects. The existence of these objects cannot represent an important part of the image itself (i.e., an object to be represented by the image itself), and therefore, in order to display the object in a more prominent manner in the image, the image material may be cut so that the object is prominent. When various types of materials are adopted for video production, the different types of materials have different functions and different reflected key points, so that various types of materials can be effectively arranged, the various types of materials are arranged at proper positions, the meaning of the materials is reflected, the produced images can better accord with the watching habit, and the watching experience is improved. When the video is produced, in order to improve the viewing experience of the user, the image materials can be beautified, for example, the image materials are beautified by colors and animations. Therefore, when a video is produced using an image material, the beautification degree of the image material may be determined first, for example, the beautification score of the image material may be acquired first, and whether or not the beautification process is required for the image material when the video is produced may be determined based on the beautification score.

As an optional embodiment, when the video production is performed by using artificial intelligence, to monitor or adjust the production process, an intermediate result may be output through an interactive device, where the intermediate result includes at least one of: and identifying the result of the information of the material by adopting an artificial intelligence identification algorithm, and visually processing the material by adopting an artificial intelligence visual algorithm. The intermediate result of the artificial intelligent video production is adjusted or monitored in an artificial adjustment or semi-artificial adjustment mode, so that the whole process is visible and controllable, and artificial intelligent black box operation and unexplainable property are avoided.

Fig. 4 is a schematic diagram of a structure of a video generated by implementing image processing according to an embodiment of the present invention, and in the video generated by using image processing, the cropping processing provided for the image may be a part of the entire generated video, for example, may be a part of the editing process of the material in the video generating process, as shown in fig. 4. In the following description, an image is described as an example of a product image.

In the related art, there is also an efficiency of improving the video clipping and editing process using the AI technique, but the entire production process thereof is still artificially dominant. In the video generation process provided by the embodiment of the invention, although an AI technology is also adopted and an editor interface is provided, the whole video generation process is AI-dominated, an algorithm can automatically process a large number of bottom layer video processing processes, a video producer only needs to perform fine adjustment on the basis of the generated video, the workload of the video producer is greatly reduced, and the capability of generating the video by one key is provided.

The video generation provided by the embodiment of the invention mainly comprises the following processing steps:

(1) and inputting a commodity link of a certain E-commerce platform to acquire a video, an image and commodity information (category, price, comment and the like) corresponding to the commodity.

(2) The input materials are identified and understood by calling an abundant AI identification algorithm, various information such as the category, the target position, the salient region, the aesthetic score and the like of the image materials are given, and a decision basis is provided for the follow-up intelligent arrangement and rendering.

(3) And screening, cutting, sequencing, rendering an intelligent visual effect, and encoding and outputting the final E-commerce video according to the provided material identification information.

It should be noted that, while outputting the final synthesized video, the intermediate result of the AI algorithm decision process may be output to the intelligent editor, so that the user may perform fine adjustment on the final result, such as replacing part of the display material, adjusting the display sequence, and the like. In addition, when the intelligent editor is adopted for adjustment, the output material analysis result is presented to the user through the intelligent editor, such as the category of the material.

By the optional embodiment, a video editor is used as a core video production process in the related technology, a new video intelligent editing and generating process mainly based on AI is provided, the whole video generating process can be automatically completed by the AI, and then the fine adjustment of video results generated by the AI is realized through a light-weight intelligent editor interactive interface. The new process can greatly improve the efficiency of video production.

The overall process is dominated by the AI, but all intermediate results in the AI processing process can be presented through an intelligent editor, such as material categories, video capture positions, special effect categories, and the like. The whole process is visible and controllable, and black box operation and inexplicability of AI are avoided.

It should be noted that the intelligent editor is not a mandatory option for the video generation system. Because the editor does not undertake the work of video rendering and synthesis and the like, the intelligent editor is not particularly specified to a visual interactive interface with an interface, and can be realized through other interactive modes such as voice conversation and the like, as long as the appeal of the user can be transferred to the AI-leading flow.

Fig. 5 is a flowchart of a third image processing method according to an embodiment of the present invention, as shown in fig. 5, the method includes the following steps:

step S502, displaying images on an interactive interface;

step S504, displaying visual characteristics of the image on the interactive interface, wherein the visual characteristics comprise one or more characteristic areas on the image, and the one or more characteristic areas are used for embodying the information content of the image;

step S506, displaying a cutting frame of the image on the interactive interface, wherein the cutting frame is determined according to the visual characteristics;

and step S508, displaying a cutting result on the interactive interface, wherein the cutting result is obtained by cutting the image according to the cutting frame.

In the above steps, the purpose of visually presenting the process of cutting the image is achieved by displaying the image, the visual characteristics of the image, the cutting frame of the image and the cutting result on the interactive interface, so that the technical effects of reasonably cutting the image according to the visual content of the image and visually presenting the image cutting process are achieved, and the technical problem that the key information content of the image is difficult to highlight when the picture frame of the image is adjusted is solved.

Fig. 6 is a flowchart illustrating an image processing method according to an alternative embodiment of the present invention, and as shown in fig. 6, the alternative embodiment proposes an application scenario of the image processing method, that is, a flow of cropping and frame adjustment for a shot image in a video. The steps involved in the method are briefly described below.

And S1, performing shot segmentation on the video. Before processing a shot image in a video, firstly, the shot image is obtained by performing shot segmentation on the video, and a plurality of image sets belonging to different shots are obtained after segmentation.

S2, face/topic/text/saliency detection. Visual features of the image, such as human faces, themes, characters and the like in the image are extracted by detecting each area in the image. After detection, the visual features are fused into a thermodynamic diagram by different weights to obtain an information thermodynamic diagram on an image, the larger the value is, the more important the corresponding area is, each characteristic area can be marked according to the thermodynamic diagram, a cutting frame is generated, and the characteristic areas are selected in a frame mode, so that the cutting frame comprises the most important area.

S3, the cropping frame is estimated/smoothed. And smoothing the trimming frame sequence of each lens through spline interpolation to avoid the dithering of the trimming lens.

S4, cut/fill. And (5) cutting or filling frame by frame according to the position of the cutting frame (when the important area is larger, the cutting frame is allowed to exceed the picture in order to ensure the length-width ratio of the cutting frame meeting the requirement, and filling operation is carried out on the exceeding part).

And S5, splicing and combining all cut lens images to output an adjusted video.

It should be noted that, for simplicity of description, the above-mentioned method embodiments are described as a series of acts or combination of acts, but those skilled in the art will recognize that the present invention is not limited by the order of acts, as some steps may occur in other orders or concurrently in accordance with the invention. Further, those skilled in the art should also appreciate that the embodiments described in the specification are preferred embodiments and that the acts and modules referred to are not necessarily required by the invention.

Through the above description of the embodiments, those skilled in the art can clearly understand that the image processing method according to the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal device (such as a mobile phone, a computer, a server, or a network device) to execute the method according to the embodiments of the present invention.

Example 2

According to an embodiment of the present invention, there is further provided an apparatus for implementing the first image processing method, and fig. 7 is a block diagram of a first image processing apparatus according to embodiment 2 of the present invention, as shown in fig. 7, the apparatus including: a first acquisition module 72, an extraction module 74, a determination module 76 and a cropping module 78, which are described in more detail below.

A first acquisition module 72 for acquiring an image; an extracting module 74, connected to the first obtaining module 72, configured to extract visual features of the image, where the visual features include one or more feature areas on the image, and the one or more feature areas are used to represent information content of the image; a determining module 76, connected to the extracting module 74, for determining a cropping frame of the image according to the visual characteristics; and a cutting module 78, connected to the determining module 76, for cutting the image according to the cutting frame to obtain a cut image.

It should be noted here that the first acquiring module 72, the extracting module 74, the determining module 76 and the cropping module 78 correspond to steps S202 to S208 in embodiment 1, and a plurality of modules are the same as the corresponding steps in the implementation example and application scenario, but are not limited to the disclosure in embodiment 1. It should be noted that the above modules may be operated in the computer terminal 10 provided in embodiment 1 as a part of the apparatus.

Example 3

According to an embodiment of the present invention, there is further provided an apparatus for implementing the second image processing method, and fig. 8 is a block diagram of a second image processing apparatus according to embodiment 3 of the present invention, as shown in fig. 8, the apparatus includes: a second acquisition module 82, an identification module 84, and a processing module 86, which are described in more detail below.

A second obtaining module 82, configured to obtain a material of the object; an identification module 84, connected to the second obtaining module 82, for identifying information of the material by using an artificial intelligence identification algorithm; and the processing module 86 is connected to the identification module 84 and is used for performing visual processing on the material by adopting an artificial intelligence visual algorithm according to the information of the material to obtain a video of the object.

It should be noted that the second acquiring module 82, the identifying module 84 and the processing module 86 correspond to steps S302 to S306 in embodiment 1, and a plurality of modules are the same as the corresponding steps in implementation examples and application scenarios, but are not limited to the disclosure in embodiment 1. It should be noted that the above modules may be operated in the computer terminal 10 provided in embodiment 1 as a part of the apparatus.

Example 4

According to an embodiment of the present invention, there is further provided an apparatus for implementing the third image processing method, and fig. 9 is a block diagram of a third image processing apparatus according to embodiment 4 of the present invention, as shown in fig. 9, the apparatus includes: a first display module 92, a second display module 94, a third display module 96 and a fourth display module 98, which will be described in detail below.

A first display module 92 for displaying an image on the interactive interface; a second display module 94, connected to the first display module 92, configured to display visual features of the image on the interactive interface, where the visual features include one or more feature areas on the image, and the one or more feature areas are used to represent information content of the image; a third display module 96, connected to the second display module 94, for displaying a trimming frame of the image on the interactive interface, wherein the trimming frame is determined according to the visual characteristics; and a fourth display module 98, connected to the third display module 96, for displaying a cutting result on the interactive interface, where the cutting result is obtained by cutting the image according to the cutting frame.

It should be noted that the first display module 92, the second display module 94, the third display module 96 and the fourth display module 98 correspond to steps S502 to S508 in embodiment 1, and the modules are the same as the corresponding steps in the implementation example and the application scenario, but are not limited to the disclosure in embodiment 1. It should be noted that the above modules may be operated in the computer terminal 10 provided in embodiment 1 as a part of the apparatus.

Example 5

The embodiment of the invention can provide a computer terminal which can be any computer terminal device in a computer terminal group. Optionally, in this embodiment, the computer terminal may also be replaced with a terminal device such as a mobile terminal.

Optionally, in this embodiment, the computer terminal may be located in at least one network device of a plurality of network devices of a computer network.

In this embodiment, the computer terminal may execute program codes of the following steps in the image processing method of the application program: acquiring an image; extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; determining a cutting frame of the image according to the visual characteristics; and cutting the image according to the cutting frame to obtain the cut image.

Alternatively, fig. 10 is a block diagram of a computer terminal according to an embodiment of the present invention. As shown in fig. 10, the computer terminal may include: one or more (only one shown) processors 102, memory 104, and the like.

The memory may be configured to store software programs and modules, such as program instructions/modules corresponding to the image processing method and apparatus in the embodiments of the present invention, and the processor executes various functional applications and data processing by running the software programs and modules stored in the memory, so as to implement the image processing method. The memory may include high speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory may further include memory located remotely from the processor, and these remote memories may be connected to the computer terminal through a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring an image; extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; determining a trimming frame of the image according to the visual characteristics; and cutting the image according to the cutting frame to obtain the cut image.

Optionally, the processor may further execute the program code of the following steps: determining a cropping frame of the image according to the visual characteristics, comprising: under the condition that the visual features comprise a plurality of feature regions, distributing different weights for the plurality of feature regions, wherein the weights represent the importance of the feature regions; and determining a cropping frame of the image according to the plurality of feature areas after the weight is distributed, wherein the cropping frame comprises the feature area with the maximum weight in the plurality of feature areas.

Optionally, the processor may further execute the program code of the following steps: determining a cropping frame of the image according to the plurality of feature areas after the weight is distributed, wherein the cropping frame comprises the following steps: fusing the plurality of characteristic regions with different weights to form a thermodynamic diagram, wherein different colors in the thermodynamic diagram represent the characteristic regions with different weights; and determining the size and the position of the cutting frame according to the thermodynamic diagram.

Optionally, the processor may further execute the program code of the following steps: cutting the image according to the cutting frame to obtain a cut image, comprising: determining an excess portion of the trimming frame beyond the image in a case where the trimming frame exceeds the image itself; and after the image is cut according to the cutting frame, filling the excess part to obtain the cut image.

Optionally, the processor may further execute the program code of the following steps: in a case where the image includes a plurality of shot images, the plurality of shot images being cut from the video, the method further includes: and splicing the cut multiple lens images to obtain a cut video.

Optionally, the processor may further execute the program code of the following steps: before splicing the cut multiple lens images to obtain a cut video, the method further comprises: and smoothing the cutting frame in each lens image in the plurality of lens images by adopting a spline interpolation method to obtain the smoothed cutting frame, wherein the smoothed cutting frame is used for cutting to obtain a plurality of cut lens images.

Optionally, the processor may further execute the program code of the following steps: extracting visual features of an image, comprising: inputting the image into an image feature model to obtain visual features of the image, wherein the image feature model is obtained by performing machine learning by adopting a first data set, and training data in the first data set comprises: images and visual features of images.

Optionally, the processor may further execute the program code of the following steps: the image comprises a plurality of material images of preset objects, and after the image is cut according to the cutting frame to obtain a cut image, the method further comprises the following steps: inputting each material image in the plurality of cut material images into an image special effect model to obtain a special effect image of each cut material image, wherein the image special effect model is obtained by adopting a second data set to perform machine learning, and training data in the second data set comprises: a material image and a special effect image of the material image; and splicing the special effect images corresponding to the plurality of material images to obtain a video of a preset object.

Optionally, the processor may further execute the program code of the following steps: splicing the special effect images corresponding to the plurality of material images to obtain a video of a preset object, wherein the method comprises the following steps: displaying the special effect images corresponding to the plurality of material images on a display interface; receiving adjustment operation for adjusting special effect images corresponding to the plurality of material images; responding to the adjustment operation, and adjusting the special effect images corresponding to the plurality of material images to obtain a plurality of adjusted special effect images; and splicing the adjusted plurality of special effect images to obtain a video of a preset object.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: displaying an image on the interactive interface; displaying visual features of the image on the interactive interface, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; displaying a cropping frame of the image on the interactive interface, wherein the cropping frame is determined according to the visual characteristics; and displaying a cutting result on the interactive interface, wherein the cutting result is obtained by cutting the image according to the cutting frame.

The processor can call the information and application program stored in the memory through the transmission device to execute the following steps: acquiring a material of an object; identifying information of the material by adopting an artificial intelligence identification algorithm; and performing visual processing on the material by adopting an artificial intelligent visual algorithm according to the information of the material to obtain the video of the object.

Optionally, the processor may further execute the program code of the following steps: under the condition that the material is an image material, identifying information of the material by adopting an artificial intelligence identification algorithm, wherein the information comprises at least one of the following: identifying the category of the pixel material by adopting a category algorithm in an artificial intelligence identification algorithm; identifying the position of a target object in the image material by adopting a position algorithm in an artificial intelligence identification algorithm; identifying a salient region in the image material by adopting a characteristic algorithm in an artificial intelligence identification algorithm; and identifying the aesthetic scores of the image materials by adopting a scoring algorithm in the artificial intelligence identification algorithm.

Optionally, the processor may further execute the program code of the following steps: and performing visual processing on the material by adopting an artificial intelligence visual algorithm, wherein the visual processing comprises at least one of the following steps: and screening the materials by adopting an artificial intelligent visual algorithm, cutting a target object in the image materials, arranging various types of materials, and rendering the image materials.

Optionally, the processor may further execute the program code of the following steps: outputting, by the interaction device, an intermediate result, wherein the intermediate result comprises at least one of: and identifying the result of the information of the material by adopting an artificial intelligence identification algorithm, and performing visual processing on the material by adopting an artificial intelligence visual algorithm.

It can be understood by those skilled in the art that the structure shown in fig. 10 is only an illustration, and the computer terminal may also be a terminal device such as a smart phone (e.g., an Android phone, an iOS phone, etc.), a tablet computer, a palmtop computer, a Mobile Internet Device (MID), a PAD, and the like. Fig. 10 is a diagram illustrating a structure of the electronic device. For example, the computer terminal 10 may also include more or fewer components (e.g., network interfaces, display devices, etc.) than shown in FIG. 10, or have a different configuration than shown in FIG. 10.

Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by a program instructing hardware associated with the terminal device, where the program may be stored in a computer-readable storage medium, and the storage medium may include: flash disks, Read-Only memories (ROMs), Random Access Memories (RAMs), magnetic or optical disks, and the like.

Example 4

The embodiment of the invention also provides a storage medium. Alternatively, in this embodiment, the storage medium may be configured to store program codes executed by the image processing method provided in embodiment 1.

Optionally, in this embodiment, the storage medium may be located in any one of computer terminals in a computer terminal group in a computer network, or in any one of mobile terminals in a mobile terminal group.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring an image; extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; determining a trimming frame of the image according to the visual characteristics; and cutting the image according to the cutting frame to obtain the cut image.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: determining a cropping frame of the image according to the visual characteristics, comprising: under the condition that the visual features comprise a plurality of feature regions, distributing different weights for the plurality of feature regions, wherein the weights represent the importance of the feature regions; and determining a cropping frame of the image according to the plurality of feature areas after the weight is distributed, wherein the cropping frame comprises the feature area with the maximum weight in the plurality of feature areas.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: determining a cropping frame of the image according to the plurality of feature areas after the weight is distributed, wherein the cropping frame comprises the following steps: fusing the plurality of characteristic regions with different weights to form a thermodynamic diagram, wherein different colors in the thermodynamic diagram represent the characteristic regions with different weights; and determining the size and the position of the cutting frame according to the thermodynamic diagram.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: cutting the image according to the cutting frame to obtain a cut image, comprising: determining an excess portion of the trimming frame beyond the image in a case where the trimming frame exceeds the image itself; and after the image is cut according to the cutting frame, filling the excess part to obtain the cut image.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: in a case where the image includes a plurality of shot images, the plurality of shot images being cut from the video, the method further includes: and splicing the cut multiple lens images to obtain a cut video.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: before splicing the cut multiple lens images to obtain a cut video, the method further comprises: and smoothing the cutting frame in each lens image in the plurality of lens images by adopting a spline interpolation method to obtain the smoothed cutting frame, wherein the smoothed cutting frame is used for cutting to obtain a plurality of cut lens images.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: extracting visual features of an image, comprising: inputting the image into an image feature model to obtain visual features of the image, wherein the image feature model is obtained by performing machine learning by adopting a first data set, and training data in the first data set comprises: images and visual features of images.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: the image comprises a plurality of material images of preset objects, and after the image is cut according to the cutting frame to obtain a cut image, the method further comprises the following steps: inputting each material image in the plurality of cut material images into an image special effect model to obtain a special effect image of each cut material image, wherein the image special effect model is obtained by adopting a second data set to perform machine learning, and training data in the second data set comprises: a material image and a special effect image of the material image; and splicing the special effect images corresponding to the plurality of material images to obtain a video of a preset object.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: splicing the special effect images corresponding to the plurality of material images to obtain a video of a preset object, wherein the method comprises the following steps: displaying the special effect images corresponding to the plurality of material images on a display interface; receiving adjustment operation for adjusting special effect images corresponding to the plurality of material images; responding to the adjustment operation, and adjusting the special effect images corresponding to the plurality of material images to obtain a plurality of adjusted special effect images; and splicing the adjusted plurality of special effect images to obtain a video of a preset object.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: displaying an image on the interactive interface; displaying visual features of the image on the interactive interface, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image; displaying a cropping frame of the image on the interactive interface, wherein the cropping frame is determined according to the visual characteristics; and displaying a cutting result on the interactive interface, wherein the cutting result is obtained by cutting the image according to the cutting frame.

Optionally, in this embodiment, the storage medium is configured to store program code for performing the following steps: acquiring a material of an object; identifying information of the material by adopting an artificial intelligence identification algorithm; and performing visual processing on the material by adopting an artificial intelligent visual algorithm according to the information of the material to obtain the video of the object.

Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: under the condition that the material is an image material, identifying information of the material by adopting an artificial intelligence identification algorithm, wherein the information comprises at least one of the following: identifying the category of the pixel material by adopting a category algorithm in an artificial intelligence identification algorithm; identifying the position of a target object in the image material by adopting a position algorithm in an artificial intelligence identification algorithm; identifying a salient region in the image material by adopting a characteristic algorithm in an artificial intelligence identification algorithm; and identifying the aesthetic scores of the image materials by adopting a scoring algorithm in the artificial intelligence identification algorithm.

Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: and performing visual processing on the material by adopting an artificial intelligence visual algorithm, wherein the visual processing comprises at least one of the following steps: and screening the materials by adopting an artificial intelligent visual algorithm, cutting a target object in the image materials, arranging various types of materials, and rendering the image materials.

Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: outputting, by the interaction device, an intermediate result, wherein the intermediate result comprises at least one of: and identifying the result of the information of the material by adopting an artificial intelligence identification algorithm, and performing visual processing on the material by adopting an artificial intelligence visual algorithm.

The above-mentioned serial numbers of the embodiments of the present invention are merely for description and do not represent the merits of the embodiments.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described apparatus embodiments are merely illustrative, and for example, a division of a unit is only a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed coupling or direct coupling or communication connection between each other may be an indirect coupling or communication connection through some interfaces, units or modules, and may be electrical or in other forms.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit may be implemented in the form of hardware, or may also be implemented in the form of a software functional unit.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the method according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic or optical disk, and other various media capable of storing program codes.

The foregoing is only a preferred embodiment of the present invention, and it should be noted that, for those skilled in the art, various modifications and decorations can be made without departing from the principle of the present invention, and these modifications and decorations should also be regarded as the protection scope of the present invention.

Claims

1. An image processing method, comprising:

acquiring an image;

extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image;

determining a trimming frame of the image according to the visual characteristics;

and cutting the image according to the cutting frame to obtain the cut image.

2. The method of claim 1, wherein determining a crop frame for the image based on the visual characteristic comprises:

under the condition that the visual features comprise a plurality of feature areas, distributing different weights for the feature areas, wherein the weights represent the importance of the feature areas;

determining a cropping frame of the image according to the plurality of feature areas after the weight is distributed, wherein the cropping frame comprises the feature area with the maximum weight in the plurality of feature areas.

3. The method of claim 2, wherein determining a cropping frame of the image according to the plurality of feature areas to which the weight values are assigned comprises:

fusing the plurality of feature areas with different weights to form a thermodynamic diagram, wherein different colors in the thermodynamic diagram represent feature areas with different weights;

and determining the size and the position of the cutting frame according to the thermodynamic diagram.

4. The method of claim 1, wherein cropping the image according to the cropping frame to obtain a cropped image comprises:

determining that the trimming frame exceeds an excess portion of the image in a case where the trimming frame exceeds the image itself;

and after the image is cut according to the cutting frame, filling the excess part to obtain the cut image.

5. The method of claim 1, wherein if the image comprises a plurality of shot images, the plurality of shot images resulting from video slicing, the method further comprises:

and splicing the cut multiple lens images to obtain a cut video.

6. The method of claim 5, further comprising, before stitching the cropped plurality of lens images to obtain the cropped video:

and smoothing the cutting frame in each lens image in the plurality of lens images by adopting a spline interpolation method to obtain a smoothed cutting frame, wherein the smoothed cutting frame is used for cutting to obtain a plurality of cut lens images.

7. The method of claim 1, wherein extracting visual features of the image comprises:

inputting the image into an image feature model to obtain visual features of the image, wherein the image feature model is obtained by performing machine learning by adopting a first data set, and training data in the first data set comprises: images and visual features of images.

8. The method according to claim 7, wherein the image includes a plurality of material images of a predetermined object, and after the image is cropped according to the cropping frame to obtain a cropped image, the method further comprises:

inputting each material image in the plurality of cut material images into an image special effect model to obtain a special effect image of each cut material image, wherein the image special effect model is obtained by adopting a second data set to perform machine learning, and training data in the second data set comprises: a material image and a special effect image of the material image;

and splicing the special effect images corresponding to the plurality of material images to obtain the video of the preset object.

9. The method according to claim 8, wherein stitching the special effect images corresponding to the plurality of material images to obtain the video of the predetermined object comprises:

displaying the special effect images corresponding to the plurality of material images on a display interface;

receiving adjustment operation for adjusting the special effect images corresponding to the plurality of material images;

responding to the adjustment operation, and adjusting the special effect images corresponding to the plurality of material images to obtain a plurality of adjusted special effect images;

and splicing the adjusted plurality of special effect images to obtain the video of the preset object.

10. An image processing method, comprising:

displaying an image on the interactive interface;

displaying visual features of the image on the interactive interface, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image;

displaying a cropping frame of the image on the interactive interface, wherein the cropping frame is determined according to the visual features;

and displaying a cutting result on the interactive interface, wherein the cutting result is obtained by cutting the image according to the cutting frame.

11. An image processing apparatus characterized by comprising:

the first acquisition module is used for acquiring an image;

the extraction module is used for extracting visual features of the image, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image;

a determining module, configured to determine a trimming frame of the image according to the visual feature;

and the cutting module is used for cutting the image according to the cutting frame to obtain the cut image.

12. An image processing apparatus characterized by comprising:

the first display module is used for displaying images on the interactive interface;

the second display module is used for displaying visual features of the image on the interactive interface, wherein the visual features comprise one or more feature areas on the image, and the one or more feature areas are used for embodying the information content of the image;

a third display module, configured to display a trimming frame of the image on the interactive interface, where the trimming frame is determined according to the visual feature;

and the fourth display module is used for displaying a cutting result on the interactive interface, wherein the cutting result is obtained by cutting the image according to the cutting frame.

13. A storage medium, characterized in that the storage medium includes a stored program, wherein an apparatus in which the storage medium is located is controlled to execute the image processing method according to any one of claims 1 to 10 when the program is executed.

14. A computer device, comprising: a memory and a processor, wherein the processor is capable of,

the memory stores a computer program;

the processor is configured to execute a computer program stored in the memory, and the computer program is configured to cause the processor to perform the image processing method according to any one of claims 1 to 10 when executed.

15. An image processing method, comprising:

acquiring a material of an object;

identifying the information of the material by adopting an artificial intelligence identification algorithm;

and performing visual processing on the material by adopting an artificial intelligence visual algorithm according to the information of the material to obtain the video of the object.

16. The method of claim 15, wherein in the case where the material is image material, identifying information of the material using the artificial intelligence recognition algorithm comprises at least one of:

identifying the category of the image material by adopting a category algorithm in the artificial intelligence identification algorithm;

identifying the position of a target object in the image material by adopting a position algorithm in the artificial intelligence identification algorithm;

identifying a salient region in the image material by adopting a characteristic algorithm in the artificial intelligence identification algorithm;

and identifying the aesthetic scores of the image materials by adopting a scoring algorithm in the artificial intelligence identification algorithm.

17. The method of claim 15, wherein visually processing the material using an artificial intelligence vision algorithm comprises at least one of:

and screening the materials by adopting an artificial intelligent vision algorithm, cutting a target object in the image materials, arranging various types of materials, and rendering the image materials.

18. The method according to any one of claims 15 to 17,

outputting, by an interaction device, an intermediate result, wherein the intermediate result comprises at least one of: and adopting the artificial intelligence vision algorithm to perform vision processing on the material according to the result of the information of the material identified by the artificial intelligence identification algorithm.

19. An image processing apparatus characterized by comprising:

the second acquisition module is used for acquiring the material of the object;

the identification module is used for identifying the information of the material by adopting an artificial intelligence identification algorithm;

and the processing module is used for carrying out visual processing on the material by adopting an artificial intelligence visual algorithm according to the information of the material to obtain the video of the object.