CN111461967B - Picture processing method, device, equipment and computer readable medium - Google Patents
Picture processing method, device, equipment and computer readable medium Download PDFInfo
- Publication number
- CN111461967B CN111461967B CN202010251643.9A CN202010251643A CN111461967B CN 111461967 B CN111461967 B CN 111461967B CN 202010251643 A CN202010251643 A CN 202010251643A CN 111461967 B CN111461967 B CN 111461967B
- Authority
- CN
- China
- Prior art keywords
- area
- picture
- information
- class
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 17
- 238000001514 detection method Methods 0.000 claims abstract description 44
- 238000000034 method Methods 0.000 claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 16
- 238000010801 machine learning Methods 0.000 claims description 28
- 238000012549 training Methods 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 9
- 238000010586 diagram Methods 0.000 description 9
- 238000013527 convolutional neural network Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 230000011218 segmentation Effects 0.000 description 5
- 238000013135 deep learning Methods 0.000 description 4
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000008901 benefit Effects 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000003709 image segmentation Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 230000004044 response Effects 0.000 description 2
- 238000009966 trimming Methods 0.000 description 2
- 241001465754 Metazoa Species 0.000 description 1
- 238000007796 conventional method Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000018109 developmental process Effects 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000007670 refining Methods 0.000 description 1
- 230000000717 retained effect Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
Images
Classifications
-
- G06T3/04—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N20/00—Machine learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10004—Still image; Photographic image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20132—Image cropping
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Software Systems (AREA)
- Data Mining & Analysis (AREA)
- Medical Informatics (AREA)
- Computing Systems (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Mathematical Physics (AREA)
- Artificial Intelligence (AREA)
- Quality & Reliability (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the disclosure discloses a picture processing method, a picture processing device, picture processing equipment and a computer readable medium. One embodiment of the method comprises the following steps: object detection is carried out on the target picture to obtain object information, wherein the object information comprises area information for representing a display area of the object; determining at least one tailorable region according to the region information; and cutting the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture. The embodiment realizes pertinence and flexibility of clipping pictures. Therefore, the experience of picture cropping of the user is improved, and convenience is provided for picture cropping of the user.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a method, an apparatus, a device, and a computer readable medium for processing a picture.
Background
With the development of internet technology and the popularization of electronic devices, a wide variety of pictures are brought into the field of view of people through various electronic devices. People can browse and download pictures through electronic equipment such as mobile phones. However, in order to meet the demand of people for higher pictures, people want more targeted and flexible pictures to improve the experience of people.
Disclosure of Invention
The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose a picture processing method, apparatus, device and computer readable medium to solve the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide a picture processing method, including: object detection is carried out on the target picture to obtain object information, wherein the object information comprises area information for representing a display area of the object; determining at least one tailorable region according to the region information; and cutting the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture.
In a second aspect, some embodiments of the present disclosure provide a picture processing apparatus, the apparatus including: a detection unit configured to perform object detection on a target picture to obtain object information, where the object information includes area information for characterizing a display area of the object; a determining unit configured to determine at least one trimmable area based on the area information; and the generating unit is configured to cut the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture.
In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; and a storage device having one or more programs stored thereon, which when executed by the one or more processors cause the one or more processors to implement the method as described in any of the first aspects.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements a method as described in any of the first aspects.
One of the above embodiments of the present disclosure has the following advantageous effects: object information including region information for characterizing a display region of the object can be obtained by performing object detection on the target picture. Thus, the region information of the object displayed in the target picture can be obtained. Thereafter, based on the above-described region information, at least one trimmable region can be determined. Finally, according to the picture cropping requirement information and the at least one croppable region, the target picture can be cropped to generate a cropped picture. Thus, a picture for the user's needs can be obtained. Furthermore, the problem that unnecessary areas are completely cut off when cutting can be solved according to the requirements of users, and pictures which are more in line with the requirements of the users can be obtained. Therefore, more flexible and targeted pictures can be provided for the user. Furthermore, the experience of picture cropping of the user is improved, so that the user can conveniently crop the picture.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
1-3 are schematic diagrams of one application scenario of a picture processing method according to some embodiments of the present disclosure;
FIG. 4 is a flow chart of some embodiments of a picture processing method according to the present disclosure;
FIG. 5 is a flow chart of further embodiments of a picture processing method according to the present disclosure;
fig. 6 is a schematic structural diagram of some embodiments of a picture processing device according to the present disclosure;
fig. 7 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.
The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1-3 are schematic diagrams of one application scenario of a picture processing method according to some embodiments of the present disclosure.
First, as shown in fig. 1, the terminal device 101 performs object detection on the object "tree", "first text" and "second text" displayed in the target picture 102, and may obtain object information 105 of the object "tree", object information 103 of the object "first text" and object information 104 of the object "second text". Wherein the object information includes area information for characterizing a display area of the object "tree", "first text" and "second text".
With continued reference to fig. 2, according to the user-set information of the cut-out objects "first text" and "second text", the terminal device 101 may first determine the area information 103 'of the area where the object "first text" is displayed and the area information 104' of the area where the object "second text" is displayed. Then, based on the above-mentioned area information 103 'and the above-mentioned area information 104', the terminal device 101 can determine 5 trimmable areas with reference numerals 106, 107, 108, 109, 110, respectively. The 5 croppable regions may be regions for cropping the target picture 102.
Finally, referring to fig. 3, according to the picture cropping proportion of the user and the 5 croppable regions with reference numerals 106, 107, 108, 109, and 110, the terminal device 101 may crop the target picture 102 to obtain cropped pictures with reference numerals 106', 107', 108', 109', and 110', respectively.
With continued reference to fig. 4, a flow 400 of some embodiments of a picture processing method according to the present disclosure is shown. The picture processing method comprises the following steps:
in step 401, object detection is performed on the target picture to obtain object information.
In some embodiments, the execution subject of the picture processing method (for example, the terminal device 101 shown in fig. 1) may perform object detection on the target picture by the deep learning detection method to obtain object information. The execution subject can also perform object detection on the target picture through the deep learning image segmentation network to obtain object information.
Wherein the object may include, but is not limited to, at least one of: humans, animals, objects. The object information may include area information for characterizing a display area of the object. The target picture may be a picture stored locally in advance, or may be a hot picture on a network.
The deep learning detection method may include, but is not limited to, at least one of: SSD algorithm (Single Shot MultiBox Detector, target detection algorithm), R-CNN algorithm (Region-Convolutional Neural Networks, target detection algorithm), fast R-CNN algorithm (Fast Region-Convolutional Neural Networks, target detection algorithm), SPP-NET algorithm (Spatial Pyramid Pooling Network, target detection algorithm), yolo algorithm (You Only Look Once, target detection algorithm), FPN algorithm (Feature Pyramid Networks, target detection algorithm), DCN algorithm (Deformable ConvNets, variable convolution algorithm), retinaNet target detection algorithm.
The deep-learning image segmentation network described above may include, but is not limited to, at least one of: FCN network (Fully Convolutional Networks, full convolution network), segNet network (Semantic Segmentation Network, image semantic segmentation network), deep lab semantic segmentation network, PSPNet network (Pyramid Scene Parsing Network, semantic segmentation network), mask-RCNN network (Mask-Region-CNN, image instance segmentation network).
In some embodiments, based on the area information obtained in step 401, the executing body may first determine, according to the information of the object that the user needs to keep, the area information of the object display area that the user needs to keep. Then, the execution body may determine at least one trimmable area according to the area information of the object display area that the user needs to reserve. The information of the object to be reserved by the user can be preset or set by the user in real time. The at least one trimmable area may include a display area of the reserved object. The at least one croppable region may be a region for cropping the target picture.
And step 403, cutting the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture.
In some embodiments, the execution subject may crop the target picture in the at least one trimmable area according to the picture cropping requirement information, to generate a cropped picture. The above-mentioned picture cropping requirement information may include, but is not limited to, at least one of the following: picture cropping width information, picture cropping area information, picture cropping proportion information. The execution body may determine a direction clipping width related to the direction based on the picture clipping ratio information. As an example, the picture cropping scale is "wide: high = 1:2", original tile dimensions are" wide: high = 200:100", then the picture cropping width is" 100/2*1 = 50".
Some embodiments of the present disclosure provide a method for obtaining object information by performing object detection on a target picture, where the object information includes area information for characterizing a display area of the object. Thus, the region information of the object displayed in the target picture can be obtained. Thereafter, based on the above-described region information, at least one trimmable region can be determined. Finally, according to the picture cropping requirement information and the at least one croppable region, the target picture can be cropped to generate a cropped picture. Thus, a picture for the user's needs can be obtained. Furthermore, the problem that unnecessary areas are completely cut off when cutting can be solved according to the requirements of users, and pictures which are more in line with the requirements of the users can be obtained. Therefore, more flexible and targeted pictures can be provided for the user. Furthermore, the experience of picture cropping of the user is improved, so that the user can conveniently crop the picture.
With further reference to fig. 5, a flow 500 of further embodiments of a picture processing method is shown. The process 500 of the picture processing method includes the following steps:
in step 501, object detection is performed on the target picture through a machine learning model to obtain object information.
In some embodiments, the executing body may perform object detection on the target picture through a machine learning model to obtain object information. The object information may include area information for characterizing a display area of the object, and may further include object category information. The object class information includes first class information and second class information. The first class information may be object class information of an object that a user designates to be retained. The second category information may be object category information in which the user designates an object to be cut out. The machine learning model described above has been trained by a set of training samples. The training sample set comprises a sample picture and sample object information corresponding to the sample picture. The speed can be improved by determining the object information through the model, and labor is saved.
As an example, the machine learning model may be obtained by performing the following training steps based on a training sample set:
step 1, sample pictures of at least one training sample in a training sample set are respectively input into an initial machine learning model, and object information corresponding to each sample picture in the at least one training sample is obtained.
And 2, comparing the object information corresponding to each sample picture in the at least one training sample with the corresponding sample object information, and determining the prediction accuracy of the initial machine learning model according to a comparison result.
And step 3, determining whether the prediction accuracy is greater than a preset accuracy threshold.
And 4, in response to determining that the accuracy is greater than the preset accuracy threshold, taking the initial machine learning model as a machine learning model with training completed.
And step 5, in response to determining that the accuracy is not greater than the preset accuracy threshold, adjusting parameters of the initial machine learning model, using unused training samples to form a training sample set, using the adjusted initial machine learning model as an initial machine learning model, and executing the training steps 1-4 again.
It will be appreciated that after the training described above, the machine learning model may be used to characterize the correspondence between the pictures and the object information of the objects displayed in the pictures. The machine learning model mentioned above may be a convolutional neural network model.
As another example, the training sample set includes a sample picture and object information of an object displayed in the sample picture, and the machine learning model is trained with the sample picture as an input and the object information of the object displayed in the sample picture as a desired output.
In some embodiments, based on the object category information obtained in step 501, the execution entity may display an object category presentation interface of each object category. Wherein the object categories may include, but are not limited to, at least one of: face, text, merchandise.
In step 503, a second class object is determined based on the selection operation for each object class on the object class display interface.
In some embodiments, the execution body may first detect a selection operation of the user for each object class on the object class display interface, and then determine the second class object. The second class object may be an object that needs to be cut by the user.
At step 504, at least one trimmable area is determined according to the area information.
In some embodiments, the execution body preference may determine the area information of the display area of the second class object based on the second class object determined in step 303. The execution body may then determine at least one trimmable area according to the area information of the display area of the second class object. Wherein the at least one trimmable area may not include a display area of the second class object.
And step 505, cutting the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture.
In some embodiments, the specific implementation of step 505 and the technical effects thereof may refer to step 403 in those embodiments corresponding to fig. 4, which are not described herein.
In alternative implementations of some embodiments, the target trimmable area may be selected from the at least one trimmable area based on the at least one trimmable area resulting from step 504. The target trimmable area may be a trimmable area meeting the trimming requirement. The cutting requirements can be set by a user according to actual needs.
As an example, the execution body may first determine an area of each of the at least one trimmable area. The one trimmable area with the largest area can then be determined by a ranking algorithm (e.g., a bubble ranking algorithm) that can be used as the target trimmable area.
Alternatively, the execution body may determine whether a display area of the first class object is included in each of the at least one trimmable area. If a trimmable area of the display area of the first class object is included, it may be a target trimmable area.
Alternatively, the execution body may first determine the number of first-class objects within each of the at least one trimmable areas, and then may determine one trimmable area having the largest number of the first-class objects through the bubble ordering algorithm. The trimmable area can be used as the target trimmable area.
Alternatively, the execution body may first determine an area sum of the first class object within each of the at least one trimmable areas, and then may determine the largest one of the area sums of the first class object through the bubble ordering algorithm. The trimmable area can be used as the target trimmable area.
As another example, first, the execution body may determine a trimmable area having a largest area through the bubble ordering algorithm according to an area of each of the at least one trimmable area. Next, the execution body may determine whether the display area of the first class object is included in the trimmable area having the largest area. If it is determined that the trimmable area having the largest area includes the display area of the first class object, the trimmable area having the largest area may be used as the target trimmable area.
Alternatively, first, the execution body may determine a trimmable area having a largest area by the bubble ordering algorithm according to an area of each of the at least one trimmable area. Second, the execution body may determine whether the number of first class objects included in the trimmable area having the largest area is the largest. If the execution body can determine that the number of first class objects included in the trimmable area having the largest area is the largest, the trimmable area having the largest area can be used as a target trim area.
Alternatively, first, the execution body may determine a trimmable area having a largest area by the bubble ordering algorithm according to an area of each of the at least one trimmable area. Second, the execution body may determine whether or not the area sum of the first class objects included in the trimmable area having the largest area of the area is largest. If the execution body can determine the area and the maximum of the first class object included in the trimmable area having the maximum area of the region, the trimmable area having the maximum area of the region can be used as the target trimming area.
As an example, the execution subject described above may determine at least one tailorable region that includes a display region of the first class object. The execution body may then determine, by the bubble ordering algorithm, whether there is a trimmable area satisfying the maximum number of objects including the first type among at least one trimmable area of the display area including the first type objects. If the execution subject can determine that there is, the trimmable area can be the target trim area.
Alternatively, the execution subject may first determine at least one trimmable area including the display area of the first class object. Thereafter, an area sum of the first class objects in the at least one trimmable area may be determined. Finally, the area of the first class object and the largest trimmable area can be determined by the bubble sorting algorithm, and the trimmable area can be used as a target trim area.
Alternatively, the execution body may first determine at least one trimmable area including the first class object. Thus, the number of the above-described first-class objects and the area sum of the above-described first-class objects can be determined. Then, a trimmable area including the largest number of the first-class objects and the area and the largest of the first-class objects may be determined from the at least one trimmable area by the bubble ordering algorithm, and the trimmable area may then be the target trim area.
As an example, the execution body may determine a tailorable region having a largest region area through the bubble ordering algorithm. Wherein the display area of the first class object and the area and maximum of the first class object may be included in the clipping area. The trimmable area may then be used as the target trim area.
Alternatively, the execution body may determine a trimmable area having the largest area by the bubble sorting algorithm. Wherein the maximum number of first class objects and the maximum area sum of the first class objects may be included in the clipping region. The trimmable area may then be used as the target trim area.
Alternatively, the execution body may first determine at least one trimmable area including a display area of the first class object. And then determining a tailorable region with the largest sum of the numbers of the first type objects included and the largest sum of the areas of the first type objects included from the at least one tailorable region through the bubble ordering algorithm, wherein the tailorable region can be used as a target tailorable region.
As an example, the execution body may determine a trimmable area from the at least one trimmable area, wherein the trimmable area may include a region area of the trimmable area being largest, a display area of the first class object being largest, and a sum of areas of the first class objects being largest. The execution body may determine a trimmable area including the largest number of the first class objects, the area of the first class objects, and the largest sum area by using the bubble sorting algorithm. The trimmable area may be referred to as a target trim area.
In an alternative implementation of some embodiments, the execution body may crop the determined target cropping area according to the picture cropping requirement information. Thereby obtaining a cut picture.
As can be seen in fig. 5, the flow 500 of the picture processing method in some embodiments corresponding to fig. 5 embodies the step of refining the object class information, as compared to the description of some embodiments corresponding to fig. 4. Therefore, the schemes described in the embodiments can quickly discriminate the object category to be cut and the object category to be reserved through the related information of the introduced object category information. Further, existing objects in the object class that need to be tailored may be tailored to the needs of the user. Thus, a picture with customization can be provided for the user. Compared to the conventional method that multiple cuts are required to cut existing objects in the same class, the solution described in these embodiments can cut out existing objects in the same class at a time through the related information of the introduced object class information. Therefore, time is saved, and efficiency is improved.
With further reference to fig. 6, as an implementation of the method shown in the above figures, the present disclosure provides some embodiments of a picture processing apparatus, which correspond to those method embodiments shown in fig. 4, and which are particularly applicable in various electronic devices.
As shown in fig. 6, a picture processing apparatus 600 of some embodiments includes: a detection unit 601, a determination unit 602, and a generation unit 603. The detection unit is configured to perform object detection on the target picture to obtain object information, wherein the object information comprises area information used for representing a display area of the object; a determining unit 602 configured to determine at least one trimmable area according to the above-mentioned area information; the generating unit 603 is configured to crop the target picture according to the picture cropping requirement information and the at least one croppable region, and generate a cropped picture.
In an alternative implementation of some embodiments, the object information further includes object category information, the object category information including first category information and second category information, and the at least one trimmable area does not include a display area of the second category object.
In an alternative implementation of some embodiments, the generating unit 603 of the picture processing apparatus 600 may be further configured to select a target croppable area from the at least one croppable area, wherein the target croppable area meets at least one of the following conditions: the area of the tailorable region is the largest; a display area including a first class object; the number of first class objects included is the greatest; the area and maximum of the first class object is included.
In an alternative implementation of some embodiments, the generating unit 603 of the picture processing apparatus 600 may be further configured to crop the target croppable area based on the picture cropping requirement information, and generate a cropped picture.
In an alternative implementation manner of some embodiments, the detection unit 601 of the picture processing apparatus 600 may be further configured to perform object detection on the target picture through a machine learning model, so as to obtain object information, where the machine learning model is trained through a training sample set.
In an alternative implementation manner of some embodiments, the detection unit 601 of the picture processing apparatus 600 may be further configured to display an object category presentation interface of each object category detected by the machine learning model; and determining a second class object based on the selection operation of each object class on the object class display interface.
It will be appreciated that the elements described in the apparatus 600 correspond to the various steps in the method described with reference to fig. 4. Thus, the operations, features and resulting benefits described above with respect to the method are equally applicable to the apparatus 600 and the units contained therein, and are not described in detail herein.
Referring now to fig. 7, a schematic diagram of an electronic device (e.g., the terminal device of fig. 1) 700 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), car terminals (e.g., car navigation terminals), and the like, as well as stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 7 is only one example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 7, the electronic device 700 may include a processing means (e.g., a central processor, a graphics processor, etc.) 701, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 702 or a program loaded from a storage means 708 into a Random Access Memory (RAM) 703. In the RAM 703, various programs and data required for the operation of the electronic device 700 are also stored. The processing device 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input/output (I/O) interface 705 is also connected to bus 704.
In general, the following devices may be connected to the I/O interface 705: input devices 706 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, and the like; an output device 707 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 708 including, for example, a memory card; and a communication device 709. The communication means 709 may allow the electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. While fig. 7 shows an electronic device 700 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 7 may represent one device or a plurality of devices as needed.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via communications device 709, or from storage 708, or from ROM 702. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing means 701.
It should be noted that the computer readable medium according to some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: object detection is carried out on the target picture to obtain object information, wherein the object information comprises area information for representing a display area of the object; determining at least one tailorable region according to the region information; and cutting the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture.
Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units described in some embodiments of the present disclosure may be implemented by means of software, or may be implemented by means of hardware. The described units may also be provided in a processor, for example, described as: a processor includes a detection unit, a determination unit, and a generation unit. The names of these units do not limit the unit itself in some cases, and for example, the detection unit may also be described as "performing object detection on a target picture to obtain object information including a unit for characterizing region information of a display region of the object".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
According to one or more embodiments of the present disclosure, there is provided a picture processing method including: object detection is carried out on the target picture to obtain object information, wherein the object information comprises area information for representing a display area of the object; determining at least one tailorable region according to the region information; and cutting the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture.
According to one or more embodiments of the present disclosure, the object information further includes object category information including first category information and second category information, and the at least one trimmable area does not include a display area of the second category object.
According to one or more embodiments of the present disclosure, the cropping of the target picture to generate a cropped picture includes: selecting a target trimmable area from the at least one trimmable area, wherein the target trimmable area meets at least one of the following conditions: the area of the tailorable region is the largest; a display area including a first class object; the number of first class objects included is the greatest; the area and maximum of the first class object is included.
According to one or more embodiments of the present disclosure, the cropping of the target picture to generate a cropped picture further includes: and cutting the target cutting area based on the picture cutting requirement information to generate a cut picture.
According to one or more embodiments of the present disclosure, the detecting an object of the target picture to obtain object information includes: and performing object detection on the target picture through a machine learning model to obtain object information, wherein the machine learning model is trained through a training sample set.
According to one or more embodiments of the present disclosure, the above method further comprises: displaying an object category display interface of each object category detected by the machine learning model; and determining a second class object based on the selection operation of each object class on the object class display interface.
According to one or more embodiments of the present disclosure, there is provided a picture processing apparatus including: a detection unit configured to perform object detection on a target picture to obtain object information, where the object information includes area information for characterizing a display area of the object; a determining unit configured to determine at least one trimmable area based on the area information; and the generating unit is configured to cut the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture.
According to one or more embodiments of the present disclosure, the object information further includes object category information including first category information and second category information, and the at least one trimmable area does not include a display area of the second category object.
According to one or more embodiments of the present disclosure, the generating unit is further configured to select a target trimmable area from the at least one trimmable area, wherein the target trimmable area meets at least one of the following conditions: the area of the tailorable region is the largest; a display area including a first class object; the number of first class objects included is the greatest; the area and maximum of the first class object is included.
According to one or more embodiments of the present disclosure, the generating unit is further configured to crop the target croppable region based on the picture cropping requirement information, and generate a cropped picture.
According to one or more embodiments of the present disclosure, the detection unit is further configured to perform object detection on the target picture through a machine learning model to obtain object information, wherein the machine learning model is trained through a training sample set.
According to one or more embodiments of the present disclosure, the detection unit is further configured to display an object class presentation interface of each object class detected by the machine learning model; and determining a second class object based on the selection operation of each object class on the object class display interface.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the invention in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the invention. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.
Claims (7)
1. A picture processing method, comprising:
object detection is carried out on the target picture to obtain object information, wherein the object information comprises area information used for representing a display area of the object; the object information further comprises object category information, wherein the object category information comprises first category information and second category information;
displaying an object category display interface of each object category obtained by performing object detection on the target picture based on a machine learning model;
determining a second class object based on a selection operation for each object class on the object class display interface;
determining at least one tailorable region according to the region information; the at least one trimmable area does not include a display area of the second class object;
and cutting the target picture according to the picture cutting requirement information and the at least one cutting area to generate a cut picture.
2. The method of claim 1, wherein the cropping the target picture to generate a cropped picture comprises:
selecting a target trimmable area from the at least one trimmable area, wherein the target trimmable area meets at least one of the following conditions: the area of the tailorable region is the largest; a display area including a first class object; the number of the first class objects included is the largest; the area and maximum of the first class object is included.
3. The method of claim 2, wherein the cropping the target picture to generate a cropped picture further comprises:
and cutting the target cutting area based on the picture cutting requirement information to generate a cut picture.
4. The method of claim 1, wherein the performing object detection on the target picture to obtain object information includes:
and performing object detection on the target picture through a machine learning model to obtain object information, wherein the machine learning model is trained through a training sample set.
5. A picture processing apparatus comprising:
the detection unit is configured to detect an object of the target picture to obtain object information, wherein the object information comprises area information used for representing a display area of the object; the object information further comprises object category information, wherein the object category information comprises first category information and second category information;
a display unit configured to display an object category display interface of each object category obtained by performing object detection on the target picture based on a machine learning model;
a first determining unit configured to determine a second class object based on a selection operation for the respective object classes on the object class presentation interface;
a second determination unit configured to determine at least one trimmable area according to the area information; the at least one trimmable area does not include a display area of the second class object;
the generating unit is configured to clip the target picture according to the picture clipping requirement information and the at least one clipping region, and generate a clipped picture.
6. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-4.
7. A computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010251643.9A CN111461967B (en) | 2020-04-01 | 2020-04-01 | Picture processing method, device, equipment and computer readable medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010251643.9A CN111461967B (en) | 2020-04-01 | 2020-04-01 | Picture processing method, device, equipment and computer readable medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111461967A CN111461967A (en) | 2020-07-28 |
| CN111461967B true CN111461967B (en) | 2023-06-27 |
Family
ID=71680549
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010251643.9A Active CN111461967B (en) | 2020-04-01 | 2020-04-01 | Picture processing method, device, equipment and computer readable medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111461967B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112541934B (en) * | 2019-09-20 | 2024-02-27 | 百度在线网络技术（北京）有限公司 | Image processing method and device |
| CN111916050A (en) * | 2020-08-03 | 2020-11-10 | 北京字节跳动网络技术有限公司 | Speech synthesis method, speech synthesis device, storage medium and electronic equipment |
| CN113256660A (en) * | 2021-06-04 | 2021-08-13 | 北京有竹居网络技术有限公司 | Picture processing method and device and electronic equipment |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104504649A (en) * | 2014-12-30 | 2015-04-08 | 百度在线网络技术（北京）有限公司 | Picture cutting method and device |
| US9270899B1 (en) * | 2012-06-27 | 2016-02-23 | Amazon Technologies, Inc. | Segmentation approaches for object recognition |
| US9684987B1 (en) * | 2015-02-26 | 2017-06-20 | A9.Com, Inc. | Image manipulation for electronic display |
| WO2018118803A1 (en) * | 2016-12-22 | 2018-06-28 | A9.Com, Inc. | Visual category representation with diverse ranking |
| CN109559300A (en) * | 2018-11-19 | 2019-04-02 | 上海商汤智能科技有限公司 | Image processing method, electronic equipment and computer readable storage medium |
| CN110136142A (en) * | 2019-04-26 | 2019-08-16 | 微梦创科网络科技（中国）有限公司 | A kind of image cropping method, apparatus, electronic equipment |
| CN110568982A (en) * | 2019-09-12 | 2019-12-13 | 北京字节跳动网络技术有限公司 | picture clipping method and device in online presentation, storage medium and equipment |
| CN110738673A (en) * | 2019-10-21 | 2020-01-31 | 哈尔滨理工大学 | Visual SLAM method based on example segmentation |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9020298B2 (en) * | 2009-04-15 | 2015-04-28 | Microsoft Technology Licensing, Llc | Automated image cropping to include particular subjects |
| US10318848B2 (en) * | 2015-12-15 | 2019-06-11 | Qualcomm Incorporated | Methods for object localization and image classification |
| EP3306527B1 (en) * | 2016-10-05 | 2021-01-06 | Canon Europa N.V. | A method of cropping an image, an apparatus for cropping an image, a program and a storage medium |
-
2020
- 2020-04-01 CN CN202010251643.9A patent/CN111461967B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9270899B1 (en) * | 2012-06-27 | 2016-02-23 | Amazon Technologies, Inc. | Segmentation approaches for object recognition |
| CN104504649A (en) * | 2014-12-30 | 2015-04-08 | 百度在线网络技术（北京）有限公司 | Picture cutting method and device |
| US9684987B1 (en) * | 2015-02-26 | 2017-06-20 | A9.Com, Inc. | Image manipulation for electronic display |
| WO2018118803A1 (en) * | 2016-12-22 | 2018-06-28 | A9.Com, Inc. | Visual category representation with diverse ranking |
| CN109559300A (en) * | 2018-11-19 | 2019-04-02 | 上海商汤智能科技有限公司 | Image processing method, electronic equipment and computer readable storage medium |
| CN110136142A (en) * | 2019-04-26 | 2019-08-16 | 微梦创科网络科技（中国）有限公司 | A kind of image cropping method, apparatus, electronic equipment |
| CN110568982A (en) * | 2019-09-12 | 2019-12-13 | 北京字节跳动网络技术有限公司 | picture clipping method and device in online presentation, storage medium and equipment |
| CN110738673A (en) * | 2019-10-21 | 2020-01-31 | 哈尔滨理工大学 | Visual SLAM method based on example segmentation |
Non-Patent Citations (1)
| Title |
|---|
| 一种面向移动终端目标图像自动裁剪的快速区域定位算法;贺辉;张泽生;肖红玉;黄静;;计算机与数字工程(第03期);570-574 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111461967A (en) | 2020-07-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111461967B (en) | Picture processing method, device, equipment and computer readable medium | |
| CN113191726B (en) | Task detail interface display method, device, equipment and computer readable medium | |
| CN110634047A (en) | Method and device for recommending house resources, electronic equipment and storage medium | |
| CN111461968B (en) | Picture processing method, device, electronic equipment and computer readable medium | |
| CN111459364A (en) | Icon updating method and device and electronic equipment | |
| CN113157153A (en) | Content sharing method and device, electronic equipment and computer readable storage medium | |
| CN111273830A (en) | Data display method and device, electronic equipment and computer readable medium | |
| CN111461965B (en) | Picture processing method and device, electronic equipment and computer readable medium | |
| CN108509442B (en) | Search method and apparatus, server, and computer-readable storage medium | |
| CN112380819A (en) | Document editing method and device and electronic equipment | |
| CN112183388A (en) | Image processing method, apparatus, device and medium | |
| CN111756953A (en) | Video processing method, device, equipment and computer readable medium | |
| CN113220922B (en) | Image searching method and device and electronic equipment | |
| CN111461964B (en) | Picture processing method, device, electronic equipment and computer readable medium | |
| CN111461969B (en) | Method, device, electronic equipment and computer readable medium for processing picture | |
| CN111625745B (en) | Recommendation method, recommendation device, electronic equipment and computer readable medium | |
| CN114417782A (en) | Display method and device and electronic equipment | |
| CN114416945A (en) | Word cloud picture display method, device, equipment and medium | |
| CN112417276A (en) | Paging data acquisition method and device, electronic equipment and computer readable storage medium | |
| CN110991312A (en) | Method, apparatus, electronic device, and medium for generating detection information | |
| CN111489286B (en) | Picture processing method, device, equipment and medium | |
| CN113283115B (en) | Image model generation method and device and electronic equipment | |
| CN112766285B (en) | Image sample generation method and device and electronic equipment | |
| CN115857753B (en) | Vehicle information display method based on different user paths in public and private domain environment | |
| CN114428823B (en) | Data linkage method, device, equipment and medium based on multidimensional variable expression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information | ||
| CB02 | Change of applicant information |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: Tiktok vision (Beijing) Co.,Ltd. |
|
| GR01 | Patent grant | ||
| GR01 | Patent grant |