CN111784710B - Image processing method, device, electronic equipment and medium - Google Patents
Image processing method, device, electronic equipment and medium Download PDFInfo
- Publication number
- CN111784710B CN111784710B CN202010648627.3A CN202010648627A CN111784710B CN 111784710 B CN111784710 B CN 111784710B CN 202010648627 A CN202010648627 A CN 202010648627A CN 111784710 B CN111784710 B CN 111784710B
- Authority
- CN
- China
- Prior art keywords
- image
- circumscribed rectangle
- rectangle
- area
- classification
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 21
- 230000011218 segmentation Effects 0.000 claims abstract description 39
- 238000000034 method Methods 0.000 claims abstract description 29
- 238000012545 processing Methods 0.000 claims abstract description 26
- 238000003709 image segmentation Methods 0.000 claims abstract description 17
- 239000003086 colorant Substances 0.000 claims description 18
- 238000004590 computer program Methods 0.000 claims description 10
- 238000013527 convolutional neural network Methods 0.000 claims description 8
- 230000005484 gravity Effects 0.000 claims description 8
- 238000013507 mapping Methods 0.000 claims description 6
- 238000010586 diagram Methods 0.000 description 8
- 238000004422 calculation algorithm Methods 0.000 description 7
- 230000006870 function Effects 0.000 description 7
- 238000004891 communication Methods 0.000 description 6
- 238000000605 extraction Methods 0.000 description 4
- 230000000694 effects Effects 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 238000001514 detection method Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000003711 image thresholding Methods 0.000 description 2
- 230000000644 propagated effect Effects 0.000 description 2
- 238000007635 classification algorithm Methods 0.000 description 1
- 239000000835 fiber Substances 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000013526 transfer learning Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G06T5/70—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20076—Probabilistic image processing
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
Abstract
The embodiment of the disclosure discloses an image processing method, an image processing device, electronic equipment and a medium. One embodiment of the method comprises the following steps: image segmentation is carried out on the image to be processed according to the pixel color values, and at least one segmentation area is generated; generating an external rectangle of each divided area to obtain at least one external rectangle of the divided area; determining a contour area circumscribed rectangle of a contour area of an object displayed in an image to be processed; and carrying out image classification on at least one of the circumscribed rectangle of the segmentation area and the circumscribed rectangle of the outline area, and generating a category information set. This embodiment utilizes the circumscribed rectangle of the object displayed in the image to achieve classification of the image surrounded by the circumscribed rectangle.
Description
Technical Field
Embodiments of the present disclosure relate to the field of computer technology, and in particular, to an image processing method, an image processing device, an electronic device, and a medium.
Background
Image processing techniques are techniques that apply certain operations and processes to an image using a computer, a camera, and other digital processing techniques to extract various information in the image. Image classification is one of image processing techniques that can distinguish between different classes of objects based on different features reflected in the image. There is a need for object classification using circumscribed rectangles of objects displayed in an image.
Disclosure of Invention
The disclosure is in part intended to introduce concepts in a simplified form that are further described below in the detailed description. The disclosure is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.
Some embodiments of the present disclosure propose an image processing method, apparatus, electronic device, and medium to solve the technical problems mentioned in the background section above.
In a first aspect, some embodiments of the present disclosure provide an image processing method, including: image segmentation is carried out on the image to be processed according to the pixel color values, and at least one segmentation area is generated; generating an external rectangle of each divided area to obtain at least one external rectangle of the divided area; determining a contour area circumscribed rectangle of a contour area of an object displayed in an image to be processed; and carrying out image classification on at least one of the circumscribed rectangle of the segmentation area and the circumscribed rectangle of the outline area, and generating a category information set.
In a second aspect, some embodiments of the present disclosure provide an image processing apparatus including: the segmentation unit is configured to segment the image to be processed according to the pixel color values to generate at least one segmentation area; the generating unit is configured to generate an external rectangle of each divided area to obtain at least one divided area external rectangle; a determination unit configured to determine a contour region circumscribing rectangle of a contour region of an object displayed in an image to be processed; the classification unit is configured to classify the at least one segmentation area circumscribed rectangle and the at least one contour area circumscribed rectangle into images and generate a category information set.
In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device having one or more programs stored thereon, which when executed by one or more processors, cause the one or more processors to implement the method as described in any of the implementations of the first aspect.
In a fourth aspect, some embodiments of the present disclosure provide a computer readable medium having a computer program stored thereon, wherein the program when executed by a processor implements a method as described in any of the implementations of the first aspect.
One of the above embodiments of the present disclosure has the following advantageous effects: image segmentation is carried out on the image to be processed according to the pixel color values, at least one segmentation area is generated, and then the circumscribed rectangle of each segmentation area is generated, so that the circumscribed rectangle of the at least one segmentation area is obtained. Thus, a first circumscribed rectangle, i.e., a split area circumscribed rectangle, is obtained. Then, a contour region circumscribed rectangle of a contour region of the object displayed in the above-described image to be processed is determined. Thus, a second circumscribed rectangle, i.e., a contour region circumscribed rectangle, is obtained. And finally, classifying the obtained external rectangles to generate a category information set. Therefore, by summarizing different circumscribed rectangles, effective classification of images included in the circumscribed rectangles is achieved.
Drawings
The above and other features, advantages, and aspects of embodiments of the present disclosure will become more apparent by reference to the following detailed description when taken in conjunction with the accompanying drawings. The same or similar reference numbers will be used throughout the drawings to refer to the same or like elements. It should be understood that the figures are schematic and that elements and components are not necessarily drawn to scale.
FIG. 1 is a schematic illustration of one application scenario of an image processing method of some embodiments of the present disclosure;
FIG. 2 is a flow chart of some embodiments of an image processing method according to the present disclosure;
FIG. 3 is a flow chart of further embodiments of an image processing method according to the present disclosure;
FIG. 4 is a schematic structural view of some embodiments of an image processing apparatus according to the present disclosure;
fig. 5 is a schematic structural diagram of an electronic device suitable for use in implementing some embodiments of the present disclosure.
Detailed Description
Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While certain embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete. It should be understood that the drawings and embodiments of the present disclosure are for illustration purposes only and are not intended to limit the scope of the present disclosure.
It should be noted that, for convenience of description, only the portions related to the present application are shown in the drawings. Embodiments of the present disclosure and features of embodiments may be combined with each other without conflict.
It should be noted that the terms "first," "second," and the like in this disclosure are merely used to distinguish between different devices, modules, or units and are not used to define an order or interdependence of functions performed by the devices, modules, or units.
It should be noted that references to "one", "a plurality" and "a plurality" in this disclosure are intended to be illustrative rather than limiting, and those of ordinary skill in the art will appreciate that "one or more" is intended to be understood as "one or more" unless the context clearly indicates otherwise.
The names of messages or information interacted between the various devices in the embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information. The present disclosure will be described in detail below with reference to the accompanying drawings in conjunction with embodiments.
Fig. 1 shows a schematic diagram of one application scenario in which the image processing method of some embodiments of the present disclosure may be applied.
As shown in fig. 1, a computing device 101 may acquire an image 102 to be processed. Here, as an example, the image to be processed 102 includes a teaching aid map 103 and a teaching aid map 104. One pixel color value that teaching aid diagram 103 has is represented by a dot in fig. 1. Another pixel color value that teaching aid map 104 has is represented in fig. 1 by a grid line. Thus, image segmentation is performed based on these two different pixel color values, and two segmentation areas corresponding to teaching aid map 103 and teaching aid map 104 can be obtained. After that, the divided region circumscribed rectangles of the two divided regions, that is, the divided region circumscribed rectangle 105 and the divided region circumscribed rectangle 106, may be generated.
Then, the computing device 101 determines the outline area bounding rectangle of the outline areas of the teaching aid map 103 and the teaching aid map 104 in the image 102 to be processed, that is, the outline area bounding rectangle 107 and the outline area bounding rectangle 108.
Finally, the computing device 101 performs image classification of the split region bounding rectangle 105, the split region bounding rectangle 106, the outline region bounding rectangle 107, and the outline region bounding rectangle 108, generating a corresponding set of category information 109.
The computing device 101 may be hardware or software. When the computing device is hardware, the computing device may be implemented as a distributed cluster formed by a plurality of servers or terminal devices, or may be implemented as a single server or a single terminal device. When the computing device is embodied as software, it may be implemented as a plurality of software or software modules, for example, to provide distributed services, or as a single software or software module. The present application is not particularly limited herein.
It should be understood that the number of computing devices 101 in fig. 1 is merely illustrative. There may be any number of computing devices 101 as desired for an implementation.
With continued reference to fig. 2, a flow 200 of some embodiments of an image processing method according to the present disclosure is shown. The image processing method comprises the following steps:
in step 201, image segmentation is performed on the image to be processed according to the pixel color values, and at least one segmentation area is generated.
In some embodiments, the execution subject of the image processing method (e.g., the computing device shown in fig. 1) may image-segment the image to be processed in various ways to obtain a plurality of image-segmented regions.
The image to be processed may be an image in which the target object is displayed. The target object includes, but is not limited to, at least one of: articles, such as teaching aids; and (5) a person. The pixel color value is the color value of a pixel point in the image to be processed. For example, the pixel color value of a pixel point may be represented by "0 (black)", "1 (white)". The pixel color values of the pixel point can also be represented by three primary colors. The above three primary colors refer to three basic colors which cannot be subdivided among colors. For example RGB (red, green and blue, red, green, blue).
Image segmentation is the process of dividing an image into multiple regions. Alternatively, the image thresholding segmentation may be performed on the image to be processed. The purpose of image thresholding may be to divide the pixel sets by gray scale, with each resulting subset forming an area corresponding to the scene.
Alternatively, the image to be processed may also be image-segmented according to a predetermined number of semantic colors based on pixel color values. Wherein the predetermined number of semantic colors is determined based on the number of mappings of pixel color values to semantic colors. For example, three categories of semantic colors may be represented by "red", "green", "blue". Since three categories are employed here, the number of mapping relationships of pixel color values to semantic colors is 3, and the number of semantic colors is 3. As an example, pixel color values in a first range may also be mapped to "semantic colors of red", pixel color values in a second range may be mapped to "semantic colors of green", and pixel color values in a third range may be mapped to "semantic colors of blue". The first range, the second range, and the third range may be distinguished according to the approximation of colors, or may be distinguished according to different objects shown in the drawing.
Step 202, generating an external rectangle of each divided area, and obtaining at least one external rectangle of the divided area.
In some embodiments, the bounding rectangle of the segmented region may be a rectangle that contains the segmented region. Optionally, the bounding rectangle is a minimum bounding rectangle of the segmented region.
In this embodiment, the coordinate values (x, y) of the pixel points in the divided regions may be counted for each divided region. The maximum and minimum values of the coordinates x and y in the divided region are determined, respectively. Then, a circumscribed rectangle of the divided region is generated using the coordinate values of the minimum value and the maximum value of x and y as vertices. As an example, the coordinate value of the pixel point in the upper left corner of the image may be noted as (0, 0).
In some alternative implementations of some embodiments, step 202 may also proceed as follows:
and a first step of removing noise from the divided regions to obtain the noise-removed divided regions.
Optionally, a mode of removing noise by median filtering can be selected, noise points of the segmentation area are removed, and the segmentation area after noise removal is obtained.
And a second step of generating an circumscribed rectangle of the divided area after noise removal.
The circumscribed rectangle of the divided region may be obtained using the coordinate information of the pixel points of the divided region from which the noise is removed.
Step 203, determining a contour area circumscribing rectangle of the contour area of the object displayed in the image to be processed.
In some embodiments, the execution subject of the image processing method may employ various contour detection algorithms to extract contours of objects (e.g., teaching aids) displayed in the image to be processed.
As an example, the picture may be first subjected to gray scale processing, such as image binarization. Then, contour information is extracted by a contour detection algorithm, for example, a binary image contour extraction algorithm. The boundary can be extracted by using an 8-neighborhood algorithm in the binary image contour extraction algorithm, and the orphan is removed, so that contour extraction is completed.
Thereafter, the outline area circumscribed rectangle of the extracted outline may be determined in a similar manner to step 202.
And 204, performing image classification on the at least one segmentation area circumscribed rectangle and the outline area circumscribed rectangle to generate a category information set.
In some embodiments, the two circumscribed rectangles obtained in the step 202 and the step 203 for the image to be processed may be mapped onto the image to be processed, to obtain the image to be processed with a rectangular frame. Then, various image classification algorithms, such as a transfer learning algorithm, can be adopted to classify the image to be processed with the rectangular frame, and a category information set is generated.
Optionally, the image to be processed may be input into a trained classification network, so as to classify the image area in the external rectangle. Thus, the classification probability corresponding to the classification information of the image in the rectangular frame and the classification information is obtained. Such as CNN (convolutional neural network, convolutional Neural Networks).
One of the above embodiments of the present disclosure has the following advantageous effects: image segmentation is carried out on the image to be processed according to the pixel color values, at least one segmentation area is generated, and then the circumscribed rectangle of each segmentation area is generated, so that the circumscribed rectangle of the at least one segmentation area is obtained. Thus, a first circumscribed rectangle, i.e., a split area circumscribed rectangle, is obtained. Then, the outline area circumscribed rectangle of the outline area of the object displayed in the above-mentioned image to be processed is determined by a conventional image outline extraction algorithm. Thus, a second circumscribed rectangle, i.e., a contour region circumscribed rectangle, is obtained. And finally, classifying the obtained external rectangles to generate a category information set. Therefore, by summarizing different circumscribed rectangles, effective classification of images included in the circumscribed rectangles is achieved.
With further reference to fig. 3, a flow 300 of further embodiments of an image processing method is shown. The flow 300 of the image processing method comprises the steps of:
in step 301, an image to be processed is segmented according to pixel color values, and at least one segmentation area is generated.
Step 302, generating an circumscribed rectangle of each divided area, and obtaining at least one divided area circumscribed rectangle.
In some embodiments, the specific implementation of steps 301 to 302 and the technical effects thereof may refer to steps 201 to 202 in the corresponding embodiment of fig. 2, which are not described herein.
Step 303, determining a contour area circumscribing rectangle of the contour area of the object displayed in the image to be processed.
In some embodiments, the execution subject of the image processing method may input the image to be processed into a pre-trained network model, and output the contour region of the object displayed in the image to be processed. Then, a circumscribed rectangle of the outline area is generated.
And step 304, for each circumscribed rectangle in at least one of the circumscribed rectangles of the divided regions and the circumscribed rectangles of the outline region, performing image classification on the circumscribed rectangles by adopting a classification network to obtain category information and classification probability corresponding to the category information.
In some embodiments, the execution subject of the image processing method may utilize a classification network, such as CNN, to implement image classification.
In step 305, a circumscribed rectangle corresponding to the classification probability with the highest probability is selected from the classification probabilities corresponding to the circumscribed rectangles meeting the predetermined conditions, and the circumscribed rectangle is used as the target circumscribed rectangle.
Wherein the predetermined condition may include: the center-of-gravity distance of each circumscribed rectangle is less than or equal to a predetermined value.
Step 306, determining the center of gravity of the target bounding rectangle as the position of the pattern corresponding to the target bounding rectangle.
Here, in the case where the pattern surrounded by the circumscribed rectangle of the target is a teaching aid, the center of gravity of the circumscribed rectangle of the target is the position of the teaching aid image.
As can be seen in fig. 3, the flow 300 of the image processing method in some embodiments corresponding to fig. 3 highlights step 305 and step 306 compared to the description of some embodiments corresponding to fig. 2. In step 305, each bounding rectangle of a certain object is selected by a constraint that the distance of the center of gravity of each bounding rectangle is equal to or smaller than a predetermined value. And then selecting the circumscribed rectangle corresponding to the classification probability with the highest probability from the classification probabilities corresponding to the circumscribed rectangles as the target circumscribed rectangle, and determining the gravity center of the target circumscribed rectangle as the position of the pattern corresponding to the target circumscribed rectangle. Thereby facilitating a more accurate determination of the class and location of the object (e.g., teaching aid).
With further reference to fig. 4, as an implementation of the method shown in the above figures, the present disclosure provides some embodiments of an image processing apparatus, which correspond to those method embodiments shown in fig. 2, and which are particularly applicable in various electronic devices.
As shown in fig. 4, the image processing apparatus 400 of some embodiments includes: a segmentation unit 401, a generation unit 402, a determination unit 403, and a classification unit 404. The segmentation unit is configured to segment the image to be processed according to the pixel color values to generate at least one segmentation area; the generating unit is configured to generate an external rectangle of each divided area to obtain at least one divided area external rectangle; a determination unit configured to determine a contour region circumscribing rectangle of a contour region of an object displayed in the image to be processed; and the classification unit is configured to classify the images of the at least one segmentation area circumscribed rectangle and the outline area circumscribed rectangle to generate a category information set.
In an alternative implementation of some embodiments, the segmentation unit 401 of the image processing apparatus 400 is further configured to: image segmentation is carried out on the image to be processed according to pixel color values and the preset semantic color number, wherein the preset semantic color number is determined according to the number of mapping relations from the pixel color values to the semantic colors.
In an alternative implementation of some embodiments, the generating unit 402 of the image processing apparatus 400 is further configured to: carrying out noise removal processing on the divided areas to obtain noise-removed divided areas; generating the circumscribed rectangle of the divided area after noise removal.
In an alternative implementation of some embodiments, the determining unit 403 of the image processing apparatus 400 is further configured to: inputting the image to be processed into a pre-trained convolutional neural network, and outputting a contour region of the image to be processed; and determining the circumscribed rectangle of the outline area of the image to be processed.
In an alternative implementation of some embodiments, the classification unit 404 of the image processing apparatus 400 is further configured to: for each circumscribed rectangle in the at least one circumscribed rectangle of the divided area and the circumscribed rectangle of the outline area, classifying the circumscribed rectangle by adopting a classification network to obtain category information and classification probability corresponding to the category information; selecting a circumscribed rectangle corresponding to the classification probability with the highest probability from the classification probabilities corresponding to the circumscribed rectangles meeting the preset conditions as a target circumscribed rectangle, wherein the preset conditions comprise: the center-of-gravity distance of each circumscribed rectangle is less than or equal to a predetermined value.
In an alternative implementation of some embodiments, the image processing apparatus 400 is further configured to: and determining the gravity center of the target circumscribed rectangle as the position of the pattern corresponding to the target circumscribed rectangle.
It will be appreciated that the elements described in the apparatus 400 correspond to the various steps in the method described with reference to fig. 2. Thus, the operations, features and resulting benefits described above with respect to the method are equally applicable to the apparatus 400 and the units contained therein, and are not described in detail herein.
Referring now to FIG. 5, a schematic diagram of an electronic device (e.g., the computing device of FIG. 1) 500 suitable for use in implementing some embodiments of the present disclosure is shown. The electronic devices in some embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), car terminals (e.g., car navigation terminals), and the like, as well as stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 5 is only one example and should not impose any limitations on the functionality and scope of use of embodiments of the present disclosure.
As shown in fig. 5, the electronic device 500 may include a processing means (e.g., a central processing unit, a graphics processor, etc.) 501, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage means 508 into a Random Access Memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic apparatus 500 are also stored. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input/output (I/O) interface 505 is also connected to bus 504.
In general, the following devices may be connected to the I/O interface 505: input devices 506 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; an output device 507 including, for example, a Liquid Crystal Display (LCD), a speaker, a vibrator, and the like; storage 508 including, for example, magnetic tape, hard disk, etc.; and communication means 509. The communication means 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. While fig. 5 shows an electronic device 500 having various means, it is to be understood that not all of the illustrated means are required to be implemented or provided. More or fewer devices may be implemented or provided instead. Each block shown in fig. 5 may represent one device or a plurality of devices as needed.
In particular, according to some embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, some embodiments of the present disclosure include a computer program product comprising a computer program embodied on a computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such embodiments, the computer program may be downloaded and installed from a network via the communications device 509, or from the storage device 508, or from the ROM 502. The above-described functions defined in the methods of some embodiments of the present disclosure are performed when the computer program is executed by the processing device 501.
It should be noted that, the computer readable medium described in some embodiments of the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In some embodiments of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In some embodiments of the present disclosure, however, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: electrical wires, fiber optic cables, RF (radio frequency), and the like, or any suitable combination of the foregoing.
In some implementations, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.
The computer readable medium may be contained in the electronic device; or may exist alone without being incorporated into the electronic device. The computer readable medium carries one or more programs which, when executed by the electronic device, cause the electronic device to: image segmentation is carried out on the image to be processed according to the pixel color values, and at least one segmentation area is generated; generating an external rectangle of each divided area to obtain at least one external rectangle of the divided area; determining a contour area circumscribed rectangle of a contour area of an object displayed in the image to be processed; and carrying out image classification on the at least one segmentation area circumscribed rectangle and the outline area circumscribed rectangle to generate a category information set.
Computer program code for carrying out operations for some embodiments of the present disclosure may be written in one or more programming languages, including an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).
The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units involved in the embodiments of the present application may be implemented in software or in hardware. The described units may also be provided in a processor, for example, described as: a processor includes a segmentation unit, a generation unit, a determination unit, and a classification unit. The names of these units do not in any way constitute a limitation of the unit itself, for example, the segmentation unit may also be described as "a unit for image segmentation of an image to be processed from pixel color values, generating at least one segmentation area".
The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: a Field Programmable Gate Array (FPGA), an Application Specific Integrated Circuit (ASIC), an Application Specific Standard Product (ASSP), a system on a chip (SOC), a Complex Programmable Logic Device (CPLD), and the like.
According to one or more embodiments of the present disclosure, there is provided an image processing method including: image segmentation is carried out on the image to be processed according to the pixel color values, and at least one segmentation area is generated; generating an external rectangle of each divided area to obtain at least one external rectangle of the divided area; determining a contour area circumscribed rectangle of a contour area of an object displayed in the image to be processed; and carrying out image classification on the at least one segmentation area circumscribed rectangle and the outline area circumscribed rectangle to generate a category information set.
According to one or more embodiments of the present disclosure, the image segmentation of the image to be processed according to the pixel color values includes: image segmentation is carried out on the image to be processed according to pixel color values and the preset semantic color number, wherein the preset semantic color number is determined according to the number of mapping relations from the pixel color values to the semantic colors.
According to one or more embodiments of the present disclosure, the generating the bounding rectangle for each of the segmented regions includes: carrying out noise removal processing on the divided regions to obtain noise-removed divided regions; and generating the circumscribed rectangle of the noise-removed dividing region.
According to one or more embodiments of the present disclosure, the determining a contour region bounding rectangle of a contour region of an object displayed in the image to be processed includes: inputting the image to be processed into a pre-trained convolutional neural network, and outputting a contour region of the image to be processed; and determining the circumscribed rectangle of the outline area of the image to be processed.
According to one or more embodiments of the present disclosure, the image classifying the at least one segmented region bounding rectangle and the contour region bounding rectangle includes: for each circumscribed rectangle in the at least one circumscribed rectangle of the divided area and the circumscribed rectangle of the outline area, carrying out image classification on the circumscribed rectangle by adopting a classification network to obtain category information and classification probability corresponding to the category information; selecting a circumscribed rectangle corresponding to the classification probability with the highest probability from the classification probabilities corresponding to the circumscribed rectangles meeting the preset conditions as a target circumscribed rectangle, wherein the preset conditions comprise: the center-of-gravity distance of each circumscribed rectangle is less than or equal to a predetermined value.
According to one or more embodiments of the present disclosure, the method further comprises: and determining the gravity center of the target circumscribed rectangle as the position of the pattern corresponding to the target circumscribed rectangle.
According to one or more embodiments of the present disclosure, there is provided an image processing apparatus including: the segmentation unit is configured to segment the image to be processed according to the pixel color values to generate at least one segmentation area; the generating unit is configured to generate an external rectangle of each divided area to obtain at least one divided area external rectangle; a determination unit configured to determine a contour region circumscribing rectangle of a contour region of an object displayed in the image to be processed; the classification unit is configured to classify the images of the at least one segmentation area circumscribed rectangle and the outline area circumscribed rectangle to generate a category information set.
According to one or more embodiments of the present disclosure, the segmentation unit is further configured to: image segmentation is carried out on the image to be processed according to pixel color values and the preset semantic color number, wherein the preset semantic color number is determined according to the number of mapping relations from the pixel color values to the semantic colors.
According to one or more embodiments of the present disclosure, the generating unit is further configured to: carrying out noise removal processing on the divided regions to obtain noise-removed divided regions; and generating the circumscribed rectangle of the noise-removed dividing region.
According to one or more embodiments of the present disclosure, the determining unit is further configured to: inputting the image to be processed into a pre-trained convolutional neural network, and outputting a contour region of the image to be processed; and determining the circumscribed rectangle of the outline area of the image to be processed.
According to one or more embodiments of the present disclosure, the classification unit is further configured to: for each circumscribed rectangle in the at least one circumscribed rectangle of the divided area and the circumscribed rectangle of the outline area, carrying out image classification on the circumscribed rectangle by adopting a classification network to obtain category information and classification probability corresponding to the category information; selecting a circumscribed rectangle corresponding to the classification probability with the highest probability from the classification probabilities corresponding to the circumscribed rectangles meeting the preset conditions as a target circumscribed rectangle, wherein the preset conditions comprise: the center-of-gravity distance of each circumscribed rectangle is less than or equal to a predetermined value.
According to one or more embodiments of the present disclosure, the above-described image processing apparatus is further configured to: and determining the gravity center of the target circumscribed rectangle as the position of the pattern corresponding to the target circumscribed rectangle.
According to one or more embodiments of the present disclosure, there is provided an electronic device including: one or more processors; a storage device having one or more programs stored thereon, which when executed by the one or more processors, cause the one or more processors to implement the method as described in any of the preceding claims.
According to one or more embodiments of the present disclosure, there is provided a computer readable medium having stored thereon a computer program, wherein the program when executed by a processor implements any of the methods described above.
The foregoing description is only of the preferred embodiments of the present disclosure and description of the principles of the technology being employed. It will be appreciated by those skilled in the art that the scope of the application in the embodiments of the present disclosure is not limited to the specific combination of the above technical features, but encompasses other technical features formed by any combination of the above technical features or their equivalents without departing from the spirit of the application. Such as the above-described features, are mutually substituted with (but not limited to) the features having similar functions disclosed in the embodiments of the present disclosure.
Claims (8)
1. An image processing method, comprising:
image segmentation is carried out on the image to be processed according to the pixel color values, and at least one segmentation area is generated; the image to be processed comprises a teaching aid chart;
generating an external rectangle of each divided area to obtain at least one external rectangle of the divided area;
determining a contour area circumscribed rectangle of a contour area of an object displayed in the image to be processed;
summarizing the at least one circumscribed rectangle of the dividing region and the circumscribed rectangle of the outline region to obtain a circumscribed rectangle; and
performing image classification on the image area in the circumscribed rectangle to generate a category information set of the image in the image area;
wherein the method further comprises:
for each circumscribed rectangle in the at least one circumscribed rectangle of the divided area and the circumscribed rectangle of the outline area, carrying out image classification on the circumscribed rectangle by adopting a classification network to obtain category information and classification probability corresponding to the category information;
selecting a circumscribed rectangle corresponding to the classification probability with the highest probability from the classification probabilities corresponding to the circumscribed rectangles meeting the preset conditions as a target circumscribed rectangle, wherein the preset conditions comprise: the center-of-gravity distance of each circumscribed rectangle is less than or equal to a predetermined value.
2. The method of claim 1, wherein the image segmentation of the image to be processed according to pixel color values comprises:
image segmentation is carried out on the image to be processed according to pixel color values and the preset semantic color number, wherein the preset semantic color number is determined according to the number of mapping relations from the pixel color values to the semantic colors.
3. The method of claim 1, wherein the generating a bounding rectangle for each segmented region comprises:
carrying out noise removal processing on the divided regions to obtain noise-removed divided regions;
and generating the circumscribed rectangle of the noise-removed dividing region.
4. The method of claim 1, wherein the determining a contour region bounding rectangle for a contour region of an object displayed in the image to be processed comprises:
inputting the image to be processed into a pre-trained convolutional neural network, and outputting a contour region of the image to be processed;
and determining the circumscribed rectangle of the outline area of the image to be processed.
5. The method of claim 4, wherein the method further comprises:
and determining the gravity center of the target circumscribed rectangle as the position of the pattern corresponding to the target circumscribed rectangle.
6. An image processing apparatus comprising:
the segmentation unit is configured to segment the image to be processed according to the pixel color values to generate at least one segmentation area; the image to be processed comprises a teaching aid chart;
the generating unit is configured to generate an external rectangle of each divided area to obtain at least one divided area external rectangle;
a determination unit configured to determine a contour region circumscribing rectangle of a contour region of an object displayed in the image to be processed;
the classifying unit is configured to summarize the at least one circumscribed rectangle of the dividing region and the circumscribed rectangle of the outline region to obtain the circumscribed rectangle; and
performing image classification on the image area in the circumscribed rectangle to generate a category information set of the image in the image area;
the classification unit is further configured to perform image classification on each circumscribed rectangle in the at least one segmented region circumscribed rectangle and the contour region circumscribed rectangle by adopting a classification network to obtain category information and classification probability corresponding to the category information;
selecting a circumscribed rectangle corresponding to the classification probability with the highest probability from the classification probabilities corresponding to the circumscribed rectangles meeting the preset conditions as a target circumscribed rectangle, wherein the preset conditions comprise: the center-of-gravity distance of each circumscribed rectangle is less than or equal to a predetermined value.
7. An electronic device, comprising:
one or more processors;
a storage device having one or more programs stored thereon,
when executed by the one or more processors, causes the one or more processors to implement the method of any of claims 1-5.
8. A non-transitory computer readable storage medium having stored thereon a computer program, wherein the program when executed by a processor implements the method of any of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010648627.3A CN111784710B (en) | 2020-07-07 | 2020-07-07 | Image processing method, device, electronic equipment and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010648627.3A CN111784710B (en) | 2020-07-07 | 2020-07-07 | Image processing method, device, electronic equipment and medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN111784710A CN111784710A (en) | 2020-10-16 |
| CN111784710B true CN111784710B (en) | 2023-10-20 |
Family
ID=72758890
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010648627.3A Active CN111784710B (en) | 2020-07-07 | 2020-07-07 | Image processing method, device, electronic equipment and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111784710B (en) |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377376A (en) * | 2012-04-13 | 2013-10-30 | 阿里巴巴集团控股有限公司 | Method and system for image classification, and method and system for image retrieval |
| CN106802113A (en) * | 2016-12-23 | 2017-06-06 | 西安交通大学 | Intelligent hit telling system and method based on many shell hole algorithm for pattern recognitions |
| CN107784301A (en) * | 2016-08-31 | 2018-03-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for identifying character area in image |
| CN108710916A (en) * | 2018-05-22 | 2018-10-26 | 重庆完美空间科技有限公司 | The method and device of picture classification |
| CN109101897A (en) * | 2018-07-20 | 2018-12-28 | 中国科学院自动化研究所 | Object detection method, system and the relevant device of underwater robot |
| CN109145746A (en) * | 2018-07-20 | 2019-01-04 | 浙江浩腾电子科技股份有限公司 | A kind of signal lamp detection method based on image procossing |
| CN109919149A (en) * | 2019-01-18 | 2019-06-21 | 平安科技（深圳）有限公司 | Object mask method and relevant device based on object detection model |
| CN111314614A (en) * | 2020-03-11 | 2020-06-19 | 北京字节跳动网络技术有限公司 | Image processing method and device, readable medium and electronic equipment |
-
2020
- 2020-07-07 CN CN202010648627.3A patent/CN111784710B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103377376A (en) * | 2012-04-13 | 2013-10-30 | 阿里巴巴集团控股有限公司 | Method and system for image classification, and method and system for image retrieval |
| CN107784301A (en) * | 2016-08-31 | 2018-03-09 | 百度在线网络技术（北京）有限公司 | Method and apparatus for identifying character area in image |
| CN106802113A (en) * | 2016-12-23 | 2017-06-06 | 西安交通大学 | Intelligent hit telling system and method based on many shell hole algorithm for pattern recognitions |
| CN108710916A (en) * | 2018-05-22 | 2018-10-26 | 重庆完美空间科技有限公司 | The method and device of picture classification |
| CN109101897A (en) * | 2018-07-20 | 2018-12-28 | 中国科学院自动化研究所 | Object detection method, system and the relevant device of underwater robot |
| CN109145746A (en) * | 2018-07-20 | 2019-01-04 | 浙江浩腾电子科技股份有限公司 | A kind of signal lamp detection method based on image procossing |
| CN109919149A (en) * | 2019-01-18 | 2019-06-21 | 平安科技（深圳）有限公司 | Object mask method and relevant device based on object detection model |
| CN111314614A (en) * | 2020-03-11 | 2020-06-19 | 北京字节跳动网络技术有限公司 | Image processing method and device, readable medium and electronic equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111784710A (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111369427B (en) | Image processing method, image processing device, readable medium and electronic equipment | |
| CN110211195B (en) | Method, device, electronic equipment and computer-readable storage medium for generating image set | |
| CN110288625B (en) | Method and apparatus for processing image | |
| CN114494298A (en) | Object segmentation method, device, equipment and storage medium | |
| CN111414879A (en) | Face shielding degree identification method and device, electronic equipment and readable storage medium | |
| CN111783777B (en) | Image processing method, apparatus, electronic device, and computer readable medium | |
| CN113658196A (en) | Method and device for detecting ship in infrared image, electronic equipment and medium | |
| CN111783889B (en) | Image recognition method and device, electronic equipment and computer readable medium | |
| CN111784709B (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| CN113435393B (en) | Forest fire smoke root node detection method, device and equipment | |
| CN111784710B (en) | Image processing method, device, electronic equipment and medium | |
| CN110852242A (en) | Watermark identification method, device, equipment and storage medium based on multi-scale network | |
| CN112085733B (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| WO2022052889A1 (en) | Image recognition method and apparatus, electronic device, and computer-readable medium | |
| CN115100536A (en) | Building identification method, building identification device, electronic equipment and computer readable medium | |
| CN114862720A (en) | Canvas restoration method and device, electronic equipment and computer readable medium | |
| CN112085035A (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| CN114120423A (en) | Face image detection method and device, electronic equipment and computer readable medium | |
| CN112101258A (en) | Image processing method, image processing device, electronic equipment and computer readable medium | |
| CN110991312A (en) | Method, apparatus, electronic device, and medium for generating detection information | |
| CN111797932B (en) | Image classification method, apparatus, device and computer readable medium | |
| CN114359673B (en) | Small sample smoke detection method, device and equipment based on metric learning | |
| CN113239943B (en) | Three-dimensional component extraction and combination method and device based on component semantic graph | |
| CN111814807B (en) | Method, apparatus, electronic device, and computer-readable medium for processing image | |
| CN112215789B (en) | Image defogging method, device, equipment and computer readable medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB02 | Change of applicant information |
Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Tiktok vision (Beijing) Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: BEIJING BYTEDANCE NETWORK TECHNOLOGY Co.,Ltd. Address after: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant after: Douyin Vision Co.,Ltd. Address before: 100041 B-0035, 2 floor, 3 building, 30 Shixing street, Shijingshan District, Beijing. Applicant before: Tiktok vision (Beijing) Co.,Ltd. |
|
| CB02 | Change of applicant information | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |