WO2020168818A1

WO2020168818A1 - Image processing method, system and device, storage medium, and blackboard device

Info

Publication number: WO2020168818A1
Application number: PCT/CN2019/127758
Authority: WO
Inventors: 邹超洋
Original assignee: 广州视源电子科技股份有限公司; 广州视睿电子科技有限公司
Priority date: 2019-02-20
Filing date: 2019-12-24
Publication date: 2020-08-27
Also published as: CN109978889A

Abstract

An image processing method, system and device, a storage medium and a blackboard device. Said method comprises: acquiring a first blackboard image (S101); performing threshold segmentation processing of a pixel point on the first blackboard image (S102); performing at least one threshold segmentation processing of an image block on the first blackboard image (S103); if it is determined that the number of times that the pixel point of the first blackboard image is classified into a first type of pixel points is within a first threshold range, taking the pixel point as a target pixel point (S104); and using the target pixel point to extract a target image from the first blackboard image (S105).

Description

Image processing method, system, device, storage medium and blackboard device

This application claims the priority of a Chinese patent application filed with the Chinese Patent Office with an application number of 201910127218.6 on February 20, 2019. The entire content of the application is incorporated into this application by reference.

Technical field

This application relates to image processing technology, such as an image processing method, system, device, storage medium, and blackboard device.

Background technique

Image threshold segmentation algorithm is a commonly used image segmentation method. Because of its simple implementation, small calculation amount, and stable performance, it has become the most basic and most widely used segmentation technology in image segmentation. It is widely used in extracting target objects from images. In the program. Currently, commonly used threshold segmentation algorithms mainly include two ideas, one is a global threshold segmentation algorithm, and the other is a local threshold segmentation algorithm. For the global threshold segmentation algorithm, it mainly uses different measurement indicators to find the optimal global threshold, so as to use this optimal global threshold to segment the image, but it is basically impossible for the target scene to have multiple peaks and valleys. Adaptation, that is, if there are multiple peaks and valleys in the image, the global threshold segmentation algorithm cannot segment the foreground and the background well. Therefore, it can be seen that for the image with reflection, the global threshold is used The segmentation algorithm cannot extract the target image from the image well, and the accuracy of the target image extraction is low.

Summary of the invention

In view of this, embodiments of the present invention provide an image processing method, system, device, storage medium, and blackboard device, which can effectively extract a target image from a blackboard image.

On the one hand, an embodiment of the present invention provides an image processing method, including:

Acquiring the first blackboard image;

Performing threshold segmentation processing on the pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels;

Perform at least one image block threshold segmentation process on the first blackboard image; wherein the threshold segmentation process of the image block is specifically performing pixel threshold segmentation on at least one image block of the first blackboard image Processing, so that the pixels of the at least one image block are divided into first type pixels or second type pixels;

It is determined that the number of times the pixels of the first blackboard image are classified into the first type of pixels is within the first threshold range, then the pixel is taken as the target pixel;

Using the target pixels, a target image is extracted from the first blackboard image.

Optionally, the threshold segmentation processing of the image block at least once is specifically: the threshold segmentation processing of the image block at least twice;

In the threshold segmentation processing of the at least two image blocks, the attributes of the image blocks used in the threshold segmentation processing of each image block are different, and the attributes include the number, size, shape, and/or selection position of the image block.

Optionally, the number of the image blocks is at least two, and the at least two image blocks are obtained by evenly dividing the first blackboard image.

Optionally, the threshold segmentation algorithm used in the step of performing pixel threshold segmentation processing on the first blackboard image is a global threshold segmentation algorithm, and/or the threshold segmentation algorithm for the first blackboard image The threshold segmentation algorithm used in the step of threshold segmentation processing for at least one image block is a global threshold segmentation algorithm.

Optionally, the step of performing threshold segmentation processing on pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels, It includes:

Perform threshold segmentation processing on the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels, and then the first type pixels Set as white pixels, and set the second type of pixels as black pixels, thereby obtaining a second blackboard image.

Optionally, the step of performing image block threshold segmentation processing at least once on the first blackboard image includes:

After performing at least one image block threshold segmentation process and binarization process on the first blackboard image, at least one third blackboard image is obtained;

Wherein, the threshold segmentation processing and binarization processing of the image block include:

Perform pixel threshold segmentation processing on at least one image block of the first blackboard image, so that the pixels of the at least one image block are divided into the first type of pixel or the second type of pixel, and then the The pixels of the first type are set as white pixels, and the pixels of the second type are set as black pixels, thereby obtaining a third blackboard image.

Optionally, the step of determining that the number of times that the pixel of the first blackboard image is classified into the first type of pixel is within a first threshold range, then the step of using the pixel as a target pixel includes:

After the second blackboard image and the at least one third blackboard image are summed to obtain the fourth blackboard image, it is determined that the pixel value of the fourth blackboard image is within the second threshold range, then the This pixel is regarded as the target pixel;

Wherein, the second threshold value range is determined according to the first threshold value range.

On the other hand, an embodiment of the present invention also provides an image processing system, including:

The first acquiring unit is configured to acquire the first blackboard image;

A first processing unit, configured to perform threshold segmentation processing on pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first-type pixels or second-type pixels;

The second processing unit is configured to perform threshold segmentation processing of image blocks at least once on the first blackboard image; wherein the threshold segmentation processing of the image blocks is specifically: performing at least one image of the first blackboard image Performing threshold segmentation processing of pixels on the block, so that the pixels of the at least one image block are divided into first type pixels or second type pixels;

The third processing unit is configured to determine that the number of times that the pixels of the first blackboard image are divided into the first type of pixels is within a first threshold range, and then the pixels are used as target pixels;

The fourth processing unit is configured to extract a target image from the first blackboard image by using the target pixel.

In another aspect, an embodiment of the present invention also provides an image processing device, which includes:

At least one processor;

At least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the image processing method.

In another aspect, an embodiment of the present invention also provides a storage medium in which instructions executable by a processor are stored, and the instructions executable by the processor are used to execute the image processing method when executed by the processor.

In another aspect, an embodiment of the present invention provides a blackboard device, including a blackboard, a camera, and a terminal device connected to the camera;

The camera is used to photograph the blackboard;

The terminal equipment includes:

At least one processor;

At least one memory for storing at least one program;

One or more technical solutions in the above-mentioned embodiments of the present invention have the following advantages: the embodiment of the present invention performs threshold segmentation processing on the first blackboard image and performs pixel points on each image block in the first blackboard image. According to the threshold segmentation processing, it is possible to determine whether the pixel is the target pixel according to the number of times the pixel is divided into the first type of pixel. It can be seen that the embodiment of the present invention has the effect of local threshold segmentation, and the pixel is divided The target pixel is determined for the number of times of the first type of pixel. Compared with the traditional solution, this can effectively solve the problem of image reflection and greatly improve the accuracy of extracting the target image from the blackboard image.

Description of the drawings

Fig. 1 is a flowchart of steps in a first specific embodiment of an image processing method according to an embodiment of the present invention;

Figure 2 is a schematic diagram of an image obtained after a graph is segmented using a traditional local threshold segmentation algorithm;

3 is a flowchart of steps in a second specific embodiment of an image processing method according to an embodiment of the present invention;

4 is a block diagram of a specific embodiment of an image processing system according to an embodiment of the present invention;

5 is a block diagram of a specific embodiment of an image processing apparatus according to an embodiment of the present invention;

Fig. 6 is a structural block diagram of a specific embodiment of a blackboard device according to an embodiment of the present invention.

detailed description

The application will be further described in detail below with reference to the drawings and specific embodiments. For the step numbers in the following embodiments, they are set only for ease of elaboration, and there is no limitation on the order between the steps. The execution order of the steps in the embodiments can be adapted according to the understanding of those skilled in the art. Sexual adjustment. In addition, the "up", "down", "left", "right", "front", and "rear" mentioned in the following embodiments are only used to clearly describe the positional relationship, and are relative positional relationships, not absolute The position relationship can be adjusted adaptively according to the understanding of those skilled in the art.

The image processing solution provided by the embodiment of the present invention can be applied to a smart blackboard scene, so as to extract a target object (such as handwriting on the blackboard) from the blackboard image. For the smart blackboard scene, it mainly includes a blackboard device. The blackboard device includes a blackboard, a camera, and a terminal device connected to the camera. The camera can be set on the blackboard and used to photograph the blackboard. The terminal device is mainly used to perform image processing on the blackboard image taken by the camera, so as to realize the operation of displaying and recording the blackboard image. Usually, in the process of displaying and recording the blackboard image, it is necessary to recognize the handwriting on the blackboard image. In order to be able to recognize the handwriting on the blackboard image well, you should choose to start from the blackboard image After extracting the handwriting image (that is, the target image) as the foreground of the blackboard image, the handwriting is recognized and adjusted. Therefore, it can be seen that a solution for extracting handwriting images (ie target images) from blackboard images is one of the very important links in the smart blackboard scene.

Based on this, an embodiment of the present invention provides an image processing method for extracting a target image from a blackboard image, as shown in FIG. 1, and the steps involved are as follows.

S101. Acquire a first blackboard image.

In an embodiment, for the first blackboard image, it may be directly the original blackboard image captured by a camera, or it may be a blackboard image obtained after image preprocessing is performed on the original blackboard image captured by the camera (wherein, the The image preprocessing can include, but is not limited to, image filtering, de-manipulation, morphological image processing, etc.). The former has the advantages of simple steps and high processing efficiency, while the latter has the advantages of improving recognition accuracy. After the original blackboard image captured by the camera is roughly positioned in the image area where the target object is located, the image area obtained by the positioning is used as the first blackboard image, which not only improves processing efficiency but also improves accuracy. Therefore, as to which blackboard image is acquired as the first blackboard image, this can be set and selected according to the actual situation, and there is no excessive limitation here.

S102: Perform threshold segmentation processing on pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels. Performing this step is equivalent to that the corresponding pixel in the first blackboard image has been subjected to a pixel type division process. If the pixel is divided into the first type of pixel, then the pixel is divided into the first type of pixel. The number of times of a type of pixel is increased by 1. If the pixel is classified into a second type of pixel, the number of times the pixel is classified into a second type of pixel is 1.

In this embodiment, for the threshold segmentation process, the specific processing steps are: threshold judgment is performed on the pixel points in the image one by one through the feature threshold, so that according to the judgment result, the pixel points in the image are divided into at least Two types of pixels. For example, when a certain pixel point i is within the threshold range corresponding to the type A pixel point, the pixel point i is classified as the type A pixel point. As for the feature mentioned in the feature threshold, it refers to the feature of the image, such as the grayscale and color of the image. Therefore, if the image subjected to the threshold segmentation process is different, the obtained feature threshold is also different. It must be the same, and how to obtain the feature threshold will vary according to the algorithm used. The method for obtaining the feature threshold includes, but is not limited to, algorithms with the largest inter-class variance (such as the Otsu algorithm), The maximum variance threshold algorithm, the bimodal selection threshold algorithm, etc., are not too limited here. It can be seen that, for the step of performing the threshold segmentation processing on the pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into the first type of pixels or the second type of pixels, the In order to threshold the pixels in the first blackboard image according to the characteristic threshold of the first blackboard image, and divide the pixels in the first blackboard image into the first type of pixels or the second type of pixels according to the judgment result point.

In addition, for the first type of pixel, it is essentially the target type of pixel, and among all the pixels of the image, except for the first type of pixel, the pixel is the second type of pixel, for example, a total of There are three threshold ranges, namely Threshold Range 1, Threshold Range 2, and Threshold Range 3. Through these three threshold ranges, the pixels in the image can be divided into three types of pixels, and the pixel values fall into Threshold Range 1. The pixels in are type 1 pixels, the pixels whose pixel values fall within the threshold range 2 are type 2 pixels, and the pixels whose pixel values fall within the threshold range 3 are type 3 pixels. If the type 1 pixel is Target type pixels, then type 1 pixels are the first type pixels, and type 2 pixels and type 3 pixels belong to the second type pixels, if type 1 pixels and type 3 pixels are the target type pixels , Then type 1 pixel and type 3 pixel are the first type pixel, and type 2 pixel is the second type pixel. If the number of pixel types to be divided is more, the same is true.

S103. Perform at least one image block threshold segmentation process on the first blackboard image; wherein the threshold segmentation process of the image block is specifically: performing pixel point segmentation on at least one image block of the first blackboard image. Threshold segmentation processing, so that the pixels of the at least one image block are divided into first type pixels or second type pixels. Regarding the threshold segmentation process, the first type pixel point, and the second type pixel point described in this step, they are the same as the analytical definition of the above step S102.

In one embodiment, this step is mainly to perform threshold segmentation processing on at least one image block in the first blackboard image, that is, in this step, the object to be subjected to threshold segmentation processing is the image block in the first blackboard image Therefore, at this time, the characteristic threshold of the corresponding image block will be different from the characteristic threshold of the corresponding first blackboard image (of course, there may also be the case where the characteristic threshold of the corresponding image block is the same as the characteristic threshold of the corresponding first blackboard image), that is, this The type of pixel points will be different.

In addition, for the threshold segmentation processing of the image block at least once, when the threshold segmentation processing of the image block is performed once, if the pixels in the image block (which are essentially the pixels in the first blackboard image) are divided into first Type pixel, the number of times that the pixel is divided into the first type pixel is increased by 1. If the pixel in the image block is divided into the second type pixel, then the pixel is divided into the second type pixel Increase the number of points by 1. Therefore, if the pixel i in the first blackboard image is divided into the first type of pixel after the step S102 is performed, after the step S103 is performed (assuming that the threshold segmentation process of the image block is performed at least once at this time) Specifically, it is the threshold segmentation processing of the image block for three times), and it is determined that the pixel i is divided into the first type of pixel during the threshold segmentation processing of the three image blocks. At this time, the pixel i is divided into the first type of pixel. The number of pixels of a type is 4. As for the image blocks in the first blackboard image, the selection and acquisition methods from the first blackboard image can be selected according to the actual situation, and when the threshold segmentation processing of each image block is performed, the selection and acquisition methods of the image blocks can be the same. Not exactly the same or completely different. For example, in the first blackboard image, there is only one image area with light reflection, then this image area can be selected as the image block, and then the image block Perform threshold segmentation processing, and then perform the remaining times of image block threshold segmentation processing, the selected acquisition position, size, and shape of the image block can be adjusted, for example, compared to the image block acquired the first time, the second time, The selection and acquisition position of the 3rd and/or 4th image block can be moved up/down/left/right by the corresponding distance, or the size or shape of the image block can be adjusted, such as the size becomes larger or smaller, and the shape changes from square Or the rectangle becomes round or irregular. If the selection and acquisition method of the image block remains unchanged, the number of executions of the threshold segmentation processing of the image block may be 1. Generally, the area with reflections on the blackboard image changes randomly. Therefore, it is difficult to determine the area with reflections. In this case, the number of image blocks should be selected as multiple, and the multiple The extraction positions of the image blocks on the first blackboard image are set as evenly as possible everywhere in the first blackboard image. Furthermore, the multiple image blocks are preferably obtained by dividing the first blackboard image. In this case, , The first blackboard image is divided into image regions, and the image regions obtained after the division are equivalent to image blocks. It can be seen that, for the step S103, it is equivalent to performing local threshold segmentation processing on the first blackboard image.

For step S102 and step S103, the execution order between the two can be interchanged, that is, step S103 is executed first and then step S102 is executed.

S104. It is determined that the number of times that the pixel of the first blackboard image is divided into the first type of pixel is within a first threshold range, and the pixel is taken as a target pixel.

In an embodiment, after the above-mentioned steps S102 and S103 are processed, the number of times the pixels in the first blackboard image are classified into the first type of pixels is counted, and then the counted times are thresholded to determine whether the times are If it falls within the first threshold range, it means that the pixel is the target pixel. For example, if the pixel point i has been subjected to a total of 3 types of classification processing (the type classification processing is the processing of dividing the pixel point i into a first type pixel point or a second type pixel point), at this time, if the pixel point When the number of times i is divided into the first type pixel is greater than or equal to a preset threshold (such as 2), that is, when it falls within the first threshold range, it means that the pixel i is the target pixel. Wherein, for the first threshold range, it can be an empirical value range, or it can be determined according to the total number of times that the pixel is subjected to type division processing, and the lower limit of the first threshold range can be selected as the pixel More than half of the total number of times of performing type division processing, that is, let the first threshold range be [a, b], where a=c*d, and c means that the pixels of the first blackboard image are executed type division processing The total number of times, d is expressed as a percentage, d is greater than or equal to 50%; b is expressed as a value greater than or equal to c. Of course, it is also possible to set d to be less than 50%, but if d is greater than or equal to 50%, the judgment accuracy of the target pixel will be higher, which is suitable for scenes with unsatisfactory external environmental factors.

S105. Using the target pixels, extract a target image from the first blackboard image. If the target pixel corresponds to the foreground of the blackboard image, that is, the handwriting, then the extracted target image is the foreground image. If the target pixel corresponds to the background of the blackboard image, then the extracted target image is Background image.

It can be seen that by using the image processing method of the above embodiment of the present invention, the blackboard image is first subjected to the threshold segmentation processing of the entire image, and then the pixels of the first blackboard image are divided into the first type of pixels or the second type of pixels, and then After threshold segmentation is performed on the image block of the blackboard image, the pixels of the image block are divided into the first type pixel point or the second type pixel point to achieve the effect of local threshold segmentation of the blackboard image, and then pass the pixel point The number of times that pixels are classified into the first type is thresholded to determine the target pixel. In this way, the determined target pixel is used to extract the corresponding image from the first blackboard image, that is, the target image. Its validity and accuracy High, and it can solve the problem of judging the foreground as the background due to the reflection of the image.

In an optional embodiment, the at least one image block threshold segmentation process is specifically: at least two image block threshold segmentation processes; each of the at least two image block threshold segmentation processes The attributes of the image blocks used in the threshold segmentation process are different, and the attributes include the number, size, shape, and/or selection position of the image block. Wherein, the difference means completely different or partly different.

In this embodiment, the step of image block threshold segmentation processing is performed at least twice, and in each processing, the attribute parameters of the image block are not the same (that is, each time the image block threshold segmentation processing is performed When the image block selection and extraction method has been adjusted), this means that each time the threshold segmentation process of the image block is performed, for the threshold judgment of the same pixel, the characteristic threshold used will be different each time , Which is equivalent to the feature threshold value used is randomly changed. It can be seen that in this case, by counting the number of times the pixel is divided into the first type of pixel and the threshold judgment, the confirmation of the target pixel can be further improved Accuracy, thereby improving the accuracy and efficiency of target extraction.

Moreover, it is important to make the attributes of the image blocks used in the threshold segmentation processing of each image block different, which can solve the problem that the traditional local threshold segmentation method cannot perform well on the filled graphics (such as "▆", "●" , "◆", etc.) for foreground segmentation, because the traditional local threshold segmentation method is based on the idea of determining a neighborhood for each pixel, and then selecting an optimal threshold in this neighborhood to use this optimal threshold To judge the pixel to achieve image segmentation, then if the size of the selected neighborhood area is too small to cover the entire graph, but is located inside the graph, then there will be When a certain part is segmented into a background image, for example, as shown in Figure 2, for a filled diamond shape, the size of the neighborhood area is too small, resulting in threshold segmentation, and a part of the diamond shape is divided into the background part , Which is the blank part of the graph. By using the method of this embodiment, since each image block threshold segmentation process, the attributes of the image block will be different, so that the characteristic threshold will change randomly, so that even if one or two of the pixels belonging to the foreground are judged as The background pixel, but as long as the pixel is judged as the foreground pixel 3 or more times (for example, the pixel has undergone a total of 5 types of classification processing), then the pixel will eventually be judged as the foreground pixel pixel. It can be seen that by using the method of this embodiment to achieve the extraction of the target image, its accuracy and effectiveness are further improved, and it can solve the problem that the traditional local threshold segmentation method is easy to judge the partial area of the filled figure as the background image. , That is, to solve the problem that the traditional local threshold segmentation method cannot handle the problem of whether the current neighborhood is divided into the global foreground or the global background.

In an optional embodiment, the number of the image blocks is at least two, and the at least two image blocks are obtained by evenly dividing the first blackboard image. Since the handwriting written on the blackboard may spread all over the blackboard, it is difficult to determine the image area with reflective image area and/or the image area where the filled figure is located. Therefore, in order to better adapt to the blackboard scene, use the even division Way to obtain at least two image blocks, so that the foreground handwriting of the blackboard image can be obtained well. Moreover, the image blocks are obtained by dividing in an average manner, which can reduce the workload of program design and utilize the design, modification and adjustment of the staff.

In an optional embodiment, the threshold segmentation algorithm used in the step of performing pixel threshold segmentation processing on the first blackboard image is a global threshold segmentation algorithm, and/or, the first blackboard image The threshold segmentation algorithm used in the step of threshold segmentation processing for at least one image block of a blackboard image is a global threshold segmentation algorithm. That is equivalent to the step of performing pixel threshold segmentation processing on the first blackboard image, which specifically includes using a first threshold segmentation algorithm to perform pixel threshold segmentation processing on the first blackboard image; The step of performing pixel threshold segmentation processing on at least one image block of the first blackboard image specifically includes using a second threshold segmentation algorithm to perform pixel pixel processing on at least one image block of the first blackboard image. Point threshold segmentation processing; wherein the first threshold segmentation algorithm and/or the second threshold segmentation algorithm are global threshold segmentation algorithms. Since in this embodiment, the first threshold segmentation algorithm and/or the second threshold segmentation algorithm adopts a global threshold segmentation algorithm, it can further reduce the traditional local threshold segmentation algorithm that can easily determine the partial area of the filled figure as a background. The problem of the image guarantees the accuracy of the segmentation of the foreground image and the background image.

In an optional embodiment, the threshold segmentation processing is performed on the pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into the first type of pixels or the second type of pixels This step S102 includes:

S1021. Perform threshold segmentation processing on pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels, and then divide the first type The pixels are set as white pixels, and the second type pixels are set as black pixels, thereby obtaining a second blackboard image. At this time, the second blackboard image is a binarized image of the first blackboard image.

It can be seen that in this embodiment, the step S1021 is implemented by performing threshold segmentation processing and binarization processing on the first blackboard image, which not only achieves the pixel processing of the first blackboard image The purpose of threshold segmentation is to mark the pixels classified as the first type as 255 or 1 (that is, the pixels of the first type are set as white pixels), and for the pixels classified as the second type Marked as 0 (that is, the second type of pixel is set as a black pixel), so that the pixel value after the pixel binarization can facilitate the subsequent statistics of the number of times the pixel is divided into the first type of pixel . It should be noted that in the color space represented by integers, the pixel value range is 0 to 255, and the pixel value corresponding to the white pixel is 255; while in the color space represented by floating point numbers, the pixel value range It is 0 to 1, and the pixel value corresponding to the white pixel is 1. In order to quickly count the number of times and reduce the amount of calculation, the pixel value 1 should be selected to represent the white pixels, that is, the color space of the image should be represented by a floating point number, so that when the first blackboard image is determined When the pixel value of the pixel in the binarized image is 1, it means that in the current binarized image, the pixel is divided into the first type of pixel, that is, the foreground pixel, which also indicates the pixel The number of times the pixels are classified into the first type plus one.

In an optional embodiment, the step S103 of performing image block threshold segmentation processing on the first blackboard image at least once includes:

S1031, after performing at least one image block threshold segmentation process and binarization process on the first blackboard image, at least one third blackboard image is obtained.

The threshold segmentation processing and binarization processing of the image block include:

It can be seen that in this embodiment, the above step S1031 is implemented by performing threshold segmentation processing and binarization processing on the at least one image block, which not only achieves the threshold segmentation processing of pixels on the image block It also realizes that the pixels classified into the first type are marked as 255 or 1 (that is, the pixels of the first type are set as white pixels), and the pixels classified as the second type are marked as 0 (That is, the second-type pixel is set as a black pixel). In this way, the pixel value after the pixel is binarized can facilitate subsequent statistics on the number of times the pixel is classified into the first-type pixel. Also optionally, a pixel value of 1 should be optionally used to represent white pixels, that is, a floating point number should be selected to represent the color space of the image, so that when the pixel in the binary image of the image block is determined When the value is 1, it means that in the current binarized image, the pixel is divided into the first type of pixel, that is, the foreground pixel, and it also indicates the number of times the pixel is divided into the first type of pixel plus 1.

In an optional embodiment, the step of determining that the number of times that the pixel of the first blackboard image is divided into the first type of pixel is within a first threshold range, then the pixel is used as the target pixel S104, which includes:

S1041, after the second blackboard image and the at least one third blackboard image are summed to obtain the fourth blackboard image, it is determined that the pixel values of the pixels of the fourth blackboard image are within the second threshold range, Then the pixel is regarded as the target pixel.

The second threshold range is determined according to the first threshold range.

In an embodiment, if the number of image blocks is multiple and is obtained by dividing the first blackboard image, that is, the multiple image blocks constitute a complete first blackboard image, at this time, the at least A third blackboard image is also a binarized image of the first blackboard image. Therefore, by performing pixel value summation processing on the second blackboard image and the at least one third blackboard image, it is equivalent to counting the number of times each pixel is judged to be the first type of pixel to obtain each pixel The total number of times H is judged to be the first type of pixel. At this time, a threshold judgment can be performed on the total number of times H, so as to finally determine whether the corresponding pixel is the target pixel. Wherein, when the pixel value corresponding to the white pixel is 1, the second threshold range is the same as the first threshold range. If the first type of pixel is not set as a white pixel, but with another color, that is, the second threshold The pixel value corresponding to one type of pixel is not 1. In this case, the first threshold range needs to be multiplied by a corresponding multiple to obtain the required second threshold range.

In an optional embodiment, the number of executions of the threshold segmentation processing of the image block can be selected from 2 to 4 times.

The application will be further elaborated below in combination with specific optional embodiments.

As shown in Figure 3, the embodiment of the present invention provides an image processing method, which is specifically applied to the writing scene of a smart blackboard device, and is mainly used to extract the handwriting image (ie, the target foreground image) on the blackboard image. The specific steps are As follows.

S201. Acquire a first blackboard image. In this embodiment, the first blackboard image is an original blackboard image captured by a camera or a blackboard image obtained after preprocessing the original blackboard image captured by a camera.

S202: Perform threshold segmentation processing on pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels.

In this embodiment, the step S202 is specifically that the Otsu algorithm is used to obtain the optimal feature threshold of the first blackboard image, and then the optimal feature threshold is used to threshold the pixels in the first blackboard image one by one, thereby According to the judgment result, the pixels in the first blackboard image are divided into foreground type pixels (i.e., first type pixels) and background type pixels (second type pixels), so as to realize the first blackboard image. Global threshold segmentation; then the pixels classified into the foreground type are set as white pixels, and the pixels classified as the background type are set as black pixels, so that the first blackboard image is binarized to obtain the first Two blackboard image. It can be seen that the second blackboard image is a binary image, that is, the first segmented mask image (mask1).

S203. Perform at least one image block threshold segmentation process on the first blackboard image; wherein the threshold segmentation process of the image block is specifically: performing pixel point segmentation on at least one image block of the first blackboard image. Threshold segmentation processing, so that the pixels of the at least one image block are divided into first type pixels or second type pixels.

In this embodiment, there are multiple image blocks, and can optionally be obtained by evenly dividing the first blackboard image. At the same time, the number of executions n of the threshold segmentation processing of the image block can be selected as 3, and each time the threshold segmentation processing of the image block is executed, the size of the image block used is different each time, that is, the first The size of the image block used in the second execution of the threshold segmentation process of the image block is size 1, the size of the image block used in the second execution of the threshold segmentation process of the image block is size 2, and the third execution of the threshold value of the image block The size of the image block used in the segmentation process is size 3. Size 1, size 2 and size 3 are all different, and because the image block described in this embodiment is obtained by evenly dividing the first blackboard image Therefore, by adjusting the number of divided image blocks, the size of the image blocks can be adjusted.

Each time the threshold segmentation processing of an image block is executed, the specific processing process is: threshold segmentation processing is performed on each image block separately, that is, at this time, the image block is used as the input image to perform the threshold segmentation processing of the pixels in the image block. That is to say, the optimal feature threshold obtained at this time is obtained after processing the image block, that is, threshold segmentation is performed on an image block, which can be selected as using Otsu algorithm to obtain the optimal feature of the image block Threshold, and then through the optimal feature threshold, threshold judgment is performed on the pixels in the image block one by one, so that according to the judgment result, the pixels in the image block are divided into foreground type pixels (i.e. first type pixels) and background type Pixels (pixels of the second type), thereby realizing global threshold segmentation of the image block. Then set the pixels classified as foreground type to white pixels (that is, set the pixel value of this pixel to 1), and set the pixels classified as background type to black (that is, set the pixel value of this pixel to 0 ) To realize the binarization of image blocks. Since all the image blocks constitute the first blackboard image, after all the image blocks are subjected to the above threshold segmentation processing and binarization processing steps, a third blackboard image is obtained. It can be seen that the third blackboard image is also one or two. The valued image is only obtained by binarizing each image block after dividing the first blackboard image into blocks. In addition, since in this embodiment, the image block threshold segmentation process is performed three times, three third blackboard images will be obtained, that is, mask_i, i=1, 2, 3 will be obtained. It can be seen that at this time, for the same pixel in the first blackboard image, such as pixel j, the total number of times the type division processing is executed is 4.

S204. After performing a summation process on the second blackboard image and the at least one third blackboard image to obtain a fourth blackboard image, it is determined that the pixel values of the pixels of the fourth blackboard image are within a second threshold range. Then the pixel is regarded as the target pixel. Wherein, the second threshold value range is determined according to the first threshold value range.

In an embodiment, the sum of mask1 and mask_i (i=1, 2, 3) is to add up the pixel values of the same pixel in the second blackboard image and the three third blackboard images. At this time, add 4 After the images are superimposed, the fourth blackboard image (ie, the second mask image mask2) will be obtained, that is, the pixel value of each pixel of the fourth blackboard image is mask1 and mask_i (i=1, 2, 3) The value after the sum of the pixel values corresponding to the same pixel, such as pixel j in the fourth blackboard image, its pixel value is the pixel value of pixel j in mask1, the pixel value of pixel j in mask_1, mask_2 The sum of the pixel value of the middle pixel j and the pixel value of the pixel j in mask_3. Then, since in this embodiment, the pixel value of pixels classified as foreground type is set to 1, and the pixel value of pixels classified as background type is set to 0, the threshold range used can be [2 , 4], at this time, after the fourth blackboard image is thresholded by using the threshold range, the pixels falling within the threshold range, that is, pixels with a pixel value greater than or equal to 2, are set to white The pixel point, that is, the pixel value is set to 1, and the pixel point that does not fall into the threshold range, that is, the pixel point with a pixel value less than 2, is set as a black pixel point, that is, the pixel value is set to 0, which is equivalent to After the four blackboard images are thresholded and then binarized, the fifth blackboard image is obtained, and the fifth blackboard image is the third mask image mask3.

S205. Acquire a foreground handwriting image of the blackboard from the first blackboard image according to the third mask image mask3.

In one embodiment, since the pixel values of the foreground pixels in mask3 are all 1, according to the pixels with the pixel value of 1 in mask3, the pixels at the corresponding positions are obtained from the first blackboard image. The final required handwriting image.

It can be seen that this embodiment can achieve the local threshold segmentation of the first blackboard image by thresholding the image block, and at the same time, the attributes of the image block used are different each time the threshold segmentation process of the image block is performed, so as to achieve multiple Scale threshold segmentation, which can not only solve the problem of the blackboard image being unable to segment the foreground and background due to reflections, but also solve the traditional local threshold segmentation because the size of the selected neighborhood is too small to cover the entire image. Lead to the problem of segmenting a certain part of the graph into the background. It can be seen that by using the method of the embodiment of the present invention, the foreground and background can be segmented effectively and accurately, and it is very suitable for the scene of the smart blackboard device. Because the blackboard is easy to reflect light, the blackboard image often has a reflective area, and In class, I often draw filling figures on the blackboard.

As shown in FIG. 4, an embodiment of the present invention also provides an image processing system, including:

The first acquiring unit 301 is configured to acquire a first blackboard image;

The first processing unit 302 is configured to perform threshold segmentation processing on pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels;

The second processing unit 303 is configured to perform threshold segmentation processing of an image block at least once on the first blackboard image; wherein the threshold segmentation processing of the image block is specifically performed on at least one of the first blackboard image Performing threshold segmentation processing of pixels on the image block, so that the pixels of the at least one image block are divided into first type pixels or second type pixels;

The third processing unit 304 is configured to determine that the number of times that the pixels of the first blackboard image are classified into the first type of pixels is within a first threshold range, and then use the pixel as a target pixel;

The fourth processing unit 305 is configured to use the target pixels to extract a target image from the first blackboard image.

It can be seen that by using the image processing system of the above embodiment of the present invention, the blackboard image is first subjected to the threshold segmentation processing of the entire image, and then the pixels of the first blackboard image are divided into the first type of pixels or the second type of pixels, and then After threshold segmentation is performed on the image block of the blackboard image, the pixels of the image block are divided into the first type pixel point or the second type pixel point to achieve the effect of local threshold segmentation of the blackboard image, and then pass the pixel point The number of times that pixels are classified into the first type is thresholded to determine the target pixel. In this way, the determined target pixel is used to extract the corresponding image from the first blackboard image, that is, the target image. Its validity and accuracy High, and it can solve the problem of judging the foreground as the background due to the reflection of the image.

In an optional embodiment, the threshold segmentation processing of the image block at least once is specifically: the threshold segmentation processing of the image block at least twice.

In an optional embodiment, the number of the image blocks is at least two, and the at least two image blocks are obtained by evenly dividing the first blackboard image.

In an optional embodiment, the threshold segmentation algorithm used in the step of performing pixel threshold segmentation processing on the first blackboard image is a global threshold segmentation algorithm, and/or, the first blackboard image The threshold segmentation algorithm used in the step of threshold segmentation processing for at least one image block of a blackboard image is a global threshold segmentation algorithm.

In an optional embodiment, the first threshold value range is [a, b], where a=c*d, and c represents the total number of times the pixel of the first blackboard image has been subjected to type division processing, d is expressed as a percentage, d is greater than or equal to 50%; b is expressed as a value greater than or equal to c.

In an optional embodiment, the first processing unit 302 includes:

The first processing module is configured to perform threshold segmentation processing on pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels, and then The pixels of the first type are set as white pixels, and the pixels of the second type are set as black pixels, thereby obtaining a second blackboard image.

In an optional embodiment, the second processing unit 303 includes:

The second processing module is configured to perform at least one image block threshold segmentation process and binarization process on the first blackboard image to obtain at least one third blackboard image.

In an optional embodiment, the third processing unit 304 includes:

The third processing module is configured to perform sum processing on the second blackboard image and the at least one third blackboard image to obtain the fourth blackboard image, and then determine that the pixel value of the fourth blackboard image is in the first Within the second threshold range, the pixel is taken as the target pixel; wherein, the second threshold range is determined according to the first threshold range.

The contents of the above method embodiments are all applicable to this system embodiment, and the specific functions implemented by this system embodiment are the same as the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

As shown in FIG. 5, an embodiment of the present invention also provides an image processing device, which includes:

At least one processor 401;

At least one memory 402, configured to store at least one program;

When the at least one program is executed by the at least one processor 401, the at least one processor 401 implements the steps of the image processing method described in the foregoing method embodiment.

It can be seen that by using the image processing device of the above-mentioned embodiment of the present invention, the blackboard image is first subjected to the threshold segmentation processing of the entire image, and then the pixels of the first blackboard image are divided into the first type of pixels or the second type of pixels, and then After threshold segmentation is performed on the image block of the blackboard image, the pixels of the image block are divided into the first type pixel point or the second type pixel point to achieve the effect of local threshold segmentation of the blackboard image, and then pass the pixel point The number of times that pixels are classified into the first type is thresholded to determine the target pixel. In this way, the determined target pixel is used to extract the corresponding image from the first blackboard image, that is, the target image. Its validity and accuracy High, and it can solve the problem of judging the foreground as the background due to the reflection of the image. In addition, all the content in the above method embodiment is applicable to this device embodiment, so all the functions implemented by this device embodiment are the same as the above method embodiment, and all the beneficial effects achieved are the same as those achieved by the above method embodiment. The beneficial effects are also the same.

In addition, an embodiment of the present invention also provides a storage medium in which instructions executable by a processor are stored, and the instructions executable by the processor are used to execute the one described in the foregoing method embodiment when executed by the processor. This image processing method steps. That is to say, the content in the above method embodiment is applicable to this storage medium embodiment, and the specific functions implemented by this storage medium embodiment are the same as the above method embodiment, and the beneficial effects achieved are the same as those achieved by the above method embodiment. The beneficial effects are also the same.

As shown in FIG. 6, an embodiment of the present invention also provides a blackboard device, including a blackboard 501, a camera 502, and a terminal device 503 connected to the camera 502.

The camera 502 is used to photograph the blackboard 501.

The terminal device 503 includes:

At least one processor;

At least one memory is used to store at least one program.

When the at least one program is executed by the at least one processor, the at least one processor implements the steps of an image processing method described in the foregoing method embodiment.

It can be seen that by using the blackboard device of the above-mentioned embodiment of the present invention, the entire image of the blackboard image is first subjected to threshold segmentation processing, and then the pixels of the first blackboard image are divided into first type pixels or second type pixels, and then After threshold segmentation is performed on the image block of the blackboard image, the pixels of the image block are divided into the first type of pixel or the second type of pixel to achieve the effect of the local threshold segmentation of the blackboard image. The number of times the pixels are divided into the first type is used for threshold judgment to determine the target pixel. In this way, the determined target pixel is used to extract the corresponding image from the first blackboard image, that is, the target image, which is highly effective and accurate , And can solve the problem of judging the foreground as the background due to the reflection of the image.

For the above-mentioned terminal device 503, it is realized by a combination of software and hardware, and it can be a computer, a mobile phone, an interactive smart tablet, a display device with smart processing functions (such as a smart TV, a smart display), and other devices. As for the memory, it may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In addition, the processor and the memory can be connected by a bus, and the processor and the memory can be integrated in the same circuit board or independently arranged in different circuit boards, and the connection between the processor and the memory can be The fixed and non-detachable connection can also be a detachable connection. These methods are not too limited in this embodiment, and can be selected according to actual requirements. In addition, the communication method between the terminal device 503 and the camera 502 can be a wired connection (such as a serial port wired connection, a universal serial bus (Universal Serial Bus, USB) interface wired connection, etc.), or a wireless connection (such as Infrared, Bluetooth, Zigbee, wireless fidelity (Wireless Fidelity, Wifi, etc.), these communication connection modes are not too limited in this embodiment, and can be selected according to actual conditions/requirements.

In an optional embodiment, the camera 502 is directly arranged on the blackboard 501 (for example, above the blackboard 501), so that the writing board can be directly installed during use, which is convenient for operation and use.

The contents in the above method embodiments are all applicable to this device embodiment, and the specific functions implemented by this device embodiment are the same as the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

The above is a specific description of the optional implementations of the application, but the invention is not limited to the embodiments. Those skilled in the art can make various equivalent modifications or substitutions without departing from the spirit of the application. These equivalent modifications or replacements are all included in the scope defined by the claims of this application.

Claims

An image processing method, including:

Acquiring the first blackboard image;

Performing threshold segmentation processing on the pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels;

Perform at least one image block threshold segmentation process on the first blackboard image; wherein the threshold segmentation process of the image block is specifically performing pixel threshold segmentation on at least one image block of the first blackboard image Processing, so that the pixels of the at least one image block are divided into first type pixels or second type pixels;

It is determined that the number of times the pixels of the first blackboard image are classified into the first type of pixels is within the first threshold range, then the pixel is taken as the target pixel;

Using the target pixels, a target image is extracted from the first blackboard image.
The image processing method according to claim 1, wherein the threshold segmentation processing of the image block at least once is specifically: the threshold segmentation processing of the image block at least twice;

In the threshold segmentation processing of the at least two image blocks, the attributes of the image blocks used in the threshold segmentation processing of each image block are different, and the attributes include the number, size, shape, and/or selection position of the image block.
4. The image processing method according to claim 1, wherein the number of the image blocks is at least two, and the at least two image blocks are obtained by evenly dividing the first blackboard image.
An image processing method according to claim 1, wherein the threshold segmentation algorithm used in the step of performing pixel threshold segmentation processing on the first blackboard image is a global threshold segmentation algorithm, and/or, The threshold segmentation algorithm used in the step of performing pixel threshold segmentation processing on at least one image block of the first blackboard image is a global threshold segmentation algorithm.
The image processing method according to any one of claims 1 to 4, wherein the threshold segmentation processing is performed on the pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into the first The step of a type of pixel or a second type of pixel includes:

Perform threshold segmentation processing on the first blackboard image, so that the pixels of the first blackboard image are divided into first type pixels or second type pixels, and then the first type pixels Set as white pixels, and set the second type of pixels as black pixels, thereby obtaining a second blackboard image.
5. The image processing method according to claim 5, wherein the step of performing threshold segmentation of image blocks at least once on the first blackboard image comprises:

After performing at least one image block threshold segmentation process and binarization process on the first blackboard image, at least one third blackboard image is obtained;

Wherein, the threshold segmentation processing and binarization processing of the image block include:

Perform pixel threshold segmentation processing on at least one image block of the first blackboard image, so that the pixels of the at least one image block are divided into the first type of pixel or the second type of pixel, and then the The pixels of the first type are set as white pixels, and the pixels of the second type are set as black pixels, thereby obtaining a third blackboard image.
The image processing method according to claim 6, wherein the number of times that it is determined that the pixel of the first blackboard image is divided into the first type of pixel is within a first threshold range, the pixel is taken as the target The pixel point step includes:

After the second blackboard image and the at least one third blackboard image are summed to obtain the fourth blackboard image, it is determined that the pixel value of the fourth blackboard image is within the second threshold range, then the This pixel is regarded as the target pixel;

Wherein, the second threshold value range is determined according to the first threshold value range.
An image processing system, including:

The first acquiring unit is configured to acquire the first blackboard image;

A first processing unit, configured to perform threshold segmentation processing on pixels of the first blackboard image, so that the pixels of the first blackboard image are divided into first-type pixels or second-type pixels;

The second processing unit is configured to perform threshold segmentation processing of image blocks at least once on the first blackboard image; wherein the threshold segmentation processing of the image blocks is specifically: performing at least one image of the first blackboard image Performing threshold segmentation processing of pixels on the block, so that the pixels of the at least one image block are divided into first type pixels or second type pixels;

The third processing unit is configured to determine that the number of times that the pixels of the first blackboard image are divided into the first type of pixels is within a first threshold range, and then the pixels are used as target pixels;

The fourth processing unit is configured to extract a target image from the first blackboard image by using the target pixel.
An image processing device including:

At least one processor;

At least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the image processing method according to any one of claims 1-7.
A storage medium storing instructions executable by a processor, wherein the instructions executable by the processor are used to execute the image processing method according to any one of claims 1-7 when executed by the processor.
A blackboard device includes a blackboard, a camera, and a terminal device connected with the camera; wherein,

The camera is used to photograph the blackboard;

The terminal equipment includes:

At least one processor;

At least one memory for storing at least one program;

When the at least one program is executed by the at least one processor, the at least one processor implements the image processing method according to any one of claims 1-7.