WO2023019681A1 - 一种图像内容的提取方法、装置、终端和存储介质 - Google Patents

一种图像内容的提取方法、装置、终端和存储介质 Download PDF

Info

Publication number
WO2023019681A1
WO2023019681A1 PCT/CN2021/119724 CN2021119724W WO2023019681A1 WO 2023019681 A1 WO2023019681 A1 WO 2023019681A1 CN 2021119724 W CN2021119724 W CN 2021119724W WO 2023019681 A1 WO2023019681 A1 WO 2023019681A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
pixel
processed
linear light
processing
Prior art date
Application number
PCT/CN2021/119724
Other languages
English (en)
French (fr)
Inventor
李�浩
Original Assignee
广东艾檬电子科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广东艾檬电子科技有限公司 filed Critical 广东艾檬电子科技有限公司
Priority to EP21953917.8A priority Critical patent/EP4345741A1/en
Publication of WO2023019681A1 publication Critical patent/WO2023019681A1/zh
Priority to US18/397,979 priority patent/US20240127404A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/20—Image enhancement or restoration using local operators
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/70—Denoising; Smoothing
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/73—Deblurring; Sharpening
    • G06T5/75—Unsharp masking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G06T5/92—Dynamic range modification of images or parts thereof based on global image properties
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/60—Extraction of image or video features relating to illumination properties, e.g. using a reflectance or lighting model
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10004—Still image; Photographic image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30176—Document

Definitions

  • the present application belongs to the field of image processing, and in particular relates to an image content extraction method, device, terminal and storage medium.
  • the handwriting board can record the characters, graffiti, etc. left by the students when they calculate and paint on the handwriting board.
  • the tablet or the device connected to the tablet can use the image of the tablet to extract the image content for data analysis, so as to achieve the effect of assisting students in learning and education.
  • threshold segmentation In practical applications, when students calculate and paint on the handwriting board, the strength is not uniform, and the pixel values of the pixels corresponding to the characters, graffiti and other image content in the image do not all fall within a certain threshold range. Therefore, the use of threshold segmentation In the solution, the extraction of image content is often incomplete and the image content is broken.
  • Embodiments of the present application provide an image content extraction method, device, terminal, and storage medium, which can improve the integrity of image content extraction.
  • the first aspect of the embodiment of the present application provides a method for extracting image content, including:
  • An image acquisition unit configured to acquire an image to be processed
  • a high-contrast preserving unit configured to perform high-contrast preserving processing on the image to be processed to obtain a high-contrast image of the image to be processed;
  • an image fusion unit configured to perform image fusion on the image to be processed and the high-contrast image to obtain a fusion image
  • a linear light enhancement unit configured to perform linear light enhancement processing on the fused image to obtain a linear light enhancement image
  • An image content extraction unit configured to use the first pixel in the linear light-enhanced image whose pixel value is within a preset pixel value range as the image content of the image to be processed.
  • the third aspect of the embodiments of the present application provides a terminal, including a memory, a processor, and a computer program stored in the memory and operable on the processor, and the above method is implemented when the processor executes the computer program A step of.
  • a fourth aspect of the embodiments of the present application provides a computer-readable storage medium, where the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the foregoing method are implemented.
  • the fifth aspect of the embodiments of the present application provides a computer program product, which, when the computer program product runs on a terminal, enables the terminal to execute the steps of the method.
  • a high-contrast image of the image to be processed is obtained, and then performing image fusion on the image to be processed and the high-contrast image to obtain a fused image, Then perform linear light enhancement processing on the fused image to obtain a linear light enhanced image, which can enhance the pixel value difference between the pixel area corresponding to the image content in the image to be processed and the pixel area corresponding to the non-image content, and strengthen the information display of the image content in the image.
  • the integrity of the image content extraction is higher.
  • FIG. 1 is a schematic diagram of an implementation flow of a method for extracting image content provided by an embodiment of the present application
  • Fig. 2 is a schematic diagram of an original image provided by an embodiment of the present application.
  • Fig. 3 is a schematic diagram of a first inversion diagram provided by an embodiment of the present application.
  • FIG. 4 is a schematic diagram of a specific implementation process for sharpening the first inverse image provided by the embodiment of the present application
  • FIG. 5 is a schematic diagram of a second sharpened image provided by an embodiment of the present application.
  • FIG. 6 is a schematic diagram of a high-contrast image provided by an embodiment of the present application.
  • FIG. 7 is a schematic diagram of a fused image provided by an embodiment of the present application.
  • Fig. 8 is a schematic diagram of an image content provided by an embodiment of the present application.
  • FIG. 9 is a schematic diagram of a target image provided by an embodiment of the present application.
  • FIG. 10 is a schematic structural diagram of an image content extraction device provided in an embodiment of the present application.
  • FIG. 11 is a schematic structural diagram of a terminal provided by an embodiment of the present application.
  • the handwriting board can record the characters, graffiti, etc. left by the students when they calculate and paint on the handwriting board.
  • the tablet or the device connected to the tablet can use the image of the tablet to extract the image content for data analysis, so as to achieve the effect of assisting students in learning and education.
  • threshold segmentation In practical applications, when students calculate and paint on the handwriting board, the strength is not uniform, and the pixel values of the pixels corresponding to the characters, graffiti and other image content in the image do not all fall within a certain threshold range. Therefore, the use of threshold segmentation In the solution, the extraction of image content is often incomplete and the image content is broken.
  • Fig. 1 shows a schematic diagram of the implementation flow of a method for extracting image content provided by an embodiment of the present application.
  • the method can be applied to a terminal and is applicable to situations where the integrity of image content extraction needs to be improved.
  • the above-mentioned terminal may be a handwriting tablet with a certain processing capability, a terminal device connected to the handwriting tablet, or other terminal equipment that needs to extract characters in the image.
  • the above method for extracting image content may include the following steps S101 to S105.
  • Step S101 acquiring an image to be processed.
  • the image to be processed refers to an image containing image content such as characters and graffiti that need to be extracted.
  • image content such as characters and graffiti that need to be extracted.
  • different character images to be processed may be selected according to actual conditions.
  • the above-mentioned terminal can obtain the original image obtained by shooting the handwriting tablet; perform inversion processing on the original image to obtain a first inversion image of the original image; and sharpen the first inversion image processing, and use the second sharpened image obtained after the sharpening process as the image to be processed.
  • the aforementioned inversion processing of the original image may include: subtracting the pixel value of each pixel in the original image from the set pixel value to obtain a first inversion image of the original image.
  • the pixel value set above can be 255, that is, for each pixel in the original image, subtract the original pixel value of the pixel in the original image from 255 to obtain the pixel in the first reflection Pixel values in the phase map.
  • FIG. 2 shows an original image obtained by capturing a tablet in the present application, and the first inverse image corresponding to the original image is shown in FIG. 3 .
  • the first inversion image can be sharpened.
  • the above-mentioned sharpening process on the first inversion image may specifically include the following steps S401 to S402.
  • Step S401 performing Gaussian filtering on the first inverse image to obtain a Gaussian blurred image of the first inverse image.
  • the above-mentioned Gaussian filter processing is an image blurring algorithm, which can make each pixel in the output Gaussian blurred image be the corresponding pixel on the original image and the surrounding pixels based on the Gaussian template (convolution kernel). weighted sum of .
  • the present application does not limit the specific manner of Gaussian filtering, and in some implementation manners of the present application, bilateral filtering may be used. More specifically, it can be realized by the bilateral filter function provided by OpenCV, wherein the parameter kernel size of the bilateral filter function can be set to 11, that is, the size of the Gaussian template (convolution kernel) can be 11; the parameter sigmaColor and parameter of the bilateral filter function sigmaSpace can be set to 0, that is, the standard deviation of the color space and the standard deviation of the coordinate space are both 0.
  • Step S402 weighting the pixel values of each second pixel in the first inversion image and the pixel values of the pixels at the same position as each second pixel in the Gaussian blurred image of the first inversion image to obtain the second Sharpen the image.
  • the product of the pixel value of each second pixel point in the first inversion image and the first weight value may be calculated, and the sum of the values of the Gaussian blurred image of the inversion image and each second pixel point in the inversion image may be calculated.
  • the product of the pixel value of the pixel at the same position of the pixel and the second weight value, and then the product of the pixel value of each second pixel in the first inversion image and the first weight value and the Gaussian blur of the first inversion image The second sharpened image is obtained by adding the product of the pixel value of the pixel at the same position as each second pixel in the image and the second weight value.
  • the pixel value of a certain pixel in the second sharpened image is the product of the pixel value of the pixel at the same position in the first inversion image and the first weight value, plus the pixel at the same position in the Gaussian blurred image The sum of the product of the point's pixel value and the second weight value.
  • the specific values of the above-mentioned first weight value and the second weight value can be adjusted according to actual conditions.
  • the above-mentioned first weight value may be 2, and the second weight value may be -1.
  • FIG. 5 shows a second sharpened image obtained after sharpening the first inverse image shown in FIG. 3 .
  • the above-mentioned terminal may also directly acquire the original image obtained by shooting the handwriting tablet, perform sharpening processing on the original image, and use the first sharpened image obtained after the sharpening processing as the to-be-processed image. image.
  • the above-mentioned terminal can select the sharpened image processed by one of the aforementioned two methods as the character to be processed based on the material of the handwriting board and the color of the characters on the handwriting board image.
  • the camera may capture the background other than the tablet into the original image when shooting the tablet, that is, only part of the pixel area in the original image is the pixel area corresponding to the tablet. Therefore, in order to facilitate subsequent processing and avoid noise in the extracted characters, in some implementations of the present application, after acquiring the original image captured by the handwriting tablet, image recognition can be performed on the original image to obtain the original The pixel area corresponding to the handwriting board in the image, and then perform perspective change processing on the pixel area corresponding to the handwriting board to obtain the handwriting board image.
  • the above-mentioned terminal can also obtain the original image obtained by taking pictures of the handwriting board; perform image recognition on the original image to obtain the pixel area corresponding to the handwriting board in the original image; The perspective change processing is performed on the corresponding pixel area to obtain the handwriting board image; the handwriting board image is sharpened, and the third sharpened image obtained after the sharpening processing is used as the image to be processed.
  • the present application does not limit the specific manner of image recognition.
  • the above-mentioned terminal may input the original image into a pre-trained handwriting board recognition model, and the handwriting board recognition model outputs the pixel area corresponding to the handwriting board in the original image.
  • the above-mentioned tablet recognition model may be a YOLO model.
  • the training process of the handwriting tablet recognition model may include: acquiring a sample image set, which includes a plurality of sample original images obtained by shooting the handwriting tablet.
  • the staff can pre-mark the pixel area corresponding to the handwriting board in each original original image.
  • select the target training samples from the sample image set input each target training sample into the network model to be trained, obtain the pixel area corresponding to the handwriting board output by the network model to be trained, and based on the pre-marked handwriting
  • the pixel area corresponding to the board, and the accuracy rate of the network model to be trained is counted.
  • the accuracy rate of the network model to be trained is less than the accuracy rate threshold, adjust the parameters in the network model to be trained, and re-execute the step of selecting the target training sample from the sample image set and subsequent steps until the accuracy of the network model to be trained
  • the accuracy rate is greater than or equal to the accuracy threshold, and the tablet recognition model is obtained.
  • the above-mentioned terminal can obtain the pixel area corresponding to the handwriting board in the original image by using the trained handwriting board recognition model.
  • the terminal may perform perspective change processing on the pixel area corresponding to the tablet to obtain the image of the tablet.
  • the perspective change processing refers to projecting the pixel area corresponding to the tablet into a new viewing plane to obtain the image of the tablet, which can realize the correction of the pixel area corresponding to the tablet in the original image and avoid the problem of the tablet area in the original image.
  • the interference of external information on the image content extraction unit is not limited to.
  • Step S102 performing high-contrast preserving processing on the image to be processed to obtain a high-contrast image of the image to be processed.
  • the terminal since there is a certain difference in pixel value between the pixel area corresponding to the character in the character image to be processed and the corresponding pixel area other than the character, the terminal can highlight the character in the character image to be processed through high-contrast preservation processing. boundary information.
  • the above-mentioned high-contrast preserving processing of the character image to be processed may specifically include: performing Gaussian filter processing on the character image to be processed to obtain a Gaussian blurred image of the character image to be processed, and then, a certain character image to be processed The pixel at the position, based on the pixel value of the pixel, the pixel value of the pixel at the same position as the pixel in the Gaussian blur image, and the set pixel value, determine the pixel at the same position as the pixel in the high-contrast image The pixel value of the point.
  • the set pixel value can be adjusted according to the actual situation, and in some embodiments of the present application, the set pixel value can be 127.
  • the pixel value of a certain pixel in the high-contrast image is equal to the pixel value of the pixel at the same position in the character image to be processed minus the pixel value of the pixel at the same position in the Gaussian blurred image, and then the obtained difference is added to The value obtained by setting the pixel value.
  • FIG. 6 shows a high-contrast image obtained after performing high-contrast preserving processing on the image to be processed shown in FIG. 5 .
  • the Gaussian blurred image obtained by smoothing the image to be processed can be mixed with the high-frequency image to be processed through high-contrast preservation processing, and the edge information in the image to be processed can be preserved, that is, the image to be processed can be highlighted. Boundary information of characters in a character image.
  • Step S103 image fusion is performed on the image to be processed and the high-contrast image to obtain a fusion image.
  • the above-mentioned image fusion refers to adding the pixel value of a certain pixel in the character image to be processed to the pixel value of the pixel at the same position as the pixel in the high-contrast image to obtain The pixel value of the pixel at the same position as the point.
  • Step S104 performing linear light enhancement processing on the fused image to obtain a linear light enhanced image.
  • the above-mentioned linear light enhancement processing specifically includes: normalizing each third pixel point in the fused image to obtain the first normalized pixel value corresponding to each third pixel point;
  • the linear light value of the four pixels add the first normalized pixel value of each fourth pixel in the fused image to the corresponding linear light value to obtain a linear light enhanced image.
  • each of the fourth pixel points is a third pixel point of each third pixel point whose first normalized pixel value is within a preset value range.
  • the linear light value of each fourth pixel in the above-mentioned fourth pixels may be based on its first normalized pixel value and the pixel at the same position in the high-contrast image The second normalized pixel value computes the resulting linear light value. This linear light value can be used to enhance the contrast of the fused image.
  • the above terminal may normalize the pixel values of each third pixel point, for example, may divide the pixel value of each third pixel point by 255 to obtain the first normalized pixel value corresponding to each third pixel point , the first normalized pixel values are all within [0, 1].
  • the terminal can also normalize the high-contrast image to obtain the second normalized pixel value of each pixel in the high-contrast image, and based on the first
  • the first normalized pixel value corresponding to the four pixels and the second normalized pixel value corresponding to the pixel at the same position in the high-contrast image are used to calculate the linear light value corresponding to the fourth pixel.
  • the preset numerical range may be selected according to actual conditions. In some embodiments of the present application, the preset numerical range may be a value other than 0.5.
  • its corresponding linear light value is equal to its first normalized pixel value, minus the pixel at the same position in the high-contrast image The product of the corresponding second normalized pixel value and 2, and then subtract 1 from the obtained difference. It can be understood that the obtained linear light value is also a normalized value.
  • the first normalized pixel value of each fourth pixel in the fused image can be added to the corresponding linear light value to obtain the corresponding The third normalized pixel value, and then numerically verifying the third normalized pixel value corresponding to each of the fourth pixel points.
  • the verification result of numerical verification is that the third normalized pixel value corresponding to a certain fourth pixel is greater than 1, set the first pixel value of the first pixel to 1; if the verification of numerical verification If the result is that the third normalized pixel value corresponding to a fourth pixel is less than 0, set the first pixel value of the first pixel to 0; if the verification result of the numerical verification is a fourth pixel If the third normalized pixel value corresponding to the point is greater than or equal to 0 and less than or equal to 1, then the corresponding third normalized pixel value is retained. Therefore, according to the verification result of the value verification, the above-mentioned terminal can determine the fourth normalized pixel values corresponding to the fourth pixel points to obtain the linear light-enhanced image.
  • FIG. 7 shows a linear light-enhanced image obtained based on the fused image obtained by performing image fusion on the image to be processed shown in FIG. 5 and the high-contrast image shown in FIG. 6 .
  • the difference in pixel values between the pixel area corresponding to the character and other pixel points can be enlarged, so that the character corresponding to the character image in the character image to be processed Pixel areas are enhanced.
  • Step S105 taking the first pixel in the linear light-enhanced image whose pixel value is within the preset pixel value range as the image content of the image to be processed.
  • the preset pixel value range refers to the range where the pixel values of the pixel points corresponding to the image content are located.
  • the pixel value of a first pixel in the linear light enhanced image is within the preset pixel value range, it means that the first pixel belongs to the pixel corresponding to the image content, and if the linear light If the pixel value of a first pixel in the enhanced image is outside the preset pixel value range, it means that the first pixel does not belong to the pixel corresponding to the image content.
  • the above-mentioned terminal can traverse each first pixel point, and if the pixel value of a certain first pixel point is outside the preset pixel value range, the pixel value of the first pixel point Set it to 0 to remove the pixel, if the pixel value of a first pixel is within the preset pixel value range, then keep the pixel value of the first pixel. After traversing all the first pixels in the linear light-enhanced image, the first pixels within the preset pixel value range can be used as the image content of the image to be processed.
  • FIG. 8 shows the image content after processing from the linear light enhanced image shown in FIG. 7 .
  • the above preset pixel value range can be adjusted according to the actual situation.
  • the preset pixel value range may be determined based on a large number of histograms, and each histogram counts the pixel values of the first pixel points in different linear light-enhanced images.
  • the preset pixel value range may be set to be greater than or equal to 230, that is, the normalized pixel value corresponding to the preset pixel value range is greater than or equal to 0.9.
  • a high-contrast image of the image to be processed is obtained, and then performing image fusion on the image to be processed and the high-contrast image to obtain a fused image, Then perform linear light enhancement processing on the fused image to obtain a linear light enhanced image, which can enhance the pixel value difference between the pixel area corresponding to the image content in the image to be processed and the pixel area corresponding to the non-image content, and strengthen the information display of the image content in the image.
  • the integrity of the image content extraction is higher.
  • the image content extraction method provided by the present application can achieve the effect of real-time and batch processing.
  • the method for extracting the above image content may further include: superimposing the image content of the character image to be processed on the target background to obtain the target image, and displaying the target image.
  • the target background may be set according to user requirements.
  • the target background may be a background picture with a pixel value of 255.
  • FIG. 9 shows a target image obtained by superimposing the content of the image shown in FIG. 8 onto a background picture with a pixel value of 255.
  • the terminal can obtain the target image by superimposing the image content of the image to be processed on the target background.
  • the color difference between the target background and the image content is relatively large. Therefore, the image content in the obtained target image will be further highlighted.
  • the terminal can enable the user to better watch the image content.
  • FIG. 10 is a schematic structural diagram of an image content extraction device 1000 provided in an embodiment of the present application, and the image content extraction device 1000 is configured on a terminal.
  • the image content extraction device 1000 may include:
  • An image acquisition unit 1001 configured to acquire an image to be processed
  • a high-contrast preserving unit 1002 configured to perform high-contrast preserving processing on the image to be processed to obtain a high-contrast image of the image to be processed;
  • An image fusion unit 1003, configured to perform image fusion on the image to be processed and the high-contrast image to obtain a fusion image
  • a linear light enhancement unit 1004 configured to perform linear light enhancement processing on the fused image to obtain a linear light enhanced image
  • the image content extraction unit 1005 is configured to use the first pixel in the linear light-enhanced image whose pixel value is within a preset pixel value range as the image content of the image to be processed.
  • the above-mentioned image acquiring unit 1001 may also be specifically configured to: acquire the original image obtained by shooting the handwriting tablet, perform sharpening processing on the original image, and sharpen the A sharpened image is used as the image to be processed.
  • the above-mentioned image acquisition unit 1001 may also be specifically configured to: acquire the original image obtained by shooting the handwriting tablet; perform inversion processing on the original image to obtain the first inverse image of the original image Phase image: performing sharpening processing on the first inverse phase image, and using a second sharpened image obtained after the sharpening processing as the image to be processed.
  • the above-mentioned image acquisition unit 1001 may also be specifically configured to: perform Gaussian filtering on the first inverse image to obtain a Gaussian blurred image of the first inverse image; Perform weighting processing on the pixel values of the second pixels in the first inversion image and the pixel values of the pixels at the same position as the second pixels in the Gaussian blur image of the first inversion image to obtain the first 2. Sharpen the image.
  • the above-mentioned image acquisition unit 1001 may also be specifically configured to: acquire the original image obtained by photographing the handwriting tablet; perform image recognition on the original image to obtain the handwriting tablet in the original image Corresponding pixel area; Perspective change processing is performed on the pixel area corresponding to the handwriting board to obtain a handwriting board image; Sharpening is performed on the handwriting board image, and the third sharpened image obtained after the sharpening process is used as the obtained Describe the image to be processed.
  • the above-mentioned image acquisition unit 1001 may also be specifically configured to: acquire the original image obtained by photographing the handwriting tablet; perform image recognition on the original image to obtain the handwriting tablet in the original image Corresponding pixel area; Perspective change processing is performed on the pixel area corresponding to the tablet to obtain the image of the tablet; Phase inversion processing is performed on the image of the tablet to obtain a second inversion image of the image of the tablet; The second inverse image is sharpened, and the fourth sharpened image obtained after sharpening is used as the image to be processed.
  • the linear light value of each fourth pixel in the above-mentioned fourth pixels is based on the first normalized pixel value and the same position in the high-contrast image The linear light value obtained by calculating the second normalized pixel value of the pixel.
  • the above-mentioned linear light enhancement unit 1004 may also be specifically configured to: add the first normalized pixel values of the fourth pixel points in the fused image to the corresponding linear light values respectively , to obtain the third normalized pixel values respectively corresponding to the respective fourth pixel points; perform numerical verification on the third normalized pixel values corresponding to the respective fourth pixel points, and verify according to the numerical values Determine the fourth normalized pixel values corresponding to the respective fourth pixel points according to the verification results, and obtain the linear light-enhanced image.
  • FIG. 11 it is a schematic diagram of a terminal provided in the embodiment of the present application.
  • the terminal may be a handwriting tablet with a certain processing capability, a terminal device connected to the handwriting tablet, or other terminal equipment that needs to extract characters in the image.
  • the terminal 11 may include: a processor 110, a memory 111, and a computer program 112 stored in the memory 111 and operable on the processor 110, such as an image content extraction program.
  • the processor 110 executes the computer program 112 , the steps in the above-mentioned embodiments of the method for extracting image content are implemented, such as steps S101 to S105 shown in FIG. 1 .
  • the processor 110 executes the computer program 112
  • it realizes the functions of the modules/units in the above-mentioned device embodiments, such as the image acquisition unit 1001, the high-contrast preservation unit 1002, the image fusion unit 1003 shown in FIG. 10 , A linear light enhancement unit 1004 and an image content extraction unit 1005 .
  • the computer program can be divided into one or more modules/units, and the one or more modules/units are stored in the memory 111 and executed by the processor 110 to complete the present application.
  • the one or more modules/units may be a series of computer program instruction segments capable of accomplishing specific functions, and the instruction segments are used to describe the execution process of the computer program in the terminal.
  • the computer program can be divided into: an image acquisition unit, a high contrast preservation unit, an image fusion unit, a linear light enhancement unit and an image content extraction unit.
  • each unit is as follows: the image acquisition unit is used to acquire the image to be processed; the high-contrast preservation unit is used to perform high-contrast preservation processing on the image to be processed to obtain a high-contrast image of the image to be processed; the image fusion unit , for image fusion of the image to be processed and the high-contrast image to obtain a fused image; a linear light enhancement unit for performing linear light enhancement processing on the fused image to obtain a linear light enhanced image; image content extraction A unit, configured to use the first pixel in the linear light-enhanced image whose pixel value is within a preset pixel value range as the image content of the image to be processed.
  • the terminal may include, but not limited to, a processor 110 and a memory 111 .
  • FIG. 11 is only an example of a terminal, and does not constitute a limitation to the terminal. It may include more or less components than those shown in the figure, or combine certain components, or different components, such as the Terminals may also include input and output devices, network access devices, buses, and so on.
  • the so-called processor 110 may be a central processing unit (Central Processing Unit, CPU), and may also be other general-purpose processors, digital signal processors (Digital Signal Processor, DSP), application specific integrated circuits (Application Specific Integrated Circuit, ASIC), Off-the-shelf programmable gate array (Field-Programmable Gate Array, FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.
  • a general-purpose processor may be a microprocessor, or the processor may be any conventional processor, or the like.
  • the storage 111 may be an internal storage unit of the terminal, such as a hard disk or memory of the terminal.
  • the memory 111 can also be an external storage device of the terminal, such as a plug-in hard disk equipped on the terminal, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash memory card (Flash Card) etc.
  • the memory 111 may also include both an internal storage unit of the terminal and an external storage device.
  • the memory 111 is used to store the computer program and other programs and data required by the terminal.
  • the memory 111 can also be used to temporarily store data that has been output or will be output.
  • the disclosed device/terminal and method may be implemented in other ways.
  • the device/terminal embodiments described above are only illustrative.
  • the division of the modules or units is only a logical function division.
  • the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be in electrical, mechanical or other forms.
  • the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.
  • each functional unit in each embodiment of the present application may be integrated into one processing unit, each unit may exist separately physically, or two or more units may be integrated into one unit.
  • the above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.
  • the integrated module/unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, all or part of the processes in the methods of the above embodiments in the present application can also be completed by instructing related hardware through computer programs.
  • the computer programs can be stored in a computer-readable storage medium, and the computer When the program is executed by the processor, the steps in the above-mentioned various method embodiments can be realized.
  • the computer program includes computer program code, and the computer program code may be in the form of source code, object code, executable file or some intermediate form.
  • the computer-readable medium may include: any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a removable hard disk, a magnetic disk, an optical disk, a computer memory, and a read-only memory (Read-Only Memory, ROM) , random access memory (Random Access Memory, RAM), electric carrier signal, telecommunication signal and software distribution medium, etc.
  • ROM Read-Only Memory
  • RAM Random Access Memory
  • electric carrier signal telecommunication signal and software distribution medium, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Quality & Reliability (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)

Abstract

一种图像内容的提取方法、装置（1000）、终端（11）和存储介质。图像内容的提取方法具体包括：获取待处理图像（S101）；对待处理图像进行高反差保留处理，得到待处理图像的高反差图像（S102）；对待处理图像和高反差图像进行图像融合，得到融合图像（S103）；对融合图像进行线性光增强处理，得到线性光增强图像（S104）；将线性光增强图像中像素值在预设像素值范围内的第一像素点作为待处理图像的图像内容（S105），能够提高图像内容提取的完整性。

Description

一种图像内容的提取方法、装置、终端和存储介质 技术领域
本申请属于图像处理领域，尤其涉及一种图像内容的提取方法、装置、终端和存储介质。
背景技术
当前用于辅导学生学习的硬件产品越来越丰富，手写板就是其中一种。手写板能够记录下学生在手写板上演算、涂画时留下的字符、涂鸦等内容。手写板或与手写板连接的设备可以利用手写板的图像将图像内容提取出来用于数据分析，以达到辅助学生学习教育的效果。
目前，常见的图像内容提取方案一般是采用阈值分割的方式分割出图像中的图像内容。实际应用中，学生在手写板上演算、涂画时力度并不均匀，图像中字符、涂鸦等图像内容对应的像素点的像素值并不都落在某一个阈值范围内，因此，利用阈值分割方案的往往会出现图像内容提取不完整、图像内容断裂的情况。
发明内容
本申请实施例提供一种图像内容的提取方法、装置、终端和存储介质，可以提高图像内容提取的完整性。
本申请实施例第一方面提供一种图像内容的提取方法，包括：
获取待处理图像；
对所述待处理图像进行高反差保留处理，得到所述待处理图像的高反差图像；
对所述待处理图像和所述高反差图像进行图像融合，得到融合图像；
对所述融合图像进行线性光增强处理，得到线性光增强图像；
将所述线性光增强图像中像素值在预设像素值范围内的第一像素点作为所述待处理图像的图像内容。
本申请实施例第二方面提供的一种图像内容的提取装置，包括：
图像获取单元，用于获取待处理图像；
高反差保留单元，用于对所述待处理图像进行高反差保留处理，得到所述待处理图像的高反差图像；
图像融合单元，用于对所述待处理图像和所述高反差图像进行图像融合，得到融合图像；
线性光增强单元，用于对所述融合图像进行线性光增强处理，得到线性光增强图像；
图像内容提取单元，用于将所述线性光增强图像中像素值在预设像素值范围内的第一像素点作为所述待处理图像的图像内容。
本申请实施例第三方面提供一种终端，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现上述方法的步骤。
本申请实施例第四方面提供一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，所述计算机程序被处理器执行时实现上述方法的步骤。
本申请实施例第五方面提供了一种计算机程序产品，当计算机程序产品在终端上运行时，使得终端执行时实现方法的步骤。
在本申请的实施方式中，通过获取待处理图像，并对待处理图像进行高反差保留处理，得到待处理图像的高反差图像，然后，对待处理图像和高反差图像进行图像融合，得到融合图像，再对融合图像进行线性光增强处理，得到线性光增强图像，能够增强待处理图像中图像内容对应的像素区域和非图像内容 对应的像素区域的像素值差异，加强图像中图像内容的信息显示，使得将线性光增强图像中像素值在预设像素值范围内的第一像素点作为待处理图像的图像内容后，图像内容提取的完整性更高。
附图说明
为了更清楚地说明本申请实施例中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1是本申请实施例提供的一种图像内容的提取方法的实现流程示意图；
图2是本申请实施例提供的一种原始图像的示意图；
图3是本申请实施例提供的一种第一反相图的示意图；
图4是本申请实施例提供的一种对第一反相图进行锐化处理的具体实现流程示意图；
图5是本申请实施例提供的一种第二锐化图像的示意图；
图6是本申请实施例提供的一种高反差图像的示意图；
图7是本申请实施例提供的一种融合图像的示意图；
图8是本申请实施例提供的一种图像内容的示意图；
图9是本申请实施例提供的一种目标图像的示意图；
图10是本申请实施例提供的一种图像内容的提取装置的结构示意图；
图11是本申请实施例提供的终端的结构示意图。
具体实施方式
为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。基于本申请的实施例，本领域技术 人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本申请保护。
当前用于辅导学生学习的硬件产品越来越丰富，手写板就是其中一种。手写板能够记录下学生在手写板上演算、涂画时留下的字符、涂鸦等内容。手写板或与手写板连接的设备可以利用手写板的图像将图像内容提取出来用于数据分析，以达到辅助学生学习教育的效果。
目前，常见的图像内容提取方案一般是采用阈值分割的方式分割出图像中的图像内容。实际应用中，学生在手写板上演算、涂画时力度并不均匀，图像中字符、涂鸦等图像内容对应的像素点的像素值并不都落在某一个阈值范围内，因此，利用阈值分割方案的往往会出现图像内容提取不完整、图像内容断裂的情况。
为了说明本申请的技术方案，下面通过具体实施例来进行说明。
图1示出了本申请实施例提供的一种图像内容的提取方法的实现流程示意图，该方法可以应用于终端，可适用于需提高图像内容提取的完整性的情形。
其中，上述终端可以为具有一定处理能力的手写板、与手写板连接的终端设备，或者其他需提取图像中字符的终端设备。
具体的，上述图像内容的提取方法可以包括以下步骤S101至步骤S105。
步骤S101，获取待处理图像。
其中，待处理图像是指包含有需要提取的字符、涂鸦等图像内容的图像。在本申请的实施方式中，根据实际情况可以选择不同的待处理字符图像。
在本申请的一些实施方式中，上述终端可以获取对手写板进行拍摄得到的原始图像；对原始图像进行反相处理，得到原始图像的第一反相图；对第一反相图进行锐化处理，并将锐化处理后得到的第二锐化图像作为待处理图像。
具体的，上述对原始图像进行反相处理可以包括：利用设定好的像素值减去原始图像中各个像素点的像素值，得到原始图像的第一反相图。其中，上述设定好的像素值可以为255，也即，对原始图像中的每个像素，用255减去该 像素点的在原始图像中的原像素值，得到该像素点在第一反相图中的像素值。
如图2示出了本申请一张采集手写板得到的原始图像，该原始图像对应的第一反相图如图3所示。
为了使第一反相图中模糊的字符边缘更加清晰，在得到第一反相图之后，可以对第一反相图进行锐化处理。具体的，如图4所示，上述对第一反相图进行锐化处理可以具体包括以下步骤S401至步骤S402。
步骤S401，对第一反相图进行高斯滤波处理，得到第一反相图的高斯模糊图像。
具体的，上述高斯滤波处理是一种图像模糊处理(bluring)算法，能够基于高斯模板(卷积核)，使输出的高斯模糊图像中每个像素点为原图像上对应像素点与周围像素点的加权和。
本申请不对高斯滤波的具体方式进行限制，在本申请的一些实施方式中可以采用双边滤波的方式实现。更具体的，可以通过OpenCV提供的双边滤波函数实现，其中，双边滤波函数的参数kernel size可以设置为11，即高斯模板(卷积核)的大小可以为11；双边滤波函数的参数sigmaColor和参数sigmaSpace可以设置为0，即色彩空间的标准方差及坐标空间的标准方差均为0。
步骤S402，将第一反相图中各个第二像素点的像素值和第一反相图的高斯模糊图像中与各个第二像素点相同位置的像素点的像素值进行加权处理，得到第二锐化图像。
具体的，在本申请的一些实施方式中，可以计算第一反相图中各个第二像素点的像素值与第一权重值的积，并计算反相图的高斯模糊图像中与各个第二像素点相同位置的像素点的像素值与第二权重值的积，然后将第一反相图中各个第二像素点的像素值与第一权重值的积和第一反相图的高斯模糊图像中与各个第二像素点相同位置的像素点的像素值与第二权重值的积相加，得到第二锐化图像。
也就是说，第二锐化图像中某个像素点的像素值为第一反相图中同一位置 的像素点的像素值与第一权重值的积，加上高斯模糊图像中同一位置的像素点的像素值与第二权重值的积的和。
其中，上述第一权重值和第二权重值的具体取值可以根据实际情况进行调整。在本申请的一些实施方式中，上述第一权重值可以为2，第二权重值可以为-1。
图5示出了对图3所示第一反相图进行锐化处理后得到的第二锐化图像。
在本申请的另一些实施方式中，上述终端还可以直接获取对手写板进行拍摄得到的原始图像，对原始图像进行锐化处理，并将锐化处理后得到的第一锐化图像作为待处理图像。
其中，对原始图像进行锐化处理的过程可以参看前述对第一反相图进行锐化处理的过程的描述，本申请对此不进行赘述。
需要说明的是，在本申请的实施方式中，上述终端可以基于手写板的材料、手写板上字符的颜色，在前述两种方式中选择其中一种方式处理得到的锐化图像作为待处理字符图像。
实际应用中，由于相机在拍摄手写板时可能将手写板以外的背景拍入到原始图像中，也即原始图像中仅有部分像素区域为手写板对应的像素区域。因此，为了方便后续的处理，同时也避免提取出的字符存在噪声，在本申请的一些实施方式中，在获取对手写板进行拍摄得到的原始图像之后，可以对原始图像进行图像识别，得到原始图像中手写板对应的像素区域，然后，对手写板对应的像素区域进行透视变化处理，得到手写板图像。
也就是说，在本申请的另一些实施方式中，上述终端还可以获取对手写板进行拍摄得到的原始图像；对原始图像进行图像识别，得到原始图像中手写板对应的像素区域；对手写板对应的像素区域进行透视变化处理，得到手写板图像；对手写板图像进行锐化处理，并将锐化处理后得到的第三锐化图像作为待处理图像。或者，获取对手写板进行拍摄得到的原始图像；对原始图像进行图像识别，得到原始图像中所述手写板对应的像素区域；对手写板对应的像素区 域进行透视变化处理，得到手写板图像；对手写板图像进行反相处理，得到手写板图像的第二反相图；对第二反相图进行锐化处理，并将锐化处理后得到的第四锐化图像作为待处理图像。
具体的，本申请不对图像识别的具体方式进行限制。在本申请的一些实施方式中，上述终端可以将原始图像输入到预先训练好的手写板识别模型，由手写板识别模型输出原始图像中手写板对应的像素区域。
其中，上述手写板识别模型可以为YOLO模型。
具体的，在本申请的一些实施方式中，手写板识别模型的训练过程可以包括：获取样本图像集，在样本图像集中包括对手写板进行拍摄得到的多张样本原始图像。同时，工作人员可以预先对每张原本原始图像中手写板对应的像素区域进行标记。然后，从样本图像集中选取目标训练样本，分别将每个目标训练样本输入到待训练的网络模型中，获取由待训练的网络模型输出的手写板对应的像素区域，并基于预先标记好的手写板对应的像素区域，统计待训练的网络模型的准确率。若待训练的网络模型的准确率小于准确率阈值，则调整待训练的网络模型中的参数，并重新执行从样本图像集中选取目标训练样本的步骤以及后续步骤，直至待训练的网络模型的准确率大于或等于准确率阈值，得到手写板识别模型。
此时，上述终端利用训练好的手写板识别模型，即可获取到原始图像中手写板对应的像素区域。
在获取到原始图像中手写板对应的像素区域之后，在本申请的一些实施方式中，上述终端可以对手写板对应的像素区域进行透视变化处理，得到手写板图像。其中，透视变化处理是指将手写板对应的像素区域投影到一个新的视平面中，得到手写板图像，能够实现对原始图像中手写板对应的像素区域的校正，避免原始图像中手写板区域外的信息对图像内容提取单元的干扰。
步骤S102，对待处理图像进行高反差保留处理，得到待处理图像的高反差图像。
在本申请的实施方式中，由于待处理字符图像中字符对应的像素区域和字符以外对应的像素区域在像素值上具有一定的差异，终端通过高反差保留处理可以突出待处理字符图像中字符的边界信息。
在本申请的一些实施方式中，上述对待处理字符图像进行高反差保留处理可以具体包括：对待处理字符图像进行高斯滤波处理，得到待处理字符图像的高斯模糊图像，然后，对待处理字符图像某个位置的像素点，基于该像素点的像素值、高斯模糊图像中与该像素点相同位置像素点的像素值，以及设定好的像素值，确定在高反差图像中与该像素点相同位置像素点的像素值。其中，设定好的像素值可以根据实际情况进行调整，在本申请的一些实施方式中，该设定好的像素值可以为127。
更具体的，在高反差图像中某个像素点的像素值等于待处理字符图像相同位置像素点像素值减去高斯模糊图像中与相同位置像素点的像素值，再将得到的差值加上设定好的像素值得到的数值。
图6示出了对图5所示的待处理图像进行高反差保留处理后得到高反差图像。
在本申请的实施方式中，通过高反差保留处理可以使对待处理图像进行平滑得到的高斯模糊图像和高频的待处理图像进行混合，能够保留待处理图像中的边缘信息，也即突出待处理字符图像中字符的边界信息。
步骤S103，对待处理图像和高反差图像进行图像融合，得到融合图像。
具体的，上述图像融合是指将待处理字符图像中某个像素点的像素值和在高反差图像中与该像素点相同位置的像素点的像素值相加，得到在融合图像中与该像素点相同位置的像素点的像素值。
步骤S104，对融合图像进行线性光增强处理，得到线性光增强图像。
在本申请的一些实施方式中，上述线性光增强处理具体包括：对融合图像中各个第三像素点进行归一化，得到各个第三像素点对应的第一归一化像素值；确定各个第四像素点的线性光值；将融合图像中各个第四像素点的第一归一化 像素值分别和与其对应的线性光值相加，得到线性光增强图像。其中，各个第四像素点中的每个第四像素点为各个第三像素点中第一归一化像素值在预设数值范围内的第三像素点。
在本申请的一些实施方式中，上述各个第四像素点中的每个第四像素点的线性光值可以为基于其第一归一化像素值，以及高反差图像中与其相同位置的像素点的第二归一化像素值计算得到的线性光值。该线性光值可以用于使融合图像的对比度增强。
具体的，上述终端可以对各个第三像素点的像素值进行归一化，例如可以将各个第三像素点的像素值除以255，得到各个第三像素点对应的第一归一化像素值，第一归一化像素值均位于[0，1]之内。对于某一个第三像素点，若其对应的第一归一化像素值不在预设数值范围内，则可以不增加线性光值；若其对应的第一归一化像素值在预设数值范围内，此时，该第三像素点为第四像素点，则终端可以对高反差图像同样进行归一化，得到高反差图像中各个像素点的第二归一化像素值，并基于该第四像素点对应的第一归一化像素值，及高反差图像中与其相同位置的像素点对应的第二归一化像素值，计算该第四像素点对应的线性光值。其中，预设数值范围可以根据实际情况进行选择，在本申请的一些实施方式中，预设数值范围可以是数值不为0.5。
更具体的，若其对应的第一归一化像素值在预设数值范围内，其对应的线性光值等于其第一归一化像素值，减去高反差图像中与其相同位置的像素点对应的第二归一化像素值与2的乘积，再将得到的差值减去1。可以理解的是，得到的线性光值同样为归一化后数值。
为了保证得到的线性光增强图像中像素值不会超出合理的范围，在本申请的一些实施方式中，上述将融合图像中各个第四像素点的第一归一化像素值分别和与其对应的线性光值相加，得到线性光增强图像，可以具体包括：将融合图像中各个第四像素点的第一归一化像素值分别和与其对应的线性光值相加，得到各个第四像素点分别对应的第三归一化像素值；对各个第四像素点分别对 应的第三归一化像素值进行数值校验，并根据数值校验的校验结果确定各个第四像素点分别对应的第四归一化像素值，得到线性光增强图像。
具体的，在本申请的一些实施方式中，可以将融合图像中各个第四像素点的第一归一化像素值分别和与其对应的线性光值相加，得到各个第四像素点分别对应的第三归一化像素值，接着对各个第四像素点分别对应的第三归一化像素值进行数值校验。若数值校验的校验结果为某个第四像素点对应的第三归一化像素值大于1，则将该第一像素点的第一像素值设置为1；若数值校验的校验结果为某个第四像素点对应的第三归一化像素值小于0，则将该第一像素点的第一像素值设置为0；若数值校验的校验结果为某个第四像素点对应的第三归一化像素值大于或等于0，且小于或等于1，则保留其对应的第三归一化像素值。因此，根据数值校验的校验结果，上述终端可以确定各个第四像素点分别对应的第四归一化像素值，得到线性光增强图像。
图7示出了基于对图5所示的待处理图像及图6所示的高反差图像进行图像融合得到的融合图像，得到的线性光增强图像。
在本申请的实施方式中，通过高反差保留处理、图像融合处理以及线性光增强，可以扩大字符对应的像素区域与其他像素点的在像素值上的差距，使待处理字符图像中字符对应的像素区域被加强。
步骤S105，将线性光增强图像中像素值在预设像素值范围内的第一像素点作为待处理图像的图像内容。
其中，上述预设像素值范围是指图像内容对应的像素点的像素值所在的范围。
在本申请的实施方式中，若线性光增强图像中某个第一像素点的像素值在预设像素值范围内，则说明该第一像素点属于图像内容对应的像素点，而若线性光增强图像中某个第一像素点的像素值在预设像素值范围之外，则说明该第一像素点不属于图像内容对应的像素点。
具体的，在本申请的一些实施方式中，上述终端可以遍历各个第一像素点， 若某个第一像素点的像素值在预设像素值范围以外，则将该第一像素点的像素值设置为0，以剔除掉该像素点，若某个第一像素点的像素值在预设像素值范围内，则保留该第一像素点的像素值。当遍历完线性光增强图像中所有第一像素点之后，可以在预设像素值范围内的第一像素点作为待处理图像的图像内容。
图8示出了从图7所示的线性光增强图像进行处理后得到的图像内容。
需要说明的是，基于手写板材料及字符颜色的不同，上述预设像素值范围可以根据实际情况进行调整。具体的，预设像素值范围可以基于对大量直方图确定出来，每个直方图统计了不同线性光增强图像中第一像素点的像素值。在本申请的一些实施方式中，预设像素值范围可以设置为大于或等于230，也即，预设像素值范围对应的归一化后的像素值大于或等于0.9。
在本申请的实施方式中，通过获取待处理图像，并对待处理图像进行高反差保留处理，得到待处理图像的高反差图像，然后，对待处理图像和高反差图像进行图像融合，得到融合图像，再对融合图像进行线性光增强处理，得到线性光增强图像，能够增强待处理图像中图像内容对应的像素区域和非图像内容对应的像素区域的像素值差异，加强图像中图像内容的信息显示，使得将线性光增强图像中像素值在预设像素值范围内的第一像素点作为待处理图像的图像内容后，图像内容提取的完整性更高。
并且，相较于人工地对待处理图像进行抠图处理，本申请提供的图像内容的提取方法可以达到实时、批量处理的效果。
进一步地，上述图像内容的提取方法还可以包括：将待处理字符图像的图像内容叠加到目标背景上，得到目标图像，并对目标图像进行显示。
其中，目标背景可以依据用户需求进行设置。
在本申请的一些实施方式中，目标背景可以为像素值为255的背景图片。例如，图9示出了将图8所示图像内容叠加到像素值为255的背景图片上得到的目标图像。
在本申请的实施方式中，终端通过将待处理图像的图像内容叠加到目标背 景上，可以得到目标图像，目标背景一般与图像内容的颜色差异较大，因此，得到的目标图像中图像内容将进一步突出。此时，终端通过对目标图像进行显示，可以使用户更好地观看到图像内容。
需要说明的是，对于前述的各方法实施例，为了简单描述，故将其都表述为一系列的动作组合，但是本领域技术人员应该知悉，本申请并不受所描述的动作顺序的限制，因为根据本申请，某些步骤可以采用其它顺序进行。
如图10所示为本申请实施例提供的一种图像内容的提取装置1000的结构示意图，所述图像内容的提取装置1000配置于终端上。
具体的，所述图像内容的提取装置1000可以包括：
图像获取单元1001，用于获取待处理图像；
高反差保留单元1002，用于对所述待处理图像进行高反差保留处理，得到所述待处理图像的高反差图像；
图像融合单元1003，用于对所述待处理图像和所述高反差图像进行图像融合，得到融合图像；
线性光增强单元1004，用于对所述融合图像进行线性光增强处理，得到线性光增强图像；
图像内容提取单元1005，用于将所述线性光增强图像中像素值在预设像素值范围内的第一像素点作为所述待处理图像的图像内容。
在本申请的一些实施方式中，上述图像获取单元1001还可以具体用于：获取对手写板进行拍摄得到的原始图像，对所述原始图像进行锐化处理，并将锐化处理后得到的第一锐化图像作为所述待处理图像。
在本申请的一些实施方式中，上述图像获取单元1001还可以具体用于：获取对手写板进行拍摄得到的原始图像；对所述原始图像进行反相处理，得到所述原始图像的第一反相图；对所述第一反相图进行锐化处理，并将锐化处理后得到的第二锐化图像作为所述待处理图像。
在本申请的一些实施方式中，上述图像获取单元1001还可以具体用于：对 所述第一反相图进行高斯滤波处理，得到所述第一反相图的高斯模糊图像；将所述第一反相图中各个第二像素点的像素值和所述第一反相图的高斯模糊图像中与所述各个第二像素点相同位置的像素点的像素值进行加权处理，得到所述第二锐化图像。
在本申请的一些实施方式中，上述图像获取单元1001还可以具体用于：获取对手写板进行拍摄得到的原始图像；对所述原始图像进行图像识别，得到所述原始图像中所述手写板对应的像素区域；对所述手写板对应的像素区域进行透视变化处理，得到手写板图像；对所述手写板图像进行锐化处理，并将锐化处理后得到的第三锐化图像作为所述待处理图像。
在本申请的一些实施方式中，上述图像获取单元1001还可以具体用于：获取对手写板进行拍摄得到的原始图像；对所述原始图像进行图像识别，得到所述原始图像中所述手写板对应的像素区域；对所述手写板对应的像素区域进行透视变化处理，得到手写板图像；对所述手写板图像进行反相处理，得到所述手写板图像的第二反相图；对所述第二反相图进行锐化处理，并将锐化处理后得到的第四锐化图像作为所述待处理图像.
在本申请的一些实施方式中，上述各个第四像素点中的每个第四像素点的线性光值为基于其所述第一归一化像素值，以及所述高反差图像中与其相同位置的像素点的第二归一化像素值计算得到的线性光值。
在本申请的一些实施方式中，上述线性光增强单元1004还可以具体用于：将融合图像中所述各个第四像素点的第一归一化像素值分别和与其对应的线性光值相加，得到所述各个第四像素点分别对应的第三归一化像素值；对所述各个第四像素点分别对应的第三归一化像素值进行数值校验，并根据所述数值校验的校验结果确定所述各个第四像素点分别对应的第四归一化像素值，得到所述线性光增强图像。
需要说明的是，为描述的方便和简洁，上述图像内容的提取装置1000的具体工作过程，可以参考图1至图9所述方法的对应过程，在此不再赘述。
如图11所示，为本申请实施例提供的一种终端的示意图。该终端可以为具有一定处理能力的手写板、与手写板连接的终端设备，或者其他需提取图像中字符的终端设备。
具体的，该终端11可以包括：处理器110、存储器111以及存储在所述存储器111中并可在所述处理器110上运行的计算机程序112，例如图像内容的提取程序。所述处理器110执行所述计算机程序112时实现上述各个图像内容的提取方法实施例中的步骤，例如图1所示的步骤S101至S105。或者，所述处理器110执行所述计算机程序112时实现上述各装置实施例中各模块/单元的功能，例如图10所示的图像获取单元1001、高反差保留单元1002、图像融合单元1003、线性光增强单元1004和图像内容提取单元1005。
所述计算机程序可以被分割成一个或多个模块/单元，所述一个或者多个模块/单元被存储在所述存储器111中，并由所述处理器110执行，以完成本申请。所述一个或多个模块/单元可以是能够完成特定功能的一系列计算机程序指令段，该指令段用于描述所述计算机程序在所述终端中的执行过程。
例如，所述计算机程序可以被分割成：图像获取单元、高反差保留单元、图像融合单元、线性光增强单元和图像内容提取单元。
各单元具体功能如下：图像获取单元，用于获取待处理图像；高反差保留单元，用于对所述待处理图像进行高反差保留处理，得到所述待处理图像的高反差图像；图像融合单元，用于对所述待处理图像和所述高反差图像进行图像融合，得到融合图像；线性光增强单元，用于对所述融合图像进行线性光增强处理，得到线性光增强图像；图像内容提取单元，用于将所述线性光增强图像中像素值在预设像素值范围内的第一像素点作为所述待处理图像的图像内容。
所述终端可包括，但不仅限于，处理器110、存储器111。本领域技术人员可以理解，图11仅仅是终端的示例，并不构成对终端的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件，例如所述终端还可以包括输入输出设备、网络接入设备、总线等。
所称处理器110可以是中央处理单元(Central Processing Unit，CPU)，还可以是其他通用处理器、数字信号处理器(Digital Signal Processor，DSP)、专用集成电路(Application Specific Integrated Circuit，ASIC)、现成可编程门阵列(Field-Programmable Gate Array，FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件等。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
所述存储器111可以是所述终端的内部存储单元，例如终端的硬盘或内存。所述存储器111也可以是所述终端的外部存储设备，例如所述终端上配备的插接式硬盘，智能存储卡(Smart Media Card，SMC)，安全数字(Secure Digital，SD)卡，闪存卡(Flash Card)等。进一步地，所述存储器111还可以既包括所述终端的内部存储单元也包括外部存储设备。所述存储器111用于存储所述计算机程序以及所述终端所需的其他程序和数据。所述存储器111还可以用于暂时地存储已经输出或者将要输出的数据。
所属领域的技术人员可以清楚地了解到，为了描述的方便和简洁，仅以上述各功能单元、模块的划分进行举例说明，实际应用中，可以根据需要而将上述功能分配由不同的功能单元、模块完成，即将所述装置的内部结构划分成不同的功能单元或模块，以完成以上描述的全部或者部分功能。实施例中的各功能单元、模块可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中，上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。另外，各功能单元、模块的具体名称也只是为了便于相互区分，并不用于限制本申请的保护范围。上述系统中单元、模块的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在上述实施例中，对各个实施例的描述都各有侧重，某个实施例中没有详述或记载的部分，可以参见其它实施例的相关描述。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示 例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对各个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
在本申请所提供的实施例中，应该理解到，所揭露的装置/终端和方法，可以通过其它的方式实现。例如，以上所描述的装置/终端实施例仅仅是示意性的，例如，所述模块或单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通讯连接可以是通过一些接口，装置或单元的间接耦合或通讯连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。
所述集成的模块/单元如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请实现上述实施例方法中的全部或部分流程，也可以通过计算机程序来指令相关的硬件来完成，所述的计算机程序可存储于一计算机可读存储介质中，该计算机程序在被处理器执行时，可实现上述各个方法实施例的步骤。其中，所述计算机程序包括计算机程序代码，所述计算机程序代码可以为源代码形式、对象代码形式、可执行文件或某些中间形式等。所述计算机可读介质可以包括： 能够携带所述计算机程序代码的任何实体或装置、记录介质、U盘、移动硬盘、磁碟、光盘、计算机存储器、只读存储器(Read-Only Memory，ROM)、随机存取存储器(Random Access Memory，RAM)、电载波信号、电信信号以及软件分发介质等。需要说明的是，所述计算机可读介质包含的内容可以根据司法管辖区内立法和专利实践的要求进行适当的增减，例如在某些司法管辖区，根据立法和专利实践，计算机可读介质不包括电载波信号和电信信号。
以上所述实施例仅用以说明本申请的技术方案，而非对其限制；尽管参照前述实施例对本申请进行了详细的说明，本领域的普通技术人员应当理解：其依然可以对前述各实施例所记载的技术方案进行修改，或者对其中部分技术特征进行等同替换；而这些修改或者替换，并不使相应技术方案的本质脱离本申请各实施例技术方案的精神和范围，均应包含在本申请的保护范围之内。

Claims (10)

  1. 一种图像内容的提取方法，其特征在于，包括：
    获取待处理图像；
    对所述待处理图像进行高反差保留处理，得到所述待处理图像的高反差图像；
    对所述待处理图像和所述高反差图像进行图像融合，得到融合图像；
    对所述融合图像进行线性光增强处理，得到线性光增强图像；
    将所述线性光增强图像中像素值在预设像素值范围内的第一像素点作为所述待处理图像的图像内容。
  2. 如权利要求1所述的图像内容的提取方法，其特征在于，所述获取待处理图像，包括：
    获取对手写板进行拍摄得到的原始图像，对所述原始图像进行锐化处理，并将锐化处理后得到的第一锐化图像作为所述待处理图像；
    或者，获取对手写板进行拍摄得到的原始图像；对所述原始图像进行反相处理，得到所述原始图像的第一反相图；对所述第一反相图进行锐化处理，并将锐化处理后得到的第二锐化图像作为所述待处理图像。
  3. 如权利要求2所述的图像内容的提取方法，其特征在于，所述对所述第一反相图进行锐化处理，包括：
    对所述第一反相图进行高斯滤波处理，得到所述第一反相图的高斯模糊图像；
    将所述第一反相图中各个第二像素点的像素值和所述第一反相图的高斯模糊图像中与所述各个第二像素点相同位置的像素点的像素值进行加权处理，得到所述第二锐化图像。
  4. 如权利要求1所述的图像内容的提取方法，其特征在于，所述获取待处理图像，还包括：
    获取对手写板进行拍摄得到的原始图像；对所述原始图像进行图像识别，得到所述原始图像中所述手写板对应的像素区域；对所述手写板对应的像素区域进行透视变化处理，得到手写板图像；对所述手写板图像进行锐化处理，并将锐化处理后得到的第三锐化图像作为所述待处理图像；
    或者，获取对手写板进行拍摄得到的原始图像；对所述原始图像进行图像识别，得到所述原始图像中所述手写板对应的像素区域；对所述手写板对应的像素区域进行透视变化处理，得到手写板图像；对所述手写板图像进行反相处理，得到所述手写板图像的第二反相图；对所述第二反相图进行锐化处理，并将锐化处理后得到的第四锐化图像作为所述待处理图像。
  5. 如权利要求1至4任意一项所述的图像内容的提取方法，其特征在于，所述对所述融合图像进行线性光增强处理，得到线性光增强图像，包括：
    对所述融合图像中各个第三像素点进行归一化，得到所述各个第三像素点对应的第一归一化像素值；
    确定各个第四像素点的线性光值，所述各个第四像素点中的每个第四像素点为所述各个第三像素点中所述第一归一化像素值在预设数值范围内的第三像素点；
    将融合图像中所述各个第四像素点的第一归一化像素值分别和与其对应的线性光值相加，得到所述线性光增强图像。
  6. 如权利要求5所述的图像内容的提取方法，其特征在于，所述各个第四像素点中的每个第四像素点的线性光值为基于其第一归一化像素值，以及所述高反差图像中与其相同位置的像素点的第二归一化像素值计算得到的线性光值。
  7. 如权利要求5所述的图像内容的提取方法，其特征在于，所述将融合图像中所述各个第四像素点的第一归一化像素值分别和与其对应的线性光值相加，得到所述线性光增强图像，包括：
    将融合图像中所述各个第四像素点的第一归一化像素值分别和与其对应的线性光值相加，得到所述各个第四像素点分别对应的第三归一化像素值；
    对所述各个第四像素点分别对应的第三归一化像素值进行数值校验，并根据所述数值校验的校验结果确定所述各个第四像素点分别对应的第四归一化像素值，得到所述线性光增强图像。
  8. 一种图像内容的提取装置，其特征在于，包括：
    图像获取单元，用于获取待处理图像；
    高反差保留单元，用于对所述待处理图像进行高反差保留处理，得到所述待处理图像的高反差图像；
    图像融合单元，用于对所述待处理图像和所述高反差图像进行图像融合，得到融合图像；
    线性光增强单元，用于对所述融合图像进行线性光增强处理，得到线性光增强图像；
    图像内容提取单元，用于将所述线性光增强图像中像素值在预设像素值范围内的第一像素点作为所述待处理图像的图像内容。
  9. 一种终端，包括存储器、处理器以及存储在所述存储器中并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时实现如权利要求1至7任一项所述方法的步骤。
  10. 一种计算机可读存储介质，所述计算机可读存储介质存储有计算机程序，其特征在于，所述计算机程序被处理器执行时实现如权利要求1至7任一 项所述方法的步骤。
PCT/CN2021/119724 2021-08-16 2021-09-22 一种图像内容的提取方法、装置、终端和存储介质 WO2023019681A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP21953917.8A EP4345741A1 (en) 2021-08-16 2021-09-22 Image content extraction method and apparatus, and terminal and storage medium
US18/397,979 US20240127404A1 (en) 2021-08-16 2023-12-27 Image content extraction method and apparatus, terminal, and storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110938232.1 2021-08-16
CN202110938232.1A CN113628196A (zh) 2021-08-16 2021-08-16 一种图像内容的提取方法、装置、终端和存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US18/397,979 Continuation US20240127404A1 (en) 2021-08-16 2023-12-27 Image content extraction method and apparatus, terminal, and storage medium

Publications (1)

Publication Number Publication Date
WO2023019681A1 true WO2023019681A1 (zh) 2023-02-23

Family

ID=78385733

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/119724 WO2023019681A1 (zh) 2021-08-16 2021-09-22 一种图像内容的提取方法、装置、终端和存储介质

Country Status (4)

Country Link
US (1) US20240127404A1 (zh)
EP (1) EP4345741A1 (zh)
CN (1) CN113628196A (zh)
WO (1) WO2023019681A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113971645A (zh) * 2021-11-17 2022-01-25 广州朗国电子科技股份有限公司 一种图像宽动态增强方法及装置

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050063586A1 (en) * 2003-08-01 2005-03-24 Microsoft Corporation Image processing using linear light values and other image processing improvements
CN102881011A (zh) * 2012-08-31 2013-01-16 北京航空航天大学 基于区域分割的肖像光照迁移方法
CN107566740A (zh) * 2017-10-18 2018-01-09 维沃移动通信有限公司 一种图像处理方法及移动终端
CN109461186A (zh) * 2018-10-15 2019-03-12 Oppo广东移动通信有限公司 图像处理方法、装置、计算机可读存储介质和电子设备
CN110675336A (zh) * 2019-08-29 2020-01-10 苏州千视通视觉科技股份有限公司 一种低照度图像增强方法及装置
US10950305B1 (en) * 2018-11-02 2021-03-16 Facebook Technologies, Llc Selective pixel output
CN112508816A (zh) * 2020-12-09 2021-03-16 中国电子科技集团公司第三研究所 一种红外图像锐化方法、锐化处理系统及终端设备

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9646366B2 (en) * 2012-11-30 2017-05-09 Change Healthcare Llc Method and apparatus for enhancing medical images
CN106228516B (zh) * 2016-07-14 2019-07-19 北京字节跳动网络技术有限公司 一种高自然度的实时美颜方法、装置
CN110163211B (zh) * 2018-09-06 2023-02-28 腾讯科技（深圳）有限公司 一种图像识别方法、装置和存储介质
CN111027556B (zh) * 2019-03-11 2023-12-22 广东小天才科技有限公司 一种基于图像预处理的搜题方法及学习设备
CN110598560A (zh) * 2019-08-15 2019-12-20 重庆特斯联智慧科技股份有限公司 基于神经网络增强的夜间监控识别方法和系统
CN112215768A (zh) * 2020-09-30 2021-01-12 广州虎牙科技有限公司 图像清晰度提升方法、装置、电子设备及可读存储介质
CN112967207B (zh) * 2021-04-23 2024-04-12 北京恒安嘉新安全技术有限公司 一种图像处理方法、装置、电子设备及存储介质

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20050063586A1 (en) * 2003-08-01 2005-03-24 Microsoft Corporation Image processing using linear light values and other image processing improvements
CN102881011A (zh) * 2012-08-31 2013-01-16 北京航空航天大学 基于区域分割的肖像光照迁移方法
CN107566740A (zh) * 2017-10-18 2018-01-09 维沃移动通信有限公司 一种图像处理方法及移动终端
CN109461186A (zh) * 2018-10-15 2019-03-12 Oppo广东移动通信有限公司 图像处理方法、装置、计算机可读存储介质和电子设备
US10950305B1 (en) * 2018-11-02 2021-03-16 Facebook Technologies, Llc Selective pixel output
CN110675336A (zh) * 2019-08-29 2020-01-10 苏州千视通视觉科技股份有限公司 一种低照度图像增强方法及装置
CN112508816A (zh) * 2020-12-09 2021-03-16 中国电子科技集团公司第三研究所 一种红外图像锐化方法、锐化处理系统及终端设备

Also Published As

Publication number Publication date
US20240127404A1 (en) 2024-04-18
EP4345741A1 (en) 2024-04-03
CN113628196A (zh) 2021-11-09

Similar Documents

Publication Publication Date Title
US20200160040A1 (en) Three-dimensional living-body face detection method, face authentication recognition method, and apparatuses
Ignatov et al. Dslr-quality photos on mobile devices with deep convolutional networks
US20100303372A1 (en) Digital image processing and enhancing system and method with function of removing noise
CN109064504B (zh) 图像处理方法、装置和计算机存储介质
CN108090511B (zh) 图像分类方法、装置、电子设备及可读存储介质
Yu et al. Efficient patch-wise non-uniform deblurring for a single image
EP3798975A1 (en) Method and apparatus for detecting subject, electronic device, and computer readable storage medium
CN107038704B (zh) 视网膜图像渗出区域分割方法、装置和计算设备
CN110062157B (zh) 渲染图像的方法、装置、电子设备和计算机可读存储介质
WO2022233185A1 (zh) 一种图像滤波方法、装置、终端和计算机可读存储介质
CN112785572B (zh) 图像质量评估方法、装置以及计算机可读存储介质
WO2020038065A1 (zh) 一种图像处理方法、终端及计算机存储介质
CN112396050B (zh) 图像的处理方法、设备以及存储介质
CN111199197B (zh) 一种人脸识别的图像提取方法及处理设备
US20240127404A1 (en) Image content extraction method and apparatus, terminal, and storage medium
CN111192241A (zh) 一种人脸图像的质量评估方法、装置及计算机存储介质
CN107194886B (zh) 一种用于相机传感器的灰尘检测方法及装置
WO2022199395A1 (zh) 人脸活体检测方法、终端设备及计算机可读存储介质
CN108764040B (zh) 一种图像检测方法、终端及计算机存储介质
CN109726613B (zh) 一种用于检测的方法和装置
CN111311610A (zh) 图像分割的方法及终端设备
CN115689947A (zh) 一种图像锐化的方法、系统、电子装置和存储介质
CN113409375B (zh) 图像处理方法、装置以及非易失性存储介质
CN114663284A (zh) 红外热成像全景图像处理方法、系统及存储介质
CN113628148A (zh) 红外图像降噪方法和装置

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21953917

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2021953917

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2021953917

Country of ref document: EP

Effective date: 20231228

NENP Non-entry into the national phase

Ref country code: DE