CN114693553A - Mobile intelligent terminal image processing method and system - Google Patents

Abstract

The invention discloses a method and a system for processing images of a mobile intelligent terminal, wherein the method comprises the following steps: the method comprises the steps of dividing a current display image of the mobile intelligent terminal into N equal-area target areas, determining the current brightness characteristic of the image in each target area, collecting the current face image of a target person watching the mobile intelligent terminal, extracting eye characteristic points from the current face image, and intelligently adjusting and processing the current display image based on the current brightness characteristic and the eye characteristic points of each area. The resolution, brightness and the like of the current display image are intelligently adjusted and enhanced by combining the brightness characteristic of the current display image of the mobile intelligent terminal with the human eye characteristic points of the target person in front of the mobile intelligent terminal, so that the intelligent processing work can be intelligently carried out according to the actual condition of a viewer, the viewing experience of the user is ensured, the practicability is improved, and the experience of the user is also improved.

Description

A kind of mobile intelligent terminal image processing method and system

technical field

The present invention relates to the technical field of image processing, and in particular, to a method and system for image processing of a mobile intelligent terminal.

Background technique

With the continuous development and maturity of intelligent terminal software and hardware technology, various mobile intelligent terminals have emerged one after another, and have been accepted by the majority of users. People can use mobile intelligent terminals to conduct video chats or watch videos and live broadcasts in their spare time. It greatly enriches people's leisure life, but due to the distortion of the image transmitted by the mobile intelligent terminal, it is necessary to process the displayed image on the mobile intelligent terminal to ensure the clarity. The enhancement is performed without considering the actual viewing situation of the user, resulting in extremely poor viewing experience for some users of the displayed image, which reduces the user's sense of experience.

SUMMARY OF THE INVENTION

In view of the above-mentioned problems, the present invention provides an image processing method and system for a mobile intelligent terminal to solve the problem that the existing image processing methods mentioned in the background art simply enhance the displayed image without considering the user Therefore, some users have a very poor viewing experience of the displayed image, which reduces the user's sense of experience.

A mobile intelligent terminal image processing method, comprising the following steps:

Divide the current display image of the mobile intelligent terminal into N target areas of equal area;

Determine the current brightness characteristics of the image within each target area;

collecting the current face image of the target person watching the mobile intelligent terminal, and extracting human eye feature points from the current face image;

Based on the current brightness characteristics and human eye feature points of each area, intelligently adjust and process the currently displayed image.

Preferably, dividing the current display image of the mobile smart terminal into N target areas of equal area, including:

Perform point cloud data detection on the currently displayed image to obtain a detection result;

Determine the point cloud data distribution in the currently displayed image according to the detection result;

Determine the segmentation form of the currently displayed image and the number of N segmentation regions according to the point cloud data distribution according to preset rules;

The currently displayed image is divided into N target areas of equal area according to the division form.

Preferably, the determining the current brightness feature of the image in each target area includes:

Extract the feature factor of each pixel of the image in each target area;

Construct the image parameter matrix of the target area according to the feature factor of each pixel of the image in each target area;

Determine the brightness parameter corresponding to each pixel according to each matrix factor parameter in the image parameter matrix;

Perform matching in the preset database according to the brightness parameter corresponding to each pixel to obtain the brightness feature value corresponding to the pixel;

Calculate the average luminance characteristic value of each target area, and determine the current luminance characteristic of the target area according to the ratio of the average luminance characteristic value to the standard luminance characteristic value.

Preferably, the current face image of the target person watching the mobile intelligent terminal is collected, and the feature points of human eyes are extracted from the current face image, including:

Extract the area where the target person's eyes are located in the current face image;

Determine the target deviation boundary of the area where the target person's eyes are located according to the pixel ratio of the current image;

Adjusting the area where the target person's eyes are located according to the target deviation boundary to obtain the adjusted human eye area;

The human eye feature points of the target person are extracted from the adjusted human eye region according to preset human eye feature parameters.

Preferably, intelligently adjusting and processing the currently displayed image based on the current brightness features and human eye feature points of each region, including:

Determine the eye gaze area of the target user according to the human eye feature points;

Determine the current visual acuity index of the target user, and perform brightness adjustment on the currently displayed image according to the current visual acuity index and the current brightness feature of each area;

Based on the current visual acuity index of the target user, evaluate the display clarity of the current brightness feature of each area to the target user, and obtain the evaluation result;

According to the evaluation result, the display scale/resolution of the currently displayed image is intelligently adjusted or the currently displayed image is enhanced.

Preferably, determining the target deviation boundary of the area where the target person's eyes are located according to the pixel ratio of the current image includes:

Determine the mask data of the area where the target person's eyes are located and its surroundings according to the pixel ratio of the current image;

Modify the first mask data of the area where the target person's eyes are located, and obtain the second mask data change parameters around the area where the target person's eyes are located;

According to the change parameter, construct a cost function in which the parameter change of the target person's eye area drives the change of the surrounding parameters;

Screening out the target second mask data whose change parameter of the second mask data around the area where the target person's eyes are located is outside the preset range, and determining the first deviation boundary according to the interval corresponding to the target second mask data;

Calculate the variation error of the second mask data around the area where the target person's eyes are located with the change of the first mask data in the area where the target person's eyes are located according to the cost function;

Correcting the first deviation boundary according to the variation error to obtain a second deviation boundary;

The second deviation boundary is identified as the target deviation boundary.

Preferably, before intelligently adjusting and processing the currently displayed image based on the current brightness feature and human eye feature points of each region, the method further includes:

Determine the spatial aggregation feature of the pixels in the currently displayed image according to the feature factor of each pixel of the currently displayed image;

Determine the aggregation characteristics of the pixels in the currently displayed image according to the spatial aggregation characteristics;

Based on the aggregation characteristics, determine the low-dimensional pixel representation distribution in the currently displayed image;

extracting the deep feature of each low-dimensional pixel in the currently displayed image according to the low-dimensional pixel representation distribution;

The deep feature of each low-dimensional pixel is used as the feature to be processed for intelligent adjustment and processing of the currently displayed image.

Preferably, the mask data of the area where the target person's eyes are located and its surroundings are determined according to the pixel ratio of the current image, including:

Determine the configuration information of the device that captures the current image according to the pixel ratio of the current image;

generating a mask matrix of the image captured by the device according to the configuration information;

Determine the mask vector of each pixel according to the mask matrix in combination with the pixel value of each pixel of the current image;

Group the mask vector of each pixel according to the vector feature of the mask vector of each pixel, and obtain the grouping result;

Determine the grouping pixel aggregation situation in each grouping result, and obtain the first pixel and the second pixel around the grouping pixel periphery;

calculating the phase coherence between each first pixel and the second pixel;

marking the target second pixel whose phase coherence calculation result between the first pixel and the second pixel is less than a preset threshold;

Determine the pixel mask bit in the current image according to the marking situation of the target second pixel;

Obtaining human eye pixel features, and performing matching in the aggregated grouped pixels of the current image according to the human eye pixel features to plan the area where the target person's human eyes are located;

Perform pixel parsing on the pixel mask bits in the current image to obtain mask data around the area where the target person's eyes are located.

A mobile intelligent terminal image processing system, the system includes:

The segmentation module is used to segment the current display image of the mobile intelligent terminal into N target areas of equal area;

A determination module for determining the current brightness feature of the image in each target area;

an extraction module, used for collecting the current face image of the target person watching the mobile intelligent terminal, and extracting human eye feature points from the current face image;

The processing module is used to intelligently adjust and process the currently displayed image based on the current brightness feature and human eye feature points of each area.

Other features and advantages of the present invention will be set forth in the description which follows, and in part will be apparent from the description, or may be learned by practice of the invention. The objectives and other advantages of the invention may be realized and attained by the structure particularly pointed out in the written description and drawings.

The technical solutions of the present invention will be further described in detail below through the accompanying drawings and embodiments.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the specification, and are used to explain the present invention together with the embodiments of the present invention, and do not constitute a limitation to the present invention.

Fig. 1 is the working flow chart of a kind of mobile intelligent terminal image processing method provided by the present invention;

Fig. 2 is another work flow chart of a kind of mobile intelligent terminal image processing method provided by the present invention;

Fig. 3 is another working flow chart of a method for processing an image of a mobile intelligent terminal provided by the present invention;

FIG. 4 is a schematic structural diagram of a mobile intelligent terminal image processing system provided by the present invention.

Detailed ways

Exemplary embodiments will be described in detail herein, examples of which are illustrated in the accompanying drawings. Where the following description refers to the drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the illustrative examples below are not intended to represent all implementations consistent with this disclosure. Rather, they are merely examples of apparatus and methods consistent with some aspects of the present disclosure as recited in the appended claims.

With the continuous development and maturity of intelligent terminal software and hardware technology, various mobile intelligent terminals have emerged one after another, and have been accepted by the majority of users. People can use mobile intelligent terminals to conduct video chats or watch videos and live broadcasts in their spare time. It greatly enriches people's leisure life, but due to the distortion of the image transmitted by the mobile intelligent terminal, it is necessary to process the displayed image on the mobile intelligent terminal to ensure the clarity. The enhancement is performed without considering the actual viewing situation of the user, resulting in extremely poor viewing experience of the displayed image for some users, which reduces the user's sense of experience. In order to solve the above problem, this embodiment discloses an image processing method for a mobile intelligent terminal.

A mobile intelligent terminal image processing method, as shown in Figure 1, includes the following steps:

Step S101, dividing the current display image of the mobile intelligent terminal into N target areas of equal area;

Step S102, determining the current brightness feature of the image in each target area;

Step S103, collecting the current face image of the target person watching the mobile intelligent terminal, and extracting human eye feature points from the current face image;

Step S104 , intelligently adjust and process the currently displayed image based on the current brightness feature and the human eye feature point of each region.

The working principle of the above technical solution is as follows: dividing the current displayed image of the mobile intelligent terminal into N target areas of equal area, determining the current brightness feature of the image in each target area, and collecting the current face of the target person watching the mobile intelligent terminal. image, extracting human eye feature points from the current face image, and intelligently adjusting and processing the currently displayed image based on the current brightness features and human eye feature points of each area.

The beneficial effects of the above technical solutions are: intelligently adjust and enhance the resolution, brightness, etc. of the currently displayed image according to the brightness feature of the currently displayed image of the mobile intelligent terminal combined with the human eye feature points of the target person in front of the mobile intelligent terminal The processing work can be intelligently performed according to the actual situation of the viewer, which ensures the user's viewing experience, improves the practicability and also improves the user's sense of experience, and solves the problem of simply enhancing the displayed image in the prior art. The actual viewing situation of the user is not considered, so that some users have extremely poor viewing experience of the displayed image, which reduces the user's sense of experience.

In one embodiment, as shown in FIG. 2 , dividing the current display image of the mobile smart terminal into N target areas of equal area, including:

Step S201, performing point cloud data detection on the currently displayed image, and obtaining a detection result;

Step S202, determining the point cloud data distribution in the currently displayed image according to the detection result;

Step S203, determining the segmentation form of the currently displayed image and the number of N segmentation regions according to the point cloud data distribution according to preset rules;

Step S204: Divide the currently displayed image into N target regions of equal area according to the division form.

The beneficial effects of the above technical solutions are: by determining the cutting method and the number of cutting areas of the currently displayed image according to the distribution of point cloud data, the uniform distribution of point cloud data in each cutting area can be ensured, thereby laying a foundation for subsequent image processing. , to ensure the consistency and objectivity of processing samples.

In one embodiment, the determining the current brightness feature of the image in each target area includes:

Extract the feature factor of each pixel of the image in each target area;

Construct the image parameter matrix of the target area according to the feature factor of each pixel of the image in each target area;

Determine the brightness parameter corresponding to each pixel according to each matrix factor parameter in the image parameter matrix;

Perform matching in the preset database according to the brightness parameter corresponding to each pixel to obtain the brightness feature value corresponding to the pixel;

Calculate the average luminance characteristic value of each target area, and determine the current luminance characteristic of the target area according to the ratio of the average luminance characteristic value to the standard luminance characteristic value.

The beneficial effects of the above technical solutions are: by using the parameter matrix to obtain the brightness characteristic value of each pixel and then determining the average brightness characteristic value of each target area, each pixel in each target area can be taken into account to comprehensively determine each pixel. The current brightness characteristics of the target area ensure the rationality and objectivity of the final result.

In one embodiment, as shown in FIG. 3 , the current face image of the target person watching the mobile smart terminal is collected, and the feature points of human eyes are extracted from the current face image, including:

Step S301, extracting the area where the target person's eyes are located in the current face image;

Step S302, determining the target deviation boundary of the area where the target person's eyes are located according to the pixel ratio of the current image;

Step S303, adjusting the area where the target person's human eye is located according to the target deviation boundary to obtain an adjusted human eye area;

Step S304 , extracting the human eye feature points of the target person from the adjusted human eye region according to the preset human eye feature parameters.

The beneficial effects of the above technical solutions are: by determining the target deviation boundary of the area where the target person's eyes are located, the influence of the error can be effectively avoided, so that the human eye feature points of the target person can be extracted more accurately, and the extraction accuracy is improved at the same time. It also effectively avoids the occurrence of errors in the determination of the area range, and further improves the practicability.

In one embodiment, the intelligently adjusting and processing the currently displayed image based on the current brightness feature and the human eye feature point of each region includes:

Determine the eye gaze area of the target user according to the human eye feature points;

Determine the current vision index of the target user, and adjust the brightness of the currently displayed image according to the current vision index and the current brightness feature of each area;

Based on the current visual acuity index of the target user, evaluate the display clarity of the current brightness feature of each area to the target user, and obtain the evaluation result;

According to the evaluation result, the display scale/resolution of the currently displayed image is intelligently adjusted or the currently displayed image is enhanced.

The beneficial effects of the above technical solutions are: by selectively processing the currently displayed image according to the current brightness feature of the currently displayed image combined with the vision index of the target person, a reasonable processing method can be further made according to the actual needs of the target user, The usability is further improved.

In one embodiment, determining the target deviation boundary of the area where the target person's eyes are located according to the pixel ratio of the current image includes:

Determine the mask data of the area where the target person's eyes are located and its surroundings according to the pixel ratio of the current image;

Modify the first mask data of the area where the target person's eyes are located, and obtain the second mask data change parameters around the area where the target person's eyes are located;

According to the change parameter, construct a cost function in which the parameter change of the target person's eye area drives the change of the surrounding parameters;

Screening out the target second mask data whose change parameter of the second mask data around the area where the target person's eyes are located is outside the preset range, and determining the first deviation boundary according to the interval corresponding to the target second mask data;

Calculate the variation error of the second mask data around the area where the target person's eyes are located with the change of the first mask data in the area where the target person's eyes are located according to the cost function;

Correcting the first deviation boundary according to the variation error to obtain a second deviation boundary;

The second deviation boundary is identified as the target deviation boundary.

The beneficial effects of the above technical solutions are: by constructing a cost function to correct the first deviation boundary of the second mask data around the area where the target person's eyes are located with the first mask data in the area where the target person's eyes are located, it can further To avoid errors and improve the accuracy, it also avoids missing areas and improves stability.

In one embodiment, before intelligently adjusting and processing the currently displayed image based on the current brightness feature and human eye feature points of each region, the method further includes:

Determine the spatial aggregation characteristics of the pixels in the currently displayed image according to the characteristic factor of each pixel of the currently displayed image;

Determine the aggregation characteristics of the pixels in the currently displayed image according to the spatial aggregation characteristics;

Based on the aggregation characteristics, determine the low-dimensional pixel representation distribution in the currently displayed image;

extracting the deep feature of each low-dimensional pixel in the currently displayed image according to the low-dimensional pixel representation distribution;

The deep feature of each low-dimensional pixel is used as the feature to be processed for intelligent adjustment and processing of the currently displayed image.

The beneficial effects of the above technical solutions are: by determining the features to be processed for intelligent adjustment and processing of the currently displayed image, the features to be processed can be emphatically processed in subsequent processing of the currently displayed image, thereby improving the processing efficiency.

In one embodiment, determining the mask data of the area where the target person's eyes are located and its surroundings according to the pixel ratio of the current image, including:

Determine the configuration information of the device that captures the current image according to the pixel ratio of the current image;

generating a mask matrix of the image captured by the device according to the configuration information;

Determine the mask vector of each pixel according to the mask matrix in combination with the pixel value of each pixel of the current image;

Group the mask vector of each pixel according to the vector feature of the mask vector of each pixel, and obtain the grouping result;

Determine the aggregation situation of grouped pixels in each grouping result, and obtain the first pixel and the second pixel around it in the periphery of the aggregated grouped pixels;

calculating the phase coherence between each first pixel and the second pixel;

marking the target second pixel whose phase coherence calculation result between the first pixel and the second pixel is less than a preset threshold;

Determine the pixel mask bit in the current image according to the marking situation of the target second pixel;

Obtaining human eye pixel features, and performing matching in the aggregated grouped pixels of the current image according to the human eye pixel features to plan the area where the target person's human eyes are located;

Perform pixel parsing on the pixel mask bits in the current image to obtain mask data around the area where the target person's eyes are located.

The beneficial effects of the above technical solutions are: by determining the area where the target person's eyes are located according to the pixel aggregation situation of the current image, the area where the human face of the current image is located can be intuitively determined according to the pixel aggregation situation, and then the area where the target person's eyes are located can be accurately determined. , the accuracy of the detection result is improved. Further, by determining the pixel mask bits in the current image, the irrelevant mask area in the current image can be determined according to the pixel distribution parameters, which provides an effective reference sample for subsequent acquisition of mask data. .

This embodiment also discloses an image processing system for a mobile intelligent terminal, as shown in FIG. 4 , the system includes:

The segmentation module 401 is used to segment the current display image of the mobile intelligent terminal into N target areas of equal area;

A determination module 402, configured to determine the current brightness feature of the image in each target area;

The extraction module 403 is used to collect the current face image of the target person watching the mobile intelligent terminal, and extract the human eye feature points from the current face image;

The processing module 404 is configured to intelligently adjust and process the currently displayed image based on the current brightness feature and human eye feature points of each region.

The working principle and beneficial effects of the above technical solutions have been described in the method claims, and will not be repeated here.

Other embodiments of the present disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or techniques in the technical field not disclosed by the present disclosure . The specification and examples are to be regarded as exemplary only, with the true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise structures described above and illustrated in the accompanying drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims (9)

1. a mobile intelligent terminal image processing method, is characterized in that, comprises the following steps: Divide the current display image of the mobile intelligent terminal into N target areas of equal area; Determine the current brightness characteristics of the image within each target area; collecting the current face image of the target person watching the mobile intelligent terminal, and extracting human eye feature points from the current face image; Based on the current brightness characteristics and human eye feature points of each area, intelligently adjust and process the currently displayed image. 2. The image processing method of a mobile intelligent terminal according to claim 1, wherein the dividing the current display image of the mobile intelligent terminal into N target areas of equal area, comprising: Perform point cloud data detection on the currently displayed image to obtain a detection result; Determine the point cloud data distribution in the currently displayed image according to the detection result; Determine the segmentation form of the currently displayed image and the number of N segmentation regions according to the point cloud data distribution according to preset rules; The currently displayed image is divided into N target areas of equal area according to the division form. 3. The image processing method of a mobile intelligent terminal according to claim 1, wherein the determining the current brightness feature of the image in each target area comprises: Extract the feature factor of each pixel of the image in each target area; Construct the image parameter matrix of the target area according to the characteristic factor of each pixel of the image in each target area; Determine the brightness parameter corresponding to each pixel according to each matrix factor parameter in the image parameter matrix; Perform matching in the preset database according to the brightness parameter corresponding to each pixel to obtain the brightness feature value corresponding to the pixel; Calculate the average luminance characteristic value of each target area, and determine the current luminance characteristic of the target area according to the ratio of the average luminance characteristic value to the standard luminance characteristic value. 4. The method for processing an image of a mobile intelligent terminal according to claim 1, wherein the current face image of the target person watching the mobile intelligent terminal is collected, and human eye feature points are extracted from the current face image, include: Extract the area where the target person's eyes are located in the current face image; Determine the target deviation boundary of the area where the target person's eyes are located according to the pixel ratio of the current image; Adjusting the area where the target person's eyes are located according to the target deviation boundary to obtain the adjusted human eye area; The human eye feature points of the target person are extracted from the adjusted human eye region according to preset human eye feature parameters. 5. The image processing method of an intelligent mobile terminal according to claim 1, wherein the current display image is intelligently adjusted and processed based on the current brightness feature and the human eye feature point of each region, comprising: Determine the eye gaze area of the target user according to the human eye feature points; Determine the current visual acuity index of the target user, and perform brightness adjustment on the currently displayed image according to the current visual acuity index and the current brightness feature of each area; Based on the current visual acuity index of the target user, evaluate the display clarity of the current brightness feature of each area to the target user, and obtain the evaluation result; According to the evaluation result, the display scale/resolution of the currently displayed image is intelligently adjusted or the currently displayed image is enhanced. 6. The image processing method for an intelligent mobile terminal according to claim 4, wherein the determining the target deviation boundary of the area where the target person's eyes are located according to the pixel ratio of the current image, comprises: Determine the mask data of the area where the target person's eyes are located and its surroundings according to the pixel ratio of the current image; Modify the first mask data of the area where the target person's eyes are located, and obtain the second mask data change parameters around the area where the target person's eyes are located; Constructing a cost function in which the parameter change of the area where the target person's eyes are located drives the change of the surrounding parameters according to the change parameter; Screening out the target second mask data whose change parameter of the second mask data around the area where the target person's eyes are located is outside the preset range, and determining the first deviation boundary according to the interval corresponding to the target second mask data; Calculate the variation error of the second mask data around the area where the target person's eyes are located with the change of the first mask data in the area where the target person's eyes are located according to the cost function; Correcting the first deviation boundary according to the variation error to obtain a second deviation boundary; The second deviation boundary is identified as the target deviation boundary. 7. The image processing method of an intelligent mobile terminal according to claim 3, characterized in that, before the current displayed image is intelligently adjusted and processed based on the current brightness feature of each region and the human eye feature points, the method further comprises: include: Determine the spatial aggregation feature of the pixels in the currently displayed image according to the feature factor of each pixel of the currently displayed image; Determine the aggregation characteristics of the pixels in the currently displayed image according to the spatial aggregation characteristics; Based on the aggregation characteristics, determine the low-dimensional pixel representation distribution in the currently displayed image; extracting the deep feature of each low-dimensional pixel in the currently displayed image according to the low-dimensional pixel representation distribution; The deep feature of each low-dimensional pixel is used as the feature to be processed for intelligent adjustment and processing of the currently displayed image. 8. The image processing method of a mobile intelligent terminal according to claim 6, wherein the mask data of the area where the target person's eyes are located and its surroundings are determined according to the pixel ratio of the current image, comprising: Determine the configuration information of the device that captures the current image according to the pixel ratio of the current image; generating a mask matrix of the image captured by the device according to the configuration information; Determine the mask vector of each pixel according to the mask matrix in combination with the pixel value of each pixel of the current image; Group the mask vector of each pixel according to the vector feature of the mask vector of each pixel, and obtain the grouping result; Determine the grouping pixel aggregation situation in each grouping result, and obtain the first pixel and the second pixel around the grouping pixel periphery; calculating the phase coherence between each first pixel and the second pixel; marking the target second pixel whose phase coherence calculation result between the first pixel and the second pixel is less than a preset threshold; Determine the pixel mask bit in the current image according to the marking situation of the target second pixel; Obtaining human eye pixel features, and performing matching in the aggregated grouped pixels of the current image according to the human eye pixel features to plan the area where the target person's human eyes are located; Perform pixel parsing on the pixel mask bits in the current image to obtain mask data around the area where the target person's eyes are located. 9. A mobile intelligent terminal image processing system, characterized in that the system comprises: The segmentation module is used to segment the current display image of the mobile intelligent terminal into N target areas of equal area; A determination module for determining the current brightness feature of the image in each target area; an extraction module, used for collecting the current face image of the target person watching the mobile intelligent terminal, and extracting human eye feature points from the current face image; The processing module is used to intelligently adjust and process the currently displayed image based on the current brightness feature and human eye feature points of each area.

Images