TW202001598A - Image search method - Google Patents

Abstract

An image search method is provided. A selection is performed on a display frame, and a region of interest (ROI) is obtained from the display frame based on the selection. A face detection is performed on the ROI. When the ROI contains a human face, an image corresponding to the human face is obtained from the display frame and the image is input to search engines.

Description

Image search method

The present invention relates to a search method, and in particular, to an image search method that can search regions of interest (ROI) in real time.

General search engines provide an interface that allows users to specify criteria for projects of interest and let search engines find matching results. In addition to entering text as a keyword to search, you can also enter pictures into the search engine to search. At present, when searching for pictures, search engines can only prepare pictures in advance and input the entire picture into the search engine.

The current image search method cannot search for specific areas in the image. If you want to search for a specific area in the picture, the user must manually crop the picture to obtain an image that includes only the specific area, and then search with the image. In addition, the current image search method cannot directly perform image search for videos. The user must capture the images of interest to search during the video playback. Accordingly, it is not convenient in use.

The invention provides an image search method, which can circle the region of interest in the display screen and search automatically.

The image search method of the present invention includes: performing a circled action in the display screen; obtaining a region of interest from the display screen based on the circled action; performing face detection on the region of interest; determining that the region of interest has a human face , The image corresponding to the human face is obtained from the display screen; and the image corresponding to the human face is input to the search engine for searching.

In an embodiment of the invention, after obtaining the region of interest, it further includes: performing image distortion correction.

In an embodiment of the present invention, after obtaining the region of interest, it further includes: performing resolution compensation on the image corresponding to the region of interest.

In an embodiment of the invention, the above image search method further includes: when it is determined that the region of interest does not have a human face, input an image corresponding to the region of interest to the search engine for searching.

In an embodiment of the present invention, the step of performing face detection on the region of interest includes: performing skin color segmentation to obtain a skin color region; determining whether a facial feature exists in the skin color region; and when a facial feature exists in the skin color region , It is determined that the region of interest includes a human face.

In an embodiment of the present invention, the above image search method further includes: after inputting an image corresponding to a human face to the search engine for searching, determining whether each of the searched images matches the human face in the region of interest.

In an embodiment of the present invention, the above image search method further includes: when N faces are detected in the region of interest, N images based on N faces are obtained from the display screen, where N is greater than 1 Integer; and input N images to the search engine to search.

In an embodiment of the invention, the display screen includes one of a video screen, a picture, and a browser screen.

Based on the above, the present invention provides a method for the user to circle the region of interest in the display screen in real time, after automatically pre-processing the image to improve or restore the resolution of the image, input the image to the search engine for searching To improve the convenience of use.

In order to make the above-mentioned features and advantages of the present invention more obvious and understandable, the embodiments are specifically described below in conjunction with the accompanying drawings for detailed description as follows.

FIG. 1 is a block diagram of an electronic device according to an embodiment of the invention. The electronic device 100 includes a processor 110, a storage unit 120, and a display 130. The processor 110 is coupled to the storage unit 120 and the display 130.

The processor 110 may be a central processing unit (Central Processing Unit, CPU), an image processing unit (Graphic Processing Unit, GPU), a physical processing unit (Physics Processing Unit, PPU), a programmable microprocessor (Microprocessor), Embedded control chip, digital signal processor (DSP), application specific integrated circuits (ASIC) or other similar devices.

The storage unit 120 is any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory (Flash memory), secure digital Card (Secure Digital Memory Card, SD), hard disk or other similar devices or a combination of these devices. The storage unit 120 stores at least one code segment, and the processor 110 reads the code segment in the storage unit 120 to execute corresponding instructions to implement the following image search method.

The display 130 may be implemented by a liquid crystal display (LCD), a plasma display (Plasma Display), or the like, or a touch screen with a touch module may be used as the display 130.

2 is a flowchart of an image search method according to an embodiment of the invention. Please refer to FIG. 1 and FIG. 2 at the same time. In step S205, a circle action is performed on the display screen. For example, the processor 110 may perform the corresponding circled action on the display screen of the display 130 by detecting the action performed by the input device when the display screen of the display 130 is presented as a video screen, picture, or browser screen. . Next, in step S210, the processor 110 obtains a region of interest (ROI) from the display screen based on the circled action.

3 is a schematic diagram of circled a region of interest according to an embodiment of the invention. In this embodiment, a notebook computer is used as an example for description. Referring to FIG. 3, assuming that the notebook computer is playing a video, the user may use an input device such as a mouse, a stylus pen, and a tablet to circle the region of interest 310 on the display screen 300 (ie, the video screen).

Returning to FIG. 2, after obtaining the region of interest, the processor 110 further performs image distortion correction. For example, if the region of interest is a distorted or flipped picture, the processor 110 performs image distortion correction to correct the image corresponding to the region of interest.

Thereafter, in step S215, the processor 110 performs face detection on the region of interest. This is to avoid affecting the search result of the image when more than one person is included in the region of interest. Face detection refers to searching for one or more areas that may be faces in an image. In the process of face detection, it can be roughly judged by using information such as facial skin color and facial features. For example, the processor 110 performs skin color segmentation to obtain a skin color area, and then determines whether a human face feature exists in the skin color area. For example, it is determined whether there are eye features, mouth features, etc. in the skin color area. When there is a face feature in the skin color area, it is determined that the area of interest includes a face. In addition, if the circled area of interest is a half-face image, the processor 110 will further extend the range of the area of interest to include the entire face area or even the whole body area.

In step S220, when it is determined that the region of interest has a human face, the processor 110 obtains an image corresponding to the human face from the display screen. After that, in step S225, the processor 110 inputs the image corresponding to the face to the search engine for searching. On the other hand, when it is determined that the region of interest does not have a human face, the processor 110 directly inputs the image corresponding to the region of interest to the search engine for searching.

When N faces are detected in the region of interest, the processor 110 obtains N images based on the N faces from the display screen, and inputs the N images to the search engine for searching. Where N is an integer greater than 1. For example, if there are two faces in the region of interest, then two corresponding images are obtained.

For example, FIGS. 4A and 4B are schematic diagrams of circled regions of interest according to another embodiment of the present invention. In FIG. 4A, the region of interest 410 is circled. After face detection, two faces are found in the region of interest 410, and images 411 and 413 based on each face are obtained from the display screen. After that, input the image 411 and the image 413 to the search engine for searching. Here, it can be further set that when the number of faces included in the region of interest exceeds a threshold, the search is still performed with the image corresponding to the entire region of interest. For example, when the number of faces exceeds 10, each face is not searched separately, but the entire image is searched directly.

After obtaining the region of interest, the processor 110 may further perform resolution compensation on the image corresponding to the region of interest. First, convert the image to grayscale image Gary ₁ . Next, an edge detection algorithm is executed on the grayscale image Gary _{1 to} find the number of edges. Here, the Sobel algorithm is used as the edge detection algorithm.

公式2

基於每一個像素的橫向及縱向梯度近似值，可使用以下公式3來計算梯度的大小（Magnitude）G_M 。 公式3

Gx ₁ and Gy _{1 in} Equation 1 and Equation 2 below represent images detected by horizontal and vertical edges, respectively. Formula 1

Formula 2

Based on the approximate horizontal and vertical gradients of each pixel, the following formula 3 can be used to calculate the magnitude of the gradient (Magnitude) G _M. Formula 3

Next, use the blank image Edge that is the same size as the grayscale image Gary ₁ to record the edge (refer to Equation 4). The threshold value T is preset to 100. Formula 4

公式6

Then, the sharpness ratio S _R is calculated with reference to the following formulas 5 and 6, and the enhancement ratio (Erancement Ratio) E _R is obtained according to the sharpness ratio S _R. Formula 5

Formula 6

Thereafter, the enhancement ratio E _R obtains the new reference threshold Equation 7, based on the obtained T. Then, a new threshold value T is used to obtain another edge image Edge, which is superimposed with the original image for optimization. Formula 7

In addition, after inputting the image corresponding to the human face to the search engine for searching, the processor 110 may further determine whether each of the searched images matches the human face in the region of interest. For example, the eye distance and the distance from the center to the mouth can be used for comparison. FIG. 5 is a schematic diagram of analyzing a human face according to an embodiment of the invention. Referring to FIG. 5, face detection is used to find the face area 510 in the image, and then the eye areas 511 and 512 and the mouth area 513 are further found in the face area. The distance L between the eye regions 511 and 512 is calculated, and the distance R from the center of the eye to the mouth region 513 is calculated.

Obtain distances L and R from the region of interest, and obtain a ratio R _ROI according to L:R. Moreover, the images searched by the search engine obtain distances L and R, and obtain a ratio R _Output according to L:R. When the comparison result meets the following formula 8, it is determined that the searched image and the region of interest are the same person. Formula 8

In addition, images corresponding to the region of interest can be input to multiple search engines, and the results obtained from each search engine can be integrated. For example, when searching for scenes, animals, plants, characters, etc., the search accuracy of different search engines is different. Therefore, integrate the data collected by various search engines, and compare the obtained data to aggregate the data with the highest accuracy (the most data) to the user.

In summary, the present invention provides a method for a user to circle a region of interest in a display screen in real time, and after automatically undergoing image processing, input an image to a search engine for searching, so as to improve the convenience of use. Moreover, before the search is performed, the resolution of the image is improved, and the accuracy of the search can be improved. In addition, the results of each search engine are consolidated, and the data with the highest accuracy is returned to the user, which can break the restrictions of using a single search engine in the past.

Although the present invention has been disclosed as above with examples, it is not intended to limit the present invention. Any person with ordinary knowledge in the technical field can make some changes and modifications without departing from the spirit and scope of the present invention. The scope of protection of the present invention shall be subject to the scope defined in the appended patent application.

100‧‧‧Electronic device 110‧‧‧Processor 120‧‧‧Storage unit 130‧‧‧Display 300‧‧‧Display screen 310,410‧‧‧Interest area 411,413‧‧‧Image 510‧‧‧ Face area 511, 512‧‧‧Eye area 513‧‧‧ Mouth area S205～S225‧‧‧Steps of image search method

FIG. 1 is a block diagram of an electronic device according to an embodiment of the invention. 2 is a flowchart of an image search method according to an embodiment of the invention. 3 is a schematic diagram of circled a region of interest according to an embodiment of the invention. 4A and 4B are schematic diagrams of circled regions of interest according to another embodiment of the invention. FIG. 5 is a schematic diagram of analyzing a human face according to an embodiment of the invention.

S205~S225‧‧‧‧Image search method steps

Claims (8)

An image search method includes: performing a circle selection action in a display screen; obtaining an area of interest from the display screen based on the circle selection action; performing a face detection on the area of interest; When the interest area has a human face, an image corresponding to the human face is obtained from the display screen; and the image corresponding to the human face is input to a search engine for searching. The image search method as described in item 1 of the patent application scope, wherein after obtaining the region of interest, further comprising: performing an image distortion correction. The image search method as described in item 1 of the patent application scope, wherein after obtaining the region of interest, further comprising: performing a resolution compensation on the image corresponding to the region of interest. The image search method described in item 1 of the scope of the patent application further includes: when it is determined that the region of interest does not have a human face, input the image corresponding to the region of interest to the search engine for searching. The image search method as described in item 1 of the patent application scope, wherein the step of performing the face detection on the region of interest includes: performing a skin color segmentation to obtain a skin color region; determining whether a human face exists in the skin color region Feature; and when there is a face feature in the skin color area, it is determined that the area of interest includes a face. The image search method described in item 1 of the scope of the patent application further includes: after inputting the image corresponding to the face to the search engine for searching, judging whether each of the searched images matches the area of interest The face of the person. The image search method as described in item 1 of the scope of the patent application further includes: when N faces are detected in the region of interest, N images based on the faces of the N persons are obtained from the display screen, wherein N is an integer greater than 1; and the N images are respectively input to the search engine for searching. The image search method as described in item 1 of the patent application scope, wherein the display screen includes one of a video screen, a picture and a browser screen.

Images