CN118116022A - Image processing method, intelligent terminal, device, medium and program product - Google Patents

Abstract

Provided are an image processing method of an intelligent terminal, an image processing method of a computing device, an electronic device, a non-transitory storage medium, and a computer program product. The image processing method of the intelligent terminal comprises the following steps: obtaining, by an intelligent terminal, a plurality of first images of a first resolution; transmitting the plurality of first images to a computing device connected to the intelligent terminal; receiving a recognition determination that the computing device recognized a first gesture in the plurality of first images and information related to a start point and an end point of a region of the plurality of first images pointed to by the first gesture; triggering the intelligent terminal to shoot a second image with a second resolution, and intercepting a region of interest in the second image according to information related to a start point and an end point of the region pointed by the first gesture, wherein the second resolution is higher than the first resolution; and the intelligent terminal sends the region of interest to the computing device so as to receive translated text which is recognized and translated by the computing device and contained in the region of interest.

Description

Image processing method, intelligent terminal, device, medium and program product

Technical Field

The present application relates to the field of smart terminals, and more specifically, to an image processing method for a smart terminal, a smart terminal, an image processing method for a computing device, a computing device, an electronic device, a non-temporary storage medium, and a computer program product.

Background technique

Traditional mobile smart devices are usually carried by mobile phones. With the development of the Internet of Everything, mobile smart devices are gradually transitioning to wearable smart devices. As a type of wearable device, smart glasses provide many smart functions, such as immersive visual rendering (virtual reality (VR), augmented reality (AR), mixed reality (MR), etc.), smart travel, smart navigation, photo translation, etc. Compared with mobile phones, smart glasses can easily obtain images according to the wearer's facial orientation. For example, in terms of photo translation, it is more convenient to use smart glasses than mobile phones.

Summary of the invention

According to one aspect of the present application, there is provided an image processing method for a smart terminal, comprising: obtaining, by the smart terminal, a plurality of first images of a first resolution; sending the plurality of first images to a computing device connected to the smart terminal; receiving a recognition judgment of the computing device recognizing a first posture in the plurality of first images and information related to the starting point and end point of an area pointed to by the first posture in the plurality of first images; triggering the smart terminal to capture a second image with a second resolution, and capturing an area of interest in the second image according to the information related to the starting point and end point of the area pointed to by the first posture, wherein the second resolution is higher than the first resolution; and sending the area of interest to the computing device by the smart terminal so as to receive a translation after the computing device recognizes and translates the text contained in the area of interest.

According to another aspect of the present application, a smart terminal is provided, comprising a shooting device and a transceiver, wherein the shooting device obtains multiple first images with a first resolution, and the transceiver sends the multiple first images to a computing device connected to the smart terminal; the transceiver receives a recognition judgment of the computing device that a first posture is recognized in the multiple first images and information related to the starting point and end point of the area pointed by the first posture in the multiple first images, the shooting device triggers shooting of a second image with a second resolution, and captures an area of interest in the second image according to the information related to the starting point and end point of the area pointed by the first posture, wherein the second resolution is higher than the first resolution; the transceiver sends the area of interest to the computing device so as to receive a translation after the computing device recognizes and translates the text contained in the image of interest.

According to another aspect of the present application, an image processing method is provided, comprising: receiving multiple first images of a first resolution sent from a smart terminal; recognizing a first posture in the multiple first images, and determining the starting point and end point of the area pointed by the first posture in the multiple first images according to the recognized first posture; triggering the smart terminal to shoot a second image using a second resolution and sending the starting point and end point of the area pointed by the first posture to the smart terminal, so that the smart terminal obtains an area of interest of a second resolution according to the starting point and end point of the area pointed by the first posture, wherein the area of interest is jointly determined by mapping the starting point and end point of the area pointed by the first posture in the first image to the second image of the second resolution; receiving the area of interest from the smart terminal so as to perform text recognition and translation according to the area of interest, wherein the second resolution is higher than the first resolution.

According to another aspect of the present application, a computing device is provided, comprising: a receiving device, configured to receive multiple first images of a first resolution sent from a smart terminal; a recognition device, configured to recognize a first posture in the multiple first images, and determine the starting point and end point of the area pointed by the first posture in the multiple first images according to the recognized first posture; a sending device, configured to trigger the smart terminal to shoot a second image using a second resolution, and send the starting point and end point of the area pointed by the first posture to the smart terminal, so that the smart terminal obtains an area of interest of a second resolution according to the starting point and end point of the area pointed by the first posture, wherein the area of interest is jointly determined by mapping the starting point and end point of the area pointed by the first posture in the first image to the second image of the second resolution; the area of interest is received from the smart terminal so as to perform text recognition and translation according to the area of interest, wherein the second resolution is higher than the first resolution.

According to another aspect of the present application, an electronic device is provided, including: a memory for storing instructions; a processor for reading the instructions in the memory and executing a method according to an embodiment of the present application.

According to another aspect of the present application, a non-transitory storage medium is provided, on which instructions are stored, wherein when the instructions are read by a processor, the processor executes a method according to an embodiment of the present application.

According to another aspect of the present application, a computer program product is provided, including computer instructions, wherein when the instructions are read by a processor, the processor is caused to execute the method according to the embodiment of the present application.

In this way, by transmitting a low-resolution first image and using the low-resolution first image to identify the pointed area, the part of the high-resolution second image involving the pointed area is translated instead of translating the entire image. This not only reduces the amount of transmitted data and the amount of data for identifying the pointed area, but also allows the pointed area to be translated in a targeted manner, while ensuring the accuracy of the translation of the pointed area.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

FIG. 1 shows a scene diagram of applying various embodiments of the present application.

FIG2 shows a flow chart of an image processing method for a smart terminal according to an embodiment of the present application.

FIG. 3 shows an example diagram of a first gesture in a case where a finger of a person's right hand is used as a pointing object according to an embodiment of the present application.

4A shows a flowchart of a process in which a computing device recognizes a first gesture in a plurality of first images and generates information related to a start point and an end point of an area pointed to by the first gesture in the plurality of first images according to an embodiment of the present application.

4B shows a flowchart of another process in which a computing device recognizes a first gesture in a plurality of first images and generates information related to a start point and an end point of an area pointed to by the first gesture in the plurality of first images according to an embodiment of the present application.

FIG5A shows a schematic diagram of using the start point and the end point of the area pointed by the first gesture as the upper left point and the lower right point of the rectangular frame of the area of interest, respectively, according to an embodiment of the present application.

5B shows a schematic diagram of using the start point and the end point of the area pointed by the first gesture as the lower left point and the lower right point of the rectangular frame of the area of interest (the rectangular frame of the area of interest has a predetermined height) according to an embodiment of the present application.

FIG6 shows a flow chart of a process of rendering and broadcasting a translation by a smart terminal according to an embodiment of the present application.

FIG. 7 shows a schematic diagram of rendering and broadcasting of translation by a smart terminal according to an embodiment of the present application.

FIG8 shows a flowchart of an image processing method of a computing device according to an embodiment of the present application.

FIG9 shows a block diagram of a smart terminal according to an embodiment of the present application.

FIG. 10 shows a block diagram of a computing device according to an embodiment of the present application.

FIG. 11 shows a block diagram of an exemplary electronic device suitable for implementing embodiments of the present application.

FIG. 12 shows a schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of the present application.

Detailed ways

Reference will now be made in detail to specific embodiments of the present application, examples of which are illustrated in the accompanying drawings. Although the present application will be described in conjunction with specific embodiments, it will be understood that it is not intended to limit the present application to the described embodiments. On the contrary, it is intended to cover changes, modifications and equivalents included in the spirit and scope of the present application as defined by the appended claims. It should be noted that the method steps described herein can all be implemented by any functional block or functional arrangement, and any functional block or functional arrangement can be implemented as a physical entity or a logical entity, or a combination of the two.

As a wearable device, smart glasses are designed to be fashionable and lightweight. The current touch-based physical interaction method of smart glasses is very inconvenient, and the computing power of smart glasses themselves is limited. Usually, they cannot directly perform reasoning calculations of artificial intelligence (AI) models on the glasses. In addition, considering the battery life of smart glasses, many time-consuming and computing-intensive processes are generally completed with the help of external device interconnection, such as the interconnection between smart glasses and external mobile phones.

In the existing photo translation process based on the interconnection of smart glasses and mobile phones, the smart glasses obtain the preview image taken and display it in real time. When photo translation is needed, the photographer aims at the object to be translated and clicks the touch pad on the side of the smart glasses to trigger the photo. The smart glasses obtain the photo image and then transmit it to the mobile phone. After the mobile phone performs image translation (including optical character recognition (OCR) and text translation), the translation result is returned to the smart glasses. After the smart glasses receive the translation result, they call the cloud text to speech (TTS) function to convert the text into speech, and then perform voice broadcast.

However, the existing entire photo translation process has the following problems:

(1) Image scaling affects the translation effect. Existing photo translation often transmits the entire image to the mobile phone through the Internet for image translation. The amount of data transmitted over the Internet is large. In order to reduce the time spent on the transmission link, sometimes images with high resolution are scaled to reduce the amount of data transmitted by reducing the resolution. However, the reduction in resolution will lead to a decrease in image clarity, affecting the text extraction results and further affecting the translation results.

(2) Redundant data is transmitted, and backfilling is deformed. Existing image translation is to transmit the entire image to the mobile phone for OCR text recognition and translation of the text in the image. However, in actual business, users may only need to translate part of the text content in the image. In addition, existing image translation involves post-processing of backfilling the translation. After translation, the character lengths of the source language and the target language may be inconsistent, resulting in deformation of the filling caused by backfilling the target language translation into the source language position.

(3) Translation speed is affected and the amount of data transmitted back is large. The existing image translation is to perform OCR text recognition and translation on the entire image. The translation speed is slow and the number of translated text results returned is large, resulting in a large amount of data transmitted back to the Internet.

(4) User operations are complex and time-consuming. Usually, after the glasses obtain the translation results, they need to call the cloud TTS to broadcast the text. The existing processing method is to display each TTS text in a list, and the user uses the touchpad to slide up and down to broadcast the selection. In the scenario where there are many texts involved in the entire image, the user needs to find the text he wants to translate before he can broadcast it.

In view of the shortcomings of the existing photo translation technology for interconnecting smart glasses with mobile phones, the present disclosure proposes an image processing method for a smart terminal, including: obtaining a plurality of first images with a first resolution by the smart terminal; sending the plurality of first images to a computing device connected to the smart terminal; receiving a recognition judgment of the computing device that a first posture is recognized in the plurality of first images and information related to the starting point and end point of the area pointed to by the first posture in the plurality of first images; triggering the smart terminal to capture a second image with a second resolution, and capturing an area of interest in the second image according to the information related to the starting point and end point of the area pointed to by the first posture, wherein the second resolution is higher than the first resolution; and sending the second image to the computing device by the smart terminal so as to receive a translation after the computing device recognizes and translates the text contained in the second image of the area of interest.

In this way, by transmitting a low-resolution first image and using the low-resolution first image to identify the pointed area, the part of the high-resolution second image involving the pointed area is translated instead of translating the entire image. This not only reduces the amount of transmitted data and the amount of data for identifying the pointed area, but also allows the pointed area to be translated in a targeted manner, while ensuring the accuracy of the translation of the pointed area.

FIG. 1 shows a scene diagram of applying various embodiments of the present application.

As shown in Figure 1, the smart terminal 110 is, for example, smart glasses, on which a camera 111 for capturing images is arranged. The computing device 120 is, for example, a mobile terminal, such as a mobile phone, a tablet computer, etc. In some embodiments, the computing device 120 may also be a cloud computer (e.g., a cloud server).

When applying various embodiments of the present application, the camera 111 of the smart terminal 110 can obtain continuous preview images (such as the third image mentioned later in this article) of the medium 130 (such as paper, book pages, electronic screens, etc.). The continuous preview images can be displayed in real time on the smart terminal at a preview resolution (such as the third resolution mentioned later in this article, such as 1080*720) (such as displaying in VR, AR, etc. or displaying on glasses, etc.). At the same time, the continuous preview images can be scaled into continuous thumbnails (such as the first image mentioned later in this article, with the first resolution mentioned later in this article, such as 320*240), and the continuous thumbnails can be transmitted to the computing device 120 so that the computing device 120 can recognize the specific gesture (such as the translation gesture) of the pointing object (such as a hand) 140 in the thumbnail, and identify the region of interest 131 to be translated pointed by the pointed object, and inform the smart terminal 110 of the information about the region of interest 131 to be translated.

The camera 111 of the smart terminal 110 (with a second resolution mentioned later in this document, for example, 1920*1080) can capture an image of an object to be translated (for example, an area of interest 131 to be translated on the medium 130), and can capture a portion of the area of interest 131 in Figure 1 (for example, a second image mentioned later in this document) and transmit it to the computing device 120.

After completing the translation of the portion of the image (including optical character recognition (OCR) and text translation), the computing device 120 returns the translation result to the smart terminal 110 .

After the smart terminal 110 receives the translation result, it can call the text-to-speech (TTS) function of the cloud or local or other devices (for example, the computing device 120) to convert the translation result into speech, and then render and display the translation result on the smart terminal 110 and perform a synchronous voice broadcast.

Next, the specific details of various embodiments of the present disclosure are further described in detail in conjunction with the drawings and embodiments.

FIG. 2 shows a flowchart of an image processing method 200 of a smart terminal according to an embodiment of the present application.

Here, the smart terminal may include a wearable device with a camera or a camera, such as smart glasses. In this article, smart glasses are used as an example for description. Relevant applications may be installed on the smart terminal to perform functions such as setting, display, taking pictures, transmitting, and voice broadcasting.

Here, the computing device may include a mobile terminal (or a cloud server) with processing capabilities such as a mobile phone or a tablet computer, and related applications may be installed on the computing device to perform functions such as image recognition, image-to-text, and text translation. In this article, a mobile phone is used as an example for description.

After the user wears the smart terminal such as smart glasses, the camera on the smart glasses can obtain images of the objects in front of the smart glasses. If the user is reading a foreign article such as English on a medium such as a book or a tablet, the camera on the smart glasses can obtain multiple continuous images of the medium including the English article. If the user points to a specific English sentence or paragraph on the medium with a finger or other pointing object such as a pointing pen, the camera on the smart glasses can obtain multiple images of the medium including the English article and the user's finger or pointing pen.

For example, the preview attribute of the camera of the smart glasses and the photo taking attribute of the camera of the smart glasses can be set in advance on the smart terminal. Here, "preview" refers to continuously acquiring the real-time image acquired by the camera and continuously displaying it as a preview image, which can be regarded as displaying a dynamic image. "Photo taking" refers to triggering the capture of (at least one) still image.

The preview attributes mainly include the resolution of the preview image, the sampling frame rate of the preview image, etc. The photo attributes mainly include the resolution of the photo image, etc.

Assuming that a user sees an English word, sentence or paragraph that he wants to translate, the most convenient way for the user is to use his finger or a pointing object such as a stylus to mark (for example, from left to right) the English part that he wants to translate. Therefore, in this article, it is hoped that some means can be used to recognize the translation posture (gesture) of the pointing object in real time to trigger text OCR recognition and translation of the English part.

In this article, it is hoped that a specific gesture such as a translation gesture can be recognized in real time at a computing device based on a preview image. If the resolution is set too low, the near-eye display of the smart glasses will not be clear. However, if the resolution is set too high, a large amount of data may be transmitted to the computing device for interconnection, resulting in a slow transmission speed and failure to reach the frame rate of the translation gesture recognition. Taking all factors into consideration, it is preferred that the resolution of the preview image (such as the third resolution in this article) is generally set to 1080*720 to meet the near-eye display clarity.

Before transmitting to the computing device, the preview image (such as the third image in this article) can be scaled to obtain a thumbnail (such as the first image in this article) of low resolution (such as the first resolution in this article) and then transmitted to reduce the amount of transmitted data and ensure a faster image transmission frame rate. The resolution of the thumbnail only needs to be able to meet the translation gesture recognition. Therefore, in practice, the resolution of the thumbnail can be set to 320*240, and the transmission frame rate can be 25 to 30fps, so that the key point detection and translation gesture recognition of the translation gesture can be completed normally.

Considering that OCR and translation need to be performed based on the photographed image (such as the second image in this article), the resolution can be set as large as possible to ensure clarity for more accurate OCR. In actual use, the resolution of the photographed image can be set to 1920*1080 (such as the second resolution in this article).

It can be seen that the second resolution of the photographed image is greater than the third resolution of the preview image, and the third resolution of the preview image is greater than the first resolution of the thumbnail of the preview image.

Therefore, in one embodiment, before step 210, the smart terminal captures multiple third images for preview in real time with a third resolution (e.g., preview images for preview); and thumbnails the multiple third images into multiple first images (e.g., thumbnails for sending to a computing device for specific gesture recognition), wherein the third resolution is greater than the first resolution and less than the second resolution.

The interconnection channel between the smart terminal and the computing device can be established when the smart terminal and the computing device are turned on, or before step 210, or before step 220 or when needed. The interconnection method can include short-range wireless communication such as Bluetooth.

As shown in FIG. 2 , the image processing method 200 at least includes steps 210 to 250 .

In step 210, a plurality of first images with a first resolution are obtained by the smart terminal.

As described above, the plurality of first images may be scaled from the plurality of (dynamically continuous) third images as preview images, where scaling refers to reducing the resolution.

Of course, the preview image can also be directly used as the first image without performing the scaling step, which may depend on the resolution of the preview image and the transmission rate requirements (for example, the resolution of the preview image is already small, or the transmission bandwidth between the smart terminal and the computing device is large enough to quickly transmit a large number of preview images, etc.).

In step 220 , a plurality of first images are sent to a computing device connected to the smart terminal.

Here, since the computing resources and computing power of a computing device such as a mobile phone are generally higher than those of an intelligent terminal such as smart glasses, a computing device can be used to perform recognition calculations for gesture recognition. The computing device can recognize a first gesture in a plurality of first images, and determine the start and end points of the area pointed to by the first gesture in the plurality of first images. The first gesture can be set to a translation action that can reflect the user's intention to translate a portion of the text. Various image recognition algorithms can be used to perform gesture recognition, including various machine learning algorithms, etc.

In this article, it is assumed that when a user wants to translate a part of a text, the user will usually extend at least one finger (such as the index finger) or use a pointing pen to stay at the starting point position of the part of the text to be translated for a period of time, and then slide the index finger or the pointing pen until the text that the user does not want to translate is reached. At the end point, the user will use the index finger or the pointing pen to stay at the ending point position of the part of the text to be translated for a period of time, and use the starting point position and the ending point position to indicate the part of the text that the user is interested in translating in the entire text.

Based on the above assumptions, multiple recognition methods can be designed to recognize the user's translation intention for a portion of text.

In one embodiment, when a human finger is used as the pointing object, the first gesture may include at least one finger extending out and staying at a fixed position in the plurality of first images for more than a first time threshold.

FIG. 3 shows an example diagram of a first gesture in a case where a finger of a person's right hand is used as a pointing object according to an embodiment of the present application.

As shown in Figure 3, this is the right hand of a person, and the index finger of the right hand is extended, which is a common pointing habit of a person's hand. Of course, the present disclosure is not limited to the right hand of a person, and if a person is used to using the left hand, identification can also be performed in a similar manner as described herein.

In order to accurately recognize this gesture, 21 parts of the hand can be set as 21 key points (not shown in the figure). For simplicity, only key points 8, 5, and 17 of the hand are marked in Figure 3, where key point 8 is located at the nail of the right index finger, key point 5 is located at the metacarpophalangeal joint of the index finger, and key point 17 is located at the metacarpophalangeal joint of the little finger.

If the following two conditions are met in the image, it can be considered that the "index finger extended" state (which can be called a translation gesture) has been recognized: (1) key point 8 is at the upper left of the other 20 key points (i.e., the remaining key points of the hand except key point 8), that is, the horizontal coordinate and vertical coordinate of point 8 in the image are both minimum values among the 21 key points; (2) key point 5 is to the left of key point 17, that is, the horizontal coordinate of key point 5 in the image is smaller than the horizontal coordinate of key point 17.

Of course, the above recognition of the "index finger extended" state adopts the key point method, but the present disclosure is not limited to this. In fact, other methods of recognizing the "index finger extended" state are also possible, such as using some machine learning algorithms, etc. It is also possible to recognize that other fingers are extended, or recognize that at least one finger is extended, etc.

After the "index finger extended" state is identified, it is further determined that the index finger stays at a fixed position of the plurality of first images for more than a first time threshold. The first time threshold is set to prevent misidentification when the user has no intention of translating but just taps a certain place with the finger and does not stay there for a long enough time.

In the case where a pointing pen is used as the pointing object, the first gesture may include the pen tip of the pointing pen staying at a fixed position of the plurality of first images for more than a predetermined time threshold. In this case, the pen tip of the pointing pen may be identified, and it may be determined that the pen tip stays at a fixed position of the plurality of first images for more than a predetermined time threshold.

Here, the predetermined time threshold may be set to t _s , and the value range may be, for example, [500, 2000], and the unit is ms. Of course, this is only an example, and the time threshold may also be other values.

To determine whether the index finger (e.g., key point 8) or the pen tip stays at a fixed position in the plurality of first images, it is possible to detect whether the coordinates of the index finger tip (e.g., key point 8) or the pen tip stay in a small area. This is because the index finger or finger tip is not necessarily completely still, and a circular area with a radius of 20 to 50 pixels and the coordinates of the key point 8 where the "translation gesture" is first detected within the _{t s} period can usually be taken as the center of the circle. If within the r _s period, the coordinates of the key point 8 of the translation gesture are always within this circular area, it can be considered that the translation gesture (index finger extension or pen tip) stays at a fixed position in the image for more than the time r _s , and it is determined that the first gesture is recognized.

Of course, the first posture of the above translation action can also be set to other postures and recognized in other ways, which are not listed here one by one. The above method of recognizing the extension of the index finger is not a limitation, and other methods can be used to recognize the extension of the index finger. In addition, the pointing object is not limited to the index finger or other fingers and the pointing pen, and other pointing objects are also applicable to various embodiments of the present disclosure.

In order to avoid misrecognition and waste of resources, the recognition of the first gesture may be enabled after the smart terminal is set to the image translation mode.

In addition, when the computing device does not recognize the first gesture indicating the translation action, other gesture recognition processes may be performed or recognition may be stopped directly.

Next, as described above, because the user may only want to translate a portion of the text, rather than the entire page of text, it is necessary to determine which portion of the text the user is interested in.

It can be assumed that the pointed area where the user passes by the pointing object is the translation area that the user is interested in. Therefore, after the computing device recognizes the recognition judgment of the first posture in multiple first images, the starting point and end point of the area pointed by the first posture in the multiple first images (also called the pointed area) can also be determined to determine the translation area that the user is interested in.

4A shows a flowchart of a process 400 in which a computing device recognizes a first gesture in a plurality of first images and generates information related to a start point and an end point of an area pointed to by the first gesture in the plurality of first images according to an embodiment of the present application.

As shown in FIG. 4A , in one embodiment, the computing device recognizes a first posture in a plurality of first images and determines information related to a start point and an end point of an area pointed to by the first posture in the plurality of first images, including: the computing device performs the following steps.

Here, if the computing device determines that the intelligent terminal has set the image translation mode, it can be initialized, for example, setting the state of the first gesture to the first state. The first state can be a set "no focus" state to indicate that the current is a new round of gesture recognition process to avoid conflict with the previous gesture recognition process.

At step 410 , a first gesture is identified in a plurality of first images.

Since the first gesture may be made by the user at the starting point of the pointed area or at the end point of the pointed area, it is necessary to determine whether the first gesture at this time is at the starting point or the end point. Specifically, in step 420, if the first gesture has not been recognized before (for example, it is the first recognition) or the state of the first gesture is the first state (i.e., a new round of gesture recognition after initialization), the position pointed to by the first gesture in the multiple first images at this time is recorded as the first starting point, and the state of the first gesture at this time is set to the second state.

Here, the position indicated by the first posture in the first state may be the center position of the aforementioned circular area.

Here, the second state may be a “focus acquisition” state.

In step 430 , if the first gesture is recognized again in the plurality of first images and the state of the first gesture is the second state, the position indicated by the first gesture in the plurality of first images at this time is recorded as the first end point.

Here, the position indicated by the first posture of the second state may be the center position of the aforementioned circular area.

In this way, the starting point and the end point of the pointed area can be determined by determining whether the first gestures recognized twice are in the first state or in the second state.

In step 440, the state of the first posture is set to the first state, which means that this round of recognition process has been completed and a new round of recognition process begins.

Next, since the resolutions of the first image, the second image, and the third image are different, the resolutions of the coordinate systems of each image are also different. Therefore, the coordinates of the start and end points of the pointed area identified in the first image are different from the coordinates of the corresponding positions in the corresponding third image (preview image) and the second image of the second resolution (image taken with high resolution), and coordinate conversion is required.

For example, the image resolution of the third image (preview image) is recorded as w _p *h _p , the coordinates of a point in the coordinate system of the third image (preview image) are recorded as (x _p , y _p )), the image resolution of the first image (thumbnail) after the third image (preview image) is scaled is recorded as w _r *h _r , the coordinates in the coordinate system are recorded as (x _r , y _r )), the image resolution of the second image (image taken with high resolution) with the second resolution is recorded as w _t *h _t , the coordinates in the coordinate system are recorded as (x _t , y _t ). Under the same camera, the coordinate systems satisfy the following:

Specifically, in step 450, according to the coordinate mapping relationship between multiple first images of the first resolution and the second image of the second resolution (such as the above formula 1), the coordinates of the first starting point and the first end point are respectively converted into the coordinates of the second starting point and the second end point in the second image.

In this way, based on the coordinates of the second starting point and the second end point in the second image of the second resolution, the part of the image (i.e., the area of interest) of the second resolution (i.e., high resolution) that the user is actually interested in translating can be obtained for OCR and translation.

In step 460, the coordinates of the second starting point and the second end point are sent to the smart terminal as the starting point and the end point of the area pointed by the first gesture.

Of course, in another embodiment, step 450 and step 460 may be changed to sending the coordinates of the first starting point and the first end point in the first image (thumbnail) to the smart terminal, so that in the smart terminal, according to the coordinate mapping relationship between the multiple first images of the first resolution and the second image of the second resolution (for example, the above formula 1), the coordinates of the first starting point and the first end point are respectively converted into the coordinates of the second starting point and the second end point in the second image. Here, in order to summarize the above multiple embodiments (sending the coordinates of the first starting point and the first end point to the smart terminal or sending the coordinates of the second starting point and the second end point to the smart terminal), the coordinates of the first starting point and the first end point and the coordinates of the second starting point and the second end point can be summarized as information related to the start point and the end point of the area pointed to by the first gesture in the multiple first images. This embodiment is explained below in conjunction with Figure 4B.

Figure 4B shows a flowchart of another process 400' in which a computing device according to an embodiment of the present application recognizes a first gesture in multiple first images and identifies information related to the start point and end point of the area pointed to by the first gesture in the multiple first images.

At step 410', a first gesture is identified in the plurality of first images.

In step 420', if the first posture has not been recognized previously or the state of the first posture is the first state, the position indicated by the first posture in the multiple first images at this time is recorded as the first starting point, and the state of the first posture at this time is set to the second state.

In step 430', if the first posture is recognized again in the multiple first images and the state of the first posture is the second state, the position indicated by the first posture in the multiple first images at this time is recorded as the first end point.

In step 440', the state of the first posture is set to the first state.

In step 450', the coordinates of the first starting point and the first end point are sent to the smart terminal.

Next, returning to FIG. 2 , in step 230 , the intelligent terminal receives a recognition judgment that the computing device recognizes a first gesture in a plurality of first images and information related to a start point and an end point of an area pointed to by the first gesture in the plurality of first images.

The information related to the starting point and the end point of the area pointed by the first gesture may include the coordinates of the first starting point and the first end point without conversion, or the coordinates of the second starting point and the second end point converted into the coordinates of the second image. Note that if the information related to the starting point and the end point of the area pointed by the first gesture includes the coordinates of the first starting point and the first end point without conversion, then the smart terminal: according to the coordinate mapping relationship between the multiple first images of the first resolution and the second image of the second resolution, the coordinates of the first starting point and the first end point are converted into the coordinates of the second starting point and the second end point in the second image as the starting point and the end point of the area pointed by the first gesture.

Next, in step 240, the smart terminal is triggered to capture a second image with a second resolution, and an area of interest is captured in the second image according to information related to the start and end points of the area pointed to by the first gesture, wherein the second resolution is higher than the first resolution.

Specifically, after receiving the recognition judgment of the computing device that a first posture is recognized in multiple first images and the information related to the starting point and end point of the area pointed by the first posture in the multiple first images, the intelligent terminal: determines a region of interest (ROI) according to the starting point and end point of the area pointed by the first posture; captures the region of interest with a second resolution, and sends it to the computing device so that the computing device can recognize and translate according to the region of interest.

In one embodiment, determining the area of interest based on the starting point and end point of the area pointed by the first gesture includes: using the starting point and end point of the area pointed by the first gesture as the upper left point and lower right point of the rectangular frame of the area of interest, respectively; or using the starting point and end point of the area pointed by the first gesture as the lower left point and lower right point of the rectangular frame of the area of interest, respectively, and the rectangular frame of the area of interest has a predetermined height.

FIG5A shows a schematic diagram of using the start point and the end point of the area pointed by the first gesture as the upper left point and the lower right point of the rectangular frame of the area of interest, respectively, according to an embodiment of the present application.

As shown in FIG5A , it is assumed that the user is accustomed to or is educated to develop the habit of sliding at least one finger from the starting point at the upper left corner of the area of interest to the end point at the lower right corner, and using the starting point and the end point as the upper left point and lower right point of the rectangular frame of the area of interest, respectively.

5B shows a schematic diagram of using the start point and the end point of the area pointed by the first gesture as the lower left point and the lower right point of the rectangular frame of the area of interest (the rectangular frame of the area of interest has a predetermined height) according to an embodiment of the present application.

As shown in FIG5B , it is assumed that the user is accustomed to or the user is educated to form a habit of sliding at least one finger from the starting point at the lower left corner of the region of interest to the end point at the lower right corner, and the starting point and the end point are used as the lower left point and the lower right point of the rectangular frame of the region of interest, respectively, and it is assumed that the rectangular frame has a predetermined height. The predetermined height can be one or more lines.

Of course, the above two methods are only exemplary methods of determining the region of interest through the starting point and the end point, but the present disclosure is not limited thereto, and the region of interest may be determined in other ways, and the region of interest may have a variety of shapes.

Referring back to FIG. 2 , in step 250 , the intelligent terminal sends the pointed area (ie, the region of interest) in the second image to the computing device so that the receiving computing device can recognize and translate the text contained in the region of interest and obtain the translated text.

In this way, by transmitting a low-resolution first image and obtaining the pointed area identified by the low-resolution first image, the translation result of the part of the pointed area in the high-resolution second image is obtained, which not only reduces the amount of transmitted data, but also reduces the amount of data for identifying the pointed area, and only translates the pointed area in a targeted manner, while ensuring the accuracy of the translation of the pointed area.

In the process of the computing device identifying and translating the region of interest, the computing device performs the following steps: identifying text from the region of interest; translating the identified text; and sending the translated text to the smart terminal.

Next, the smart terminal renders and broadcasts the translated text.

Fig. 6 shows a flow chart of a process 600 of rendering and broadcasting a translation by a smart terminal according to an embodiment of the present application. Fig. 7 shows a schematic diagram of rendering and broadcasting a translation by a smart terminal according to an embodiment of the present application.

In step 601, according to the coordinate mapping relationship between the multiple first images of the first resolution and the multiple third images of the third resolution, the coordinates of the first starting point and the first end point in the multiple first images are respectively converted into the coordinates of the third starting point and the third end point in the image of the third resolution (as shown by the two black dots in Figure 7).

Here, as mentioned above, the third image is a preview image. Since it is desired to render the portion of the translation in the region of interest in the preview image that the user can see on the smart glasses, the coordinates of the region of interest in the preview image must be obtained.

In step 602, the third starting point and the third end point and the line between the third starting point and the third end point (such as the diagonal line of the region of interest shown in FIG7 ) are marked in the third image, and the area jointly determined by the third starting point and the third end point is rendered as a translation area. The text to be translated is included in the translation area.

In step 603, a translation result frame (as shown in FIG7 ) is superimposed on the translation area in the third image for preview. As shown in FIG7 , the translation result frame can be superimposed in a pop-up window form in a pop-up window image area.

In step 604, the translated text is displayed in a translation result box.

When a synchronous voice broadcast of the translation is required, the smart terminal can also perform the following steps.

In step 605, initially, the translated text is initialized to a non-broadcast state. The micro-broadcast state may cause the translated text to be in a text format, such as gray.

In step 606, the corresponding speech audio is obtained according to the translated text.

In step 607, the length of the translation and the length of the speech audio in the unbroadcasted state are obtained.

In step 608, the voice audio is broadcasted. Here, the translated text can be used to call a TTS service (eg, a mobile phone or a cloud) to generate the voice audio corresponding to the translated text.

In step 609, the duration of the currently broadcasted voice audio (denoted as t _s ) is obtained.

In step 610, the length of the translated text that has been announced (referred to as l r ) is calculated based on the length of the unannounced translation (referred to as l _c ) (e.g., the number of characters), the length of the speech audio (referred to as _{t s )} , and the duration of the currently announced speech audio (referred to as _tu ). The calculation formula is as follows:

In step 611, the translated text of the length that has been broadcast (the broadcast translated text shown in FIG. 7 ) is rendered synchronously with the currently broadcast voice audio. For example, it can be rendered in a highlighted text color, or in a bold format, etc. The remaining translated text is the text to be broadcast as shown in FIG. 7 .

In step 612, if the voice audio broadcast is completed, the broadcast is terminated. At this time, the translation result box can also be set to automatically disappear after the voice broadcast is completed. Of course, the display of the translation result box can also be terminated in advance through other process controls.

In step 613, if the voice audio broadcast is interrupted due to the recognition of the first gesture of the pointing object, the broadcast is terminated and a new round of first gesture recognition process of the pointing object can be started.

In this way, the advantage of using the translation result box display is that it can only focus on the translation area that the user is interested in, and the translation result display has less content, and the voice broadcast only processes the translation area of interest selected by the user, rather than the translation result of the entire image content, avoiding the user's broadcast selection operation. Moreover, according to an embodiment of the present application, the voice is broadcast synchronously with the translation rendering, allowing the user to see the translation more intuitively.

Thus, according to the embodiments of the present application, the amount of image data transmitted between devices can be reduced by transmitting low-resolution images and identifying specific postures in the low-resolution images, and the translation area in the high-resolution image is obtained as the region of interest for transmission based on the starting point and end point of the specific posture identified in the low-resolution image, thereby reducing the amount of image data transmitted between devices. Furthermore, OCR and translation can be performed from the high-resolution image of the region of interest taken, ensuring the clarity and translation accuracy of the translation area. Only translating the region of interest reduces the amount of OCR and translation calculations and the amount of data transmitted back from the interconnection. The translation result of the region of interest is the translation content that the user is concerned about, and after obtaining the translated text, the smart glasses can directly render and perform speech synthesis broadcasting, reducing the operation of user selection broadcasting.

FIG. 8 shows a flowchart of an image processing method 800 at a computing device according to an embodiment of the present application.

The computing device includes, for example, a mobile phone, a tablet computer, a computer, and a cloud server.

As shown in FIG8 , the image processing method 800 includes: step 810, receiving a plurality of first images of a first resolution sent from a smart terminal; step 810, identifying a first posture in the plurality of first images, and determining the starting point and the end point of the area pointed to by the first posture in the plurality of first images according to the identified first posture; step 810, triggering the smart terminal to shoot a second image using a second resolution, and sending the starting point and the end point of the area pointed to by the first posture to the smart terminal, so that the smart terminal obtains a region of interest of a second resolution according to the starting point and the end point of the area pointed to by the first posture, wherein the region of interest is jointly determined by mapping the starting point and the end point of the area pointed to by the first posture in the first image to the second image of the second resolution; step 810, receiving the region of interest from the smart terminal for identification and translation according to the region of interest, wherein the second resolution is higher than the first resolution.

Thus, according to the embodiments of the present application, the amount of image data transmitted between devices can be reduced by transmitting low-resolution images and identifying specific postures in the low-resolution images, and the translation area in the high-resolution image is obtained as the region of interest for transmission based on the starting point and end point of the specific posture identified in the low-resolution image, thereby reducing the amount of image data transmitted between devices. Furthermore, OCR and translation can be performed from the high-resolution image of the region of interest taken, ensuring the clarity and translation accuracy of the translation area. Only translating the region of interest reduces the amount of OCR and translation calculations and the amount of data transmitted back from the interconnection. The translation result of the region of interest is the translation content that the user is concerned about, and after obtaining the translated text, the smart glasses can directly render and perform speech synthesis broadcasting, reducing the operation of user selection broadcasting.

FIG. 9 shows a block diagram of a smart terminal 900 according to an embodiment of the present application.

Here, the smart terminal 900 may include smart glasses and the like.

As shown in Fig. 9, the smart terminal 900 includes a photographing device 910 and a transceiver 920. The photographing device 910 obtains a plurality of first images of a first resolution, and the transceiver 920 sends the plurality of first images to a computing device connected to the smart terminal 900.

The transceiver 920 receives a recognition judgment of a computing device recognizing a first posture in a plurality of first images and information related to the start point and end point of the area pointed to by the first posture in the plurality of first images, and the shooting device 910 triggers shooting of a second image with a second resolution, and captures an area of interest in the second image based on the information related to the start point and end point of the area pointed to by the first posture, wherein the second resolution is higher than the first resolution.

The transceiver 920 sends the second image to the computing device so as to receive the translated text after the computing device recognizes and translates the second image of the text contained in the region of interest.

Fig. 10 shows a block diagram of a computing device 1000 according to an embodiment of the present application. Here, the computing device 1000 may include a mobile terminal such as a mobile phone, a tablet computer, etc.

The computing device 1000 includes: a receiving device 1010, configured to receive multiple first images of a first resolution from a smart terminal; an identifying device 1020, configured to identify a first posture in the multiple first images, and determine the starting point and end point of the area pointed by the first posture in the multiple first images according to the identified first posture; a sending device 1030, configured to trigger the smart terminal to shoot a second image using a second resolution, and send the starting point and end point of the area pointed by the first posture to the smart terminal, so that the smart terminal obtains a region of interest of a second resolution according to the starting point and end point of the area pointed by the first posture, wherein the region of interest is jointly determined by mapping the starting point and end point of the area pointed by the first posture in the first image to the second image of the second resolution; receiving the region of interest from the smart terminal for identification and translation according to the region of interest, wherein the second resolution is higher than the first resolution.

In this way, by transmitting a low-resolution first image and obtaining the pointed area identified by the low-resolution first image, the translation result of the part of the pointed area in the high-resolution second image is obtained, which not only reduces the amount of transmitted data, but also reduces the amount of data for identifying the pointed area, and only translates the pointed area in a targeted manner, while ensuring the accuracy of the translation of the pointed area.

FIG. 11 shows a block diagram of an exemplary electronic device suitable for implementing embodiments of the present application.

The electronic device may include a processor (H1); a storage medium (H2) coupled to the processor (H1) and storing computer executable instructions therein for performing the steps of each method of an embodiment of the present application when executed by the processor.

The processor (H1) may include, but is not limited to, one or more processors or microprocessors, etc.

The storage medium (H2) may include, but is not limited to, for example, random access memory (RAM), read-only memory (ROM), flash memory, EPROM memory, EEPROM memory, registers, computer storage media (such as hard disk, floppy disk, solid-state drive, removable disk, CD-ROM, DVD-ROM, Blu-ray disc, etc.).

In addition, the electronic device may also include (but not limited to) a data bus (H3), an input/output (I/O) bus (H4), a display (H5), and input/output devices (H6) (for example, a keyboard, a mouse, a speaker, etc.), etc.

The processor (H1) may communicate with external devices (H5, H6, etc.) through an I/O bus (H4) via a wired or wireless network (not shown).

The storage medium (H2) may also store at least one computer executable instruction for executing the various functions and/or method steps in the embodiments described in the present technology when the instruction is executed by the processor (H1).

In one embodiment, the at least one computer executable instruction may also be compiled into or constitute a software product, wherein one or more computer executable instructions are executed by a processor to perform the various functions and/or method steps in the embodiments described in the present technology.

FIG. 12 shows a schematic diagram of a non-transitory computer-readable storage medium according to an embodiment of the present application.

As shown in FIG12 , instructions are stored on a computer-readable storage medium 1220, and the instructions are, for example, computer-readable instructions 1210. When the computer-readable instructions 1210 are executed by a processor, the various methods described above can be executed. The computer-readable storage medium includes, but is not limited to, for example, volatile memory and/or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and/or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. For example, the computer-readable storage medium 1220 may be connected to a computing device such as a computer, and then, when the computing device runs the computer-readable instructions 1210 stored on the computer-readable storage medium 1220, the various methods described above may be performed.

Note that the advantages, strengths, effects, etc. mentioned in this disclosure are only examples and not limitations, and it cannot be considered that these advantages, strengths, effects, etc. are required by each embodiment of this application. In addition, the specific details disclosed above are only for the purpose of illustration and facilitation of understanding, not limitation, and the above details do not limit this application to being implemented by adopting the above specific details.

The block diagrams of the devices, apparatuses, equipment, and systems involved in this disclosure are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. As will be appreciated by those skilled in the art, these devices, apparatuses, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "including," "comprising," "having," and the like are open words, referring to "including but not limited to," and can be used interchangeably therewith. The words "or" and "and" used herein refer to the words "and/or," and can be used interchangeably therewith, unless the context clearly indicates otherwise. The word "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably therewith.

The step flow charts and the above method descriptions in this disclosure are only illustrative examples and are not intended to require or imply that the steps of each embodiment must be performed in the order given. As will be appreciated by those skilled in the art, the order of the steps in the above embodiments can be performed in any order. Words such as "thereafter", "then", "next", etc. are not intended to limit the order of steps; these words are only used to guide the reader through the description of these methods. In addition, any reference to a singular element, such as using the article "one", "an", or "the", is not to be construed as limiting the element to the singular.

In addition, the steps and devices in the various embodiments of this document are not limited to being implemented in a certain embodiment. In fact, according to the concept of this application, relevant partial steps and partial devices in the various embodiments of this document can be combined to conceive new embodiments, and these new embodiments are also included in the scope of this application.

Each operation of the method described above may be performed by any suitable means capable of performing the corresponding functions. The means may include various hardware and/or software components and/or modules, including but not limited to hardware circuits, application specific integrated circuits (ASICs) or processors.

The various illustrated logic blocks, modules and circuits may be implemented or described using a general purpose processor, a digital signal processor (DSP), an ASIC, a field programmable gate array signal (FPGA) or other programmable logic device (PLD), discrete gate or transistor logic, discrete hardware components or any combination thereof designed to perform the functions described herein. A general purpose processor may be a microprocessor, but as an alternative, the processor may be any commercially available processor, controller, microcontroller or state machine. The processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, multiple microprocessors, a microprocessor cooperating with a DSP core, or any other such configuration.

The methods disclosed herein include gestures for implementing the described methods. Methods and/or gestures may be interchangeable with one another without departing from the scope of the claims. In other words, unless a specific order of gestures is specified, the order and/or use of specific gestures may be modified without departing from the scope of the claims.

The above functions can be implemented by hardware, software, firmware or any combination thereof. If implemented in software, the functions can be stored as instructions on a tangible computer-readable medium. The storage medium can be any available tangible medium that can be accessed by a computer. By way of example and not limitation, such a computer-readable medium may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage device or any other tangible medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer. As used herein, a disk and a disc include a compact disc (CD), a laser disc, an optical disc, a digital versatile disc (DVD), a floppy disc and a blue disc, wherein the disc usually reproduces data magnetically, and the disc reproduces data optically using a laser.

Therefore, the present disclosure may also include computer program products, wherein the computer program products can perform the methods, steps and operations presented herein. For example, such computer program products can be computer software packages, computer code instructions, computer-readable tangible media having computer instructions tangibly stored (and/or encoded) thereon, which instructions can be executed by a processor to perform the operations described herein. The computer program product can include packaging materials.

In addition, the modules and/or other appropriate means for performing the methods and techniques described herein can be downloaded and/or otherwise obtained by the user terminal and/or base station when appropriate. For example, such a device can be coupled to a server to facilitate the transmission of the means for performing the methods described herein. Alternatively, the various methods described herein can be provided via a storage component so that the user terminal and/or base station can obtain the various methods when the storage component is coupled to the device or provided to the device. In addition, any other appropriate technology for providing the methods and techniques described herein to the device can be utilized.

The above description has been given for the purpose of illustration and description. In addition, this description is not intended to limit the embodiments of the present application to the forms disclosed herein. Although multiple example aspects and embodiments have been discussed above, those skilled in the art will recognize certain variations, modifications, changes, additions and sub-combinations thereof.

Claims (14)

1. An image processing method for an intelligent terminal, comprising: The smart terminal obtains a plurality of first images with a first resolution; Sending the plurality of first images to a computing device connected to the smart terminal; receiving a recognition determination that the computing device recognizes a first gesture in the plurality of first images and information related to a start point and an end point of an area pointed to by the first gesture in the plurality of first images; triggering the smart terminal to capture a second image with a second resolution, and intercepting an area of interest in the second image according to information related to a start point and an end point of the area pointed by the first gesture, wherein the second resolution is higher than the first resolution; The intelligent terminal sends the region of interest to the computing device so as to receive the translation after the computing device recognizes and translates the text contained in the region of interest. 2 . The method according to claim 1 , wherein the first gesture comprises at least one finger extending out and staying at a fixed position of the plurality of first images for more than a first time threshold. 3 . The method according to claim 1 , wherein the first gesture comprises a tip of a pointing pen staying at a fixed position of the plurality of first images for more than a predetermined time threshold. 4. The method according to claim 1, wherein the receiving of the recognition judgment that the computing device recognizes the first gesture in the plurality of first images and the information related to the start point and the end point of the area pointed by the first gesture in the plurality of first images comprises: By the computing device: recognizing a first gesture in the plurality of first images; If the first posture has not been recognized before or the state of the first posture is the first state, the position indicated by the first posture in the plurality of first images at this time is recorded as the first starting point, and the state of the first posture at this time is set to the second state; If the first gesture is recognized again in the plurality of first images and the state of the first gesture is the second state, recording the position indicated by the first gesture in the plurality of first images at this time as the first end point; Setting the state of the first posture to a first state; According to a coordinate mapping relationship between the plurality of first images of the first resolution and the second image of the second resolution, converting the coordinates of the first starting point and the first end point into the coordinates of the second starting point and the second end point in the second image respectively; Sending the coordinates of the second starting point and the second end point to the smart terminal as the starting point and the end point of the area pointed by the first gesture; By the intelligent terminal: The region of interest is determined according to the starting point and the end point of the pointed region, wherein: The starting point and the end point of the area pointed by the first gesture are used as the upper left point and the lower right point of the rectangular frame of the area of interest respectively; or the starting point and the end point of the area pointed by the first gesture are used as the lower left point and the lower right point of the rectangular frame of the area of interest respectively, and the rectangular frame of the area of interest has a predetermined height; The region of interest is captured using a second resolution and sent to the computing device so that the computing device can recognize and translate according to the region of interest. 5. The method according to claim 4, wherein determining the region of interest according to the start point and the end point of the region pointed by the first gesture comprises: The receiving device's recognition judgment that the computing device recognizes the first gesture in the plurality of first images and information related to the start point and the end point of the area pointed by the first gesture in the plurality of first images includes: By the computing device: recognizing a first gesture in the plurality of first images; If the first posture has not been recognized before or the state of the first posture is the first state, the position indicated by the first posture in the plurality of first images at this time is recorded as the first starting point, and the state of the first posture at this time is set to the second state; If the first gesture is recognized again in the plurality of first images and the state of the first gesture is the second state, recording the position indicated by the first gesture in the plurality of first images at this time as the first end point; Setting the state of the first posture to a first state; Sending the coordinates of the first starting point and the first end point to the smart terminal; By the intelligent terminal: According to a coordinate mapping relationship between the plurality of first images of the first resolution and the second image of the second resolution, the coordinates of the first starting point and the first end point are converted into the coordinates of the second starting point and the second end point in the second image respectively as the starting point and the end point of the area pointed by the first gesture; The region of interest is determined according to the starting point and the end point of the pointed region, wherein: The starting point and the end point of the area pointed by the first gesture are used as the upper left point and the lower right point of the rectangular frame of the area of interest respectively; or the starting point and the end point of the area pointed by the first gesture are used as the lower left point and the lower right point of the rectangular frame of the area of interest respectively, and the rectangular frame of the area of interest has a predetermined height; The region of interest is captured using a second resolution and sent to the computing device so that the computing device can recognize and translate according to the region of interest. 6. The method according to claim 1, further comprising: The intelligent terminal collects a plurality of third images for preview in real time using a third resolution; reducing the plurality of third images to the plurality of first images, The third resolution is greater than the first resolution and smaller than the second resolution. 7. The method according to claim 6, wherein the intelligent terminal further performs the following steps: According to a coordinate mapping relationship between the plurality of first images of the first resolution and the plurality of third images of the third resolution, the coordinates of the first starting point and the first end point in the plurality of first images are converted into coordinates of a third starting point and a third end point in the image of the third resolution respectively; marking a third starting point and a third end point and a line between the third starting point and the third end point in the third image, and rendering an area jointly determined by the third starting point and the third end point as a translation area; superimposing a translation result frame on the translation area in the third image for preview; The translated text is displayed in the translation result box. 8. The method according to claim 7, wherein the intelligent terminal further performs the following steps: Initializing the translated text to an unbroadcasted state; Obtaining corresponding voice audio according to the translated text; Obtaining the length of the unbroadcasted translation and the length of the voice audio; broadcasting the voice audio; Get the duration of the currently played voice audio; Calculate the announced length of the translation according to the length of the unannounced translation, the length of the voice audio, and the duration of the currently announced voice audio; Rendering the translation of the length of the broadcasted speech synchronously with the currently broadcasted speech audio; If the voice audio broadcast is completed, end the broadcast; If the voice audio broadcast is interrupted due to recognition of the first gesture of the pointing object, the broadcast is terminated. 9. A smart terminal, comprising a photographing device and a transceiver device, The photographing device obtains a plurality of first images of a first resolution, and the transceiver device sends the plurality of first images to a computing device connected to the smart terminal; The transceiver receives a recognition judgment that the computing device recognizes a first gesture in the plurality of first images and information related to a start point and an end point of an area pointed to by the first gesture in the plurality of first images, and the shooting device triggers shooting of a second image with a second resolution, and captures an area of interest in the second image according to the information related to the start point and the end point of the area pointed to by the first gesture, wherein the second resolution is higher than the first resolution; The transceiver sends the region of interest to the computing device so as to receive the translation after the computing device recognizes and translates the text contained in the image of interest. 10. An image processing method, comprising: Receiving a plurality of first images with a first resolution sent from a smart terminal; A first gesture is recognized in the multiple first images, and a start point and an end point of an area pointed by the first gesture in the multiple first images are determined according to the recognized first gesture; the smart terminal is triggered to shoot a second image using a second resolution and the start point and the end point of the area pointed by the first gesture are sent to the smart terminal, so that the smart terminal obtains an area of interest with a second resolution according to the start point and the end point of the area pointed by the first gesture, and the area of interest is jointly determined by mapping the start point and the end point of the area pointed by the first gesture in the first image to the second image with the second resolution; The region of interest is received from the smart terminal so as to perform text recognition and translation according to the region of interest, wherein the second resolution is higher than the first resolution. 11. A computing device comprising A receiving device, configured to receive a plurality of first images of a first resolution sent from a smart terminal; a recognition device configured to recognize a first gesture in the plurality of first images, and determine a start point and an end point of an area pointed to by the first gesture in the plurality of first images according to the recognized first gesture; a sending device configured to trigger the smart terminal to capture a second image using a second resolution, and to send the start point and the end point of the area pointed by the first gesture to the smart terminal, so that the smart terminal obtains a region of interest at the second resolution according to the start point and the end point of the area pointed by the first gesture, wherein the region of interest is jointly determined by mapping the start point and the end point of the area pointed by the first gesture in the first image to the second image at the second resolution; The region of interest is received from the smart terminal so as to perform text recognition and translation according to the region of interest, wherein the second resolution is higher than the first resolution. 12. An electronic device, comprising: A memory for storing instructions; A processor, configured to read instructions in the memory and execute the method as claimed in any one of claims 1-8 and 10. 13. A non-transitory storage medium having stored thereon instructions, When the instruction is read by a processor, the processor executes the method as claimed in any one of claims 1-8 and 10. 14. A computer program product comprising computer instructions, When the instruction is read by a processor, the processor executes the method as claimed in any one of claims 1-8 and 10.