WO2023020334A1

WO2023020334A1 - Image direction adjustment method and apparatus, and storage medium and electronic device

Info

Publication number: WO2023020334A1
Application number: PCT/CN2022/111133
Authority: WO
Inventors: 王红梅; 郑侠松
Original assignee: 广州视源电子科技股份有限公司; 广州视睿电子科技有限公司
Priority date: 2021-08-16
Filing date: 2022-08-09
Publication date: 2023-02-23
Also published as: CN115941859A

Abstract

Disclosed in the present application are an image direction adjustment method and apparatus, and a storage medium and an electronic device. The method comprises: acquiring a first image of a photographed subject that is captured by a camera; rotating the first image according to different designated angles to obtain a second image, a third image, and a fourth image; using a pre-trained object test model to respectively acquire object probability values corresponding to the first image, the second image, the third image, and the fourth image, and determining the maximum probability value among the object probability values, wherein the object test model is obtained by means of training based on a forward target object sample image and a non-forward target object sample image; and identifying the rotation angle of a target image that is indicated by the maximum probability value, and then adjusting the display direction of the first image on the basis of the rotation angle. By means of the present application, an image direction can be adjusted to be forward, without additionally adding a gravity sensor, thereby saving on the cost of the adjustment of the image direction.

Description

Image orientation adjustment method, device, storage medium and electronic equipment

This application claims the priority of the Chinese patent application with the application number 202110940091.7 and the title of the invention "image orientation adjustment method, device, storage medium and electronic equipment" submitted to the China Patent Office on August 16, 2021, the entire contents of which are incorporated by reference incorporated in this application.

technical field

The present application relates to the field of computer technology, and in particular to an image orientation adjustment method, device, storage medium and electronic equipment.

Background technique

When the mobile terminal takes pictures of the subject, since the camera is only responsible for taking pictures, and the shooting application does not consider whether the mobile terminal will rotate to make a corresponding response, this will occur when the mobile terminal is rotated by 90°, 180° Or the rotation of 270° and other angles, but the image does not follow the rotation, or even if the camera is facing the subject to shoot and the mobile terminal does not rotate after shooting, the captured image still has 90°, 180° ° or 270° and other angles of rotation, it is impossible to present an image in which the subject is facing forward.

In order to adjust the image to be positive, nowadays, the mainstream adjustment method is realized through gravity sensing. Some mobile terminals have a built-in gravity sensor, which measures the acceleration caused by the movement of the mobile terminal through the gravity sensor, so as to calculate the tilt angle of the mobile terminal relative to the horizontal plane, and then adjust the image accordingly according to the tilt angle. The method to adjust the image direction requires an additional gravity sensor, which increases the cost of image direction adjustment.

Contents of the invention

Embodiments of the present application provide an image orientation adjustment method, device, storage medium, and electronic equipment, which can adjust the image orientation to the positive orientation without adding an additional gravity sensor, saving the cost of image orientation adjustment. This technical scheme is as follows:

In the first aspect, an embodiment of the present application provides a method for adjusting an image direction, the method including:

Obtaining the first image collected by the camera for the subject;

Rotating the first image according to different specified angles to obtain a second image, a third image and a fourth image, wherein the first image, the second image, the third image and the The fourth image includes a forward target object image;

Obtaining object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively by using a pre-trained object detection model, and determining a maximum probability value among the object probability values, The object detection model is obtained by training based on forward target object sample images and non-positive target object sample images;

Identifying the rotation angle of the target image indicated by the maximum probability value, and adjusting the display direction of the first image based on the rotation angle.

In a second aspect, an embodiment of the present application provides a method for adjusting an image direction, the method comprising:

receiving a shooting instruction, and turning on the camera in response to the shooting instruction;

displaying the first image collected by the camera for the subject;

Rotating the first image according to different specified angles to obtain a second image, a third image and a fourth image, wherein the first image, the second image, the third image and the first The four images include a forward target object image;

Outputting the object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively obtained by using a pre-trained object detection model, the object detection model is based on the forward target object The sample image and the sample image of the non-positive target object are trained;

determining the maximum probability value among the object probability values, identifying the rotation angle of the target image indicated by the maximum probability value, adjusting the display direction of the first image based on the rotation angle, and displaying the adjusted first image image.

In a third aspect, an embodiment of the present application provides an image orientation adjustment device, the device comprising:

An image acquisition module, configured to acquire the first image collected by the camera for the subject;

An image rotation module, configured to rotate the first image according to different specified angles to obtain a second image, a third image, and a fourth image, wherein the first image, the second image, and the first image The three images and the fourth image include a forward target object image;

A probability value determination module, configured to obtain object probability values corresponding to the first image, the second image, the third image, and the fourth image by using a pre-trained object detection model, and determine the object probability The maximum probability value in the value, the object detection model is obtained based on the training of the forward target object sample image and the non-positive target object sample image;

A direction adjustment module, configured to identify the rotation angle of the target image indicated by the maximum probability value, and adjust the display direction of the first image based on the rotation angle.

In a fourth aspect, the embodiment of the present application provides an image orientation adjustment device, the device comprising:

A camera opening module, configured to receive a shooting instruction, and turn on the camera in response to the shooting instruction;

A picture display module, configured to display the first image collected by the camera for the shooting object;

The image rotation module is used to rotate the first image according to different specified angles to obtain a second image, a third image, and a fourth image, wherein the first image, the second image, and the The third image and the fourth image include a forward target object image;

A probability value determination module, configured to output object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively obtained by using a pre-trained object detection model, the object The detection model is trained based on the sample image of the forward target object and the sample image of the non-positive target object;

A direction adjustment module, configured to determine the maximum probability value among the object probability values, identify the rotation angle of the target image indicated by the maximum probability value, adjust the display direction of the first image based on the rotation angle, and display the adjusted after the first image.

In a fifth aspect, an embodiment of the present application provides a computer storage medium, where a plurality of instructions are stored in the computer storage medium, and the instructions are adapted to be loaded by a processor and execute the above-mentioned method steps.

In a sixth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein, the memory stores a computer program, and the computer program is adapted to be loaded by the processor and execute the above-mentioned method steps .

The beneficial effects brought by the technical solutions provided by the embodiments of the present application at least include:

In the embodiment of the present application, the mobile terminal obtains the first image captured by the camera for the subject, and rotates the first image according to different specified angles to obtain the second image, the third image, and the fourth image. The object detection model obtains the object probability values corresponding to the first image, the second image, the third image, and the fourth image, and determines the maximum probability value among the object probability values, because the pre-trained object detection model is based on the forward target object sample images and non-forward target object sample images, then the image corresponding to the maximum probability value obtained by using the trained object detection model is the forward target object image, and then identify the rotation of the target image indicated by the maximum probability value Angle, adjust the display direction of the first image based on the rotation angle, so that the direction of the final displayed image can be positive, so as to conform to the user's usage habits and facilitate the user's use. During this process, no additional gravity is required sensor, saving the cost of image orientation adjustment.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only These are some embodiments of the present application. Those skilled in the art can also obtain other drawings based on these drawings without creative work.

FIG. 1 is a schematic flow chart of an image orientation adjustment method provided in an embodiment of the present application;

Fig. 2a is a schematic structural diagram of an image orientation adjustment system provided by an embodiment of the present application;

Fig. 2b is a schematic structural diagram of another image orientation adjustment system provided by an embodiment of the present application;

Fig. 3a is a schematic diagram of an example of a first image provided by an embodiment of the present application;

Fig. 3b is an example schematic diagram of another first image provided by the embodiment of the present application;

Fig. 4 is a schematic diagram of an example of image size conversion provided by an embodiment of the present application;

Fig. 5 is a schematic diagram of an example of a first image, a second image, a third image, and a fourth image provided by an embodiment of the present application;

Fig. 6 is a schematic diagram of an example before and after image direction adjustment provided by the embodiment of the present application;

FIG. 7 is a schematic flowchart of a method for adjusting an image direction provided by an embodiment of the present application;

Fig. 8 is a schematic diagram of an example of a prompt information display interface provided by an embodiment of the present application;

FIG. 9 is a schematic flowchart of a method for adjusting an image direction provided by an embodiment of the present application;

FIG. 10 is a schematic diagram of an example of enabling a camera function provided by an embodiment of the present application;

Fig. 11 is a schematic diagram of an example of a prompt information display interface provided by an embodiment of the present application;

FIG. 12 is a schematic structural diagram of an image orientation adjustment device provided in an embodiment of the present application;

FIG. 13 is a schematic structural diagram of an image orientation adjustment device provided in an embodiment of the present application;

Fig. 14 is a schematic structural diagram of a direction adjustment module provided by an embodiment of the present application;

Fig. 15 is a schematic structural diagram of an image orientation adjustment device provided by an embodiment of the present application;

Fig. 16 is a schematic structural diagram of an image orientation adjustment device provided by an embodiment of the present application;

Fig. 17 is a schematic structural diagram of a direction adjustment module provided by an embodiment of the present application;

FIG. 18 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the implementation manners of the present application will be further described in detail below in conjunction with the accompanying drawings.

When the following description refers to the accompanying drawings, the same numerals in different drawings refer to the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with aspects of the present application as recited in the appended claims.

In the description of the present application, it should be understood that the terms "first", "second" and so on are used for descriptive purposes only, and should not be understood as indicating or implying relative importance. Those of ordinary skill in the art can understand the specific meanings of the above terms in this application in specific situations. In addition, in the description of the present application, unless otherwise specified, "plurality" means two or more. "And/or" describes the association relationship of associated objects, indicating that there may be three types of relationships, for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists independently. The character "/" generally indicates that the contextual objects are an "or" relationship.

The present application will be described in detail below in conjunction with specific embodiments.

The method can be implemented relying on a computer program, and can run on an image orientation adjustment device based on the von Neumann system. The computer program can be integrated in the application, or run as an independent utility application. Wherein, the image orientation adjustment device in the embodiment of the present application may be a mobile terminal, including but not limited to: personal computer, tablet computer, handheld device, vehicle-mounted device, wearable device, computing device or other processing device connected to a wireless modem, etc. . User terminals can be called by different names in different networks, for example: user equipment, access terminal, subscriber unit, subscriber station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication Equipment, user agent or user device, cellular phone, cordless phone, personal digital assistant (PDA), terminal equipment in 5G network or future evolution network, etc.

This method includes but is not limited to application to student learning machines. This learning machine refers to a tablet computer that is generally used for students to take online classes or conduct other learning projects. It has a camera set on the top of the learning machine and has a shooting function.

Please refer to FIG. 1 , which provides a schematic flowchart of a method for adjusting an image direction according to an embodiment of the present application.

In this embodiment of the present application, a learning machine is taken as an example for description, and the image direction adjustment method may include the following steps:

S101, acquiring the first image collected by the camera for the subject;

Specifically, as shown in Figure 2a, the camera includes a camera, and the camera can be a device that communicates with the learning machine in a wireless or wired manner, wherein the wireless way for the learning machine to communicate with the camera includes but is not limited to, for example, a cellular network, a wireless local area network, Infrared network, near-field communication network or Bluetooth network, etc., wired methods include but not limited to Universal Serial Bus (Universal Serial Bus, USB). As shown in FIG. 2b, the camera can also be a part of the learning machine, that is, a device installed on the camera of the learning machine.

Wherein, the shooting object can be any object that the user wants to shoot, for example, it can include animals, people, books, etc., and can include one or more.

Wherein, the first image may be a pre-browsing image captured by the camera of the camera based on the shooting picture, or may be a pre-browsing image obtained based on the capturing of the shooting picture by the camera installed on the learning machine in response to the shooting instruction of the learning machine. The first image can be displayed on the screen.

For example, as shown in FIG. 2 b , the user triggers to turn on the camera of the learning machine, and the camera of the learning machine collects images of the subject to obtain the first image. The acquisition of the first image by the learning machine can be based on a preset timing acquisition mechanism, for example, acquisition in the second second after the camera is turned on, or it can be acquired in a preset certain step length (eg, once every 2 seconds).

In some cases, the image direction of the first image is forward, as shown in Figure 3a, the first image displayed in the avatar area of the sidebar of the learning machine in the figure is a forward image, in this case, it is not necessary to adjust the image direction Adjustment.

It can be understood that, before the direction adjustment of the first image is performed, the first image is an image rotated by a certain angle or an image not rotated with the device. One possible situation is as shown in FIG. 3b , the learning machine The first image is an image rotated by a certain angle before adjustment.

Optionally, the first image can also be resized according to a certain ratio without changing the aspect ratio, for example: if the initial size ratio of the image is M×N, reduce the image without changing the aspect ratio size, so as to transform it into M0×N0, if there is a vacant area after the transformation, fill in 0 pixels, and obtain the first image I0 with the size of M0×N0, as shown in Figure 4. Therefore, under the condition that the image information is not lost, the size of the image can be reduced, thereby reducing the amount of calculation.

S102. Rotate the first image according to different specified angles to obtain a second image, a third image, and a fourth image, wherein the first image, the second image, the third image, and The fourth image includes a forward target object image;

The solution provided by the embodiment of the present application adjusts the display direction of the image based on the analysis of the content of the image. It needs a forward target object image as a basis. In order to facilitate the detection of the direction of the target object in the first image, the first image is divided into Rotate by the specified angle.

Wherein, the first image is an image of the target object facing a horizontal direction or a vertical direction, then different specified angles may be 90°, 180° and 270°. For example, the second image, the third image and the fourth image obtained after rotating the first image by 90°, 180° and 270° respectively are shown in FIG. 5 , wherein the first image is the image of the forward target object.

It should be noted that the different specified angles can be understood as angles that rotate in the same direction, such as clockwise or counterclockwise.

After rotation, the second image I1, the third image I2 and the fourth image I3 are obtained, and then the mapping relationship between the image and the angle can be obtained γ(I,D)={I ₀ :0°,I ₁ :90°,I ₂ : 180°, I ₃ : 270°}.

Optionally, the resized first image is correspondingly rotated according to different specified angles to obtain a second image, a third image and a fourth image having the same size as the first image.

S103. Obtain object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively by using a pre-trained object detection model, and determine a maximum probability among the object probability values Value, the object detection model is obtained based on the training of the sample image of the forward target object and the sample image of the non-positive target object;

The object detection model is generated based on training of positive target object sample images and non-positive target object sample images, specifically, it can be a classifier for a two-class classification problem. When an image is input into the object detection model, the probability value that the target object in the image is positive is output, that is, the object probability value. The larger the object probability value, the more likely the target object in the image is positive.

Wherein, the sample image of the forward target object may be a forward image of any target object.

The target object can be any object that the user wants to identify, such as animals, people, and books. The sample image of the non-forward target object can be any image except the forward target object. For example, if the target object is user 1, then the non-direct target object can be any object except user 1, such as user 2, cat, dog, etc.

When the first image includes the target object, the first image, the second image, the third image and the fourth image are identified using a pre-trained object detection model, thereby obtaining the first image, the second image, the third image and the The object probability values corresponding to the fourth images respectively, and determine the maximum probability value, because the image indicated by the maximum probability value has the highest probability of being the image of the forward target object, and the first image, the second image, the third image and the fourth image includes a forward target object image, so it can be determined that the image indicated by the maximum probability value is the forward target object image.

S104. Identify the rotation angle of the target image indicated by the maximum probability value, and adjust the display direction of the first image based on the rotation angle;

The target image indicated by the maximum probability value is one of the first image, the second image, the third image, and the fourth image. After determining which image the target image is, the rotation angle of the image is used as the first image. The rotation angle of the image.

For example, if the target image is the first image, then determine that the rotation angle is 0°; if the target image is the second image, then determine that the rotation angle is the rotation angle of the second image (ie 90°), and so on. The basis for determining the rotation angle is the mapping relationship between the angle and the image.

After the rotation angle of the target image is determined, the first image is rotated according to the rotation angle, so as to adjust the display direction of the first image.

For example, if the rotation angle is 0°, it means that the target image is the first image, that is, the first image is the image of the forward target object, then keep the display direction of the first image unchanged; if the rotation angle is the rotation angle of the second image , it means that the target image is the second image, that is, the second image is the forward target object image, then the angle of the first image is adjusted according to the rotation angle of the second image, and so on.

Because the target image is the forward target object image obtained by rotating the first image according to a certain angle, adjust the display direction of the first image according to the rotation angle of the target image, and the adjusted first image is the forward target object image, adjust The before and after effects are shown in Figure 6.

In the embodiment of the present application, the learning machine acquires the first image captured by the camera for the subject, and when the first image includes the target object, the first image is rotated according to different specified angles to obtain the second image, the third image and For the fourth image, use the pre-trained object detection model to identify the first image, the second image, the third image and the fourth image to obtain the corresponding object probability values, and determine the maximum probability value of the object probability value, because the pre-trained The object detection model is trained based on the sample image of the forward target object and the sample image of the non-positive target object. If the object probability value obtained by using the trained object detection model is the largest, the image corresponding to the maximum probability value can be considered as the forward target. An object image, identifying a rotation angle of the target image indicated by the maximum probability value, and adjusting a display direction of the first image based on the rotation angle. The technical solution provided by the embodiment of the present application is based on adjusting the image direction of the image content, and the image direction can be adjusted to the positive direction without adding an additional gravity sensor, which saves the cost of image direction adjustment.

Please refer to FIG. 7 , which provides a schematic flowchart of an image adjustment method according to an embodiment of the present application. In this embodiment of the present application, a learning machine is taken as an example for description, and the image direction adjustment may include the following steps:

S201, collecting a sample image of a forward target object and a sample image of a non-positive target object;

Among them, the target object can be any object that the user wants to shoot, for example, it can include animals, people, books, etc., and can include one or more. The sample image of the non-positive target object can be any In the image, for example, the target object is user 1, then the non-direct target object can be any object except user 1, such as user 2, cat, dog, etc.

Collect sample images of positive target objects and sample images of non-positive target objects to provide material basis for subsequent training of the object detection model. In order to improve the recognition rate and reliability of the object detection model, it is necessary to ensure that the sample images of positive target objects and non-positive The amount of positive target object sample images is sufficiently large.

S202. Create an initial object detection model, and train the initial object detection model based on the forward target object sample image and the non-forward target object sample image to obtain a trained object detection model;

Specifically, because the purpose of the object detection model is to obtain the object probability value corresponding to the image, the object detection model can be based on conventional face detection networks such as retinaface and mtcnn, or can combine different deep learning blocks to form a network for training. In this way, a classifier that detects whether there is a target object in the image and outputs a corresponding object probability value is obtained. The training samples are a sample image of a positive target object and a sample image of a non-positive target object.

Therefore, when the trained object detection model is used to identify the direction of the target object in the image, the corresponding object probability value is output, and a probability threshold is set in advance according to the training result, and the probability threshold is combined with the object probability value output by the object detection model. If the maximum probability value is less than the probability threshold, it means that the target object is not included in the image; if the maximum probability value is greater than or equal to the probability threshold, it means that the target object is included in the image.

When the maximum probability value is greater than the probability threshold, the larger the object probability value, the greater the probability that the image is a positive target object image; the smaller the probability value, the smaller the probability that the image is a positive target object image.

S203, acquiring the first image collected by the camera for the subject;

Taking the camera as a device installed on the camera of the learning machine as an example, the first image collected by the camera on the learning machine for the subject is shown in Figure 3b. The first image has been rotated, which does not match the direction of the user's actual needs, which is inconvenient for the user. use.

Refer to S101 for details of acquiring the first image captured by the camera with respect to the subject, which will not be repeated here.

S204. Rotate the first image according to different specified angles to obtain a second image, a third image, and a fourth image, wherein the first image, the second image, the third image, and The fourth image includes a forward target object image;

The first image is rotated according to different specified angles to obtain the second image, the third image and the fourth image. For details, refer to S102, which will not be repeated here.

S205. Obtain object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively by using a pre-trained object detection model, and determine a maximum probability among the object probability values Value, the object detection model is obtained based on the training of the sample image of the forward target object and the sample image of the non-positive target object;

The probability value that the image is a positive target object image can be obtained through the pre-trained object detection model. The larger the probability value, the greater the possibility that the image is a positive target object image.

When the target object is included in the first image, the first image, the second image, the third image and the fourth image are identified using a pre-trained object detection model, and the first image, the second image, the third image and the fourth image are obtained. The object probability values corresponding to the four images, such as P1, P2, P3, P4, and determine the maximum probability value in P1, P2, P3, P4, because the image indicated by the maximum probability value has the highest probability of being the forward target object image, so It may be determined that the fourth image with the maximum probability value indicated by P4 is the forward target object image.

S206. When the maximum probability value is greater than or equal to a probability threshold, determine a target image indicated by the maximum probability value;

When the obtained maximum probability value is greater than or equal to the probability threshold, based on the settings of the object detection model in the creation and training process, it can be considered that the image includes the target object, and because the image indicated by the maximum probability value can be considered as the positive target object image, Therefore, it is determined that the image indicated by the maximum probability value is the target image, and the target image is one of the first image, the second image, the third image, and the fourth image.

S207. If the target image is the first image, determine that the rotation angle is 0°, and keep the display direction of the first image unchanged;

If the target image determined based on the maximum probability value is the first image, since the rotation angle of the first image is 0° compared to the first image, the rotation angle is determined to be 0°, and the first image itself is considered to be the forward target object image, There is no need to adjust the image orientation, keeping the display orientation of the first image unchanged.

S208. If the target image is the second image, determine that the rotation angle is the rotation angle of the second image, and adjust the display direction of the first image according to the rotation angle of the second image ;

If the target image determined based on the maximum probability value is the second image, since the second image is obtained by rotating the first image according to a specified angle, the rotation angle is determined to be the rotation angle of the second image, and the first image is rotated according to the rotation angle of the second image The rotation angle is adjusted so that the first image is the image of the forward target object.

S209. If the target image is the third image, determine that the rotation angle is the rotation angle of the third image, and adjust the display direction of the first image according to the rotation angle of the third image ;

If the target image determined based on the maximum probability value is the third image, since the third image is obtained by rotating the first image according to a specified angle, the rotation angle is determined to be the rotation angle of the third image, and the first image is rotated according to the rotation angle of the third image The rotation angle is adjusted so that the first image is the image of the forward target object.

S210, if the target image is the fourth image, determine that the rotation angle is the rotation angle of the fourth image, and adjust the display direction of the first image according to the rotation angle of the fourth image ;

If the target image determined based on the maximum probability value is the fourth image, because the fourth image is obtained by rotating the first image according to a specified angle, the rotation angle is determined to be the rotation angle of the fourth image, and the first image is rotated according to the rotation angle of the fourth image. The rotation angle is adjusted so that the first image is the image of the forward target object.

S211. When the maximum probability value is less than the probability threshold, output a specified value, the specified value indicating that there is no target object in the first image;

If the maximum probability value is less than the probability threshold, based on the settings of the object detection model during creation and training, it can be considered that the target image is not included in the image, and a specified value is output, and the specified value is used to indicate that there is no target object in the first image.

Among them, the specified value can be any number, letter or symbol, etc., which can be set in advance during the training of the object detection model. For example, if the specified value is set to -1 during training, then if the maximum probability value is less than the probability threshold, the specified value is output -1 means that there is no target object in the first image.

S212. Acquire the next frame of image collected by the camera for the subject, use the next frame of image as the first image, and execute the step of rotating the first image according to different specified angles , the step of obtaining the second image, the third image and the fourth image;

After the object detection model outputs the specified value indicating that there is no target object in the first image, the camera is triggered to acquire the next frame of image collected for the subject again, and the next frame of image is used as the new first image for detection, and the first image is executed. An image is rotated according to different specified angles to obtain the second image, the third image and the fourth image, and the steps of S205-S213 are continued to obtain the first image adjusted to the forward target object image.

S213. When there is no target object in the first images of the preset number of frames collected continuously, output a prompt message of a camera failure, where the prompt message is used to prompt the user to adjust the shooting angle of the camera;

If no target object is detected in the first images of the preset number of frames collected continuously, it is considered that the camera may be malfunctioning or there is a problem with the shooting angle of the camera, and a prompt message is output to prompt the user to detect the shooting angle of the camera.

Wherein, the preset number of frames may be 3 frames, 4 frames, etc., and the specific number is not limited.

The prompt information can be output in the form of audio, text or animation, or multiple ways in parallel, and the specific ways and contents are not limited.

As shown in FIG. 8 , a feasible prompting method may be a text box prompt, a text box pops up, and the content of the text box is "no portrait is recognized in the current image, please take a new photo".

Among them, the possible faults of the camera include but are not limited to: the camera is damaged, the camera is blocked by other objects, and so on.

Among them, the objects that exist at the shooting angle of the camera include but are not limited to: the target object cannot be captured at the current shooting angle of the camera, and the wrong camera is used, for example, the rear camera is used instead of the front camera.

Wherein, the front camera can be a camera placed on the upper end of the panel where the display screen of the learning machine is located.

Wherein, the rear camera may be a camera placed on the back panel of the learning machine.

After the user has readjusted the camera, repeat the acquisition of the first image and the following other steps.

In the embodiment of the present application, when the first image includes the target object, the first image is rotated according to different angles to obtain four images including a forward target object image, and the pre-trained object detection model is used to obtain four images. The object probability value corresponding to each image, determine the rotation angle of the target image based on the maximum probability value, and adjust the display direction of the first image based on the rotation angle; if the maximum probability value is less than the preset threshold, it means that there is no object in the first image target object, re-acquire the next frame of image as the first image for identification, and finally obtain the first image whose display direction is positive. The technical solution provided by the embodiment of the present application is to analyze and adjust the image direction based on the image content, which can The image direction can be adjusted to the positive direction without adding an additional gravity sensor, which saves the cost of image direction adjustment.

Please refer to FIG. 9 , which provides a schematic flowchart of a method for adjusting an image direction according to an embodiment of the present application.

The embodiment of the present application is described on the learning machine side, and the image direction adjustment method may include the following steps:

S301. Receive a shooting instruction, and turn on the camera in response to the shooting instruction;

As shown in Figure 2a, the camera can be a device that communicates with the learning machine in a wireless or wired manner, and as shown in Figure 2b, the camera can also be a part of the learning machine, that is, a device installed on the learning machine.

Taking the camera as a device installed on the learning machine as an example, as shown in Figure 10, when the user triggers the learning machine to turn on the shooting function of the camera, the learning machine sends a shooting command, and when the learning machine receives the shooting command, it responds to the shooting command and turns on The shooting function of the camera, the camera is aimed at the subject to shoot.

Wherein, the shooting object can be any object that the user wants to shoot, for example, it can include animals, people, books, etc., and can also include one or more.

S302, displaying the first image collected by the camera for the shooting object;

The first image refers to the first image obtained by the camera in response to the start shooting instruction sent by the learning machine based on the acquisition of the shooting picture.

For details of the first image obtained based on the acquisition of the photographed picture, refer to S101, which will not be repeated here.

S303, displaying whether to enable a prompt message for direction detection of the target object;

The first image collected by the camera is displayed in the avatar area of the learning machine, and a prompt message is displayed to prompt the user whether to enable the direction detection of the target object, and the user can choose according to the actual situation.

Wherein, the prompt information may be output in the form of audio, text, or animation, or in parallel in multiple ways, and the specific ways and contents are not limited.

As shown in FIG. 11 , a feasible prompting method may be a text box prompt, a text box pops up, and the content of the text box is "whether to enable image orientation detection".

S304. Receive a confirmation instruction for the target object in the prompt information, and enable a function of detecting the direction of the target object in response to the confirmation instruction;

If the user clicks the "Yes" button in the prompt text box shown in Figure 11, it means that the user agrees to enable the direction detection of the target image. The detection function; if the user clicks the "No" button, the image detection function will not be enabled.

Among them, the method of clicking the button depends on the specific conditions of the learning machine. If the learning machine is a touch screen, you can directly click with your fingers; if the learning machine supports voice control, you can also use voice to select options; if the learning machine supports mouse control, You can use the mouse to operate.

S305. Rotate the first image according to different specified angles to obtain a second image, a third image, and a fourth image, wherein the first image, the second image, the third image, and the The fourth image includes a forward target object image;

The technical solution provided by the embodiment of the present application is to adjust the display direction of the image based on the analysis of the content of the image. A forward target object image is required as a basis. In order to facilitate the detection of the target object in the first image, the first The image is rotated by a different specified angle.

For details of rotating the first image according to different specified angles, refer to S102 for details, which will not be repeated here.

S306. Output the object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively acquired by using the pre-trained object detection model, and determine the maximum probability value, the The object detection model is trained based on the sample image of the forward target object and the sample image of the non-positive target object;

The pre-trained object detection model can be a classifier based on the two-class classification problem trained based on the sample image of the forward target object and the sample image of the non-positive target object. Through this model, the probability value of the image is the image of the forward target object, and the probability value The larger the , the more likely the image is the image of the forward target object.

For the training process of the object detection model, refer to S201 and S202, which will not be repeated here.

The maximum probability value is determined by using the object probability value of the image acquired by the pre-trained object detection model. For details, refer to S205, which will not be repeated here.

Wherein, the object probability value may be a probability value that the image includes a forward target object.

S307. When the maximum probability value is greater than or equal to a probability threshold, identify the rotation angle of the target image indicated by the maximum probability value, and determine the target direction of the first image based on the rotation angle;

Wherein, the probability threshold may be a threshold set according to the training effect during the training process of the object detection model.

When the maximum probability value is greater than or equal to the probability threshold, it can be considered that the image includes the target object.

The second image, the third image, and the fourth image are obtained by rotating the first image according to different specified angles. The four images include a forward target object image, and the pre-trained object detection model is used to obtain the four images. The object probability value, the image indicated by the maximum probability value can be considered as the forward target object image, and the indicated image is used as the target image to determine the rotation angle of the target image.

For example: if the target image is the first image, determine the rotation angle as 0°; if the target image is the second image, determine the rotation angle as the rotation angle of the second image, and so on.

After determining the rotation angle of the target image, determine the target direction of the first image based on the rotation angle of the target image, because the target image is a positive target object image obtained by rotating the first image according to the specified angle, so the display direction of the target image is considered to be The first image is to be adjusted to face the target direction of the target object image

S308. Determine the target image indicated by the maximum probability value;

When the obtained maximum probability value is greater than or equal to the probability threshold, based on the settings of the object detection model in the creation and training process, it can be considered that the image includes the target object, and because the image indicated by the maximum probability value can be considered as the positive target object image, Therefore, it is determined that the image indicated by the maximum probability value is the target image.

S309. If the target image is the first image, keep the display direction of the first image unchanged, and display the first image;

If the target image is the first image, it can be considered that the first image is the forward target object image, because the rotation angle of the first image is 0° compared with the first image, and there is no need to adjust the display direction of the first image, so the first image is maintained The display orientation of is unchanged, and the first image is displayed.

S310. If the target image is the second image, adjust the display direction of the first image according to the rotation angle of the second image, and display the adjusted first image;

If the target image is the second image, it can be considered that the second image is the forward target object image, then the rotation angle is determined to be the rotation angle of the second image, and the display direction of the first image is adjusted according to the rotation angle of the second image, and The adjusted first image is displayed.

S311. If the target image is the third image, adjust the display direction of the first image according to the rotation angle of the third image, and display the adjusted first image;

If the target image is the third image, it can be considered that the third image is the forward target object image, then the rotation angle is determined to be the rotation angle of the third image, and the display direction of the first image is adjusted according to the rotation angle of the third image, and The adjusted first image is displayed.

S312. If the target image is the fourth image, adjust the display direction of the first image according to the rotation angle of the fourth image, and display the adjusted first image;

If the target image is the fourth image, it can be considered that the fourth image is the forward target object image, then the rotation angle is determined to be the rotation angle of the fourth image, and the display direction of the first image is adjusted according to the rotation angle of the fourth image, and The adjusted first image is displayed.

S313. When the maximum probability value is less than the probability threshold, output a specified value, where the specified value indicates that there is no target object in the first image;

S314. Acquire the next frame of image collected by the camera for the subject, use the next frame of image as the first image, and execute the step of rotating the first image according to different specified angles , the step of obtaining the second image, the third image and the fourth image;

After the object detection model outputs the specified value indicating that there is no target object in the first image, the camera is triggered to acquire the next frame of image collected for the subject again, and the next frame of image is used as the new first image for detection, and the first image is executed. An image is rotated according to different specified angles to obtain the second image, the third image and the fourth image, and the steps of S306-S315 are continued to obtain the first image adjusted to the forward target object image.

S315. When there is no target object in the first images of the preset number of frames continuously collected, output a prompt message of camera failure, where the prompt message is used to prompt the user to adjust the shooting angle of the camera;

Wherein, the prompt information can be outputted in the form of audio, text or animation, and multiple ways can be used in parallel, and the specific ways and contents are not limited.

Among them, the objects that exist at the shooting angle of the camera include but are not limited to: the target object cannot be captured at the current shooting angle of the camera, the wrong camera is used, for example, the rear camera is used instead of the front camera, etc.

After the user adjusts and adjusts the camera, the steps of acquiring the first image and performing S303-S315 are repeated.

In the embodiment of the present application, after the user triggers the function of turning on the camera, the learning machine turns on the camera to obtain the first image and displays it for processing. When the user turns on the detection function, a pop-up window prompts the user to confirm to turn on the detection, and turns on the detection after receiving the confirmation command Function, through the object detection model to obtain the object probability values of the second image, the third image and the fourth image obtained by rotating the first image at a specified angle, when the maximum probability value is greater than or equal to the probability threshold, identify the target image based on the maximum probability value Rotation angle of the first image, and adjust the display direction of the first image to display the adjusted first image; when the maximum probability value is less than the probability threshold, it means that there is no target object in the first image, then re-acquire the next frame of image as the first image One image is recognized. If the first image collected continuously does not include the target object, the user is prompted to check the camera and its shooting angle, and the image is acquired again for detection, so as to finally obtain the first image whose display direction is positive. This application implements The technical solution provided by the example is to analyze and adjust the image direction based on the image content, and the image direction can be adjusted to the positive direction without adding an additional gravity sensor, which saves the cost of image direction adjustment.

The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.

Please refer to FIG. 12 , which shows a schematic structural diagram of image direction adjustment provided by an exemplary embodiment of the present application. The device for adjusting the image direction can be implemented as all or a part of the terminal through software, hardware or a combination of the two. The device 1 includes an image acquisition module 11 , an image rotation module 12 , a probability value determination module 13 and a direction adjustment module 14 .

An image acquisition module 11, configured to acquire the first image collected by the camera for the object to be photographed;

The image rotation module 12 is used to rotate the first image according to different specified angles to obtain the second image, the third image and the fourth image, wherein the first image, the second image, the third image and the fourth image Include a positive target object image;

The probability value determination module 13 is used to obtain the object probability values corresponding to the first image, the second image, the third image and the fourth image respectively by using the pre-trained object detection model, and determine the maximum probability value in the object probability values, and the object detection The model is trained based on the sample image of the forward target object and the sample image of the non-positive target object;

The direction adjustment module 14 is configured to identify the rotation angle of the target image indicated by the maximum probability value, and adjust the display direction of the first image based on the rotation angle.

Optionally, as shown in Figure 13, the device 1 further includes:

A sample collection module 15, configured to collect a forward target object sample image and a non-positive target object sample image;

The model training module 16 is used to create an initial object detection model, and train the initial object detection model based on the forward target object sample image and the non-positive target object sample image to obtain a trained object detection model.

Optionally, the direction adjustment module 14 is specifically used for:

When the maximum probability value is greater than or equal to the probability threshold, identifying the rotation angle of the target image indicated by the maximum probability value, and adjusting the display direction of the first image based on the rotation angle.

Optionally, as shown in Figure 14, the direction adjustment module 14 includes:

An image determining unit 141, configured to determine the target image indicated by the maximum probability value;

A direction adjustment unit 142, configured to determine that the rotation angle is 0° if the target image is the first image, and keep the display direction of the first image unchanged;

The direction adjusting unit 142 is further configured to determine the rotation angle as the rotation angle of the second image if the target image is the second image, and adjust the rotation angle of the second image according to the rotation angle of the second image. adjusting the display direction of the first image;

The direction adjusting unit 142 is further configured to determine the rotation angle as the rotation angle of the third image if the target image is the third image, and adjust the rotation angle of the third image according to the rotation angle of the third image. adjusting the display direction of the first image;

The direction adjustment unit 142 is further configured to determine the rotation angle as the rotation angle of the fourth image if the target image is the fourth image, and adjust the rotation angle of the fourth image according to the rotation angle of the fourth image. The display orientation of the first image is adjusted.

Optionally, as shown in Figure 13, the device 1 further includes:

A specified value output module 17, configured to output a specified value when the maximum probability value is smaller than the probability threshold, the specified value indicating that there is no target object in the first image.

Optionally, the image acquisition module 11 is further configured to acquire the next frame of image collected by the camera for the subject, use the next frame of image as the first image, and trigger the image rotation module 12 The first image is rotated respectively according to different specified angles to obtain the second image, the third image and the fourth image.

Optionally, as shown in FIG. 13 , the device further includes an information output module 18, configured to output a prompt message of camera failure when there is no target object in the first images of the preset number of frames collected continuously , the prompt information is used to prompt the user to adjust the shooting angle of the camera.

Please refer to FIG. 15 , which shows a schematic structural diagram of an image orientation adjustment device provided by an exemplary embodiment of the present application. The device 2 includes a camera opening module 21 , a picture display module 22 , an image rotation module 23 , a probability value determination module 24 , and a direction adjustment module 25 .

The camera opening module 21 is configured to receive a shooting instruction, and open the camera in response to the shooting instruction;

A picture display module 22, configured to display the first image collected by the camera for the subject;

An image rotation module 23, configured to rotate the first image according to different specified angles to obtain a second image, a third image and a fourth image, wherein the first image, the second image, the The third image and the fourth image include a forward target object image;

A probability value determination module 24, configured to output the object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively obtained by using a pre-trained object detection model, the The object detection model is trained based on the sample image of the forward target object and the sample image of the non-positive target object;

a direction adjustment module 25, configured to determine the maximum probability value among the object probability values, and identify the rotation angle of the target image indicated by the maximum probability value, adjust the display direction of the first image based on the rotation angle, and and displaying the adjusted first image.

Optionally, as shown in Figure 16, the device 2 further includes:

The detection prompt display module 26 is used to display whether to open the prompt information for direction detection of the target object;

The detection and activation module 27 is configured to receive a confirmation instruction for the prompt information, and respond to the confirmation instruction to enable the function of detecting the direction of the target object.

Optionally, as shown in FIG. 16, the device 2 further includes:

A specified value output module 28, configured to output a specified value when the maximum probability value is less than the probability threshold, the specified value indicating that there is no target object in the first image.

Optionally, the image acquisition module 22 is also configured to acquire the next frame of image collected by the camera for the subject, use the next frame of image as the first image, and trigger the image rotation module 23 The first image is rotated respectively according to different specified angles to obtain the second image, the third image and the fourth image.

Optionally, as shown in FIG. 13 , the device further includes an information output module 29, configured to output a prompt message of camera failure when there is no target object in the first images of the preset number of frames collected continuously , the prompt information is used to prompt the user to check the camera and its shooting angle.

Optionally, the direction adjustment module 25 is specifically used for:

Optionally, as shown in Figure 17, the angle adjustment unit 25 includes:

An image determining unit 251, configured to determine the target image indicated by the maximum probability value;

a direction adjustment unit 252, configured to keep the display direction of the first image unchanged and display the first image if the target image is the first image;

The direction adjustment unit 252 is further configured to adjust the display direction of the first image according to the rotation angle of the second image if the target image is the second image, and display the adjusted the first image;

The direction adjusting unit 252 is further configured to adjust the display direction of the first image according to the rotation angle of the third image if the target image is the third image, and display the adjusted the first image;

The direction adjustment unit 252 is further configured to adjust the display direction of the first image according to the rotation angle of the fourth image if the target image is the fourth image, and display the adjusted Describe the first image.

It should be noted that, when the image orientation adjustment device provided in the above-mentioned embodiments executes the image orientation adjustment method, the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be assigned to different function modules as required Module completion means that the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the image orientation adjustment device and the image orientation adjustment method embodiment provided by the above embodiment belong to the same idea, and the implementation process thereof is detailed in the method embodiment, and will not be repeated here.

The serial numbers of the above embodiments of the present application are for description only, and do not represent the advantages and disadvantages of the embodiments.

The embodiment of the present application also provides a computer storage medium, the computer storage medium can store a plurality of instructions, and the instructions are suitable for being loaded by the processor and executing the method steps of the above-mentioned embodiments shown in Figures 1-11 For the specific execution process, refer to the specific description of the embodiments shown in FIGS. 1-11 , and details are not repeated here.

The present application also provides a computer program product, the computer program product stores at least one instruction, and the at least one instruction is loaded by the processor and executes the specific descriptions of the embodiments shown in FIGS. 1-11 above. Herein No further elaboration.

Please refer to FIG. 18 , which provides a schematic structural diagram of an electronic device according to an embodiment of the present application. As shown in FIG. 18 , the mobile terminal 1000 may include: at least one processor 1001 , at least one network interface 1004 , a user interface 1003 , a memory 1005 , and at least one communication bus 1002 .

Wherein, the communication bus 1002 is used to realize connection and communication between these components.

Wherein, the user interface 1003 may include a display screen (Display) and a camera (Camera), and the optional user interface 1003 may also include a standard wired interface and a wireless interface.

Wherein, the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).

Wherein, the processor 1001 may include one or more processing cores. The processor 1001 uses various interfaces and lines to connect various parts of the entire electronic device 1000, and by running or executing instructions, programs, code sets or instruction sets stored in the memory 1005, and calling data stored in the memory 1005, execute Various functions of the electronic device 1000 and processing data. Optionally, the processor 1001 may use at least one of Digital Signal Processing (Digital Signal Processing, DSP), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), and Programmable Logic Array (Programmable Logic Array, PLA). implemented in the form of hardware. The processor 1001 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), an image processor (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used to render and draw the content that needs to be displayed on the display screen; the modem is used to handle wireless communication. It can be understood that the above modem may also not be integrated into the processor 1001, but implemented by a single chip.

Wherein, the memory 1005 may include a random access memory (Random Access Memory, RAM), and may also include a read-only memory (Read-Only Memory). Optionally, the memory 1005 includes a non-transitory computer-readable storage medium (non-transitory computer-readable storage medium). The memory 1005 may be used to store instructions, programs, codes, sets of codes or sets of instructions. The memory 1005 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), Instructions and the like for implementing the above method embodiments; the storage data area can store the data and the like involved in the above method embodiments. Optionally, the memory 1005 may also be at least one storage device located away from the aforementioned processor 1001 . As shown in FIG. 18 , the memory 1005 as a computer storage medium may include an operating system, a network communication module, a user interface module, and an image orientation adjustment application program.

In the mobile terminal 1000 shown in FIG. 18 , the user interface 1003 is mainly used to provide the user with an input interface to obtain the data input by the user; and the processor 1001 can be used to call the generated image orientation adjustment application program stored in the memory 1005, And specifically do the following:

Obtaining the first image collected by the camera for the subject;

In one embodiment, before the processor 1001 executes acquiring the first image captured by the camera for the subject, the following operations are further performed:

Collect sample images of positive target objects and sample images of non-positive target objects;

An initial object detection model is created, and the initial object detection model is trained based on the sample image of the forward target object and the sample image of the non-positive target object to obtain a trained object detection model.

In one embodiment, the processor 1001 identifies the rotation angle of the target image indicated by the maximum probability value, determines the target direction of the first image based on the rotation angle, and determines the target direction of the first image based on the rotation angle of the target image indicated by the maximum probability value. When adjusting the display direction of the first image, specifically perform the following operations:

When the maximum probability value is greater than or equal to a probability threshold, identifying a rotation angle of the target image indicated by the maximum probability value, and adjusting a display direction of the first image based on the rotation angle.

In one embodiment, when the processor 1001 identifies the rotation angle of the target image indicated by the maximum probability value, and adjusts the display direction of the first image based on the rotation angle, specifically perform the following operations:

determining the target image indicated by the maximum probability value;

If the target image is the first image, then determine that the rotation angle is 0°, and keep the display direction of the first image unchanged;

If the target image is the second image, determining that the rotation angle is the rotation angle of the second image, and adjusting the display direction of the first image according to the rotation angle of the second image;

If the target image is the third image, determining that the rotation angle is the rotation angle of the third image, and adjusting the display direction of the first image according to the rotation angle of the third image;

If the target image is the fourth image, determine that the rotation angle is the rotation angle of the fourth image, and adjust the display direction of the first image according to the rotation angle of the fourth image.

In one embodiment, the processor 1001 also performs the following operations:

When the maximum probability value is less than the probability threshold, outputting a specified value, the specified value indicating that there is no target object in the first image;

In one embodiment, the processor 1001 outputs a specified value when the maximum probability value is smaller than the probability threshold, and the specified value indicates that there is no target object in the first image, the processing The device 1001 also performs the following operations:

Acquiring the next frame of image collected by the camera for the subject, using the next frame of image as the first image, and executing the step of rotating the first image according to different specified angles, to obtain The steps of the second image, the third image and the fourth image.

In one embodiment, when the processor 1001 executes acquiring the next frame of image captured by the camera for the subject, using the next frame of image as the first image, and executing the When the first image is rotated according to different specified angles to obtain the second image, the third image and the fourth image, the processor 1001 further performs the following operations:

When there is no target object in the first images of the preset number of frames continuously collected, a prompt message of camera failure is output, and the prompt message is used to prompt the user to adjust the shooting angle of the camera.

Optionally, the processor 1001 also performs the following operations:

displaying the first image collected by the camera for the subject;

Determine the maximum probability value among the object probability values, identify the rotation angle of the target image indicated by the maximum probability value, and adjust the display direction of the first image based on the rotation angle.

In one embodiment, the processor 1001 also performs the following operations:

Display whether to open the prompt information for the direction detection of the target object;

A confirmation instruction for the prompt information is received, and a function of detecting the direction of the target object is turned on in response to the confirmation instruction.

In one embodiment, when the processor 1001 determines the target direction of the first image based on the maximum probability value, it specifically performs the following operations:

When the maximum probability value is greater than or equal to a probability threshold, identifying the rotation angle of the target image indicated by the maximum probability value, and determining the target direction of the first image based on the rotation angle.

In one embodiment, the processor 1001 also performs the following operations:

When the maximum probability value is less than the probability threshold, output a specified value indicating that there is no target object in the first image.

In one embodiment, the processor 1001 performs identifying the rotation angle of the target image indicated by the maximum probability value, adjusts the display direction of the first image based on the rotation angle, and displays the adjusted first image. For an image, perform the following operations:

If the target image is the first image, keeping the display direction of the first image unchanged and displaying the first image;

If the target image is the second image, adjusting the display direction of the first image according to the rotation angle of the second image, and displaying the adjusted first image;

If the target image is the third image, adjusting the display direction of the first image according to the rotation angle of the third image, and displaying the adjusted first image;

If the target image is the fourth image, adjusting the display direction of the first image according to the rotation angle of the fourth image, and displaying the adjusted first image.

In the embodiment of the present application, after the user triggers the function of turning on the camera, the learning machine turns on the camera to obtain the first image and displays it for processing. When the user turns on the detection function, a pop-up window prompts the user to confirm to turn on the detection, and turns on the detection after receiving the confirmation command Function, through the object detection model to obtain the object probability values of the second image, the third image and the fourth image obtained by rotating the first image at a specified angle, when the maximum probability value is greater than or equal to the probability threshold, identify the target image based on the maximum probability value rotation angle of the first image, and adjust the display direction of the first image; when the maximum probability value is less than the probability threshold, it means that there is no target object in the first image, and then re-acquire the next frame image as the first image for identification. If the first image does not include the target object, the user is prompted to check the camera and its shooting angle, and the image is acquired again for detection, so as to finally obtain the first image whose display direction is positive. The technical solution provided by the embodiment of this application is based on the image The content is analyzed and the image direction is adjusted, and the image direction can be adjusted to positive without adding an additional gravity sensor, which saves the cost of image direction adjustment.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be implemented through computer programs to instruct related hardware, and the programs can be stored in a computer-readable storage medium. During execution, it may include the processes of the embodiments of the above-mentioned methods. Wherein, the storage medium may be a magnetic disk, an optical disk, a read-only memory or a random access memory, and the like.

The above disclosures are only preferred embodiments of the present application, which certainly cannot limit the scope of the present application. Therefore, equivalent changes made according to the claims of the present application still fall within the scope of the present application.

Claims

A method for adjusting an image direction, comprising:

Obtaining the first image collected by the camera for the subject;

Rotating the first image according to different specified angles to obtain a second image, a third image and a fourth image, wherein the first image, the second image, the third image and the The fourth image includes a forward target object image;

Obtaining object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively by using a pre-trained object detection model, and determining a maximum probability value among the object probability values, The object detection model is obtained by training based on forward target object sample images and non-positive target object sample images;

Identifying the rotation angle of the target image indicated by the maximum probability value, and adjusting the display direction of the first image based on the rotation angle.
The method according to claim 1, wherein, before the acquisition of the first image collected by the camera for the object to be photographed, further comprising:

Collect sample images of positive target objects and sample images of non-positive target objects;

An initial object detection model is created, and the initial object detection model is trained based on the sample image of the forward target object and the sample image of the non-positive target object to obtain a trained object detection model.
The method according to claim 1, wherein the identifying the rotation angle of the target image indicated by the maximum probability value, and adjusting the display direction of the first image based on the rotation angle comprises:

When the maximum probability value is greater than or equal to a probability threshold, identifying a rotation angle of the target image indicated by the maximum probability value, and adjusting a display direction of the first image based on the rotation angle.
The method according to claim 1 or 3, wherein the identifying the rotation angle of the target image indicated by the maximum probability value, and adjusting the display direction of the first image based on the rotation angle comprises:

determining the target image indicated by the maximum probability value;

If the target image is the first image, then determine that the rotation angle is 0°, and keep the display direction of the first image unchanged;

If the target image is the second image, determining that the rotation angle is the rotation angle of the second image, and adjusting the display direction of the first image according to the rotation angle of the second image;

If the target image is the third image, determining that the rotation angle is the rotation angle of the third image, and adjusting the display direction of the first image according to the rotation angle of the third image;

If the target image is the fourth image, determine that the rotation angle is the rotation angle of the fourth image, and adjust the display direction of the first image according to the rotation angle of the fourth image.
The method according to claim 3, further comprising:

When the maximum probability value is less than the probability threshold, output a specified value indicating that there is no target object in the first image.
The method according to claim 5, wherein the method further comprises:

Acquiring the next frame of image collected by the camera for the subject, using the next frame of image as the first image, and executing the step of rotating the first image according to different specified angles, to obtain The steps of the second image, the third image and the fourth image.
The method according to claim 6, further comprising:

When there is no target object in the first image of the preset number of frames collected continuously, a prompt message of camera failure is output, and the prompt message is used to prompt the user to adjust the shooting angle of the camera.
A method for adjusting image direction, characterized in that the method comprises:

receiving a shooting instruction, and turning on the camera in response to the shooting instruction;

displaying the first image collected by the camera for the subject;

Rotating the first image according to different specified angles to obtain a second image, a third image and a fourth image, wherein the first image, the second image, the third image and the first The four images include a forward target object image;

Outputting the object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively obtained by using a pre-trained object detection model, the object detection model is based on the forward target object The sample image and the sample image of the non-positive target object are trained;

determining the maximum probability value among the object probability values, identifying the rotation angle of the target image indicated by the maximum probability value, adjusting the display direction of the first image based on the rotation angle, and displaying the adjusted first image image.
The method according to claim 8, wherein after the camera is turned on, further comprising:

Display whether to open the prompt information for the direction detection of the target object;

A confirmation instruction for the prompt information is received, and the function of detecting the direction of the target object is turned on in response to the confirmation instruction.
The method according to claim 8, wherein the determining the target direction of the first image based on the maximum probability value comprises:

When the maximum probability value is greater than or equal to a probability threshold, identifying the rotation angle of the target image indicated by the maximum probability value, and determining the target direction of the first image based on the rotation angle.
The method according to claim 10, characterized in that the method further comprises:

When the maximum probability value is less than the probability threshold, output a specified value indicating that there is no target object in the first image.
The method according to claim 11, characterized in that the method further comprises:

Acquiring the next frame of image collected by the camera for the subject, using the next frame of image as the first image, and executing the step of rotating the first image according to different specified angles, to obtain The steps of the second image, the third image and the fourth image.
The method according to claim 12, characterized in that the method further comprises:

When there is no target object in the first images of the preset number of frames continuously collected, a prompt message of camera failure is output, and the prompt message is used to prompt the user to adjust the shooting angle of the camera.
The method according to claim 8, characterized in that identifying the rotation angle of the target image indicated by the maximum probability value, adjusting the display direction of the first image based on the rotation angle, and displaying the adjusted The first image, including:

determining the target image indicated by the maximum probability value;

If the target image is the first image, keeping the display direction of the first image unchanged and displaying the first image;

If the target image is the second image, adjusting the display direction of the first image according to the rotation angle of the second image, and displaying the adjusted first image;

If the target image is the third image, adjusting the display direction of the first image according to the rotation angle of the third image, and displaying the adjusted first image;

If the target image is the fourth image, adjusting the display direction of the first image according to the rotation angle of the fourth image, and displaying the adjusted first image.
An image orientation adjustment device, characterized in that it comprises:

An image acquisition module, configured to acquire the first image collected by the camera for the subject;

An image rotation module, configured to rotate the first image according to different specified angles to obtain a second image, a third image and a fourth image, wherein the first image, the second image, the The third image and the fourth image include a forward target object image;

A probability value determination module, configured to obtain object probability values corresponding to the first image, the second image, the third image, and the fourth image by using a pre-trained object detection model, and determine the object probability The maximum probability value in the value, the object detection model is obtained based on the training of the forward target object sample image and the non-positive target object sample image;

A direction adjustment module, configured to identify the rotation angle of the target image indicated by the maximum probability value, and adjust the display direction of the first image based on the rotation angle.
An image orientation adjustment device, characterized in that the device comprises:

A camera opening module, configured to receive a shooting instruction, and turn on the camera in response to the shooting instruction;

A picture display module, configured to display the first image collected by the camera for the shooting object;

An image rotation module, configured to rotate the first image according to different specified angles to obtain a second image, a third image, and a fourth image, wherein the first image, the second image, and the first image The three images and the fourth image include a forward target object image;

A probability value determination module, configured to output object probability values corresponding to the first image, the second image, the third image, and the fourth image respectively obtained by using a pre-trained object detection model, the object The detection model is trained based on the sample image of the forward target object and the sample image of the non-positive target object;

A direction adjustment module, configured to determine the maximum probability value among the object probability values, identify the rotation angle of the target image indicated by the maximum probability value, adjust the display direction of the first image based on the rotation angle, and display the adjusted after the first image.
A computer storage medium, characterized in that the computer storage medium has a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps according to any one of claims 1-7 or 8-14.
An electronic device, characterized by comprising: a processor and a memory; wherein, the memory stores a computer program, and the computer program is suitable for being loaded and executed by the processor according to claims 1-7 or 8-14 Any one of the method steps.