CN110868550B

CN110868550B - Photographing method and terminal device electronic device

Info

Publication number: CN110868550B
Application number: CN201911168964.6A
Authority: CN
Inventors: 胡鹏翔
Original assignee: Vivo Mobile Communication Hangzhou Co Ltd
Current assignee: Vivo Mobile Communication Hangzhou Co Ltd
Priority date: 2019-11-25
Filing date: 2019-11-25
Publication date: 2021-04-06
Anticipated expiration: 2039-11-25
Also published as: CN110868550A

Abstract

The invention provides a photographing method and electronic equipment. The method is applied to the electronic equipment, a shooting module of the electronic equipment comprises a first sensor and a second sensor, and the first sensor is used for collecting a contour image of a moving object; the method comprises the following steps: acquiring at least two frame contour images of a target moving object acquired by a first sensor; acquiring a target movement speed of a target movement object based on at least two frames of contour images; when the target movement speed is smaller than or equal to a preset threshold value, controlling a second sensor to perform exposure according to a target exposure frame rate matched with the target movement speed, and generating a first target image of a target moving object; and when the target movement speed is greater than a preset threshold value, controlling the second sensor to expose to generate a first intermediate image, and generating a second target image of the target moving object based on the first intermediate image and the second intermediate image, wherein the second intermediate image is a contour image of the target moving object. The invention can optimize the photographing effect on the high-speed moving object.

Description

Photographing method and terminal device electronic device

Technical Field

The embodiment of the invention relates to the technical field of image processing, in particular to a photographing method and electronic equipment.

Background

The exposure mode of the photosensitive element of the current electronic equipment is rolling shutter exposure, that is, the Sensor (Sensor) of the camera scans line by line and exposes line by line until all the pixels are exposed.

The electronic device of this imaging method needs a certain time for exposure, and when a user needs to photograph a fast moving object, the photographing shutter needs to be triggered manually. In the case that the moving speed of the target moving object is high, it is often difficult to capture the moving object, or the captured image is too blurred, so that the moving object is captured with poor effect.

Disclosure of Invention

The embodiment of the invention provides a photographing method and electronic equipment, and aims to solve the problem that a photographing effect is poor due to the fact that a moving object is difficult to capture in a photographing picture or the captured image of the moving object is fuzzy in the photographing method in the related art.

In order to solve the technical problem, the invention is realized as follows:

in a first aspect, an embodiment of the present invention provides a photographing method applied to an electronic device.

The shooting module of the electronic equipment comprises a first sensor and a second sensor, wherein the first sensor is used for acquiring a contour image of a moving object; the method comprises the following steps:

acquiring at least two frame contour images of a target moving object acquired by the first sensor;

acquiring a target motion speed of the target motion object based on the at least two frames of contour images;

under the condition that the target movement speed is smaller than or equal to a preset threshold value, controlling the second sensor to perform exposure according to a target exposure frame rate matched with the target movement speed, and generating a first target image of the target moving object;

and under the condition that the target movement speed is greater than a preset threshold value, controlling the second sensor to expose to generate a first intermediate image, and generating a second target image of the target moving object based on the first intermediate image and the second intermediate image, wherein the second intermediate image is a contour image of the target moving object.

In a second aspect, an embodiment of the present invention further provides an electronic device.

The shooting module of the electronic equipment comprises a first sensor and a second sensor, wherein the first sensor is used for acquiring a contour image of a moving object; the electronic device includes:

the first acquisition module is used for acquiring at least two frames of contour images of the target moving object acquired by the first sensor;

the second acquisition module is used for acquiring the target movement speed of the target moving object based on the at least two frames of contour images;

the first control module is used for controlling the second sensor to perform exposure according to a target exposure frame rate matched with the target motion speed under the condition that the target motion speed is smaller than or equal to a preset threshold value, and generating a first target image of the target motion object;

and the second control module is used for controlling the second sensor to be exposed to generate a first intermediate image under the condition that the target movement speed is greater than a preset threshold value, and generating a second target image of the target moving object based on the first intermediate image and the second intermediate image, wherein the second intermediate image is a contour image of the target moving object.

In a third aspect, an embodiment of the present invention further provides an electronic device, including: a memory, a processor and a computer program stored on the memory and executable on the processor, the computer program, when executed by the processor, implementing the steps of the photographing method.

In a fourth aspect, the embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when being executed by a processor, the computer program implements the steps of the photographing method.

In the embodiment of the invention, the first sensor is used for acquiring at least two frames of contour images matched with contour information of a target moving object, the at least two frames of contour images are used for acquiring a target moving speed matched with the target moving object, and under the condition that the target moving speed is less than or equal to a preset threshold value, the second sensor is controlled to expose at a target exposure frame rate matched with the target moving speed, so that a first target image generated by the exposure of the second sensor can acquire a clear image of the target moving object, and the photographing effect of the moving object is optimized. When the moving object is shot quickly, the user can be ensured to shoot the moving object, and the moving object is shot clearly as much as possible. In addition, under the condition that the target moving speed is greater than the preset threshold value, a first intermediate image which does not contain the target moving object is acquired by the second sensor, and then a second target image of the target moving object is generated by combining a frame of contour image of the target moving object acquired by the first sensor, namely the second intermediate image, so that when a high-speed moving object is shot, the contour of the moving object is shot clearly, the clear contour of the target moving object moving at high speed is captured, and the shooting effect of the moving object is optimized.

Drawings

FIG. 1 is a flow chart of a method of taking a picture in accordance with one embodiment of the present invention;

FIG. 2 is a block diagram of an electronic device of one embodiment of the invention;

FIG. 3 is a block diagram of an electronic device of another embodiment of the invention;

fig. 4 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention.

Detailed Description

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, not all, embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.

Referring to fig. 1, a flowchart of a photographing method according to an embodiment of the present invention is shown, and is applied to an electronic device, where a photographing module of the electronic device includes a first sensor and a second sensor, and the first sensor is used for acquiring a contour image of a moving object;

the two sensors can be configured in the same camera of the electronic device, and can also be configured in different cameras. For ease of understanding, the first sensor and the second sensor are configured in the same camera for illustration purposes hereinafter.

In one example, the first sensor of an embodiment of the present invention is a neuro-mimicry visual sensor (NVS, now commonly referred to as DVS); the second sensor is a roller shutter type exposure sensor.

DVS has very little demand for data storage and computational resources and it has very low latency, which can reach the microsecond level, with higher recognition and recording for fast moving objects. The DVS can only record moving objects in a shot scene, and has no perception capability on a static background picture. Specifically, the DVS may capture the moving object and obtain contour information of the moving object. However, a series of frame pictures are generated by the conventional rolling shutter sensor at a fixed frequency, and due to the limitation of the frame rate, more key information is lost between two adjacent frames, so that a moving object is difficult to capture during photographing, or a photographed image of the moving object is blurred.

DVS uses an event-driven based approach to capture dynamic changes in a photographic scene. Unlike conventional cameras, DVS does not have the concept of "frames". When a change occurs in the shooting scene, the DVS generates some pixel-level outputs (i.e., events), and one event specifically includes (t, x, y, p), where x, y are pixel coordinates of the event in a 2D (two-dimensional) space (i.e., two-dimensional contour coordinates of the moving object being shot), t is a time stamp of the event, and p is a polarity of the event. The polarity of the event represents a change in brightness of the scene, either up (positive) or down (negative). One event corresponds to one frame of image, which may include two-dimensional coordinates of a moving object.

In the embodiment of the present invention, the purpose of photographing a fast moving object can be achieved by monitoring and recording a dynamic object through DVS and combining pictures taken by a Sensor with rolling shutter exposure, which is described below by taking an electronic device as an example of a mobile phone, and for convenience of description, a first Sensor (DVS) is named as Sensor B and a second Sensor (a Sensor with rolling shutter exposure) is named as Sensor a.

The method shown in fig. 1 may comprise the steps of:

step 101, acquiring at least two frames of contour images of a target moving object acquired by a first sensor;

the target moving object is a moving object needing to acquire a contour image.

The at least two frames of contour images are at least two frames of images of the target moving object acquired by Sensor B. The time for the Sensor B to acquire the at least two frames of images may be acquired in real time or may be acquired in advance, which is not limited in the present invention.

Optionally, before step 101, after the user starts the camera application of the mobile phone, the method according to the embodiment of the present invention may start Sensor a and Sensor B of the shooting module, where the preview interface of the Sensor a is a picture of the shooting scene (for example, a front picture of a camera), and the preview interface of the Sensor B may only display a moving object in the shooting scene, and when there is no moving object in the shooting scene, that is, when there is no moving object in the shooting picture, the preview interface of the Sensor B does not display any content.

Because the shooting module is provided with two sensors, when images collected by the two sensors are displayed, the display method I or the display method II can be adopted for realizing the display method I or the display method II.

The display method comprises the following steps:

displaying a contour image of the moving object acquired by the first sensor in a first area of a shooting preview interface, and displaying a preview picture acquired by the second sensor in a second area of the shooting preview interface;

for example, a shooting preview interface of a mobile phone is divided into left and right areas: the left area displays the contour image of the moving object acquired by the Sensor B, and the right area displays the preview picture acquired by the Sensor A; or the right area displays the outline image of the moving object acquired by the Sensor B, and the left area displays the preview picture acquired by the Sensor A. Of course, the present invention does not limit the dividing manner of the two regions of the shooting preview interface.

And a second display method:

and under the condition that the first sensor does not acquire the contour image of the moving object, displaying a preview picture acquired by the second sensor on a shooting preview interface, and under the condition that the first sensor acquires the contour image of the moving object, displaying the contour image acquired by the first sensor on the shooting preview interface.

For example, after the camera application is started, if the Sensor B does not detect a moving object in the shooting scene, the mobile phone may receive a notification from the Sensor B that there is no moving object, and then the mobile phone switches the display content of the shooting preview interface to the preview screen captured by the Sensor a. When the Sensor B detects a moving object in the shooting scene, the mobile phone may receive a notification from the Sensor B that the moving object exists, and the mobile phone switches the display content of the shooting preview interface to the contour image of the moving object acquired by the Sensor B.

Namely: and if no moving object is in the shooting picture, displaying a normal preview. When the moving object appears, a preview picture of the Sensor B is displayed, and a user can clearly observe the moving state of the moving object by viewing the preview interface of the Sensor B.

According to the embodiment of the invention, the images acquired by the two sensors are respectively displayed in the areas in the shooting preview interface, or when a moving object exists in the shooting picture, the outline image of the moving object acquired by the first sensor is displayed in the shooting preview interface, and when the moving object does not exist in the shooting picture, the preview image acquired by the second sensor is displayed in the shooting preview interface, so that two image display modes of the shooting preview interface are provided, and a user can conveniently view a target moving object to be shot from the shooting preview interface.

Optionally, in an embodiment, in the case that the at least two frames of images in step 101 are acquired in real time by Sensor B, when step 101 is executed, the following steps may be implemented:

firstly, receiving a shooting request, wherein the shooting request comprises contour information of a target moving object;

the shooting request may be triggered automatically by the system or by the user.

When the shooting request is automatically triggered by the system, the profile information in the shooting request can be profile information preset by the system;

when the shooting request is triggered by the user, the profile information in the shooting request can be profile information input by the user or profile information determined according to the input of the user.

When the contour information is contour information input by a user, referring to the above embodiment, in the first display method, the user can simultaneously view a contour image of a moving object and a preview picture of a shooting scene in a shooting preview interface of a camera application; in the second display method, the user can view the outline image of the moving object or the preview picture of the shooting scene in a shooting preview interface of the camera application in time.

The user can select a target moving object desired to be photographed after seeing the display contents in the photographing preview interface, and thus, the user can input the contour information of the target moving object.

In one embodiment, the profile information may be input by the system providing a list of profiles of a plurality of types of photographic subjects for selection by a user who selects a profile of one type of photographic subject from the list. The selected contour is carried into a photographing request, and thus, the photographing request includes contour information of the target moving object selected by the user.

For example, the list includes contour images of a photographed object such as a cow, a horse, a sheep, a man, a woman, a child, etc., and the user selects a contour of the horse, for example. Of course, in some embodiments, a plurality of profiles may be included for one type of photographic subject, and the number of profiles preset for one type of photographic subject is not limited by the present invention.

In another embodiment, as for the input mode of the outline information, the user may also trigger the floating display of the drawing board window on the preview interface through a preset input when shooting the preview interface.

Then, the user can use the virtual brush of the drawing board window to draw the shape of the target moving object in a self-defined manner in the drawing board, and then the method of the embodiment of the invention can match with big data about the shape after receiving the shape information, so as to determine the type of the contour information of the object, which is the shape information input by the user, and obtain the contour information of the target moving object. The outline information may be more accurate than shape information input by a user using a brush.

After the user inputs the contour information of the target moving object in any one of the two manners, the method of the embodiment of the invention can automatically trigger the shooting request, or manually trigger the shooting request by pressing and inputting the shooting button in the camera preview interface when the user needs to shoot the target moving object. The system may load the profile information of the target moving object previously input by the user into the photographing request so that the photographing request includes the profile information of the target moving object.

Since the Sensor B (i.e., the first Sensor) can capture any moving object in the captured scene to generate an event, in order to avoid erroneous judgment of the output of the Sensor B caused by interference of other moving objects except for the target moving object, the capture request needs to carry the contour information of the target moving object.

And then controlling the first sensor to acquire at least two frames of contour images of the target moving object matched with the contour information.

In this step, the first Sensor, i.e. Sensor B, may record the moving object to a subtle level, and it is very easy for Sensor B to detect whether a moving object appears in the captured picture, therefore, in this step, only the contour information of the target moving object needs to be input to Sensor B, so that Sensor B is controlled to collect only an event about the target moving object, where the contour information matches (e.g. the similarity is greater than a preset threshold) the contour information of the target moving object, here, at least two frames of images of the target moving object, and since the image output by Sensor B depicts the contour information, at least two frames of images are named as at least two frames of contour images.

In the embodiment of the invention, when at least two frames of contour images of a target moving object acquired by a first sensor are acquired, a shooting request is received, and the first sensor can shoot any moving object in a shooting scene to generate an event, so that in order to avoid erroneous judgment of SensorB output caused by interference of other moving objects except the target moving object, in the embodiment of the invention, the shooting request carries contour information of the target moving object, so that the first sensor can be controlled to acquire at least two frames of images of the target moving object matched with the contour information in response to the shooting request, and the image shooting accuracy of the target moving object is improved.

102, acquiring a target movement speed of the target moving object based on the at least two frames of contour images;

since the first Sensor, that is, the Sensor B, can record at a level of microseconds, after the target moving object enters the shooting picture of the Sensor B, the Sensor B can capture one frame of contour image of the target moving object in each unit time (at least one microsecond), and therefore, the target moving speed of the target moving object can be obtained according to the displacement condition of the target moving object between the at least two frames of contour images and the unit time.

The target movement speed may be a speed value or a speed interval, which is not limited in the present invention.

103, controlling the second sensor to perform exposure according to a target exposure frame rate matched with the target motion speed under the condition that the target motion speed is less than or equal to a preset threshold value, and generating a first target image of the target motion object;

the motion speed and the exposure frame rate have a corresponding relationship, and therefore, the preset threshold may be the motion speed corresponding to the maximum exposure frame rate supported by the second sensor.

In the case that the target moving speed is less than or equal to the preset threshold, that is, in the case that the second sensor supports the target exposure frame rate, the second sensor may be controlled to perform exposure at the target exposure frame rate matching the target moving speed, so as to generate a first target image of the target moving object.

For example, if the maximum frame rate supported by the second Sensor, i.e. Sensor a, is greater than or equal to the target exposure frame rate, it indicates that Sensor a supports the target exposure frame rate, and therefore, Sensor a may be controlled to perform exposure at the target exposure frame rate, so that the first target image generated by Sensor a exposure may include a clear target moving object.

When the Sensor a is controlled to perform exposure at the target exposure frame rate, since the target exposure frame rate can be determined according to the exposure time and ISO (sensitivity), the exposure time and ISO of the Sensor a can be adjusted to make the frame rate of the Sensor a reach the target exposure frame rate.

And 104, controlling the second sensor to expose to generate a first intermediate image when the target movement speed is greater than a preset threshold, and generating a second target image of the target moving object based on the first intermediate image and the second intermediate image, wherein the second intermediate image is a contour image of the target moving object.

The preset threshold may be a motion speed corresponding to a maximum exposure frame rate supported by the second Sensor, and if the target motion speed is greater than the preset threshold, it indicates that the maximum exposure frame rate supported by the second Sensor is also smaller than the target exposure frame rate, so that if the second Sensor, i.e., Sensor a, is used for exposure, it is also difficult to capture a clear image of the target moving object, and therefore, in this step, the second Sensor may be controlled to expose and generate a first intermediate image (where the adopted exposure frame rate may be a system default exposure frame rate), where the first intermediate image corresponds to a background image where the target moving object is located. And the second intermediate image can be a frame of contour image of the target moving object acquired by the first sensor in advance, and the first intermediate image and the second intermediate image are used for generating a second target image of the target moving object.

Optionally, when the second sensor is controlled to perform exposure at the target exposure frame rate matched with the target motion speed in step 103, the target exposure frame rate matched with the target motion speed may be obtained according to a preset corresponding relationship between the exposure frame rate and the speed, so as to control the second sensor to perform exposure at the target exposure frame rate.

The exposure frame rate in the corresponding relationship may be a frame rate value or a frame rate range, and similarly, the speed in the corresponding relationship may also be a speed value or a speed range. The correspondence relationship may be a direct correspondence relationship between the frame rate and the speed, or an indirect correspondence relationship between the frame rate and the speed.

The description is given by taking the correspondence relationship as an indirect correspondence relationship between the frame rate range and the speed range as an example: table 1 shows the correspondence between the speed range and the speed level, and table 2 shows the correspondence between the frame rate range and the speed level.

Speed rating	Speed range
		1	20～30
2	31～50
		3	51～70
4	71～90
		5	Over 90

TABLE 1

The unit of the speed in the speed range can be flexibly set according to the requirement, and the invention is not limited to this.

Frame rate range	Speed rating
		0～60	1
61～100	2
		101～150	3
151～200	4
		201～240	5

TABLE 2

The direct correspondence between the frame rate and the speed means that the frame rate and the speed directly correspond to each other, and the indirect correspondence is not performed through the speed level. Of course, in the direct correspondence, the frame rate may be a frame rate value or a frame rate range, and the speed may be a speed value or a speed range.

In addition, when the target motion speed is a target speed interval, the speed in the corresponding relation between the frame rate and the speed is a speed range, and when the target exposure frame rate corresponding to the target motion speed is determined, a target speed range with the highest matching degree with the target speed interval in the corresponding relation is acquired, and then, the frame rate value corresponding to the target speed range is determined as the target exposure frame rate, or the maximum frame rate in the frame rate range corresponding to the target speed range is determined as the target exposure frame rate.

In one embodiment, when the frame rate in the preset correspondence relationship between the exposure frame rate and the speed is a frame rate range, the target exposure frame rate is the maximum frame rate in the target exposure frame rate range corresponding to the target movement speed.

In the embodiment of the invention, a target exposure frame rate matched with the target motion speed is acquired based on the corresponding relation between the preset exposure frame rate and the speed, and the second sensor is controlled to perform exposure at the target exposure frame rate. Since the frame rate and the speed in the corresponding relationship are matched, the exposure speed of the target exposure frame rate can be matched with the target movement speed of the target moving object, so that a clear static image of the target moving object can be captured conveniently by using the second sensor.

Alternatively, in one embodiment, the correspondence between the exposure frame rate and the speed may be configured according to the frame rate range supported by the second Sensor, i.e., Sensor a. For example, when the electronic device is shipped from the factory, the correspondence relationship is configured according to the frame rate range supported by Sensor a. Therefore, the target exposure frame rate obtained by using the above correspondence relationship is necessarily the frame rate supported by the Sensor a.

Alternatively, in another embodiment, the correspondence between the exposure frame rate and the speed may be set regardless of the frame rate range supported by Sensor a, which is generated by directly referring to the image of the moving object at which speed can be captured at a certain frame rate, without referring to the frame rate range supported by Sensor a. In this embodiment scenario, the obtained target exposure frame rate matching the target motion speed may exceed the maximum frame rate supported by Sensor a, so that there is a case that Sensor a does not support the target exposure frame rate, that is, the target motion speed is greater than a preset threshold, in which case the step 104 is adopted to implement the photographing method according to the embodiment of the present invention.

The preset corresponding relation between the exposure frame rate and the speed can be automatically and dynamically adjusted according to the actual shooting effect, so that a clearer moving object can be shot through the adjusted corresponding relation. The correspondence between the preset exposure frame rate and the speed may also be dynamically adjusted manually by the user.

Alternatively, when step 102 is executed, it may be implemented by: acquiring displacement information of the target moving object based on the at least two frames of contour images; acquiring a frame interval duration T of the first sensor; and calculating the target movement speed of the target moving object according to the displacement information and the frame interval duration.

Specifically, since the interval time (i.e., the unit time described above) of each frame of image acquired by the first sensor, i.e., the sensor B, is in the order of microseconds and is fixed, the time interval (i.e., the frame interval duration) between two adjacent frames of profile images is denoted as T.

In addition, the movement displacement of the target moving object between two frame contour images is also measurable, and the displacement of the target moving object between at least two frame contour images is recorded as Lx.

For example, the at least two frames of contour images sequentially comprise an image 1, an image 2, an image 3 and an image 4 from morning to evening according to the exposure time; and the frame interval between two adjacent frames of images is T.

In the embodiment of the invention, the displacement of the target moving object is obtained by utilizing at least two frames of contour images, and the target moving speed of the target moving object can be effectively calculated by combining the frame interval duration of the first sensor. Since the data for calculating the target moving speed is derived from the parameter information of the first sensor for capturing the contour image, i.e., the frame interval duration, and the contour image captured by the first sensor, the accuracy and the acquisition efficiency of the target moving speed can be ensured to some extent.

Alternatively, in one embodiment, the displacement information includes at least two target displacements of the target moving object at different time periods, and then when the target moving speed of the target moving object is calculated based on the displacement information and the frame interval duration, at least two moving speeds of the target moving object may be calculated based on the at least two target displacements and the frame interval duration, respectively; and determining the average value of the at least two motion speeds as the target motion speed of the target motion object.

Specifically, in an example, the time intervals corresponding to the different time periods are the same, for example, all of T, or all of 2T, or all of 3T. Taking the time interval as T as an example, when calculating the target movement speed of the target moving object, the target movement speed may be calculated using a plurality of sets of two adjacent frame images. For example, if the displacement of the target moving object from the position in the image 1 to the position in the image 2 is L1, the speed of the target moving object from the position in the image 1 to the position in the image 2 is L1/T ═ v 1;

similarly, the displacement of the target moving object from the position in the image 2 to the position in the image 3 is L2, and the speed of the target moving object from the position in the image 2 to the position in the image 3 is L2/T-v 2;

similarly, the displacement of the target moving object from the position in the image 3 to the position in the image 4 is L3, and the speed of the target moving object from the position in the image 3 to the position in the image 4 is L3/T-v 3;

the target moving speed of the target moving object may be a speed section (i.e., a speed section consisting of the minimum speed and the maximum speed among the three) consisting of v1, v2 and v3, or the target moving speed may be an average value of v1, v2 and v3, i.e., (v1+ v2+ v 3)/3.

In another example, the time intervals corresponding to the different time periods may also be different, for example, the displacement information includes displacement 1, displacement 2, and displacement 3, where the time interval corresponding to displacement 1 is 3T, the time interval corresponding to displacement 2 is 2T, and the time interval corresponding to displacement 3 is T.

That is, when calculating the target moving speed of the target moving object, it may be calculated from two non-adjacent frame contour images, for example, the displacement 1 of the target moving object from the position in the image 1 to the position in the image 4 is calculated as L4, and the time interval from the image 1 to the image 4 is calculated as 3 × T, so that the speed of the target moving object from the position in the image 1 to the position in the image 4 is calculated as L4/3T — v 4. The displacement 2 is L5, the corresponding speed is L5/2T-v 5, the displacement 3 is L6, and the corresponding speed is L6/T-v 6, so the target movement speed is (v4+ v5+ v 6)/3.

In the embodiment of the present invention, when the target movement speed is calculated by using the displacement information and the frame interval time length, since the displacement information includes at least two target displacements of the target moving object in different time lengths, in order to improve the calculation accuracy of the target movement speed, it is possible to avoid data unreliability in which a single speed is used as the target movement speed by calculating at least two movement speeds and taking the average value thereof as the target movement speed.

Optionally, in another embodiment, the displacement information may also be a target displacement of the target moving object in the same time period. Therefore, the calculation efficiency of the target movement speed can be improved.

Optionally, in an embodiment, when the step of obtaining the displacement information of the target moving object based on the at least two frames of contour images is performed, at least two sets of contour coordinate information corresponding to the target moving object in the at least two frames of contour images may be obtained, where any set of contour coordinate information includes coordinate information of each contour point of the target moving object; then, based on the at least two sets of contour coordinate information, displacement information of the target moving object is determined.

Specifically, when determining the displacement information of the target moving object, the displacement information of the target moving object between at least two frames of contour images can be determined according to at least two sets of contour coordinate information respectively corresponding to the target moving object in at least two frames of contour images, because the sizes of the contour images of different frames are consistent, and therefore, the displacement information of the target moving object between at least two frames of contour images can be directly determined by calculating the displacement between at least two sets of contour coordinate information.

The contour coordinate information may be two-dimensional contour coordinate information or three-dimensional contour coordinate information, and thus, the displacement information may be two-dimensional displacement information or three-dimensional displacement information.

Since the contour image output by the first Sensor, i.e., Sensor B, can carry the two-dimensional contour coordinates of the target moving object, when calculating the two-dimensional displacement information, the two-dimensional displacement can be calculated by directly using the two-dimensional contour coordinates.

If the mobile phone can also acquire the depth information of the target moving object, the depth coordinates corresponding to the contour of the target moving object in at least two frame contour images can be acquired, so that the three-dimensional contour coordinates corresponding to the target moving object in the same frame image can be determined by combining the depth coordinates and the two-dimensional contour coordinates of the target moving object in the same frame image, and the three-dimensional displacement of the target moving object between at least two frame contour images can be calculated according to at least two sets of three-dimensional contour coordinates corresponding to the target moving object in at least two frame contour images.

Optionally, when obtaining the depth information of the target moving object, if the two sensors are integrated on the same first camera and the electronic device is further provided with a second camera, that is, the electronic device has at least two cameras, then the depth coordinate of the contour of the target moving object can be obtained by using double shooting; or, in another embodiment, the electronic device has only one camera, and the TOF (Time of flight ranging) may be used to obtain the depth coordinates of the contour of the target moving object.

In the embodiment of the present invention, the displacement information of the target moving object between the at least two frames of contour images can be determined by using at least two sets of contour coordinate information corresponding to the target moving object in the at least two frames of contour images, so that the calculation accuracy of the displacement information can be improved.

Optionally, when step 104 is executed, in a case that the target moving speed is greater than a preset threshold and the first sensor detects that the target moving object leaves the shooting scene, the second sensor may be controlled to expose to generate a first intermediate image, and a second target image of the target moving object may be generated based on the first intermediate image and the second intermediate image, where the second intermediate image is a contour image of the target object.

If the target motion speed is greater than the preset threshold, it indicates that the maximum exposure frame rate supported by the second Sensor, i.e. Sensor a, is less than the target exposure frame rate, i.e. Sensor a performs exposure with the maximum frame rate, and it is also difficult to capture a clear image of the target moving object with the target motion speed. Then, in order to make the first intermediate image generated by exposing the second Sensor, i.e. Sensor a, clear without the blurred image of the target moving object moving at a high speed, the second intermediate image can be exposed by Sensor a again when Sensor B detects that the target moving object leaves the shooting scene, so that the generated first intermediate image is a clear background image without the target moving object. The finally generated second target image then comprises a clear outline of the target moving object, and a clear background image of the target moving object.

When Sensor B detects that the target moving object leaves the shooting scene (i.e. when the preview interface of Sensor B has no content), the electronic device may receive a notification from Sensor B indicating that the target moving object does not exist in the shooting scene, and then the electronic device may control Sensor a to perform exposure, i.e. shoot the shooting scene, where the generated first intermediate image does not include the target moving object, for example, a static background image except the target moving object. The background image of the target moving object of the second target image generated by the embodiment of the invention is clearer.

Alternatively, in another embodiment, when performing step 104, it can be implemented by:

selecting one frame of contour image in the at least two frames of contour images as a second intermediate image;

before step 104 is executed, the electronic device already controls the Sensor B to acquire at least two frames of contour images of the target moving object, so that in this step, the electronic device can randomly select one frame of image from the at least two frames of contour images as a second intermediate image; or selecting an image with the clearest outline from the at least two frame outline images as a second intermediate image.

Acquiring target contour coordinate information of the target moving object in the second intermediate image;

wherein the electronic device acquires target contour coordinate information (here, two-dimensional contour coordinates) of the target moving object in the target contour image.

The execution sequence between the steps of selecting the second intermediate image and acquiring the target contour coordinate information is not limited by the invention.

Drawing the contour of the target moving object in the first intermediate image based on the target contour coordinate information, and generating a second target image of the target moving object.

Since the Sensor a and the Sensor B are configured on the same camera, the sizes and coordinate systems of the images acquired by the two sensors are the same, and therefore, in this step, the contour of the target moving object can be drawn in a static background image (i.e., a first intermediate image) shot by the Sensor a according to the target contour coordinate information, so as to generate a second target image including the target moving object.

During rendering, a Graphics Processing Unit (GPU) may be used to render the contour in the first intermediate image according to the target contour coordinate information, thereby generating a second target image.

In the embodiment of the present invention, when the first intermediate image and the second intermediate image are synthesized, the image processing apparatus may be implemented in a manner that the contour information of the target moving object in the second intermediate image is drawn into the first intermediate image, so that the obtained second target image includes a position where the target moving object has stopped shortly before, and the obtained second target image ensures a high degree of restoration of the position of the target moving object in the background image.

In addition, in the embodiment of the present invention, a photographing strategy may be flexibly selected according to the magnitude of the target motion speed of the target moving object, and if the target exposure frame rate corresponding to the target motion speed is supported by the second sensor (that is, the target motion speed is less than or equal to the preset threshold), the second sensor may be directly controlled to expose at the target exposure frame rate to generate the first target image including the sharp target moving object; if the target exposure frame rate corresponding to the target motion speed cannot be supported by the second sensor, that is, the target exposure frame rate exceeds the maximum frame rate of the second sensor (that is, the target motion speed is greater than the preset threshold), a second target image containing a clear outline of the target moving object may be captured.

Optionally, in another embodiment, the triggering of the step 103 and the step 104 may also be that the user manually selects a corresponding photographing strategy for photographing by manually resolving the movement speed of the target moving object. For example, if the user determines that the moving speed of the target moving object is very high, the photographing strategy 2 may be selected to trigger the execution of step 104, whereas if the user determines that the moving speed of the target moving object is not very high, the photographing strategy 1 may be selected to trigger the execution of step 103.

Optionally, when the step of generating the second target image of the target moving object by drawing the contour of the target moving object in the first intermediate image based on the target contour coordinate information is performed, the following steps may be performed:

drawing the contour of the target moving object in the first intermediate image according to the target contour coordinate information;

receiving a color input to a target moving object, the color input comprising a target color;

in response to the color input, rendering the color of the region in the first intermediate image that is within the outline to the target color, generating a second target image of the target moving object.

In the embodiment of the invention, when the fast moving object is shot, the shape of the moving object (even the fast moving object) can be shot clearly. And the user is supported to input the custom color of the target moving object, so that the target image generated by the embodiment of the invention not only can capture the clear shape contour of the target moving object moving rapidly, but also can carry out custom configuration on the color of the target moving object, thereby improving the photographing characteristic of the moving object.

Optionally, the method of the embodiment of the present invention may provide an adjustment operation of different colors of the body region of the target moving object in the UI interface.

In the embodiment of the present invention, not only the shape of the target moving object (even the high-speed moving object) can be clearly photographed when the high-speed moving object is photographed, but also the color of the photographed target moving object can be flexibly adjusted.

Referring to FIG. 2, a block diagram of an electronic device 300 of one embodiment of the invention is shown. The shooting module of the electronic device 300 includes a first sensor and a second sensor, and the first sensor is used for collecting the contour image of the moving object. The electronic device 300 according to the embodiment of the present invention can implement the details of the photographing method in the above-described embodiments, and achieve the same effect. The electronic device 300 shown in fig. 2 includes:

a first obtaining module 301, configured to obtain at least two frame contour images of a target moving object acquired by the first sensor;

a second obtaining module 302, configured to obtain a target motion speed of the target moving object based on the at least two frames of contour images;

the first control module 303 is configured to control the second sensor to perform exposure according to a target exposure frame rate matched with the target motion speed when the target motion speed is less than or equal to a preset threshold value, so as to generate a first target image of the target moving object;

a second control module 304, configured to control the second sensor to expose to generate a first intermediate image if the target moving speed is greater than a preset threshold, and generate a second target image of the target moving object based on the first intermediate image and the second intermediate image, where the second intermediate image is a contour image of the target moving object.

Optionally, the first obtaining module 301 includes:

the receiving submodule is used for receiving a shooting request, and the shooting request comprises contour information of a target moving object;

and the first control sub-module is used for controlling the first sensor to collect at least two frames of contour images of the target moving object matched with the contour information.

Optionally, the first control module 303 includes:

the first acquisition submodule is used for acquiring a target exposure frame rate matched with the target motion speed according to the corresponding relation between the preset exposure frame rate and the speed;

and the second control submodule is used for controlling the second sensor to carry out exposure according to the target exposure frame rate.

Optionally, the second obtaining module 302 includes:

the second obtaining submodule is used for obtaining displacement information of the target moving object based on the at least two frames of contour images;

a third obtaining submodule, configured to obtain a frame interval duration of the first sensor;

and the calculating submodule is used for calculating the target movement speed of the target moving object based on the displacement information and the frame interval duration.

Optionally, the second obtaining sub-module includes:

the acquisition unit is used for acquiring at least two groups of contour coordinate information corresponding to the target moving object in the at least two frames of contour images, wherein any one group of contour coordinate information comprises coordinate information of each contour point of the target moving object;

and the first determining unit is used for determining the displacement information of the target moving object based on the at least two sets of contour coordinate information.

Optionally, the displacement information includes at least two target displacements of the target moving object at different time periods;

the calculation submodule includes:

a calculating unit, configured to calculate at least two motion speeds of the target moving object based on the at least two target displacements and the frame interval duration, respectively;

a second determining unit, configured to determine an average value of the at least two motion speeds as a target motion speed of the target moving object.

Optionally, the second control module 304 includes:

the selecting submodule is used for selecting one frame of contour image in the at least two frames of contour images as a second intermediate image;

the fourth obtaining submodule is used for obtaining target contour coordinate information of the target moving object in the second intermediate image;

and the drawing sub-module is used for drawing the contour of the target moving object in the first intermediate image based on the target contour coordinate information and generating a second target image of the target moving object.

Optionally, as shown in fig. 3, the electronic device 300 further includes:

a first display module 305, configured to display a contour image of the moving object acquired by the first sensor in a first area of a shooting preview interface, and display a preview image acquired by the second sensor in a second area of the shooting preview interface;

the second display module 306 is configured to display a preview picture acquired by the second sensor on a shooting preview interface when the first sensor does not acquire the contour image of the moving object, and display the contour image acquired by the first sensor on the shooting preview interface when the first sensor acquires the contour image of the moving object.

The electronic device 300 provided in the embodiment of the present invention can implement each process implemented by the electronic device in the above method embodiments, and is not described here again to avoid repetition.

The electronic equipment acquires at least two frames of contour images matched with contour information of a target moving object by using the first sensor through the module, acquires a target moving speed of the target moving object by using the at least two frames of contour images, and controls the second sensor to expose according to a target exposure frame rate matched with the target moving speed under the condition that the target moving speed is less than or equal to a preset threshold value, so that a first target image generated by the exposure of the second sensor can acquire a clear image of the target moving object, and the photographing effect of the moving object is optimized. When the moving object is shot quickly, the user can be ensured to shoot the moving object, and the moving object is shot clearly as much as possible. In addition, under the condition that the target moving speed is greater than the preset threshold value, a first intermediate image which does not contain the target moving object is acquired by the second sensor, and then a second target image of the target moving object is generated by combining a frame of contour image of the target moving object acquired by the first sensor, namely the second intermediate image, so that when a high-speed moving object is shot, the contour of the moving object is shot clearly, the clear contour of the target moving object moving at high speed is captured, and the shooting effect of the moving object is optimized.

Fig. 4 is a schematic hardware configuration diagram of an electronic device implementing various embodiments of the present invention, the electronic device has a shooting module 412, the shooting module 412 includes a first sensor 4121 and a second sensor 4122, and the first sensor 4121 is used for collecting a contour image of a moving object.

The electronic device 400 also includes, but is not limited to: radio frequency unit 401, network module 402, audio output unit 403, input unit 404, sensor 405, display unit 406, user input unit 407, interface unit 408, memory 409, processor 410, and power supply 411. Those skilled in the art will appreciate that the electronic device configuration shown in fig. 4 does not constitute a limitation of the electronic device, and that the electronic device may include more or fewer components than shown, or some components may be combined, or a different arrangement of components. In the embodiment of the present invention, the electronic device includes, but is not limited to, a mobile phone, a tablet computer, a notebook computer, a palm computer, a vehicle-mounted terminal, a wearable device, a pedometer, and the like.

A processor 410 for acquiring at least two frames of contour images of the target moving object acquired by the first sensor 4121; acquiring a target motion speed of the target motion object based on the at least two frames of contour images; when the target movement speed is less than or equal to a preset threshold value, controlling the second sensor 4122 to perform exposure according to a target exposure frame rate matched with the target movement speed, and generating a first target image of the target moving object; and when the target movement speed is greater than a preset threshold value, controlling the second sensor 4122 to expose to generate a first intermediate image, and generating a second target image of the target moving object based on the first intermediate image and the second intermediate image, wherein the second intermediate image is a contour image of the target moving object.

It should be understood that, in the embodiment of the present invention, the radio frequency unit 401 may be used for receiving and sending signals during a message sending and receiving process or a call process, and specifically, receives downlink data from a base station and then processes the received downlink data to the processor 410; in addition, the uplink data is transmitted to the base station. Typically, radio unit 401 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. Further, the radio unit 401 can also communicate with a network and other devices through a wireless communication system.

The electronic device provides wireless broadband internet access to the user via the network module 402, such as assisting the user in sending and receiving e-mails, browsing web pages, and accessing streaming media.

The audio output unit 403 may convert audio data received by the radio frequency unit 401 or the network module 402 or stored in the memory 409 into an audio signal and output as sound. Also, the audio output unit 403 may also provide audio output related to a specific function performed by the electronic apparatus 400 (e.g., a call signal reception sound, a message reception sound, etc.). The audio output unit 403 includes a speaker, a buzzer, a receiver, and the like.

The input unit 404 is used to receive audio or video signals. The input Unit 404 may include a Graphics Processing Unit (GPU) 4041 and a microphone 4042, and the Graphics processor 4041 processes image data of a still picture or video obtained by an image capturing apparatus (such as a camera) in a video capturing mode or an image capturing mode. The processed image frames may be displayed on the display unit 406. The image frames processed by the graphic processor 4041 may be stored in the memory 409 (or other storage medium) or transmitted via the radio frequency unit 401 or the network module 402. The microphone 4042 may receive sound, and may be capable of processing such sound into audio data. The processed audio data may be converted into a format output transmittable to a mobile communication base station via the radio frequency unit 401 in case of the phone call mode.

The electronic device 400 also includes at least one sensor 405, such as light sensors, motion sensors, and other sensors. Specifically, the light sensor includes an ambient light sensor that adjusts the brightness of the display panel 4061 according to the brightness of ambient light, and a proximity sensor that turns off the display panel 4061 and/or the backlight when the electronic apparatus 400 is moved to the ear. As one type of motion sensor, an accelerometer sensor can detect the magnitude of acceleration in each direction (generally three axes), detect the magnitude and direction of gravity when stationary, and can be used to identify the posture of an electronic device (such as horizontal and vertical screen switching, related games, magnetometer posture calibration), and vibration identification related functions (such as pedometer, tapping); the sensors 405 may also include a fingerprint sensor, a pressure sensor, an iris sensor, a molecular sensor, a gyroscope, a barometer, a hygrometer, a thermometer, an infrared sensor, etc., which will not be described in detail herein.

The display unit 406 is used to display information input by the user or information provided to the user. The Display unit 406 may include a Display panel 4061, and the Display panel 4061 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like.

The user input unit 407 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the electronic device. Specifically, the user input unit 407 includes a touch panel 4071 and other input devices 4072. Touch panel 4071, also referred to as a touch screen, may collect touch operations by a user on or near it (e.g., operations by a user on or near touch panel 4071 using a finger, a stylus, or any suitable object or attachment). The touch panel 4071 may include two parts, a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 410, receives a command from the processor 410, and executes the command. In addition, the touch panel 4071 can be implemented by using various types such as a resistive type, a capacitive type, an infrared ray, and a surface acoustic wave. In addition to the touch panel 4071, the user input unit 407 may include other input devices 4072. Specifically, the other input devices 4072 may include, but are not limited to, a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a track ball, a mouse, and a joystick, which are not described herein again.

Further, the touch panel 4071 can be overlaid on the display panel 4061, and when the touch panel 4071 detects a touch operation thereon or nearby, the touch operation is transmitted to the processor 410 to determine the type of the touch event, and then the processor 410 provides a corresponding visual output on the display panel 4061 according to the type of the touch event. Although in fig. 4, the touch panel 4071 and the display panel 4061 are two independent components to implement the input and output functions of the electronic device, in some embodiments, the touch panel 4071 and the display panel 4061 may be integrated to implement the input and output functions of the electronic device, and the implementation is not limited herein.

The interface unit 408 is an interface for connecting an external device to the electronic apparatus 400. For example, the external device may include a wired or wireless headset port, an external power supply (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, an audio input/output (I/O) port, a video I/O port, an earphone port, and the like. The interface unit 408 may be used to receive input (e.g., data information, power, etc.) from an external device and transmit the received input to one or more elements within the electronic apparatus 400 or may be used to transmit data between the electronic apparatus 400 and an external device.

The memory 409 may be used to store software programs as well as various data. The memory 409 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the cellular phone, and the like. Further, the memory 409 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 410 is a control center of the electronic device, connects various parts of the entire electronic device using various interfaces and lines, performs various functions of the electronic device and processes data by operating or executing software programs and/or modules stored in the memory 409 and calling data stored in the memory 409, thereby performing overall monitoring of the electronic device. Processor 410 may include one or more processing units; preferably, the processor 410 may integrate an application processor, which mainly handles operating systems, user interfaces, application programs, etc., and a modem processor, which mainly handles wireless communications. It will be appreciated that the modem processor described above may not be integrated into the processor 410.

The electronic device 400 may further include a power supply 411 (e.g., a battery) for supplying power to various components, and preferably, the power supply 411 may be logically connected to the processor 410 through a power management system, so as to implement functions of managing charging, discharging, and power consumption through the power management system.

In addition, the electronic device 400 includes some functional modules that are not shown, and are not described in detail herein.

Preferably, an embodiment of the present invention further provides an electronic device, which includes a processor 410, a memory 409, and a computer program that is stored in the memory 409 and can be run on the processor 410, and when being executed by the processor 410, the computer program implements each process of the above-described photographing method embodiment, and can achieve the same technical effect, and in order to avoid repetition, details are not described here again.

The embodiment of the present invention further provides a computer-readable storage medium, where a computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the computer program implements the processes of the above-mentioned photographing method embodiment, and can achieve the same technical effects, and in order to avoid repetition, the descriptions thereof are omitted here. The computer-readable storage medium may be a Read-Only Memory (ROM), a Random Access Memory (RAM), a magnetic disk or an optical disk.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.

Through the above description of the embodiments, those skilled in the art will clearly understand that the method of the above embodiments can be implemented by software plus a necessary general hardware platform, and certainly can also be implemented by hardware, but in many cases, the former is a better implementation manner. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which is stored in a storage medium (such as ROM/RAM, magnetic disk, optical disk) and includes instructions for enabling a terminal (such as a mobile phone, a computer, a server, an air conditioner, or a network device) to execute the method according to the embodiments of the present invention.

While the present invention has been described with reference to the embodiments shown in the drawings, the present invention is not limited to the embodiments, which are illustrative and not restrictive, and it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A shooting method is applied to electronic equipment and is characterized in that a shooting module of the electronic equipment comprises a first sensor and a second sensor, wherein the first sensor is used for collecting a contour image of a moving object; the method comprises the following steps:

2. The method of claim 1, wherein said acquiring at least two frames of contour images of a target moving object acquired by said first sensor comprises:

receiving a shooting request, wherein the shooting request comprises contour information of a target moving object;

and controlling the first sensor to acquire at least two frames of contour images of the target moving object matched with the contour information.

3. The method of claim 1, wherein the controlling the second sensor to perform exposure at a target exposure frame rate corresponding to the target motion speed comprises:

acquiring a target exposure frame rate matched with the target motion speed according to the corresponding relation between the preset exposure frame rate and the speed;

and controlling the second sensor to perform exposure according to the target exposure frame rate.

4. The method according to claim 1, wherein the obtaining the target motion speed of the target moving object based on the at least two frames of contour images comprises:

acquiring displacement information of the target moving object based on the at least two frames of contour images;

acquiring the frame interval duration of the first sensor;

and calculating the target movement speed of the target moving object based on the displacement information and the frame interval duration.

5. The method according to claim 4, wherein the obtaining displacement information of the target moving object based on the at least two frames of contour images comprises:

acquiring at least two groups of contour coordinate information corresponding to the target moving object in the at least two frames of contour images, wherein any one group of contour coordinate information comprises coordinate information of each contour point of the target moving object;

and determining displacement information of the target moving object based on the at least two sets of contour coordinate information.

6. The method of claim 4, wherein the displacement information includes at least two target displacements of the target moving object at different time periods;

the calculating the target movement speed of the target moving object based on the displacement information and the frame interval duration comprises:

respectively calculating at least two movement speeds of the target moving object based on the at least two target displacements and the frame interval duration;

and determining the average value of the at least two motion speeds as the target motion speed of the target motion object.

7. The method of claim 1, wherein generating a second target image of the target moving object based on the first and second intermediate images comprises:

8. The method of claim 1, further comprising:

alternatively, the first and second electrodes may be,

9. The electronic equipment is characterized in that a shooting module of the electronic equipment comprises a first sensor and a second sensor, wherein the first sensor is used for acquiring a contour image of a moving object; the electronic device includes: