CN117768766A

CN117768766A - Shooting method, video processing method and shooting device

Info

Publication number: CN117768766A
Application number: CN202311558042.2A
Authority: CN
Inventors: 王根源; 胡江坤
Original assignee: Insta360 Innovation Technology Co Ltd
Current assignee: Insta360 Innovation Technology Co Ltd
Priority date: 2023-11-20
Filing date: 2023-11-20
Publication date: 2024-03-26

Abstract

The application relates to a photographing method and photographing apparatus. The shooting method comprises the following steps: controlling shooting equipment to enter a video recording state so as to record video; in the video recording process, responding to the triggering operation of a user, and controlling the image sensor to be converted from a first output mode to a second output mode; at least one transition video frame is inserted at a corresponding cut-off position during mode switching of the image sensor during or after the video recording. The video processing method comprises the following steps: inserting at least one transition video frame at a cut-off position of the recorded video; and in the recording process of the recorded video, the image sensor of the shooting equipment pauses to acquire images at least once. The method can improve the video recording quality.

Description

Shooting method, video processing method and shooting device

Technical Field

The present disclosure relates to the field of video processing technologies, and in particular, to a shooting method, a video processing method, and a shooting device.

Background

With the rapid development of digital photography and video technology, the motion camera technology is also gradually reaching the corner of the world, so as to meet the demands of more and more outdoor sports, adventure and sports enthusiasts. The technology provides an intuitive interface and a plurality of shooting modes, so that a user can easily customize shooting effects and record wonderful moments. As motion camera technology continues to evolve, it has become a power tool for sports fans, travelers, and photographers to record their adventure and activity, thereby enabling more people to share exciting visual stories.

External factors may have an impact on the performance of a recording process using a motion camera, including sudden power fluctuations, network disturbances, or other wireless signal disturbances. These factors can cause the motion camera to switch modes or zoom, resulting in cut-out of recorded video, which in turn can affect recorded video quality and continuity.

Disclosure of Invention

In view of the foregoing, it is necessary to provide a shooting method, a video processing method, and a shooting apparatus capable of improving video recording quality.

In a first aspect, the present application provides a photographing method, applied to a photographing apparatus, the method including:

controlling shooting equipment to enter a video recording state so as to record video;

in the video recording process, responding to the triggering operation of a user, and controlling the image sensor to be converted from a first output mode to a second output mode;

at least one transition video frame is inserted at a corresponding cut-off position during mode switching of the image sensor during or after the video recording.

In one embodiment, the method comprises: and during the mode switching of the image sensor, acquiring an audio signal acquired by the sound receiving device.

In one embodiment, the method comprises:

and after the image sensor is switched to the second output mode, synchronizing the recorded video with the audio signal acquired by the radio receiving device.

In an embodiment, the method further comprises:

and acquiring at least one frame output video frame output by the first output mode, and generating the transition video frame according to the output video frame.

In an embodiment, the obtaining at least one frame output video frame output by the first output mode includes:

and acquiring a last frame of video frame output by the first output mode.

In an embodiment, the method further comprises:

and carrying out Gaussian blur processing on the output video frame to generate Gaussian blur video clips.

In an embodiment, the method further comprises:

after switching to the second output mode, processing a local image of an output video frame of the second output mode into a target video frame so as to realize zooming of the shooting equipment with a preset zooming magnification; the resolution of the target video frame is smaller than that of the output video frame of the second output mode; wherein,

the resolution of the image output by the first output mode is lower than that of the image output by the second output mode; and/or, the combining rate of the first output mode is higher than the combining rate of the second output mode.

In an embodiment, the resolution of the target video frame is substantially equal to the resolution of the output video frame of the first output mode.

In an embodiment, the method further comprises:

and responding to clicking operation of the interaction component of the shooting equipment, controlling the shooting equipment to zoom at a preset zooming multiplying power, wherein the preset zooming multiplying power is an integer.

In an embodiment, the interaction component of the photographing apparatus includes a touch screen;

the clicking operation includes: and executing clicking operation on the touch screen.

In an embodiment, the click operation includes a double click.

In one embodiment, the touch screen has a size of less than or equal to 3 inches.

In an embodiment, the photographing method further includes:

controlling the touch screen to display a first preview image, wherein the zooming magnification corresponding to the first preview image is a first magnification;

acquiring a target point in a first preview image according to the clicking position of the clicking operation; the target point is the center point of the target area;

and processing a local image of the target output video frame into the target video frame, wherein the local image corresponds to the target area.

In an embodiment, the photographing method further includes:

Controlling the touch screen to display a first preview image, wherein the first preview image is a zooming magnification which is a first magnification;

determining a target point in the first preview image according to the clicking position of the clicking operation; the target point is the center point of the target area;

and displaying a second preview image, wherein the center point of the second preview image is the target point, the first preview image is a zooming magnification which is a second magnification, and the second magnification is larger than the first magnification.

In an embodiment, the determining the target point in the first preview image according to the click position of the click operation includes:

determining a candidate area according to the click position, wherein the candidate area comprises the click position;

identifying salient objects in the candidate region;

and taking the position of the remarkable target as the target point.

In a second aspect, the present application further provides a photographing apparatus, including a camera, a memory, and a processor, where the memory stores a computer program, and the processor implements the steps of the photographing method when executing the computer program.

According to the shooting method and the shooting device, the shooting device is controlled to enter a video recording state to record video; then, in the video recording process, responding to the triggering operation of a user, and controlling the image sensor to be converted from a first output mode to a second output mode; inserting at least one transition video frame at a corresponding cut-off position during mode switching of the image sensor during video recording or after recording is finished; in the process of recording video, seamless picture switching is realized by controlling the image sensor to switch from a first output mode to a second output mode and inserting transition video frames during mode switching. The seamless switching can avoid the user from perceiving the picture incoherence and improve the watching comfort level. In addition, inserting transition video frames may add some special effects to the picture switching, such as fades, etc. The transition effect not only enables the picture to be smoother, but also improves the watching experience, and enables the whole video to be more vivid and interesting. Finally, the insertion of transitional video frames during image sensor mode switching helps to avoid abrupt picture changes, especially when there is a large difference between the first output mode and the second output mode. This can alleviate discomfort that a user may feel when switching modes, making the overall process more natural.

Drawings

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the related art, the drawings that are required to be used in the embodiments or the related technical descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and other drawings may be obtained according to the drawings without inventive effort for a person having ordinary skill in the art.

FIG. 1 is an internal block diagram of an electronic device in one embodiment;

FIG. 2 is a flow chart of a shooting method in one embodiment;

FIG. 3 is a flowchart of a shooting method according to another embodiment;

fig. 4 is a flowchart of a photographing method according to another embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more apparent, the present application will be further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the present application.

In an exemplary embodiment, a photographing apparatus, which may be a terminal, is provided, and an internal structure thereof may be as shown in fig. 1. The photographing apparatus includes a processor, a memory, an input/output interface, a communication interface, a display unit, and an input device. The processor, the memory and the input/output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein the processor of the photographing apparatus is used to provide computing and control capabilities. The memory of the photographing apparatus includes a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage media. The input/output interface of the photographing apparatus is used to exchange information between the processor and the external apparatus. The communication interface of the photographing device is used for performing wired or wireless communication with an external terminal, and the wireless communication can be realized through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a photographing method. The display unit of the shooting device is used for forming a visual picture, and can be a display screen, wherein the display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the shooting device can be a touch layer covered on the display screen or a key, a track ball or a touch pad arranged on the shell of the shooting device.

It will be appreciated by those skilled in the art that the structure shown in fig. 1 is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation of the photographing apparatus to which the present application is applied, and that a specific photographing apparatus may include more or less components than those shown in the drawings, or may combine some components, or have a different arrangement of components.

In an exemplary embodiment, as shown in fig. 2, a photographing method is provided, which is illustrated as an example of application of the method to the photographing apparatus of fig. 1, including the following S202 to S206. Wherein:

s202, controlling the shooting equipment to enter a video recording state so as to record videos.

In the embodiment of the application, the user can trigger the video recording through a touch screen interaction control set by the shooting equipment, or trigger the video recording through a video recording button of the shooting equipment. When the video mode is selected, a mode switch button on the camera device or corresponding setting on the touch screen is used, typically marked with a camera icon or similar identification. This helps to clarify the working state of the camera. Starting the recording may be accomplished by pressing a dedicated recording button on the capture device or a recording option on a touch screen. Such an operation is simple and intuitive, ensuring the start of recording. Care is taken to avoid rapid and erratic movements during recording, which helps to maintain video stability. If possible, the use of camera stabilizers can be considered, in particular when mobile shooting is required.

It should be noted that the settings of the video parameters, such as frame rate, aperture and exposure, ensure that they remain consistent throughout the recording process. This helps to avoid unnatural transitions in post editing.

S204, responding to the triggering operation of a user in the video recording process, and controlling the image sensor to switch from the first output mode to the second output mode.

Illustratively, the triggering operation may be a zoom triggering operation, the resolution of the image output by the first output mode being lower than the resolution of the image output by the second output mode; the first output mode has a lower image resolution than the second output mode. This means that in the first output mode the number of pixels is smaller, possibly for applications requiring lower resolution. The second output mode has a higher image resolution, providing more detail and more pixels. This applies to scenes with higher requirements for image quality. Alternatively, taking the first output mode as a 4k pixel merging output mode and the second output mode as an example, the image resolution corresponding to the 4k pixel merging output mode is 4k, and the image resolution corresponding to the 8k common output mode is 8k. In this example, the first output mode is a 4K pixel-wise combined output mode, corresponding to an image resolution of 4K. In this mode, adjacent pixel information is combined into one pixel, thereby generating a 4K image. The second output mode is an 8K normal output mode, corresponding to an image resolution of 8K. In this mode, the sensor captures the original 8K image without pixel binning.

In some embodiments, the first output mode has a higher merge rate than the second output mode; the first output mode has a higher merge rate than the second output mode. The merge rate refers to the number of frames per second (fps) or other indicator of the rate of image capture and transmission. This suggests that the first output mode may be more suitable in applications requiring higher frame rates or faster image transmission, but at the expense of some image quality.

The triggering operation comprises the following steps: a double click operation is performed on the photographing apparatus.

In the embodiment of the present application, in normal video recording, the photographing apparatus generally uses the first output mode, and the image resolution of this mode is relatively low. This helps reduce the size of the generated video file and improves recording efficiency because fewer pixels need to be processed. However, when the user triggers a zoom operation, such as double clicking on the screen or otherwise sending a zoom command, the capture device may sense this command and immediately stop recording. Once the video encoder stops recording, the photographing apparatus performs a key operation of switching the operation mode of the image sensor from the first output mode to the second output mode according to a zoom command of the user. This second output mode has a higher image resolution, typically higher than the first output mode. It therefore contains more pixels, so that the image has more detail and higher image quality. The key to this handover procedure is to add technical details. In the second output mode, the image sensor captures more pixels of data, which results in an image with more detail and sharpness. This is critical to the zoom operation because it allows the user to zoom in on a particular area without losing image quality. This also ensures that the user can capture more details and information to obtain a higher quality image. In the whole switching process, the logic of the shooting equipment is to improve the image quality according to the user instruction so as to meet the requirements of users, thereby providing more excellent visual experience.

Alternatively, in general, the preset zoom magnification is twice, and the first output mode may be a 4k pixel combined output mode, and the second output mode may be an 8k normal output mode. When lossless zooming is desired during recording of video, the screen is double-clicked. At this point, the video encoder stops recording. At this time, the photographing apparatus will perform switching of the sensor mode. It will switch from the previous 4K pixel binning output mode to the higher resolution 8K normal mode. In this mode, the sensor will begin capturing the original 8K image instead of the previously merged 4K image.

S206, inserting at least one transition video frame at a corresponding cut-off position during mode switching of the image sensor during or after video recording.

In the embodiment of the present application, before the image sensor is switched in mode, the photographing apparatus needs to detect the signal of the mode switching. This may be triggered by the user or the photographing device may be triggered based on certain conditions, such as sudden power supply fluctuations, network interference or other wireless signal interference, etc.

After the mode switching signal is detected, the photographing apparatus needs to prepare one or more transition video frames. These frames may be pre-designed to smooth transitions and may include fade-in fade-out effects, or other transition effects. At the key time of mode switching, the shooting device inserts transition video frames into the video stream. This may be achieved by mixing or replacing the transition video frames with the normal video frames.

The insertion of the transition video frame causes the picture not to be suddenly changed when the mode is switched, but gradually switched to a new mode through the transition effect. This helps to avoid the user perceiving discontinuities in the picture. Once the transitional video frame insertion is complete, the capture device may continue recording the normal video stream, at which point it has switched to a new mode.

In the shooting method, shooting equipment is controlled to enter a video recording state to record video; then, in the video recording process, responding to the triggering operation of a user, and controlling the image sensor to be converted from a first output mode to a second output mode; inserting at least one transition video frame at a corresponding cut-off position during mode switching of the image sensor during video recording or after recording is finished; in the process of recording video, seamless picture switching is realized by controlling the image sensor to switch from a first output mode to a second output mode and inserting transition video frames during mode switching. The seamless switching can avoid the user from perceiving the picture incoherence and improve the watching comfort level. In addition, inserting transition video frames may add some special effects to the picture switching, such as fades, etc. The transition effect not only enables the picture to be smoother, but also improves the watching experience, and enables the whole video to be more vivid and interesting. Finally, the insertion of transitional video frames during image sensor mode switching helps to avoid abrupt picture changes, especially when there is a large difference between the first output mode and the second output mode. This can alleviate discomfort that a user may feel when switching modes, making the overall process more natural.

In an exemplary embodiment, the method may further include: and during the mode switching of the image sensor, acquiring an audio signal acquired by the sound receiving device.

In the embodiment of the application, a trigger signal may be defined to indicate that the shooting device needs to switch the working mode of the image sensor. When the trigger signal is detected, the photographing apparatus sends an instruction to the image sensor, asking it to switch to another operation mode. This may involve configuration registers, communication protocols or other control means. And when the image sensor is switched to a mode, the sound receiving device is ensured to start to collect the audio signal. This may need to be achieved by a synchronization signal with the image sensor or a separate audio trigger signal. During the mode switching, it is ensured that the operation of the image sensor and the audio acquisition device is synchronized. This may be achieved by ensuring clock synchronization, sample rate agreement, etc. of the two. Ensuring that the acquired audio data and image data can be synchronously processed inside or outside the device. This may involve buffering, timestamp matching, data format conversion, etc. After processing the audio and video data, the shooting device can select to store the data on a storage medium or transmit the data to other devices through an output interface. This may include recording video files, real-time streaming, etc. Error handling mechanisms are implemented to handle problems that may occur during mode switching, such as data synchronization errors, lost data, etc.

It should be noted that, before the actual application, sufficient testing and tuning were performed. By simulating the mode switching situation, the normal collection and processing of the audio data is ensured, and the audio data cannot be influenced by the mode switching of the image sensor.

In the embodiment, seamless audio/video switching is realized by synchronizing the audio acquisition and the image sensor mode switching. The method can avoid the inconsistency perceived by the user during the mode switching, and improve the viewing experience of the user. The transition video frames are inserted during the mode switching period, and special effects can be added for picture switching in combination with synchronous acquisition of audio signals. The transition effect not only makes the picture smoother, but also improves the professional degree and makes the recorded video more attractive.

In an exemplary embodiment, the method may further include: after the image sensor is switched to the second output mode, the recorded video and the audio signals acquired by the radio receiving device are synchronized.

In the embodiment of the present application, after the image sensor is switched to the second output mode, the current audio/video recording process is stopped first. This ensures synchronicity of the recorded video and audio data. The photographing apparatus needs to monitor whether the image sensor successfully switches to the second output mode. This may be accomplished by monitoring the mode switch complete signal or waiting for a certain period of time. Once the mode switch is confirmed, calibration of the audio-video data is started. This involves calibrating the time stamps of the audio and video to ensure that they are aligned on the time axis. For possible delays in the recording process, the recording device needs to set appropriate buffering and alignment processes. This ensures the synchronicity of the audio and video, preventing mismatch between the two. Once the audio-video data calibration is completed, the recording process is restarted. Ensure that the image sensor and the radio device work normally in the new mode.

It should be noted that, in the whole process, an error handling mechanism is implemented to cope with possible problems, such as mode switching failure, data calibration error, etc. This may include rolling back to a previous state to avoid unsynchronized recorded data. In the recording process, the shooting equipment needs to monitor the synchronicity of the audio and video in real time and adjust the synchronicity. This helps to cope with changes and uncertainties that may occur when the device is running.

In the above embodiment, the video and the audio recorded synchronously can avoid the asynchronous or abrupt effect generated during the mode switching, so as to create a more seamless and natural audio-video experience and improve the viewing experience of the user. After the mode switch, if the audio and video are not synchronized, discomfort may be caused to the user. The synchronous audio/video can avoid the uncomfortable feeling and ensure the comfort and pleasure of the user in the watching process.

In an exemplary embodiment, the method may further include: and acquiring at least one frame of output video frame output by the first output mode, and generating a transition video frame according to the output video frame.

In an embodiment of the present application, the captured video frames of the first output mode are stored in a suitable location, such as a memory buffer. This ensures subsequent access to and processing of these frames. And generating a transition video frame by using the stored first output mode video frame and adopting a generation algorithm of the transition effect. This may include the following several common transitional effects: (1) fade in and fade out: by gradually changing the transparency of the first frame and the second frame, a smooth fade-in and fade-out effect is achieved. (2) erasure: starting from one area of the screen, gradually expanding or erasing, the first frame is transitioned to the second frame. (3) cutting: the screen is divided into a plurality of areas, and the second frame is switched from area to area, so that a cutting effect is formed. (4) deformation: and performing certain geometric deformation on the first frame to gradually change the first frame into the second frame, so as to form a deformation effect. (5) synthesizing a transition video frame: and synthesizing the generated transition video frame with the video frame of the second output mode. This may be done by image processing techniques such as pixel-by-pixel blending to ensure smooth and natural transitions.

In the above embodiment, by acquiring the video frame of the first output mode and generating the transition video frame, the smooth picture switching effect is achieved. This helps to avoid abrupt or inconsistent sensations upon mode switching, enhancing the user viewing experience. The generation of transition video frames may apply various transition effects such as fade-in, fade-out, erasure, cutting, etc., thereby increasing the artistry and ornamental value of the video content. At the time of mode switching, if the screen is suddenly changed, discomfort may be caused to the user. By generating the transition video frame, the impact of picture switching on a user can be reduced, discomfort is reduced, and transition is more natural.

In an exemplary embodiment, the acquiring the at least one frame output video frame output by the first output mode includes: and acquiring a last frame of video frame output by the first output mode.

In the embodiment of the present application, after stopping video capturing, the video data that has been captured needs to be processed. This may include storing the video frames in a buffer or other memory for subsequent access and processing. The last frame in the captured video is determined by the video processing unit or the storage control unit. This may be done by querying a storage device or looking up the last frame in memory. After the last frame is determined, it is stored in an appropriate location, such as a memory buffer, hard disk, or other storage device. This may be done by direct storage or transmission to a designated location.

After the last frame is acquired, subsequent processing or analysis work can be performed. This may include special effects processing of the image, extraction of key information, or other related video processing operations. Once the last frame is acquired and stored, the relevant resources need to be released in due time, e.g. buffer is closed, memory is released, so that the system can reallocate these resources.

In the above embodiment, by acquiring the last frame of video frame, a smooth transition effect can be achieved at the time of mode switching. This helps to avoid abrupt feel when switching pictures, improving the user viewing experience. The acquisition of the last frame helps to maintain consistency of the picture during the switching process, reducing the perceived asynchronism or unnatural sensation of the user. This is critical to improving overall video quality. Acquiring the last frame helps to avoid the sense of breakage of the picture at the time of mode switching. This is important to maintain the attention of the user and to provide a more coherent visual experience

In an exemplary embodiment, the method further comprises: the output video frames are subjected to a gaussian blur process to generate a gaussian blurred video segment.

The Gaussian blur processing is to blur an image by convolution operation using a Gaussian-check image to obtain a complementary frame image. This helps to eliminate discontinuities in sensor mode switching, providing smooth transitions and visual consistency. The degree of gaussian blur can be adjusted by the size and distribution of the gaussian kernels to meet specific requirements.

In the embodiment of the application, the shooting device can perform blurring processing on the output video frame by using a gaussian blurring processing algorithm to generate a gaussian-blurred video segment. Wherein the gaussian blur processing algorithm uses a special matrix called gaussian kernel. This gaussian kernel defines the degree of blurring. The size and mathematical distribution of the gaussian kernel determines the effect of blurring. In general, the larger the gaussian kernel, the stronger the blur. The gaussian kernel is applied to the last frame image. This is achieved by convolution operations. Specifically, for each pixel, the gaussian kernel is weighted averaged with the color values of surrounding pixels. The central pixel value of the kernel takes the maximum weight while the surrounding pixel values gradually decrease in weight, according to the shape of the gaussian distribution. Convolution results in a blurring effect in which the color value of each pixel is affected by its surrounding pixels. This causes the output video frame to become blurred as if the details in the image were blurred. After the gaussian blur process, a gaussian blurred video segment is obtained, which is a blurred version. This gaussian blur video segment is used for smooth transitions during sensor mode switching. When playing video, the gaussian blur video segment gradually transitions to the next sharp image, which makes the transition look smooth and natural.

In the above-described embodiment, the gaussian blur process causes the last frame image to become blurred, which helps create a smooth transitional effect. When the sensor mode is switched, the blurred last frame image gradually transits to the next clear image, so that the switching process is more natural and seamless. Sensor mode switching can lead to a change in image quality, often accompanied by momentary interruption. By using a gaussian blur process, blurring during switching does not give the user a noticeable feeling of interruption or jump in the image, thereby improving the viewing experience. The gaussian blur process helps keep the recorded video visually continuous, especially when zooming is required. This makes the transition from one scene to another smoother without distracting or uncomfortable for the user. The final objective of this technique is to provide a better user experience. By ensuring that there is no significant image interruption during the switch, and a smooth transition, the user can better enjoy the recorded video content without being disturbed by a discontinuity or abrupt sensation. In general, the gaussian blur process plays a key role during sensor mode switching, which helps to maintain image continuity and smoothness, improves the user's viewing experience, and ensures that video quality does not degrade during zooming.

In an exemplary embodiment, the method further comprises:

and responding to the triggering operation of the user, and controlling the shooting equipment to zoom at a preset zoom multiplying power.

For example, in response to a click operation of an interactive component of the photographing apparatus, the photographing apparatus is controlled to zoom at a preset zoom magnification, which is an integer. Illustratively, the triggering operation includes performing a double click operation on the photographing apparatus, setting the zoom magnification to be twice. The zooming may be referred to as digital zooming.

In this embodiment, the setting of the preset zoom magnification may be switched to the first zoom magnification when the user clicks (for example, double clicks) at the second zoom magnification, and may be switched to the second zoom magnification when the user clicks (for example, double clicks) at the first zoom magnification.

In the embodiment of the application, the shooting device needs to monitor the triggering operation of the user in real time, which may be triggered by performing a double-click operation on the shooting device. Next, the type of user-triggered operation is determined, as well as a preset zoom magnification desired by the user. This may include a user clicking, double clicking, or long clicking triggers with different gestures or buttons, each trigger type corresponding to a different zoom magnification. And adjusting according to the preset zoom multiplying power set by a user by controlling a zooming mechanism of the shooting equipment.

In the above-described embodiments, the user can easily realize zooming by a simple trigger operation without having to make a cumbersome manual adjustment in the camera setting. This simplifies the user's operational flow, providing a more friendly interactive experience.

In some embodiments, the size of the touch screen is less than or equal to 3 inches, and the clicking operation is a single click operation, a long press operation, or a double click operation on the touch screen, and the size of the touch screen is less than or equal to 3 inches. By the arrangement, the shooting device can be triggered to execute zooming operation on a smaller screen.

Illustratively, the touch screen has a size less than or equal to 2.5 inches, e.g., 1.5 inches, 2.0 inches, 2.25 inches, 2.27 inches, 2.3 inches, etc.

In an exemplary embodiment, the method may further include: after switching to the second output mode; processing a local image of the output video frame of the second output mode into a target video frame so as to realize zooming of the shooting equipment with a preset zooming multiplying power; the resolution of the target video frame is smaller than that of the output video frame of the second output mode; the resolution of the target video frame is approximately equal to the resolution of the output video frame of the first output mode.

In the embodiment of the application, it is necessary to monitor and confirm that the photographing apparatus has successfully switched to the second output mode, and ensure that the subsequent image processing is performed based on the correct output mode. In the second output mode, the capture device is required to capture the current output video frame. This may be done by an image sensor and a corresponding data processing unit. And calculating the resolution of the target video frame according to the preset zoom multiplying power. In general, the resolution of the target video frame should be smaller than the original output video frame of the second output mode to achieve a zoom effect. In the output video frame of the second output mode, an area where local image processing is required is determined. This area may be a central portion, or an area that varies according to a difference in zoom magnification. Image processing is performed on the defined local area, which may include reducing the image size, applying a blurring effect, or other image processing techniques suitable for achieving a zooming effect. The locally image processed region is combined with the rest of the original output video frame to create the target video frame. Ensuring that the resolution of the target video frame meets the previously calculated target resolution.

In the zooming process, the image on the shooting device can be updated and displayed in real time, and the zooming effect is displayed for the user. This helps to provide a more intuitive operating experience.

In the above embodiment, after switching to the second output mode; and processing the local image of the output video frame of the second output mode into a target video frame so as to realize zooming of the shooting equipment with a preset zooming multiplying power. Therefore, compared with the amplification mode of the traditional method, the image quality is not lost when zooming is performed. In addition, since the image resolution of the second output mode is significantly higher, more detail and a clearer image can be provided, enabling the user to capture more detail, such as fine textures or remote objects, which is important for photography and video recording.

In one exemplary embodiment, as shown in FIG. 3, the triggering operation includes: performing double-click operation on a touch screen of the shooting device, and displaying a preview video picture of the recorded video on the touch screen; the processing the local image of the output video frame of the second output mode into the target video frame includes: s302 to S304. Wherein:

s302, acquiring the double-click position of the touch screen and the corresponding preview video picture position.

In the embodiment of the present application, after the double-click operation is detected, the photographing apparatus needs to accurately capture the coordinate information of the double-click position. The coordinates of the double click position are mapped onto a preview video picture of the recorded video. This requires converting the screen coordinates to corresponding coordinates on the video frame, ensuring that subsequent image processing is performed at the correct location.

S304, processing the local image corresponding to the preview video picture position of the output video frame of the second output mode into a target video frame.

In the embodiment of the application, in the output video frame of the second output mode, according to the determined local area coordinates, the local image to be processed is intercepted. And performing image processing on the intercepted partial image. This may include applying special effects, adjusting brightness, contrast, etc. to achieve a zoom effect. And synthesizing the local image subjected to the local image processing with the rest part of the original output video frame to create a target video frame. And in the zooming process, updating the preview video picture on the display device in real time, and displaying the zooming effect to the user. Ensuring that the user can intuitively perceive the change in the image. When the target zoom magnification set by the user is reached or the user finishes the zooming operation, the zooming process is finished, and the stability of the equipment state is ensured.

In the above embodiment, the photographing device may capture the operation position of the user and map the operation position to the preview video screen in response to the double click operation on the touch screen, and then perform image processing on the local area corresponding to the position, so as to achieve the effect of zooming with the preset zoom magnification. This helps to promote the user interaction experience and flexibility of video editing.

In an exemplary embodiment, as shown in fig. 4, the above includes S304: s402 to S406.

Wherein:

s402, confirming a target area based on the position of the preview video picture.

S404, obtaining the salient points of the target area.

And S406, processing the local image corresponding to the salient point into a target video frame.

In the embodiment of the application, a range of a target area is defined according to the mapped preview video picture position. This range may be fixed or dynamically adjusted depending on the design of the application. Salient points may be extracted within the target region using computer vision techniques, such as feature point detection algorithms. These salient points may represent key features within the target area. And according to the position of the extracted salient point, intercepting a local image taking the salient point as a center. Ensuring that the truncated image covers a range around the salient point to preserve sufficient context information. And performing image processing on the intercepted partial image. This may include adjusting color, contrast, sharpening, etc. to achieve an optimized treatment of the target area. And synthesizing the local image subjected to the local image processing with the rest part of the original output video frame to create a target video frame. And in the zooming process, updating the preview video picture on the display device in real time, and displaying the zooming effect to the user. Ensuring that the user can intuitively perceive the change in the image. When the target zoom magnification set by the user is reached or the user finishes the zooming operation, the zooming process is finished, and the stability of the equipment state is ensured.

In the above-described embodiments, the region where the user wishes to zoom can be located more accurately by the position information acquired by the double-click operation and the salient point extraction based on the position of the preview video picture. This helps to ensure that the zooming effect occurs on the object of interest to the user. By carrying out local image processing by taking the salient point as the center, the system can keep the key information of the target area and avoid the information loss or distortion possibly caused by processing the whole picture.

In some embodiments, the photographing method further comprises: controlling the touch screen to display a first preview image, wherein the zooming magnification corresponding to the first preview image is a first magnification; acquiring a target point in a first preview image according to the clicking position of the clicking operation; the target point is the center point of the target area; and processing a local image of the target output video frame into the target video frame, wherein the local image corresponds to the target area.

For example, taking a double-click and fixed-focus lens as an example, when a user clicks at one position of the touch screen twice continuously, a zooming instruction is triggered, the clicking position has a corresponding relation with a picture of the first preview image, and the processor can cut a picture corresponding to the clicking position, corresponding to the coordinate of the clicking position, and the zooming magnification, and a next frame output video frame (i.e., a target output video frame) and the picture corresponding to the clicking position, so as to realize zooming. The principle of the zoom lens is similar, and only the picture after zooming is the picture corresponding to the clicking position, and therefore, the description is omitted. In this embodiment, the first magnification and the second magnification may be achieved by digital zooming, or may be achieved by switching the output modes of the image sensor as given in the above embodiment.

By the embodiment, the user can click which position, and the screen is positioned to the clicking position while zooming. Compared with the zooming of the corresponding positions realized by the double-finger separation and the double-finger gathering, the embodiment is more suitable for being realized on a small screen, and the dependence on the precision of the touch screen is reduced.

In some embodiments, the image needs to be previewed in the shooting process, and the shooting method further comprises: controlling the touch screen to display a first preview image, wherein the first preview image is a zooming magnification which is a first magnification; determining a target point in the first preview image according to the clicking position of the clicking operation; the target point is the center point of the target area; and displaying a second preview image, wherein the center point of the second preview image is the target point, the first preview image is a zooming magnification which is a second magnification, and the second magnification is larger than the first magnification.

In this embodiment, the second preview image may be an enlarged area of the next frame image of the first preview image. When realizing the preview, the user double clicks where to zoom to the position where the user is positioned.

For the above two embodiments, in some embodiments, the positioning of the target point may be implemented according to the position clicked by the user, if the clicked position can be accurately identified, the position is positioned, if the clicked position portion can be identified, the nearest position is positioned, and if the target area can be confirmed according to the preset zoom magnification and the target point. In addition, in the small-screen device, the finger of the user is thicker than the screen, and the embodiment can realize that the invalid positions around the clicking position are all adjusted to be the positions of the remarkable targets, so that the accuracy of the user for selecting the zooming position is improved.

In some other embodiments, the determining the target point in the first preview image according to the click position of the click operation includes: determining a candidate area according to the click position, wherein the candidate area comprises the click position; identifying salient objects in the candidate region; and taking the position of the remarkable target as the target point.

In this embodiment, the image of the first preview image may be identified, or the output video frame corresponding to the first preview image may be identified, and the target point may be automatically identified and displayed, so as to automatically focus on the target that the user wants to focus on. In addition, in the small-screen device, the finger of the user is thicker than the screen, and the click positions around the remarkable targets can be adjusted to the positions of the remarkable targets, so that the accuracy of selecting the zooming positions by the user is improved.

In some embodiments, a click operation may refer to a double click, single click, long press, etc. operation. For example, in the case of the first zoom magnification, when a click is detected, the zoom position may be confirmed and zoomed; when a click is detected in the case where the second zoom magnification is possible, the first zoom magnification is directly restored.

In some embodiments, the click operation may refer to a double click, where a single click may be used as a focus operation and a double click may be used as a zoom operation, better suited for small screen devices.

In an exemplary embodiment, a video processing method is provided, and the method is applied to the photographing apparatus in fig. 1, for example, and includes:

inserting at least one transition video frame at a cut-off position of the recorded video;

the image sensor of the shooting equipment pauses to collect images at least once in the recording process of the recorded video.

In the embodiment of the application, the shooting device needs to monitor the recording process of the recorded video in real time so as to process at the cut-off position. During the recording of the video, the image sensor of the photographing device is monitored for at least one pause in capturing the image. At the moment when the image sensor is detected to pause the image acquisition, the moment when the transition frame is inserted at the cut-off position of the recorded video is determined. This helps to ensure that the inserted transition frames match the fluency of the recorded video. At the determined occasion, at least one frame transition video frame is generated. The transition frame may be an image similar to the previous frame and the next frame, or a smooth transition effect may be achieved by special effects or gradual changes. The generated transition frame is inserted into a cut-out position of the recorded video. It is ensured that the insertion of transition frames does not cause a discontinuity in the video playback. Depending on the specific requirements of inserting transition frames, it may be necessary to adjust the playback parameters, such as frame rate, transition time, etc., to ensure consistency of the video playback effect. After the transition frame is inserted, the video may be previewed or played back in real time to confirm whether the transition effect is expected. This helps to adjust the parameters for optimal results. And after the cut-off position of the recorded video is inserted into at least one transition video frame, ending the whole processing process. Ensuring the integrity and playability of the video file.

The embodiment can ensure that the transition video frame is inserted in the cut-off position of the recorded video so as to smooth the playing of the transition video. Under the condition that the image sensor pauses to collect images, the experience of a user when watching videos can be improved through proper transition frame insertion.

A detailed embodiment is given below to explain the process of the shooting method and the video processing method in the embodiment of the present application, and on the basis of the above embodiment, the implementation process of the method may include the following:

s1, controlling shooting equipment to enter a video recording state so as to record videos.

S2, in the video recording process, responding to double-click operation of a user, and controlling the image sensor to be converted into a second output mode from a first output mode; the resolution of the image output by the first output mode is lower than that of the image output by the second output mode; and/or the combining rate of the first output mode is higher than the combining rate of the second output mode.

S3, obtaining a last frame of video frame output by the first output mode, and carrying out Gaussian blur processing on the output video frame to generate a Gaussian blur video segment.

S4, inserting Gaussian blur video clips at corresponding cut-off positions during mode switching of the image sensor during video recording or after recording is finished.

S5, during the mode switching period of the image sensor, the audio signals collected by the sound receiving device are obtained at the same time.

S7, after the image sensor is switched to the second output mode, synchronizing the recorded video and the audio signals acquired by the radio receiving device.

S8, after switching to the second output mode, processing a local image of an output video frame of the second output mode into a target video frame so as to realize zooming of the shooting equipment with a preset zooming multiplying power; acquiring a double-click position of the touch screen and a corresponding preview video picture position; obtaining salient points of a target area, and processing local images corresponding to the salient points into target video frames; the resolution of the target video frame is approximately equal to the resolution of the output video frame of the first output mode.

It should be understood that, although the steps in the flowcharts related to the above embodiments are sequentially shown as indicated by arrows, these steps are not necessarily sequentially performed in the order indicated by the arrows. The steps are not strictly limited to the order of execution unless explicitly recited herein, and the steps may be executed in other orders. Moreover, at least some of the steps in the flowcharts described in the above embodiments may include a plurality of steps or a plurality of stages, which are not necessarily performed at the same time, but may be performed at different times, and the order of the steps or stages is not necessarily performed sequentially, but may be performed alternately or alternately with at least some of the other steps or stages.

In one exemplary embodiment, a photographing apparatus is provided, including a camera, a memory, and a processor, the memory storing a computer program therein, the processor implementing the following steps when executing the computer program:

in the video recording process, responding to the triggering operation of a user, and controlling the image sensor to be converted from a first output mode to a second output mode; the resolution of the image output by the first output mode is lower than that of the image output by the second output mode; and/or the combining rate of the first output mode is higher than the combining rate of the second output mode;

In one embodiment, the processor when executing the computer program further performs the steps of:

and during the mode switching of the image sensor, acquiring an audio signal acquired by the sound receiving device.

after the image sensor is switched to the second output mode, the recorded video and the audio signals acquired by the radio receiving device are synchronized.

and acquiring at least one frame of output video frame output by the first output mode, and generating a transition video frame according to the output video frame.

and acquiring a last frame of video frame output by the first output mode.

the output video frames are subjected to a gaussian blur process to generate a gaussian blurred video segment.

a double click operation is performed on the photographing apparatus.

after switching to the second output mode, processing a local image of an output video frame of the second output mode into a target video frame so as to realize zooming of the shooting equipment with a preset zooming multiplying power; the resolution of the target video frame is smaller than that of the output video frame of the second output mode; the resolution of the target video frame is approximately equal to the resolution of the output video frame of the first output mode.

acquiring a double-click position of the touch screen and a corresponding preview video picture position;

and processing the local image corresponding to the preview video picture position of the output video frame of the second output mode into a target video frame.

confirming a target area based on the preview video picture position;

obtaining salient points of a target area;

and processing the local image corresponding to the salient point as a target video frame.

inserting at least one transition video frame at a cut-off position of the recorded video; the image sensor of the shooting equipment pauses to collect images at least once in the recording process of the recorded video.

It should be noted that, the user information (including, but not limited to, user equipment information, user personal information, etc.) and the data (including, but not limited to, data for analysis, stored data, presented data, etc.) referred to in the present application are information and data authorized by the user or sufficiently authorized by each party, and the collection, use, and processing of the related data are required to meet the related regulations.

Those skilled in the art will appreciate that implementing all or part of the above-described methods in accordance with the embodiments may be accomplished by way of a computer program stored on a non-transitory computer readable storage medium, which when executed may comprise the steps of the embodiments of the methods described above. Any reference to memory, database, or other medium used in the various embodiments provided herein may include at least one of non-volatile and volatile memory. The nonvolatile Memory may include Read-Only Memory (ROM), magnetic tape, floppy disk, flash Memory, optical Memory, high density embedded nonvolatile Memory, resistive random access Memory (ReRAM), magnetic random access Memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric Memory (Ferroelectric Random Access Memory, FRAM), phase change Memory (Phase Change Memory, PCM), graphene Memory, and the like. Volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, and the like. By way of illustration, and not limitation, RAM can be in the form of a variety of forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), and the like. The databases referred to in the various embodiments provided herein may include at least one of relational databases and non-relational databases. The non-relational database may include, but is not limited to, a blockchain-based distributed database, and the like. The processors referred to in the embodiments provided herein may be general purpose processors, central processing units, graphics processors, digital signal processors, programmable logic units, quantum computing-based data processing logic units, etc., without being limited thereto.

The technical features of the above embodiments may be arbitrarily combined, and all possible combinations of the technical features in the above embodiments are not described for brevity of description, however, as long as there is no contradiction between the combinations of the technical features, they should be considered as the scope of the description.

The foregoing examples represent only a few embodiments of the present application, which are described in more detail and are not thereby to be construed as limiting the scope of the present application. It should be noted that it would be apparent to those skilled in the art that various modifications and improvements could be made without departing from the spirit of the present application, which would be within the scope of the present application. Accordingly, the scope of protection of the present application shall be subject to the appended claims.

Claims

1. A photographing method applied to a photographing apparatus, the method comprising:

2. The photographing method according to claim 1, characterized in that the method comprises: and during the mode switching of the image sensor, acquiring an audio signal acquired by the sound receiving device.

3. The photographing method according to claim 2, characterized in that the method comprises:

4. The photographing method according to claim 1, characterized in that the method further comprises:

5. The photographing method of claim 4, wherein said acquiring at least one frame output video frame output by said first output mode comprises:

and acquiring a last frame of video frame output by the first output mode.

6. The photographing method of claim 4, wherein the method further comprises:

7. The photographing method according to claim 1, characterized in that the method further comprises:

8. The photographing method of claim 7, wherein,

the resolution of the target video frame is approximately equal to the resolution of the output video frame of the first output mode.

9. The photographing method according to any one of claims 1 to 8, characterized in that the method further comprises:

10. The photographing method of claim 9, wherein the interactive component of the photographing apparatus comprises a touch screen;

11. The photographing method of claim 10, wherein the click operation comprises a double click.

12. The method of photographing as claimed in claim 10, wherein the size of the touch screen is less than or equal to 3 inches.

13. The photographing method of claim 10, wherein the photographing method further comprises:

14. The photographing method of claim 10, wherein the photographing method further comprises:

15. The photographing method of claim 13 or 14, wherein the determining the target point in the first preview image according to the click position of the click operation includes:

identifying salient objects in the candidate region;

and taking the position of the remarkable target as the target point.

16. A photographing apparatus comprising a camera, a memory and a processor, the memory storing a computer program, characterized in that the processor implements the steps of the method of any of claims 1 to 15 when executing the computer program.