WO2014067422A1

WO2014067422A1 - Processing method and apparatus for image data collection

Info

Publication number: WO2014067422A1
Application number: PCT/CN2013/085914
Authority: WO
Inventors: 葛建强
Original assignee: 华为终端有限公司
Priority date: 2012-11-01
Filing date: 2013-10-24
Publication date: 2014-05-08
Also published as: CN103795892A; CN103795892B

Abstract

Disclosed are a processing method and apparatus for image data collection, including: acquiring the collected image data, and the inclination angle when the image collection device collecting the image data; adjusting the direction of the collected image data according to the inclination angle, the direction of the adjusted image data being vertical forward when displaying the output; saving the adjusted image data. Embodiments of the present invention can ensure that regardless whether the image data collection device collecting the image data is in the vertical forward direction, the collected image data stored and outputted to the display device always displays in the vertical forward direction state. Users conveniently apply the image data collection device to obtain the video or picture data for easy watching.

Description

Processing method and device for collecting image data

The present invention relates to an image processing technology, and in particular, to a method and an apparatus for processing image data. Background technique

With the development of electronic technology, there are more and more electronic devices with image data acquisition functions. These electronic devices may be in any non-vertical forward direction during the acquisition of image data. This causes the acquired image data to be in a non-vertical forward direction during the output display.

Suppose someone uses a mobile phone to shoot a video of a concert, or use a mobile phone to walk along the roadside landscape, because the phone is not strictly in the vertical or horizontal direction during the shooting (may tilt a large angle, or even reverse) , will cause the recorded video to be skewed during playback, which greatly reduces the visual experience of watching the video. For example, as shown in Figure 1, the phone is initially tapped, and the captured video is also forward when played on the computer; after that, the phone is tilted or rotated, and the video captured is on the computer. Deflection occurs during playback, even completely reversed. Among them, the left side of the small arrow in Figure 1 is the angle diagram of the mobile phone in real-time shooting, the right side of the small arrow is the corresponding screen when playing on the computer, the big arrow in Figure 1 indicates the direction change during the shooting of the mobile phone. The process and the corresponding change process of the screen displayed on the computer.

If the photographer is always positive relative to gravity in order to play the captured video, the handset must be kept in a vertical or horizontal direction and cannot be freely rotated, which in turn will result in a decrease in the photographer's electronic device application experience.

It can be seen that, in the prior art, it is not convenient to ensure that the image data collected by the electronic device is displayed in the correct direction during output display, thereby reducing the user's application experience. Summary of the invention

SUMMARY OF THE INVENTION It is an object of the present invention to provide a method and apparatus for processing image data to ensure that image data acquired by a tilted image capture device can be displayed in the correct orientation during output display.

The object of the invention is achieved by the following technical solutions:

A processing method for collecting image data, comprising:

Acquiring acquired image data, and a tilt angle when the image collecting device collects the image data;

Performing direction adjustment on the collected image data according to the tilt angle, and the direction of the adjusted image data in the output display is a vertical forward direction; Save the adjusted image data.

A processing device for collecting image data, comprising:

An image data acquiring module, configured to acquire the collected image data;

a tilt angle detecting module, configured to detect a tilt angle when the image collecting device collects the image data;

And an image data direction adjustment module, configured to perform direction adjustment on the acquired image data acquired by the image data acquiring module according to the tilt angle detected by the tilt angle detecting module, and the adjusted image data is vertically oriented in the output display manner. Positive

And a storage module, configured to save the image data adjusted by the image data direction adjustment module.

It can be seen from the technical solution provided by the present invention that the technical solution provided by the embodiment of the present invention can ensure that the image capturing device is in the vertical direction when the image data is collected, and the collected image data is output to the display device after being stored. The time is in a vertical forward state, which is convenient for the user to obtain video or picture data for viewing by using the image capturing device. DRAWINGS

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention, Those skilled in the art can also obtain other drawings based on these drawings without any creative work.

1 is a schematic diagram of an effect of acquiring an image and displaying it in the prior art;

2 is a schematic diagram 1 of a process flow according to an embodiment of the present invention;

FIG. 3 is a schematic diagram 2 of a process flow according to an embodiment of the present disclosure;

4 is a schematic diagram of an effect of collecting an image and displaying the image according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of an apparatus according to an embodiment of the present invention. detailed description

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.

In the embodiment of the present invention, the image data collected at the time of shooting can be corrected to ensure the collection. The video or picture corresponding to the image data can always maintain the correct view direction during playback or browsing. The method for processing image data collected by the embodiment of the present invention, as shown in FIG. 2, may include: Step 21: acquiring acquired image data, and a tilt angle when the image capturing device collects the image data, and corresponding image data. Can be a video or a picture;

Step 22: Perform direction adjustment on the collected image data according to the tilt angle, and the direction of the adjusted image data in the output display is a vertical forward direction;

In this step, the processing method for performing direction adjustment on the collected image data may adopt any one of the following processing methods:

Method 1: According to the obtained tilt angle, the collected image data is rotated to rotate the image data to a vertical forward direction, that is, the video image or the picture corresponding to the output image data is vertically positive after the rotation Afterwards, the rotated image data is also subjected to size cropping processing so that the corresponding image data meets the predetermined display size requirement;

Method 2: According to the obtained tilt angle, using computer graphics polygon cutting technology, the image data is subjected to size cropping along the direction of the tilt angle, and assuming that the tilt angle is a degree, the computer graphics polygon cutting technique is utilized. The captured image data is cropped along a direction of tilting a degree; wherein, the corresponding computer graphics polygon cropping technique comprises: rearranging the image data after the size cropping process, to arrange the cropped image data The cloth is vertically forward image data, that is, the image data is subjected to size cropping processing by computer graphics polygon cropping technology, and then the cropped vertical forward image data can be obtained.

Specifically, the corresponding size cropping process may determine the size of the captured image data according to the size of the predetermined image data, or may determine the collection according to the size of the narrowest part of the image corresponding to the collected image data. The size of the image data is cropped, or the size of the captured image data may be determined according to other predetermined manners to obtain image data of a desired size while ensuring that the cropping process can be performed.

Further, the adjusted image data can also be saved.

The image data processed by the above step 22 will be displayed in the vertical direction in the display screen during the output display, thereby facilitating the user to view; Moreover, the adjustment process of the entire image data is automatically completed without manual participation, and is convenient. The user application image acquisition device improves the user's application experience.

Optionally, in the specific application process, the embodiment of the present invention may further include: determining whether the direction adjustment function is enabled before performing direction adjustment on the image data, and only touching the pair of images when the direction adjustment function is enabled. Data processing for direction adjustment. That is, when the user wants to adjust the direction of the image data according to the tilt angle, the direction adjustment function is turned on, otherwise, the direction adjustment function is turned off. Further, in order to facilitate the user to preview the collected image data in the process of acquiring the image data, the method may further include a process of previewing the collected image data, the process may include:

(1) according to the tilt angle at which the image acquisition device collects the corresponding image data, the adjusted image data obtained through the processing in step 22 is rotated in the opposite direction, that is, the degree value of the tilt angle is reversely rotated, assuming When the tilt angle is a degree, the inverse rotation is a degree (or rotation - a degree);

(2) According to the size of the display screen of the image acquisition device, the image data after the rotation processing is resized so that the corresponding image data can match the size of the effective display area in the display screen to ensure the optimal preview display. Effect

Specifically, the diagonal length of the picture corresponding to the image data may be adjusted to be smaller than or equal to the size of the narrowest display area of the display screen of the image capturing device;

(3) Display the image data after the size adjustment in step (2).

The technical solution provided by the embodiment of the present invention can solve the problem that the captured video or photo is tilted or even reversed during playback or browsing due to tilting or inversion when the electronic device is photographed. That is, the image data such as the captured video or photo is corrected to ensure that the image data such as video or photo can always be in the vertical direction during playback or browsing, so that the user can face the corresponding video or photo. Moreover, in the shooting preview phase, the processing by the above scheme can also be displayed with the same effect.

The electronic device (i.e., image capturing device) in the embodiment of the present invention may be, but not limited to, a mobile phone, a tablet computer, a camera or a video camera, and the like. For ease of understanding, the embodiment of the present invention will be further described in detail below with reference to the accompanying drawings, taking a mobile phone as an image capturing device as an example.

The application scenario of this embodiment is shown in the upper part of the dotted line in FIG. 3, that is, the camera captures the image data of one picture frame when the mobile phone is tilted at a, and the image of the image data in the unprocessed state is also shown in FIG. The display of the data, as well as the display when the phone is previewed.

In this embodiment, the process of processing the collected image data by the mobile phone is still as shown in FIG. 3, and specifically includes the following steps:

Step 31: Turn on the gravity synchronization function (ie, the direction adjustment function), so as to be able to trigger the direction adjustment processing process of executing subsequent image data;

During the imaging process, the corresponding gravity synchronization function can be turned on or off to facilitate control whether to perform subsequent processing to adjust the corresponding image data, that is, the embodiment of the present invention can be used as an optional function (on or off) Close), in order to facilitate the user to choose whether to apply or not, has a strong practicality;

In other words, the user can selectively turn the gravity synchronization function on or off according to his own needs: If the user turns on the gravity synchronization function to shoot, such as shooting a concert or walking while walking on the street, there is no need to worry about the tilt of the captured video. When the netizens watch the video uploaded on the Internet, they will not appreciate it because of the tilt of the video, and even if they want to take a personalized tilt image, they can choose to turn off the gravity synchronization function. For the acquired image data, the correction processing will not be performed any more;

Step 32, detecting a tilt angle of the mobile phone during shooting;

Specifically, the gravity signal (gravity sensing function) set in the mobile phone can be used to obtain the gravity signal to ensure the accuracy and sensitivity of the gravity signal, and then the tilt angle of the mobile phone when the mobile phone is facing in the direction of gravity is calculated according to the gravity signal. (ie, the tilt angle of the image acquisition device with respect to the positive direction of gravity) a, that is, the detection determines the deviation angle a of the camera from the original gravity forward;

Specifically, the tilt angle of the mobile phone can be calculated according to the output of the gravity sensor provided in the mobile phone. For example, the acceleration caused by gravity is measured by the gravity sensor, and the tilt angle of the device equipped with the gravity sensor with respect to the horizontal plane can be calculated. .

Step 33: Perform a cropping process on a picture frame corresponding to the image data obtained by the mobile phone shooting;

Specifically, the cutting processing manner in the step may include:

First, on the basis of detecting the obtained tilt angle of the mobile phone, a tilted rectangular area is formed in the captured picture data (ie, image data) area according to the set preview aspect ratio, as shown in step 33 in FIG. The side portrait display screen is shown; in the process of calculating the cropped rectangular area, the diagonal length of the rectangular area is at least smaller than the narrowest part of the image (picture) corresponding to the image data, so that the mobile phone can rotate at any angle. Perform correct cropping of image data;

Then, using the arbitrary polygon cropping technique in computer graphics, the captured image data is cropped according to the oblique rectangular area, and the features and attributes of the cropped image data are updated, that is, the data format type, size and time of the updated image data are updated. Such a flag or information, determining the corresponding features and attributes for the cropped image data; rearranging the image data in the oblique rectangular region in the process of cropping the captured image data by using computer graphics polygon clipping technology The cloth is processed to obtain the vertical forward image data, that is: the data in the oblique rectangular area is acquired and rearranged into the new buffer (the process of rearranging is equivalent to the process of rotating the image data) , to obtain vertical forward image data, that is, a gravity forward picture frame), and update the features and attributes of the rearranged image data (ie, update the image data type, size, and time, etc., or other information, Determining the corresponding features for the rearranged image data And attributes) to get what you need when playing or browsing a picture frame that is positive in the direction of gravity;

In the above processing, the corresponding rearrangement processing may specifically include: mapping the image data of the rectangular area into the new data area one by one according to the calculated tilt angle and the oblique rectangular area, that is, each pixel The data is calculated by the position of the row and column one by one and accesses the corresponding pixel data to the corresponding offset in the new buffer, thereby arranging the respective pixels in the oblique rectangular region in the vertically positive rectangular region, and The image data in the rectangular region of the vertical direction obtained by the arrangement is equivalent to the image data obtained by rotating the image data of the oblique rectangular region by the degree value of the above-described tilt angle.

Step 34: Store the image data after the cropping process;

The cropped image data and feature attribute information obtained in step 33 can be directly stored in the same manner as the normal storage method.

Further, in order to implement a preview effect synchronized with the stored image data in the mobile phone, the following processing is also required in the embodiment, including:

Step 35: Rotate the previously obtained gravity forward picture frame with a rotation angle of -a to obtain a picture frame that is forward in the gravity direction during preview;

Step 36, scale and adjust to fit the size of the display;

That is, the step is to adapt the picture frame in the forward direction of the gravity direction to the display screen. For example, if a picture of 1920×1080 needs to be completely displayed on the display screen of 800×480, it needs to be scaled accordingly ( That is, adapted)

Specifically, the adjustment in the step may be based on the premise that the diagonal length of the screen is less than or equal to the narrowest part of the effective display area of the mobile phone screen, so that the mobile phone performs arbitrary rotation, and the image can be The image preview effect can be displayed correctly and completely on the screen of the mobile phone.

Step 37: Display the frame frame after the zoom adjustment processing on the display screen to implement preview processing for one frame of the screen.

Through the processing of this embodiment, when the video or photo taken by the mobile phone is played or browsed, whether the mobile phone is placed in the forward direction of gravity or not during the shooting, the corresponding video or photo ratio and picture can be guaranteed. In the state of gravity forward. For example, referring to FIG. 4, FIG. 4 is a schematic diagram showing the effect of the video captured on the computer when the mobile phone is tilted or rotated in the middle when the mobile phone is initially tapped, and the small arrow in FIG. On the left is the real-time shooting of the mobile phone, the right side of the small arrow is the corresponding screen when playing on the computer monitor, and the big arrow is the process indication of the change of the direction of the mobile phone. Moreover, it is also ensured by this embodiment that the stored image data is displayed in the same manner as when the image data is previewed. It should be noted that, in this embodiment, the gravity sensing function and the camera photographing or camera function existing in the mobile phone can be utilized, or the corresponding gravity sensing function and the camera photographing or camera function can be set in advance in the mobile phone, so that It is applied in the embodiment of the present invention. If the former method is adopted, the embodiment of the present invention may only expand the software function of the mobile phone to reduce the cost of implementing the embodiment of the present invention.

The application process of the embodiment of the present invention will be described by taking a mobile phone as an example, but this does not constitute a limitation of the present invention. The embodiments of the present invention can also be applied to other electronic devices including a photographing function. A person skilled in the art can understand that all or part of the process of implementing the above embodiment method can be completed by a computer program to instruct related hardware, and the program can be stored in a computer readable storage medium. In execution, the flow of an embodiment of the methods as described above may be included. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

The embodiment of the present invention further provides a processing device for collecting image data, and the specific implementation structure thereof is as shown in FIG. 5, and may include the following modules:

An image data obtaining module 51, configured to acquire the collected image data;

The tilt angle detecting module 52 is configured to detect a tilt angle when the image collecting device collects the image data;

The image data direction adjustment module 53 is configured to perform direction adjustment on the acquired image data acquired by the image data acquiring module 51 according to the tilt angle detected by the tilt angle detecting module 52, and the adjusted image data is displayed at the time of output display. The direction is vertical forward.

Optionally, the storage module 54 is further included in the device, for storing the image data adjusted by the image data direction adjustment module 53.

Specifically, the corresponding tilt angle detecting module 52 may include:

a gravity signal acquisition module 521, configured to obtain a gravity signal by using a set gravity sensing function;

The tilt angle calculation module 522 is configured to calculate a tilt angle of the image capture device relative to the gravity forward direction according to the gravity signal acquired by the gravity signal acquisition module 521.

Optionally, the device may further include a gravity sensor, and the gravity signal acquisition module 521 obtains a gravity signal through the gravity sensor.

Further, in order to control whether the image data direction adjustment module in the device performs direction adjustment on the image data, the following modules may also be included in the device:

The direction adjustment function control module 55 is configured to set whether the direction adjustment function is turned on, so as to facilitate the image acquisition device. The user can conveniently select whether to use the image data direction adjustment module in the device to perform direction adjustment processing on the collected image data;

The determining module 56 is configured to determine whether the direction adjustment function set by the direction adjustment function control module 55 is turned on before the direction adjustment of the image data, and trigger the image data direction only when the direction adjustment function is turned on. The adjustment module performs the process of adjusting the direction of the image data.

The processing mode adopted by the corresponding image data direction adjusting module 53 may include any one of the following: Mode 1: According to the obtained tilt angle, the collected image data is rotated, and the rotated image data is performed. Size cutting processing;

Method 2: Performing a size cropping process on the collected image data along a direction of the tilt angle according to the obtained tilt angle, using a computer graphics polygon cropping technique, where the computer graphics polygon cropping technique includes the size cropping process The arrangement of the image data is adjusted to obtain vertical forward image data.

Further, the size cropping process employed in the image data direction adjustment module 53 may include, but is not limited to, determining: a size for cropping the collected image data according to a size of the predetermined image data, or an image corresponding to the captured image data. The size at the narrowest point determines the size at which the captured image data is cropped.

Furthermore, in order to facilitate the user to preview the collected image data, the device may further include:

The preview image rotation module 57 is configured to rotate the image data adjusted by the direction adjustment function control module 53 in the opposite direction of the tilt angle by the tilt angle detected by the tilt angle detecting module 52, so as to facilitate the A screen corresponding to the vertically forward image data is displayed in the tilted display screen;

The zooming module 58 is configured to resize the image data after the rotation of the preview image rotation module 57 according to the size of the display screen of the image capturing device, so that the corresponding image data can be compared with the size of the effective display area of the display screen. The zoom module 58 may specifically include a screen size determining module 581 and a size adjusting module 582, wherein the corresponding screen size determining module 581 is configured to determine a size of a narrowest display area of the display screen of the image capturing apparatus; The corresponding size adjustment module 582 is configured to adjust a diagonal length of the screen corresponding to the image data to be smaller than or equal to a size of a narrowest display area of the display screen of the image collection device determined by the display screen size determining module 581;

The display module 59 is configured to display the image data after the scaling module 58 performs resizing.

With the device embodiment, when the image data such as a video or a photo taken by the electronic device is played or browsed, the electronic device can be placed in the direction of gravity regardless of the shooting, and the corresponding video and photo can be guaranteed during the display. The scale and picture are in a positive gravity state. Moreover, the embodiment of the apparatus can also ensure that the stored image data is consistent in display performance when previewing the image data. It should be noted that the specific implementation manners of the functions implemented by the processing units included in the foregoing apparatus have been described in detail in the foregoing various embodiments, and thus are not described herein again.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the division of each functional module described above is exemplified. In practical applications, the above-mentioned function assignment can be completed by different functional modules as needed. The internal structure of the device is divided into different functional modules to perform all or part of the functions described above. For the specific working process of the device and the module described above, reference may be made to the corresponding process in the foregoing method embodiments, and details are not described herein.

In the several embodiments provided herein, it should be understood that the disclosed apparatus and method can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the modules is only a logical function division. In actual implementation, there may be another division manner, for example, multiple modules or components may be combined or Can be integrated into another system, or some features can be ignored, or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection through some interface, device or module, and may be in an electrical, mechanical or other form.

The modules described as separate components may or may not be physically separate, and the components displayed as modules may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, each functional module in each embodiment of the present invention may be integrated into one processor, or each module may exist physically separately, or two or more modules may be integrated into one processor. The above integrated processor can be implemented in the form of hardware or in the form of a software function module; the corresponding processor can be

The CPU, and the processor is used to perform the functions that each module included in the above device needs to complete. The above is only a preferred embodiment of the present invention, but the scope of the present invention is not limited thereto, and any person skilled in the art can easily think of changes or within the technical scope of the present disclosure. Alternatives are intended to be covered by the scope of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

claims

1. An image data processing method, characterized by including:

Obtain the collected image data and the tilt angle when the image acquisition device collected the image data;

The direction of the collected image data is adjusted according to the tilt angle, and the direction of the adjusted image data when output and displayed is a vertical positive direction.

2. The method according to claim 1, characterized in that, the method further includes:

Before adjusting the direction of the image data, it is determined whether the direction adjustment function is turned on, and only when the direction adjustment function is turned on, the process of adjusting the direction of the image data is triggered.

3. The method according to claim 1, wherein the step of obtaining the inclination angle includes:

The gravity signal is obtained through the set gravity sensing function, and the tilt angle of the image acquisition device relative to the positive direction of gravity is calculated based on the gravity signal.

4. The method according to claim 3, characterized in that the gravity signal is obtained through a gravity sensor provided in the image acquisition device.

5. The method according to any one of claims 1 to 4, characterized in that the method also includes a process of previewing the collected image data in an image acquisition device, and the process includes:

According to the tilt angle, the adjusted image data is rotated in the opposite direction of the tilt angle by the degree value of the tilt angle;

According to the size of the display screen of the image acquisition device, the size of the image data after rotation is adjusted; and the image data after the size adjustment is displayed on the display screen of the image acquisition device.

6. The method according to claim 5, characterized in that the size adjustment includes:

Adjust the diagonal length of the picture corresponding to the image data to be less than or equal to the size of the narrowest point of the display area of the display screen of the image acquisition device.

7. A processing device for collecting image data, characterized in that it includes:

Image data acquisition module, used to acquire the collected image data;

The tilt angle detection module is used to detect the tilt angle of the image acquisition device when collecting the image data; The image data direction adjustment module is used to adjust the direction of the collected image data obtained by the image data acquisition module according to the inclination angle detected by the inclination angle detection module. The direction of the adjusted image data when output and displayed is vertical. Positive.

8. The device according to claim 7, characterized in that, the device further includes:

Direction adjustment function control module, used to set whether the direction adjustment function is turned on;

A judgment module, used to judge whether the direction adjustment function set by the direction adjustment function control module is turned on before adjusting the direction of the image data, and trigger the image data direction adjustment module only when the direction adjustment function is turned on. Processing of orientation adjustment of image data.

9. The device according to claim 7, characterized in that the tilt angle detection module includes:

Gravity signal acquisition module, used to obtain gravity signals through the set gravity sensing function;

An inclination angle calculation module, configured to calculate the inclination angle of the image acquisition device relative to the positive direction of gravity based on the gravity signal acquired by the gravity signal acquisition module.

10. The device according to claim 9, wherein the device further includes a gravity sensor, and the gravity signal acquisition module obtains the gravity signal through the gravity sensor.

11. The device according to any one of claims 7 to 10, characterized in that, the device further includes:

A preview image rotation module, configured to detect the tilt angle obtained by the tilt angle detection module and rotate the image data adjusted by the direction adjustment function control module by the degree value of the tilt angle in the opposite direction of the tilt angle;

A scaling module configured to adjust the size of the image data after rotation processing by the preview image rotation module according to the size of the display screen of the image acquisition device;

A display module, configured to display the image data after size adjustment by the scaling module.

12. The device according to claim 11, characterized in that the scaling module includes:

The display screen size determination module is used to determine the size of the narrowest part of the display area of the image acquisition device's display screen; the size adjustment module is used to adjust the diagonal length of the picture corresponding to the image data to be less than or equal to the display screen The size determination module determines the size of the narrowest point of the display area of the display screen of the image acquisition device.