WO2022037479A1

WO2022037479A1 - Photographing method and photographing system

Info

Publication number: WO2022037479A1
Application number: PCT/CN2021/112362
Authority: WO
Inventors: 刘亮
Original assignee: 华为技术有限公司
Priority date: 2020-08-19
Filing date: 2021-08-12
Publication date: 2022-02-24
Also published as: CN114079730B; CN114079730A

Abstract

Disclosed in the present application are a photographing method and a photographing system. The photographing method is applied to the photographing system. The photographing system comprises an electronic device and a first wearable device. A connection is established between the electronic device and the first wearable device. The electronic device receives a first operation, and in response to the first operation, obtains a multimedia file acquired by a camera, the multimedia file comprising an image file, a video file, and the like; the first wearable device detects first sensor data by means of at least one sensor; the electronic device obtains first information, which corresponds to the first sensor data. The electronic device stores the multimedia file, which is associated with the first information. The first information comprises biologic characteristic information of a user. The electronic device associates the biologic characteristic information with image/video information, and performs more precise characteristic identification on the image/video, facilitating subsequent classification on the stored image/video according to the biologic characteristic information.

Description

A shooting method and shooting system

This application claims the priority of the Chinese patent application with the application number 202010839365.9 and the application title "A Shooting Method and Shooting System" filed with the China Patent Office on August 19, 2020, the entire contents of which are incorporated into this application by reference .

technical field

The present application relates to the field of electronic technology, and in particular, to a photographing method and a photographing system.

Background technique

In the era of network digital information, consumers' resources (such as pictures and videos) are increasing exponentially, which poses challenges to the storage and use of resources. Usually involves a search for a resource of a specified type. For example, through a search engine, search for pictures based on preset characteristics. For another example, the user searches the resource library stored on the electronic device to achieve the purpose of browsing and sharing. Such as: searching pictures of specified characteristics through mobile phone albums, pictures of children dancing, playing football, etc.

However, feature extraction and labeling depend on a lot of hardware resources, especially when the amount of resources is large, the computing time also increases exponentially; and the existing image recognition technology can only identify and extract some external features, and also No technology discloses how to classify images according to the psychological and physiological characteristics of characters.

SUMMARY OF THE INVENTION

The embodiments of the present application provide a shooting method and a shooting system, which realize the association of biometric information with the information of the pictures/videos in the process of generating pictures/videos, and make more accurate feature recognition for the pictures/videos.

In a first aspect, the present application provides a shooting system, which is characterized by comprising: an electronic device and a first wearable device, and the electronic device includes a camera; wherein the electronic device is used to establish a connection with the first wearable device; the electronic device, The electronic device is also used for receiving the first operation; the electronic device is also used for acquiring the multimedia file collected by the camera in response to the first operation; the first wearable device is used for detecting the first sensor data through at least one sensor; the electronic device is also used for The first information is acquired, and the first information corresponds to the first sensor data; the electronic device is further configured to save a multimedia file, and the multimedia file is associated with the first information. The multimedia files include picture files, video files, and the like. The first operation may include, but is not limited to, operations such as clicking, double-clicking, long-pressing, sliding, and the like, and the first operation is used to trigger the electronic device to take pictures/videos. The electronic device establishes a connection with the first wearable device; the electronic device captures pictures/videos through the camera in response to the first operation. The electronic device acquires the user's biometric information (eg, heart rate, blood pressure, exercise posture, etc.) detected by the first wearable device through the sensor, and the biometric information is also referred to as first information. The electronic device associates the biometric information with the photographed picture/video, or associates the biometric information with a person in the photographed picture/video, and the person corresponds to the biometric information. The electronic device generates a picture/video including biometric information indicative of the user's biometrics. Through the shooting system provided by the present application, in the process of picture/video generation, the electronic device and the first wearable device perform information interaction, the electronic device obtains biometric information, and the electronic device correlates the biometric information with the information of the picture/video The electronic device saves the pictures/videos with biometric information, which facilitates the subsequent classification of the stored pictures/videos according to the biometric information.

In a possible implementation manner, the first wearable device is further configured to determine the first information according to the first sensor data; and send the first information to the electronic device. The first information is biometric information. This approach describes a way for the electronic device to acquire the first information. The first wearable device detects sensor data (first sensor data) through one or more sensors, and the first wearable device determines the first information based on the first sensor data. , the first wearable device sends the first information to the electronic device, and the electronic device obtains the first information. The first wearable device processes the first sensor data, which saves time and resources for the electronic device to process the first sensor data.

In a possible implementation manner, the first wearable device is further configured to send the first sensor data to the electronic device; and the electronic device is further configured to determine the first information according to the first sensor data. The first information is biometric information. This method describes another method for the electronic device to obtain the first information. The first wearable device detects sensor data (first sensor data) through one or more sensors, and sends the first sensor data to the electronic device. The electronic device Biometric information is determined based on the acquired first sensor data.

In a possible implementation manner, the electronic device is specifically configured to: in response to the first operation, acquire first information, where the first information corresponds to the first sensor data. This way describes the timing when the electronic device obtains the first information. The first operation includes a user operation that triggers the electronic device to take pictures/videos. After the electronic device receives the first operation, the electronic device acquires the first information based on the first operation. The electronic device processes the first sensor data, which saves time and resources for the first wearable device to process the first sensor data.

In a possible implementation manner, the electronic device is further configured to: in response to the first operation, send a first request message to the first wearable device; the first wearable device is specifically configured to: in response to the first request message, send a first request message to the electronic device The device sends the first information. This manner describes a manner in which the electronic device acquires the first information in response to the first operation. After the electronic device receives the first operation, the electronic device requests the first wearable device to obtain biometric information (first information) based on the first operation, the first wearable device sends the first information to the electronic device, and the electronic device Obtain the first information.

In a possible implementation manner, the electronic device is further configured to: in response to the first operation, send a first request message to the first wearable device; the first wearable device is specifically configured to: in response to the first request message, send a first request message to the electronic device The device sends first sensor data. This manner describes another manner in which the electronic device acquires the first information in response to the first operation. After the electronic device receives the first operation, the electronic device requests the first wearable device to acquire sensor data (first sensor data) based on the first operation, and the electronic device determines biometric information (first sensor data) based on the acquired first sensor data information).

In a possible implementation manner, the electronic device is further configured to: display a shooting preview interface, where the shooting preview interface includes a shooting button, and the first operation includes an input operation acting on the shooting button. An application scenario is provided here. The electronic device receives the first operation on the shooting preview interface, and triggers the electronic device to shoot pictures/videos.

In a possible implementation manner, the attribute information of the multimedia file includes the first information. The electronic device saves the multimedia file, performs fusion coding on the multimedia file and the first information, and the user can check the first information by checking the attribute information of the multimedia file.

In a possible implementation manner, the electronic device for establishing a connection with the first wearable device includes: in response to the electronic device entering a preset shooting mode, establishing a connection between the electronic device and the first wearable device. This method describes the timing of establishing the connection between the electronic device and the first wearable device. In the preset shooting mode, the electronic device displays a shooting preview interface. When the electronic device detects a user operation entering the preset shooting mode, the electronic device automatically turns on Bluetooth, and automatically establishes a Bluetooth connection with the first wearable device.

In a possible implementation manner, the first wearable device is further configured to: receive the second operation; the first wearable device is further configured to instruct the electronic device to turn on the camera in response to the second operation; the first electronic device is further configured to use For displaying a shooting preview interface, the shooting preview interface displays a preview image collected by the camera; the first operation includes an operation acting on the shooting preview interface. This way describes a way of triggering the shooting of pictures/videos on the side of the first wearable device. Wherein, the second operation may include, but is not limited to, operations such as clicking, double-clicking, long-pressing, sliding, etc. The second operation acts on the first wearable device and is used to trigger the first wearable device to instruct the electronic device to turn on the camera, thereby taking pictures/ video.

In a possible implementation manner, the electronic device is further configured to display the multimedia file and at least part of the first information. The user can enter the gallery to view the multimedia file, and the electronic device displays the multimedia file and at least part of the first information, where the first information indicates the biometric information of the user. Optionally, the display interface of the electronic device displays the multimedia file and part of the first information, and the user can view all the first information by viewing the detailed information or the like.

In a possible implementation manner, the electronic device is further configured to display the multimedia file and at least part of the first information, including: the electronic device is further configured to display the multimedia file and at least part of the first information in response to the preset facial image included in the multimedia file The first information; the preset facial image corresponds to the first wearable device. Information of one or more facial images is stored in the electronic device, the electronic device determines the preset facial image corresponding to the first wearable device through the identity information of the first wearable device, and associates the preset facial image with one or more of the multimedia files. The personal characters are matched for similarity. If the preset facial image is successfully matched with one of the characters, it means that the character in the multimedia file is the user of the first wearable device. At this time, the electronic device displays at least part of the first information, otherwise it will not be displayed. User privacy is protected.

The preset facial image is associated with the first wearable device, and the facial information may be preset by the user in the electronic device, or uploaded to the electronic device in the form of an image or video, or may be preset by the user in the electronic device. In the first wearable device, the first wearable device is then provided to the electronic device, which is not limited in this application.

In a possible implementation manner, the electronic device is further configured to display the multimedia file and at least part of the first information, including: the electronic device is further configured to respond to the multimedia file including the first facial image and the second facial image, And the first facial image matches the preset facial image, and at least part of the first information is displayed in the first area of the multimedia file; wherein, the preset facial image corresponds to the first wearable device; The distance of the display area is smaller than the distance between the first area and the display area of the second facial image. This approach describes a way of determining the display position of at least part of the first information on the multimedia file. Similarity matching is performed between the preset facial image and one or more characters in the multimedia file, and if the preset facial image is successfully matched with one of the characters, the photographing device displays at least part of the first information near the character. In terms of display effect, the correspondence between the user and the first information is strengthened, and the user corresponding to the first information can be seen intuitively, which can improve the user experience.

In a possible implementation manner, the electronic device is further configured to: before receiving the first operation, display a shooting preview interface, where the shooting preview interface includes a preview image captured by a camera; the electronic device is further configured to display on the preview image At least part of the second information corresponds to the second sensor data detected by the first wearable device. The second information is biometric information. The electronic device displays at least part of the second information on the preview image, so that the biometric information is displayed on the preview interface in real time, and the user can view the biometric information in real time. Even more biometric information may be included in the image or video file acquired by the electronic device, and the biometric information is derived from sensor data collected by different sensors.

In a possible implementation manner, the electronic device is further configured to display at least part of the second information on the preview image, including: the electronic device is further configured to display the preview image and at least a preset face image in response to the preview image. Part of the second information, the preset facial image corresponds to the first wearable device. Information of one or more facial images is stored in the electronic device, the electronic device determines the preset facial image corresponding to the first wearable device through the identity information of the first wearable device, and compares the preset facial image with one or more of the facial images on the preview image. The personal characters are matched for similarity. If the preset facial image is successfully matched with one of the characters, it means that the character on the preview image is the user of the first wearable device. At this time, the electronic device displays at least part of the second information, otherwise it will not be displayed. User privacy is protected.

In a possible implementation manner, the electronic device is further configured to display at least part of the second information on the preview image, including: the electronic device is further configured to respond that the preview image includes the third facial image and the fourth facial image, And the third facial image matches the preset facial image, and at least part of the second information is displayed in the second area of the preview image; wherein the preset facial image corresponds to the first wearable device; the second area corresponds to the display area of the third facial image The distance is smaller than the distance between the second area and the display area of the fourth face image. This approach describes a way of determining the display position of at least part of the second information on the preview image. The preset facial image is matched with one or more characters in the preview image for similarity, and if the preset facial image is successfully matched with one of the characters, the photographing device displays at least part of the biometric information near the character in the preview image. In the display effect, the correspondence between the user and the second information is strengthened, and the user corresponding to the second information on the preview image can be intuitively seen, thereby improving the user experience.

In a possible implementation manner, the electronic device is further configured to: in response to the preview image not including the preset face image, output a first prompt, where the first prompt is used to prompt the user to align the face. The electronic device performs similarity matching between the preset facial image and one or more characters in the picture. If the matching is unsuccessful, it means that the user of the first wearable device is not included in the preview image, and the electronic device outputs prompt information to prompt the user to change the shooting angle. Aiming at the face avoids the situation that the user of the first wearable device is not included in the photographed multimedia file, and improves the user experience.

In a possible implementation manner, the first information includes at least one of the following: health state information, exercise state information, or emotional state information.

In a possible implementation manner, the first sensor data includes data detected by at least one sensor, and the at least one sensor includes at least one of the following: an acceleration sensor, a gyroscope sensor, a geomagnetic sensor, an atmospheric pressure sensor, a heart rate sensor, Blood pressure sensor, ECG sensor, EMG sensor, body temperature sensor, skin electrical sensor, air temperature and humidity sensor, light sensor, or bone conduction sensor.

In a possible implementation manner, the system further includes a second wearable device; the electronic device is further configured to establish a connection with the second wearable device; the second wearable device is configured to detect fourth sensor data through at least one sensor; wherein , the first information also corresponds to the fourth sensor data. This method describes that when the electronic device establishes a connection with two wearable devices (the first wearable device and the second wearable device), the electronic device obtains the first information, which is related to the first wearable device and the second wearable device. The sensor data (the first sensor data and the fourth sensor data) of the wearable device are all corresponding. The same is true for establishing a connection between the electronic device and two or more wearable devices.

In a second aspect, a shooting method is applied to an electronic device including a camera. The method includes: establishing a connection between the electronic device and a first wearable device; the electronic device receives a first operation; the electronic device responds to the first operation and acquires multimedia collected by the camera. file; the electronic device acquires first information, and the first information corresponds to first sensor data detected by at least one sensor of the first wearable device; the electronic device saves a multimedia file, and the multimedia file is associated with the first information. The multimedia files include picture files, video files, and the like. The first operation may include, but is not limited to, operations such as clicking, double-clicking, long-pressing, sliding, and the like, and the first operation is used to trigger the electronic device to take pictures/videos. The electronic device captures pictures/videos through the camera in response to the first operation. The electronic device acquires the user's biometric information (such as heart rate, blood pressure, exercise posture, etc.) detected by the first wearable device through the sensor, and the electronic device associates the biometric information with the photographed picture/video, or associates the biometric information with the photographed image/video. associated with the person in the picture/video of , the person corresponds to the biometric. The biometric information is also referred to as the first information. The electronic device generates a picture/video including biometric information indicative of the user's biometrics. This method associates the biometric information with the information of the picture/video in the process of picture/video generation, makes more accurate feature recognition for the picture/video, and provides a new picture/video format, making The electronic device saves pictures/videos with biometric information, which facilitates subsequent classification of the stored pictures/videos according to the biometric information.

In a possible implementation manner, acquiring the first information by the electronic device includes: acquiring the first sensor data by the electronic device; and determining the first information based on the first sensor data by the electronic device. This way describes a way for the electronic device to obtain the first information, where the first information is biometric information. The first wearable device detects sensor data (first sensor data) through one or more sensors, sends the first sensor data to the electronic device, and the electronic device determines biometric information based on the acquired first sensor data. The electronic device processes the first sensor data, which saves time and resources for the first wearable device to process the first sensor data.

In a possible implementation manner, acquiring the first information by the electronic device includes: the electronic device acquiring the first information determined by the first wearable device based on the first sensor data. This method describes another method for the electronic device to obtain the first information, where the first information is biometric information. The first wearable device detects sensor data (first sensor data) through one or more sensors, the first wearable device determines first information based on the first sensor data, the first wearable device sends the first information to the electronic device, and the electronic The device obtains the first information. The first wearable device processes the first sensor data, which saves time and resources for the electronic device to process the first sensor data.

In a possible implementation manner, the electronic device acquiring the first information includes: in response to the first operation, the electronic device acquiring the first information. This method describes the timing when the electronic device acquires the first information, and the first operation includes a user operation that triggers the electronic device to capture a picture/video. After the electronic device receives the first operation, the electronic device acquires the first information based on the first operation.

In a possible implementation manner, acquiring the first information by the electronic device in response to the first operation includes: in response to the first operation, the electronic device sends a first request to the first wearable device, where the first request is used to request to acquire the first information. First sensor data detected by at least one sensor of a wearable device; the electronic device acquires first information, and the first information corresponds to the first sensor data. This manner describes a manner in which the electronic device acquires the first information in response to the first operation. After the electronic device receives the first operation, based on the first operation, the electronic device sends a request to the first wearable device to acquire sensor data (first sensor data), and the electronic device determines biometric information based on the acquired first sensor data (first message).

In a possible implementation manner, acquiring the first information by the electronic device in response to the first operation includes: in response to the first operation, the electronic device sends a first request to the first wearable device, where the first request is used to request to acquire the first information. A wearable device determines the first information based on the first sensor data; the electronic device obtains the first information, and the first information corresponds to the first sensor data. This manner describes another manner in which the electronic device acquires the first information in response to the first operation. After the electronic device receives the first operation, the electronic device sends a request to the first wearable device based on the first operation to obtain biometric information (first information), and the first wearable device sends the first information to the electronic device , the electronic device obtains the first information.

In a possible implementation manner, the electronic device receiving the first operation includes: the electronic device displays a shooting preview interface, and the shooting preview interface includes a shooting button; the electronic device receives a first operation, and the first operation includes an input operation acting on the shooting button . An application scenario is provided here. The electronic device receives the first operation on the shooting preview interface, and triggers the electronic device to shoot pictures/videos.

In a possible implementation manner, establishing a connection between the electronic device and the first wearable device includes: in response to the electronic device entering a preset shooting mode, establishing a connection between the electronic device and the first wearable device. This method describes the timing of establishing the connection between the electronic device and the first wearable device. In the preset shooting mode, the electronic device displays a shooting preview interface. When the electronic device detects a user operation entering the preset shooting mode, the electronic device automatically turns on Bluetooth, and automatically establishes a Bluetooth connection with the first wearable device.

In a possible implementation manner, the method further includes: the electronic device displays the multimedia file and at least part of the first information. The user can enter the gallery to view the multimedia file, and the electronic device displays the multimedia file and at least part of the first information, where the first information indicates the biometric information of the user. Optionally, the display interface of the electronic device displays the multimedia file and part of the first information, and the user can view all the first information by viewing the detailed information or the like.

In a possible implementation manner, the electronic device displays the multimedia file and at least part of the first information, specifically including: in response to the multimedia file including a preset face image, the electronic device displays the multimedia file and at least part of the first information; preset face image The image corresponds to the first wearable device. Information of one or more facial images is stored in the electronic device, the electronic device determines the preset facial image corresponding to the first wearable device through the identity information of the first wearable device, and associates the preset facial image with one or more of the multimedia files. The personal characters are matched for similarity. If the preset facial image is successfully matched with one of the characters, it means that the character in the multimedia file is the user of the first wearable device. At this time, the electronic device displays at least part of the first information, otherwise it is not displayed. User privacy is protected.

In a possible implementation manner, the electronic device displays the multimedia file and at least part of the first information, which specifically includes: in response to the multimedia file including the first facial image and the second facial image, and the first facial image and the preset The facial image is matched, and the electronic device displays at least part of the first information in the first area of the multimedia file; wherein, the preset facial image corresponds to the first wearable device; the distance between the first area and the display area of the first facial image is smaller than the first area. The distance of an area from the display area of the second facial image. This approach describes a way of determining the display position of at least part of the first information on the multimedia file. Similarity matching is performed between the preset facial image and one or more characters in the multimedia file, and if the preset facial image is successfully matched with one of the characters, the photographing device displays at least part of the first information near the character. In terms of display effect, the correspondence between the user and the first information is strengthened, and the user corresponding to the first information can be seen intuitively, which can improve the user experience.

In a possible implementation manner, before the electronic device receives the first operation, it further includes: the electronic device displays a shooting preview interface, and the shooting preview interface includes a preview image captured by a camera; and the electronic device displays at least part of the second information on the preview image , the second information corresponds to the second sensor data detected by the first wearable device. The second information is the biometric information displayed on the preview interface. The electronic device displays at least part of the second information on the preview image, so that the biometric information is displayed on the preview interface in real time, and the user can view the biometric information in real time.

In a possible implementation manner, the electronic device displays at least part of the second information on the preview image, which specifically includes: in response to the preview image including a preset facial image, the electronic device displays the preview image and at least part of the second information, preset The facial image corresponds to the first wearable device. Information of one or more facial images is stored in the electronic device, the electronic device determines the preset facial image corresponding to the first wearable device through the identity information of the first wearable device, and compares the preset facial image with one or more of the facial images on the preview image. The personal characters are matched for similarity. If the preset facial image is successfully matched with one of the characters, it means that the character on the preview image is the user of the first wearable device. At this time, the electronic device displays at least part of the second information, otherwise it will not be displayed. User privacy is protected.

In a possible implementation manner, the electronic device displays at least part of the second information on the preview interface, which specifically includes: in response to the preview image including the third facial image and the fourth facial image, and the third facial image is the same as the preset face Image matching, the electronic device displays at least part of the second information in the second area of the preview image; the preset facial image corresponds to the first wearable device; the distance between the second area and the display area of the third facial image is smaller than the second area The distance from the display area of the fourth face image. This approach describes a way of determining the display position of at least part of the second information on the preview image. The preset facial image is matched with one or more characters in the preview image for similarity, and if the preset facial image is successfully matched with one of the characters, the photographing device displays at least part of the biometric information near the character in the preview image. In terms of display effect, the correspondence between the user and the second information is strengthened, and the user corresponding to the second information on the preview image can be intuitively seen, thereby improving the user experience.

In a possible implementation manner, the method further includes: in response to the preview image not including the preset face image, the electronic device outputs a first prompt, where the first prompt is used to prompt the user to aim at the face. The electronic device performs similarity matching between the preset facial image and one or more characters in the picture. If the matching is unsuccessful, it means that the user of the first wearable device is not included in the preview image, and the electronic device outputs prompt information to prompt the user to change the shooting angle. Aiming at the face avoids the situation that the user of the first wearable device is not included in the photographed multimedia file, and improves the user experience.

In a possible implementation manner, the first sensor data includes data detected by at least one sensor, and the at least one sensor includes at least one of the following: an acceleration sensor, a gyroscope sensor, a geomagnetic sensor, an atmospheric pressure sensor, a heart rate sensor, Blood pressure sensor, ECG sensor, EMG sensor, body temperature sensor, skin electric sensor, air temperature and humidity sensor, light sensor, or bone conduction sensor.

In a possible implementation manner, the method further includes: establishing a connection between the electronic device and the second wearable device; the second wearable device is configured to detect fourth sensor data through at least one sensor, and the first information also corresponds to the fourth sensor data . This method describes that when the electronic device establishes a connection with two wearable devices (the first wearable device and the second wearable device), the electronic device obtains the first information, which is related to the first wearable device and the second wearable device. The sensor data (the first sensor data and the fourth sensor data) of the wearable device are all corresponding. The same is true for establishing a connection between the electronic device and two or more wearable devices.

In a third aspect, the present application provides an electronic device, comprising: one or more processors and one or more memories; the one or more memories are coupled with the one or more processors; the one or more memories are used for The computer program code is stored in the computer program code, and the computer program code includes computer instructions; when the computer instructions are executed on the processor, the electronic device enables the electronic device to execute the photographing method in any possible implementation manner of any of the above aspects.

In a fourth aspect, the embodiments of the present application provide a computer storage medium, including computer instructions, when the computer instructions are executed on the electronic device, the communication apparatus can execute the photographing method in any possible implementation manner of any of the foregoing aspects.

In a fifth aspect, the present application provides a chip system, which is applied to an electronic device including a memory, a display screen and a sensor; the chip system includes: one or more interface circuits and one or more processors; the interface circuit and the processor interconnected by lines; the interface circuit is used to receive signals from the memory and send signals to the processor, where the signals include computer instructions stored in the memory; when the processor executes the computer instructions, the electronic device executes any one of the first aspect and the first aspect Possible implementations of task handling methods.

In a sixth aspect, an embodiment of the present application provides a computer program product that, when the computer program product runs on a computer, enables the computer to execute the photographing method in any of the possible implementations of any one of the foregoing aspects.

Description of drawings

Fig. 1 is a kind of system diagram that the embodiment of this application provides;

FIG. 2 is a schematic structural diagram of an electronic device provided by an embodiment of the present application;

3 is a software architecture diagram of an electronic device provided by an embodiment of the present application;

FIG. 4 is a schematic structural diagram of a wearable device according to an embodiment of the present application;

5a-5e are interface schematic diagrams of a group of shooting methods provided by an embodiment of the present application;

6a-6b are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

7a-7b are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

8a-8d are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

9a-9c are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

10 is a schematic interface diagram of another set of shooting methods provided by an embodiment of the present application;

11a-11c are interface schematic diagrams of another group of shooting methods provided by the embodiments of the present application;

12 is a schematic interface diagram of another group of shooting methods provided by an embodiment of the present application;

13a-13b are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

14a-14b are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

15a-15b are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

16a-16b are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

17a-17b are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

18a-18b are interface schematic diagrams of another group of shooting methods provided by an embodiment of the present application;

19a-19b are technical schematic diagrams of a shooting method provided by an embodiment of the application;

20a-20b are method flowcharts of a shooting method provided by an embodiment of the present application;

FIG. 21 is a method flowchart of another shooting method provided by an embodiment of the present application;

FIG. 22 is another system diagram provided by an embodiment of the present application;

FIGS. 23-24 are method flowcharts of still another photographing method provided by an embodiment of the present application.

detailed description

The technical solutions in the embodiments of the present application will be described clearly and in detail below with reference to the accompanying drawings. Wherein, in the description of the embodiments of the present application, unless otherwise specified, “/” means or, for example, A/B can mean A or B; “and/or” in the text is only a description of an associated object The association relationship indicates that there can be three kinds of relationships, for example, A and/or B can indicate that A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of this application , "plurality" means two or more than two.

Hereinafter, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as implying or implying relative importance or implying the number of indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, the "multiple" The meaning is two or more.

The electronic equipment/user equipment involved in the embodiments of the present application may be mobile phones, tablet computers, desktops, laptops, notebook computers, Ultra-mobile Personal Computers (UMPCs), handheld computers, netbooks, personal digital Assistant (Personal Digital Assistant, PDA), virtual reality device, PDA (Personal Digital Assistant, also known as PDA), portable Internet device, data storage device, camera or wearable device (for example, wireless headset, smart Watches, smart bracelets, smart glasses, head-mounted display (HMD), electronic clothing, electronic bracelets, electronic necklaces, electronic accessories, electronic tattoos and smart mirrors) and so on.

The embodiment of the present application provides a shooting method, which is applied to a system including at least an electronic device 100 and a wearable device 201. The electronic device 100 establishes a connection with the wearable device 201. The biometrics of the user detected by the sensor (such as heart rate, blood pressure, exercise posture, etc.), the electronic device 100 associates the biometrics with the captured pictures/videos, or associates the biometrics with the characters in the captured pictures/videos , the character corresponds to the biological feature. The electronic device 100 generates a picture/video including biometric information indicating the user's biometrics. This method associates the biometric information with the information of the picture/video in the process of picture/video generation, makes more accurate feature recognition for the picture/video, and provides a new picture/video format, making The electronic device 100 saves pictures/videos with biometric information, so as to facilitate subsequent classification of the stored pictures/videos according to the biometric information. Wherein, in order to implement the solutions involved in the embodiments of the present application, the number of the electronic device and the wearable device may be one or more, which is not limited in the present application.

FIG. 1 exemplarily shows a system diagram provided by the present application.

As shown in FIG. 1 , the system may include an electronic device 100 and one or more wearable devices (eg, wearable device 201 , wearable device 202 ). The electronic device 100 and the wearable device 201 (wearable device 202 ) may be connected by wireless communication. For example, the connection can be established by at least one of the following wireless connection manners: Bluetooth (blue tooth, BT), near field communication (near field communication, NFC), wireless fidelity (wireless fidelity, WiFi), or WiFi direct connection.

Wherein, optionally, the electronic device 100 may be connected with a plurality of different types of wearable devices. For example, the electronic device 100 can connect a smart watch and a wireless earphone through Bluetooth at the same time.

In the embodiments of the present application, the electronic device 100 and the wearable device 201 are connected via Bluetooth as an exemplary illustration.

The electronic device 100 is an electronic device having an imaging function. Such as mobile phones, tablets, or cameras, etc.

Wearable devices 201 include wireless headphones, smart watches, smart bracelets, smart glasses, smart rings, smart sports shoes, virtual reality display devices, smart headbands, electronic clothing, electronic bracelets, electronic necklaces, electronic accessories, electronic tattoos, or smart mirrors etc.

The wearable device 201 can detect the user's health state information, exercise state information, emotional state information, and the like through sensors. Health status information includes heart rate, blood pressure, blood sugar, EEG, ECG, EMG, body temperature and other information; exercise status information includes walking, running, cycling, swimming, badminton, skating, surfing, dancing and other common sports postures , can also include some more fine-grained motion gestures, such as: forehand, backhand, Latin dance, mechanical dance, etc.; emotional state information includes tension, anxiety, sadness, stress, excitement, joy, etc.

The electronic device 100 and the wearable device 201 are connected through Bluetooth. When taking pictures/videos, the electronic device 100 obtains the user's biometric information, which corresponds to the user's health state information, motion state information, emotional state information, etc. detected by the wearable device 201 through sensors. For example, the wearable device 201 detects heart rate data through a heart rate sensor and blood pressure data through a blood pressure sensor; the biometric information may be heart rate data, blood pressure data, or normal heart rate and blood pressure based on the heart rate data and blood pressure data. normal information.

The electronic device associates the biometric information with the information of the picture/video, and saves the picture/video with the biometric information. By classifying such pictures/videos with biometric information, the electronic device can quickly and accurately query and filter pictures/videos with different characteristics. Moreover, the sensors of the wearable device can provide more intrinsic features, and the feature recognition of pictures/videos can be more accurate.

First, the exemplary electronic device 100 provided in the following embodiments of the present application is introduced.

FIG. 2 shows a schematic structural diagram of the electronic device 100 .

The embodiment will be described in detail below by taking the electronic device 100 as an example. It should be understood that the electronic device 100 shown in FIG. 2 is only an example, and the electronic device 100 may have more or fewer components than those shown in FIG. 2, two or more components may be combined, or Different component configurations are possible. The various components shown in the figures may be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and/or application specific integrated circuits.

The electronic device 100 may include: a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2. Mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, buttons 190, motor 191, indicator 192, camera 193, display screen 194, And a subscriber identification module (subscriber identification module, SIM) card interface 195 and so on. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, and ambient light. Sensor 180L, bone conduction sensor 180M, etc.

It can be understood that the structures illustrated in the embodiments of the present application do not constitute a specific limitation on the electronic device 100 . In other embodiments of the present application, the electronic device 100 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (application processor, AP), a modem processor, a graphics processor (graphics processing unit, GPU), an image signal processor (image signal processor, ISP), controller, memory, video codec, digital signal processor (digital signal processor, DSP), baseband processor, and/or neural-network processing unit (NPU) Wait. Wherein, different processing units may be independent devices, or may be integrated in one or more processors. In some embodiments, electronic device 100 may also include one or more processors 110 .

The controller can generate an operation control signal according to the instruction operation code and timing signal, and complete the control of fetching and executing instructions.

A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in processor 110 is cache memory. This memory may hold instructions or data that have just been used or recycled by the processor 110 . If the processor 110 needs to use the instruction or data again, it can be called directly from the memory. Repeated accesses are avoided and the latency of the processor 110 is reduced, thereby increasing the efficiency of the system.

The charging management module 140 is used to receive charging input from the charger. The charger may be a wireless charger or a wired charger. The power management module 141 is used for connecting the battery 142 , the charging management module 140 and the processor 110 . The power management module 141 receives input from the battery 142 and/or the charging management module 140 and supplies power to the processor 110 , the internal memory 121 , the external memory, the display screen 194 , the camera 193 , and the wireless communication module 160 .

The wireless communication function of the electronic device 100 may be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, the baseband processor, and the like.

Antenna 1 and Antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 may be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve antenna utilization.

The mobile communication module 150 may provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 . The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA) and the like. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, filter and amplify the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and then turn it into an electromagnetic wave for radiation through the antenna 1 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the processor 110 . In some embodiments, at least part of the functional modules of the mobile communication module 150 may be provided in the same device as at least part of the modules of the processor 110 .

The modem processor may include a modulator and a demodulator. Wherein, the modulator is used to modulate the low frequency baseband signal to be sent into a medium and high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low frequency baseband signal. Then the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. The low frequency baseband signal is processed by the baseband processor and passed to the application processor. The application processor outputs sound signals through audio devices (not limited to the speaker 170A, the receiver 170B, etc.), or displays images or videos through the display screen 194 . In some embodiments, the modem processor may be a stand-alone device. In other embodiments, the modem processor may be independent of the processor 110, and may be provided in the same device as the mobile communication module 150 or other functional modules.

The wireless communication module 160 can provide applications on the electronic device 100 including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellites Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2 , frequency modulates and filters the electromagnetic wave signals, and sends the processed signals to the processor 110 . The wireless communication module 160 can also receive the signal to be sent from the processor 110 , perform frequency modulation on it, amplify it, and radiate it into electromagnetic waves through the antenna 2 . Exemplarily, the wireless communication module 160 may include a Bluetooth module, a Wi-Fi module, and the like.

In some embodiments, the antenna 1 of the electronic device 100 is coupled with the mobile communication module 150, and the antenna 2 is coupled with the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include the 5th Generation (the 5th Generation, 5G) system, the New Radio (New Radio, NR) system, the Global System for Mobile Communications (GSM), the General Packet Radio Service ( general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (time-division code division multiple access) access, TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and/or IR technologies, etc. The GNSS may include a global positioning system (global positioning system, GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (quasi -zenith satellite system, QZSS) and/or satellite based augmentation systems (SBAS).

Optionally, in some embodiments, a Bluetooth (BT) module and a WLAN module included in the wireless communication module 160 can transmit signals to detect or scan for devices near the electronic device 100, so that the electronic device 100 can use wireless communication such as Bluetooth or WLAN. The technology discovers nearby devices, establishes wireless communication connections with nearby devices, and shares data to nearby devices through the aforementioned connections. Among them, the Bluetooth (BT) module can provide a solution including one or more Bluetooth communications in classic Bluetooth (Bluetooth 2.1) or Bluetooth low energy (Bluetooth low energy, BLE). The WLAN module can provide a solution including one or more WLAN communications in Wi-Fi direct, Wi-Fi LAN or Wi-Fi softAP.

Optionally, in some embodiments, the wireless communication solution provided by the mobile communication module 150 may enable the electronic device to communicate with a device (such as a server) in the network, and the WLAN wireless communication solution provided by the wireless communication module 160 may also be used. The electronic device can be made to communicate with a device (such as a server) in a network, and can communicate with a cloud device through the device (such as a server) in the network. In this way, the electronic device can discover the cloud device and transmit data to the cloud device.

The electronic device 100 implements a display function through a GPU, a display screen 194, an application processor, and the like. The GPU is a microprocessor for image processing, and is connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs that execute program instructions to generate or alter display information.

Display screen 194 is used to display images, videos, and the like. The display screen 194 may be a flat display screen, a curved display screen, or a folding screen. When the display screen 194 is a folding screen, the folding screen is in a folding state, and the folding screen at least includes a first display area and a second display area. Wherein, the light emitting surfaces of the first display area and the second display area are different. The first display area is located in the first area of the folding screen, and the second display area is located in the second area of the folding screen. When the folding screen is in the folded state, the angle between the first area and the second area is greater than or equal to 0 degrees and less than 180 degrees. .

Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (active-matrix organic light). emitting diode, AMOLED), flexible light-emitting diode (flex light-emitting diode, FLED), Miniled, MicroLed, Micro-oLed, quantum dot light-emitting diode (quantum dot light emitting diodes, QLED) and so on. In some embodiments, the electronic device 100 may include one or N display screens 194 , where N is a positive integer greater than one. When the electronic device 100 includes two or more display screens, the shapes of the display screens may be different. For example, one of the displays may be a folding display and the other display may be a flat display. For example, one of the displays could be a color display and the other could be a black and white display.

The electronic device 100 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

The ISP is used to process the data fed back by the camera 193 . For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the camera 193 .

Camera 193 is used to capture still images or video. The object is projected through the lens to generate an optical image onto the photosensitive element. The photosensitive element may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals. In some embodiments, the electronic device 100 may include 1 or N cameras 193 , where N is a positive integer greater than 1.

The camera 193 may be a 3D camera, and the electronic device 100 may implement a camera function through the 3D camera, ISP, video codec, GPU, display screen 194, application processor AP, neural network processor NPU, and the like.

3D cameras can be used to capture color image data as well as depth data of the subject. The ISP can be used to process the color image data captured by the 3D camera. For example, when taking a photo, the shutter is opened, the light is transmitted to the camera photosensitive element through the lens, the light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing, and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on image noise, brightness, and skin tone. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, the ISP may be provided in the 3D camera.

Optionally, in some embodiments, the 3D camera may be composed of a color camera module and a 3D sensing module.

Optionally, in some embodiments, the photosensitive element of the camera of the color camera module may be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (complementary metal-oxide-semiconductor,

CMOS) phototransistors. The photosensitive element converts the optical signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. DSP converts digital image signals into standard RGB, YUV and other formats of image signals.

Optionally, in some embodiments, the 3D sensing module may be a time of flight (TOF) 3D sensing module or a structured light (structured light) 3D sensing module. The structured light 3D sensing is an active depth sensing technology, and the basic components of the structured light 3D sensing module may include an infrared (Infrared) emitter, an IR camera module, and the like. The working principle of the structured light 3D sensing module is to first emit a light spot of a specific pattern on the object to be photographed, and then receive the light coding of the light spot pattern on the surface of the object, and then compare the similarities and differences with the original projected light spot. And use the principle of trigonometry to calculate the three-dimensional coordinates of the object. The three-dimensional coordinates include the distance between the electronic device 100 and the object to be photographed. Among them, TOF 3D sensing is also an active depth sensing technology, and the basic components of the TOF 3D sensing module may include an infrared (Infrared) transmitter, an IR camera module, and the like. The working principle of the TOF 3D sensing module is to calculate the distance (ie depth) between the TOF 3D sensing module and the object to be photographed through the time of infrared reentrant, so as to obtain a 3D depth of field map.

A digital signal processor is used to process digital signals, in addition to processing digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy and so on.

Video codecs are used to compress or decompress digital video. The electronic device 100 may support one or more video codecs. In this way, the electronic device 100 can play or record videos in various encoding formats, such as: Moving Picture Experts Group (moving picture experts group, MPEG)-1, MPEG-2, MPEG-3, MPEG-4 and so on.

The NPU is a neural-network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transfer mode between neurons in the human brain, it can quickly process the input information, and can continuously learn by itself. Applications such as intelligent cognition of the electronic device 100 can be realized through the NPU, such as: image recognition, facial recognition, speech recognition, text understanding, etc.

The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100 . The external memory card communicates with the processor 110 through the external memory interface 120 to realize the data storage function. For example, save data such as music, photos, videos, etc. in an external memory card.

Internal memory 121 may be used to store computer executable program code, which includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by executing the instructions stored in the internal memory 121 . The internal memory 121 may include a storage program area and a storage data area. The storage program area can store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), and the like. The storage data area may store data (such as audio data, phone book, etc.) created during the use of the electronic device 100 and the like. In addition, the internal memory 121 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, universal flash storage (UFS), and the like.

The electronic device 100 may implement audio functions through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, an application processor, and the like. Such as music playback, recording, etc.

The audio module 170 is used for converting digital audio information into analog audio signal output, and also for converting analog audio input into digital audio signal. Audio module 170 may also be used to encode and decode audio signals. Speaker 170A, also referred to as a "speaker", is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music through the speaker 170A, or listen to a hands-free call. The receiver 170B, also referred to as "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 answers a call or a voice message, the voice can be answered by placing the receiver 170B close to the human ear. The microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by approaching the microphone 170C through a human mouth, and input the sound signal into the microphone 170C. The electronic device 100 may be provided with at least one microphone 170C.

The earphone jack 170D is used to connect wired earphones. The earphone interface 170D can be the USB interface 130, or can be a 3.5mm open mobile terminal platform (OMTP) standard interface, a cellular telecommunications industry association of the USA (CTIA) standard interface.

The pressure sensor 180A is used to sense pressure signals, and can convert the pressure signals into electrical signals.

The gyro sensor 180B may be used to determine the motion attitude of the electronic device 100 .

The air pressure sensor 180C is used to measure air pressure.

The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can detect the opening and closing of the flip holster using the magnetic sensor 180D.

The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in various directions (generally three axes). The magnitude and direction of gravity can be detected when the electronic device 100 is stationary. It can also be used to identify the posture of electronic devices, and can be used in applications such as horizontal and vertical screen switching, pedometers, etc.

Distance sensor 180F for measuring distance. The electronic device 100 can measure the distance through infrared or laser.

Proximity light sensor 180G may include, for example, light emitting diodes (LEDs) and light detectors, such as photodiodes. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power.

The ambient light sensor 180L is used to sense ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket, so as to prevent accidental touch.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to realize fingerprint unlocking, accessing application locks, taking pictures with fingerprints, answering incoming calls with fingerprints, and the like.

The temperature sensor 180J is used to detect the temperature.

Touch sensor 180K, also called "touch panel". The touch sensor 180K may be disposed on the display screen 194 , and the touch sensor 180K and the display screen 194 form a touch screen, also called a “touch screen”. The touch sensor 180K is used to detect a touch operation on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to touch operations may be provided through display screen 194 .

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire the vibration signal of the vibrating bone mass of the human voice. The bone conduction sensor 180M can also contact the pulse of the human body and receive the blood pressure beating signal. In some embodiments, the bone conduction sensor 180M can also be disposed in the earphone, combined with the bone conduction earphone. The audio module 170 can analyze the voice signal based on the vibration signal of the vocal vibration bone block obtained by the bone conduction sensor 180M, so as to realize the voice function. The application processor can analyze the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M, and realize the function of heart rate detection.

The keys 190 include a power-on key, a volume key, and the like. Keys 190 may be mechanical keys. It can also be a touch key. The electronic device 100 may receive key inputs and generate key signal inputs related to user settings and function control of the electronic device 100 .

Motor 191 can generate vibrating cues. The motor 191 can be used for vibrating alerts for incoming calls, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, playing audio, etc.) can correspond to different vibration feedback effects. The motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of the display screen 194 . Different application scenarios (for example: time reminder, receiving information, alarm clock, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.

The indicator 192 can be an indicator light, which can be used to indicate the charging state, the change of the power, and can also be used to indicate a message, a missed call, a notification, and the like.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be contacted and separated from the electronic device 100 by inserting into the SIM card interface 195 or pulling out from the SIM card interface 195 . The electronic device 100 may support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM card, Micro SIM card, SIM card and so on. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the plurality of cards may be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as call and data communication. In some embodiments, the electronic device 100 employs an eSIM, ie: an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100 .

The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a microkernel architecture, a microservice architecture, or a cloud architecture. The embodiments of the present application take an Android system with a layered architecture as an example to exemplarily describe the software structure of the electronic device 100 . The Android system is only a system example of the electronic device 100 in the embodiment of the present application, and the present application may also be applicable to other types of operating systems, such as IOS, windows, etc., which is not limited in the present application. The following only takes the Android system as an example of the operating system of the electronic device 100 .

FIG. 3 is a block diagram of the software structure of the electronic device 100 according to the embodiment of the present application.

A layered architecture divides software into several layers. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, which are, from top to bottom, an application layer, an application framework layer, an Android runtime (Android runtime) and a system library, and a kernel layer.

The application layer can include a series of application packages.

As shown in Figure 3, the application package can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, short message and so on.

The application framework layer provides an application programming interface (application programming interface, API) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

As shown in Figure 3, the application framework layer may include window managers, content providers, view systems, telephony managers, resource managers, notification managers, and the like.

A window manager is used to manage window programs. The window manager can get the size of the display screen, determine whether there is a status bar, lock the screen, take screenshots, etc.

Content providers are used to store and retrieve data and make these data accessible to applications. The data may include video, images, audio, calls made and received, browsing history and bookmarks, phone book, etc.

The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, and so on. View systems can be used to build applications. A display interface can consist of one or more views. For example, the display interface including the short message notification icon may include a view for displaying text and a view for displaying pictures.

The phone manager is used to provide the communication function of the electronic device 100 . For example, the management of call status (including connecting, hanging up, etc.).

The resource manager provides various resources for the application, such as localization strings, icons, pictures, layout files, video files and so on.

The notification manager enables applications to display notification information in the status bar, which can be used to convey notification-type messages, and can disappear automatically after a brief pause without user interaction. For example, the notification manager is used to notify download completion, message reminders, etc. The notification manager can also display notifications in the status bar at the top of the system in the form of graphs or scroll bar text, such as notifications of applications running in the background, and notifications on the screen in the form of dialog windows. For example, text information is prompted in the status bar, a prompt sound is issued, the electronic device vibrates, and the indicator light flashes.

Android Runtime includes core libraries and a virtual machine. Android runtime is responsible for scheduling and management of the Android system.

The core library consists of two parts: one is the function functions that the java language needs to call, and the other is the core library of Android.

The application layer and the application framework layer run in virtual machines. The virtual machine executes the java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, safety and exception management, and garbage collection.

A system library can include multiple functional modules. For example: image processing module, video processing module, surface manager (surface manager), media library (Media Libraries), 3D graphics processing library (eg: OpenGL for Embedded Systems, OpenGL ES)), 2D graphics Engine (for example: Skia Graphics Library (Skia Graphics Library, SGL)), etc.

The image processing module is used to encode, decode and render the image, so that the application can display the image on the display screen. The conversion of image formats and the generation of image files can be realized.

The video processing module is used to encode, decode, and render video frames, so that the application can display the video on the display screen. It can realize the conversion of video formats and the generation of video files.

The Surface Manager is used to manage the display subsystem and provides a fusion of 2D and 3D layers for multiple applications.

The media library supports playback and recording of a variety of commonly used audio and video formats, as well as still image files. The media library can support a variety of audio and video encoding formats, such as: Motion Picture Expert Group 4 (Motion Picture Expert Group, MPEG4), Advanced Video Coding (MPEG-4 Part 10 Advanced Video Coding, MPEG-4 AVC/H.264), Motion Picture Expert Compression Standard Audio Layer 3 (MPEG Audio Layer 3, MP3), Advanced Audio Coding (Advanced Audio Coding, AAC), Adaptive Multi-Rate (AMR), Joint Photographic Experts Group (Joint Photographic Experts Group, JPEG/JPG), Portable Network Graphics (PNG), etc.

The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

2D graphics engine is a drawing engine for 2D drawing.

The kernel layer is the layer between hardware and software. The kernel layer contains at least display drivers, camera drivers, audio drivers, and sensor drivers.

The software system shown in FIG. 3 involves application presentation (eg, gallery, file manager) using photographing capabilities, and the application framework layer provides WLAN services, Bluetooth services, and the kernel and bottom layer provide WLAN Bluetooth capabilities and basic communication protocols.

In the following, the workflow of the software and hardware of the electronic device 100 is exemplarily described in conjunction with the capturing and photographing scene.

When the touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes touch operations into raw input events (including touch coordinates, timestamps of touch operations, etc.). Raw input events are stored at the kernel layer. The application framework layer obtains the original input event from the kernel layer, and identifies the control corresponding to the input event. Taking the control corresponding to the touch operation as the control of the camera application icon as an example, the camera application calls the interface of the application framework layer to start the camera service corresponding to the camera application, and then the camera service starts the camera driver by calling the kernel layer, and captures static images through the camera 193. image or video.

The camera service synchronously calls the kernel layer to start the Bluetooth driver, sends a request message to the connected wearable device through the Bluetooth antenna, and receives the sensor data or biometric information sent by the above-mentioned wearable device based on the request message; the camera service calls the image processing module or video processing module The module writes the biometric information into the image frame of the image or video, and generates a picture file or video file with the biometric information.

In this application, picture files and video files may be referred to as multimedia files.

FIG. 4 exemplarily shows a schematic structural diagram of the wearable device 201 provided by the present application.

As shown in FIG. 4 , the wearable device 201 may include a processor 102 , a memory 103 , a wireless communication processing module 104 , a mobile communication processing module 105 , a touch display screen 106 and a sensor module 107 . These components can be connected via a bus. in:

The processor 102 may be used to read and execute computer readable instructions. In a specific implementation, the processor 102 may mainly include a controller, an arithmetic unit and a register. Among them, the controller is mainly responsible for instruction decoding, and sends out control signals for the operations corresponding to the instructions. The arithmetic unit is mainly responsible for performing fixed-point or floating-point arithmetic operations, shift operations, and logical operations, and can also perform address operations and conversions. Registers are mainly responsible for saving register operands and intermediate operation results temporarily stored during instruction execution. In specific implementation, the hardware architecture of the processor 102 may be an application specific integrated circuit (Application Specific Integrated Circuits, ASIC) architecture, a MIPS architecture, an ARM architecture, an NP architecture, or the like.

In some embodiments, the processor 102 may be configured to parse the signals received by the wireless communication processing module 104 and the wired LAN communication processing module 116, such as a request message sent by the electronic device 100 to obtain sensor data or biometric information, the electronic device 100 A request message sent to stop acquiring sensor data or biometric information. The processor 102 may be configured to perform corresponding analysis processing according to the information collected by the sensor module 106, such as analyzing the user's health state, exercise state, emotional state, and the like.

Memory 103 is coupled to processor 102 for storing at least one of various software programs or sets of instructions. In specific implementations, memory 103 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic disk storage devices, flash memory devices, or other non-volatile solid-state storage devices. The memory 103 can store an operating system, such as an embedded operating system such as uCOS, VxWorks, RTLinux, and the like. Memory 103 may also store communication programs that may be used to communicate with electronic device 100, one or more servers, or additional devices.

The wireless communication processing module 104 may include one or more of a Bluetooth (BT) communication processing module 104A, a WLAN communication processing module 104B.

In some embodiments, one or more of the Bluetooth (BT) communication processing module and the WLAN communication processing module may listen to signals transmitted by other devices (eg, electronic device 100 ), such as probe requests, scan signals, etc., and A response signal, such as a probe response, a scan response, etc., can be sent, so that other devices (such as the electronic device 100) can discover the wearable device 201 and establish a wireless communication connection with other devices (such as the electronic device 100) through a Bluetooth or a WLAN. One or more wireless communication technologies to communicate with other devices, such as electronic device 100 .

In other embodiments, one or more of the Bluetooth (BT) communication processing module and the WLAN communication processing module may also transmit signals, such as broadcasting Bluetooth signals, beacon signals, so that other devices (eg, electronic device 100 ) can The wearable device 201 is discovered, and a wireless communication connection is established with other devices (eg, the electronic device 100 ), and communicates with other devices (eg, the electronic device 100 ) through one or more wireless communication technologies in Bluetooth or WLAN.

The mobile communication processing module 105 can provide wireless communication solutions including 2G/3G/4G/5G etc. applied on the electronic device 100 . For performing cellular communications and/or data communications, for example, the mobile communications module 106 may include a circuit switched module ("CS" module) for performing cellular communications and a packet switched module ("PS" module) for performing data communications . In this application, the mobile communication processing module 105 may communicate with other devices (such as servers) through the fourth generation mobile communication technology (4th generation mobile networks) or the fifth generation mobile communication technology (5th generation mobile networks).

The touch screen 106, also known as a touch panel, is an inductive liquid crystal display device that can receive input signals such as contacts, and can be used to display images, videos, and the like. The touch screen 106 can use a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, an active-matrix organic light emitting diode (AMOLED) ) display, flexible light-emitting diode (flexible light-emitting diode, FLED) display, quantum dot light emitting diode (quantum dot light emitting diodes, QLED) display and so on.

Sensor module 107 may include motion sensor 107A, biosensor 107B, environmental sensor 107C, and the like. in,

The motion sensor 107A is a component that converts changes in non-electricity (eg, speed, pressure) into changes in electricity. It may include at least one of the following: an acceleration sensor, a gyro sensor, a geomagnetic sensor (also known as an electronic compass sensor), or an atmospheric pressure sensor. Among them, the acceleration sensor can detect the magnitude of acceleration in various directions (generally three axes, ie x, y and z axes). Gyroscopic sensors can be used to determine motion attitude. Electronic compass sensors can be used to measure direction, enable or assist navigation. The atmospheric pressure sensor is used to measure the air pressure. In some embodiments, the altitude change of the location can be calculated through the weak air pressure change during the movement, and the accuracy can be controlled within 10CM during the height movement of the 10-story building. Data ranging from rock climbing to small stair climbing can be monitored.

In this application, the motion sensor 107A can measure the user's activity, such as running steps, speed, swimming laps, cycling distance, exercise posture (such as playing ball, swimming, running) and the like.

Biosensor 107B is an instrument that is sensitive to biological substances and converts their concentration into electrical signals for detection. It consists of immobilized biologically sensitive materials as identification elements (including enzymes, antibodies, antigens, microorganisms, cells, tissues, nucleic acids and other biologically active substances) and appropriate physical and chemical transducers (such as oxygen electrodes, photosensitive tubes, field effect tubes, Piezoelectric crystal, etc.), that is, an analysis tool or system composed of a signal amplifying device. Biosensors function as receivers and converters. The biosensor 107B may include at least one of the following: a blood sugar sensor, a blood pressure sensor, an electrocardiogram sensor, an electromyography sensor, a body temperature sensor, a brain wave sensor, etc. The main functions of these sensors include health and medical monitoring, entertainment, and the like.

Among them, the blood sugar sensor is used to measure blood sugar. Blood pressure sensors are used to measure blood pressure. ECG sensors, for example, use silver nanowires to monitor electrophysiological signals, such as electrocardiograms. EMG sensors are used to monitor EMG. Body temperature sensors are used to measure body temperature, and brain wave sensors are used to monitor brain waves. In this application, various physiological indicators of the user (such as blood sugar, blood pressure, body temperature, electrocardiogram, etc.) can be measured by the biosensor 107B, and the wearable device 201 can calculate the health status of the user according to the physiological indicators.

The biosensor 107B may also include a heart rate sensor, a galvanic sensor. in,

The heart rate sensor can track the user's exercise intensity, different exercise training modes, etc. by detecting the user's heart rate, and can calculate the user's sleep cycle, sleep quality and other health data. When the capacitive light strikes the skin, the light reflected back through the skin tissue is received by the photosensitive sensor and converted into an electrical signal, which is then converted into a digital signal, and then the heart rate can be measured according to the absorbance of the blood.

Galvanic sensors are used to measure the user's arousal, which is closely linked to the user's attention and engagement, and is usually equipped on some devices that can monitor sweat levels. The skin resistance and conductance of the human body change with the changes in the function of the skin sweat glands, and these measurable skin galvanic changes are called electrodermal activity (EDA).

In the embodiment of the present application, the wearable device 201 measures the psychologically induced sweat gland activity through the electrodermal sensor to determine the user's psychological activity, such as the user's mood index. For example, feeling happy, nervous, fearful, stressed, etc.

The environmental sensor 107C may include at least one of the following: an air temperature and humidity sensor, a rain sensor, a light sensor, a wind speed and direction sensor, a particle sensor, and the like. The environmental sensor 107C can detect air quality, such as the degree of haze, indoor formaldehyde concentration, PM2.5 detection, and so on. In this application, weather changes, air humidity, air quality, etc. can be measured by the environmental sensor 107C.

It can be understood that the structure shown in FIG. 4 does not constitute a specific limitation on the wearable device 201 . Optionally, in other embodiments of the present application, the wearable device 101 may include more or less components than shown, or combine some components, or separate some components, or arrange different components. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

In some specific scenarios, the user wears the wearable device, and the electronic device establishes a connection with the wearable device. The user triggers the electronic device to start the camera to take a picture, the electronic device obtains the picture information, and obtains the biometric information. The biometric information corresponds to sensor data detected by the wearable device through at least one sensor. Wherein, the sensor data involved in this application includes, but is not limited to, data detected by at least one of the motion sensor 107A, the biosensor 107B, and the environmental sensor 107C mentioned above.

Optionally, the above-mentioned biometric information may be sent to the electronic device after the wearable device performs analysis and processing based on sensor data; The sensor data is analyzed and processed.

The electronic device associates the biometric information with the picture information, or associates the biometric information with a person in the photographed picture, and the person corresponds to the biometric information. The electronic device generates a picture file with biometric information, and in response to the user viewing the picture file, the electronic device may display the biometric information or display a portion of the biometric information.

In the following, a smartphone is taken as an example of the above-mentioned electronic device, and an implementation form of the photographing method provided by the present application on the display interface of the smartphone is exemplarily described.

First, the startup annotation mode is introduced.

Method 1: Start the camera of the electronic device and select the labeling mode.

A camera application is an application software for taking pictures of an electronic device. When the user wants to shoot an image or video, the camera application can be started, and the electronic device calls at least one camera to shoot. As shown in Figure 5a, Figure 5a illustrates an exemplary user interface on an electronic device for displaying a list of applications. Figure 5a includes a status bar 201 and a display interface 202, wherein the status bar 201 may include: one or more signal strength indicators 203 of mobile communication signals (also known as cellular signals), wireless fidelity (Wi-Fi) ) signal of one or more of signal strength indicator 207, Bluetooth indicator 208, battery status indicator 209, time indicator 211. When the Bluetooth module of the electronic device is in an on state (ie, the electronic device supplies power to the Bluetooth module), a Bluetooth indicator 208 is displayed on the display interface of the electronic device.

The display interface 202 displays a plurality of application icons. The display interface 202 includes an application icon of the camera 205 . When the electronic device detects a user operation acting on the application icon of the camera 205, the electronic device displays an application interface provided by the camera application.

Referring to Figure 5b, Figure 5b shows a possible user interface provided by a camera application. The application interface of the camera 205 is shown in FIG. 5b . The application interface may include: a display area 30 , a flash icon 301 , a setting icon 302 , a mode selection area 303 , a gallery icon 304 , a shooting icon 305 , and a switching icon 306 .

The display content of the display area 30 is the preview display interface of the image captured by the camera currently used by the electronic device. The camera currently used by the electronic device may be the default camera set by the camera application, and the camera currently used by the electronic device may also be the camera used when the camera application was closed last time.

The flash icon 301 can be used to indicate the working status of the flash.

The setting icon 302, when a user operation acting on the setting icon 302 is detected, in response to the operation, the electronic device can display other shortcut functions, such as adjusting the resolution, time-lapse shooting (also known as time-lapse shooting, which can be controlled to start taking pictures) time), shooting mute, voice-activated photo, smile capture (when the camera detects a smile feature, automatically focus on the smile) and other functions.

The mode selection area 303 is used to provide different shooting modes. According to the different shooting modes selected by the user, the cameras and shooting parameters enabled by the electronic device are also different. An annotation mode 303A, a night scene mode 303B, a photographing mode 303C, a video recording mode 303D, and more 303E may be included. In FIG. 5b, the icon of the photographing mode 303C is marked to prompt the user that the current mode is the photographing mode. in,

Labeling mode 303A, in this mode, when the electronic device detects a user operation for taking pictures/videos, the electronic device obtains the image information currently collected by the camera, and obtains biometric information, which is detected by the wearable device through the sensor. Corresponding to the sensor data (such as the heart rate data detected by the heart rate sensor, the blood pressure data detected by the blood pressure sensor), the electronic device fuses and encodes the biometric information and the image information obtained by shooting, and generates a picture/video file with the biometric information. . For example, the biometric information may be information such as the user's heart rate data, blood pressure data, etc.; it may also be information such as normal heart rate and normal blood pressure.

Optionally, in some embodiments, if the electronic device does not turn on Bluetooth, when the electronic device detects the user operation 307 acting on the labeling mode 303A, in response to the user operation 307, the electronic device automatically turns on Bluetooth, and automatically searches for connectable devices. The connection is established according to the user's choice, or the electronic device automatically establishes a connection with the Bluetooth device that has previously established a connection.

Optionally, in some embodiments, the electronic device may simultaneously receive sensor data or biometric information of the wearable devices of two users, and perform fusion coding with the two biometric information and the image information obtained by shooting, and then combine the two biometric information in one picture/video. The biometric information of the two users is associated in the file.

When a user operation acting on the annotation mode 303A is detected, in response to the operation, the icon of the photographing mode 303C in the mode selection area 303 of the electronic device is no longer marked, but the annotation mode 303A is marked.

The night scene mode 303B can improve the detail rendering ability of bright and dark parts, control noise, and present more picture details. The photographing mode 303C is suitable for most photographing scenes, and can automatically adjust photographing parameters according to the current environment. Video mode 303D, used to shoot a video. More 303E, when detecting a user operation acting on more 303E, in response to the operation, the electronic device may display other selection modes, such as panorama mode (to achieve automatic stitching, the electronic device stitches multiple photos taken continuously into one 3 photos to achieve the effect of expanding the viewing angle of the picture), HDR mode (automatic continuous shooting underexposure, normal exposure, overexposure three photos, and select the best part to combine into one photo) and so on.

When detecting a user operation acting on an application icon of any mode in the mode selection area 303 (for example, the label mode 303A, the night scene mode 303B, the photo mode 303C, the video mode 303D, the panorama mode, the HDR mode, etc.) Operation, the photographing device/electronic device can enter the corresponding mode. Correspondingly, the image displayed in the display area 30 is the image processed in the current mode.

The mode icons in the mode selection area 303 are not limited to virtual icons, and can also be selected through physical buttons deployed on the photographing device/electronic device, so that the photographing device enters the corresponding mode.

The gallery icon 304, when a user operation acting on the gallery icon 304 is detected, in response to the operation, the electronic device may enter a gallery of the electronic device, and the gallery may include photos and videos that have been taken. The gallery icon 304 may be displayed in different forms. For example, after the electronic device saves the image currently captured by the camera, the gallery icon 304 displays a thumbnail of the image.

Shooting icon 305, when a user operation (such as a touch operation, voice operation, gesture operation, etc.) acting on the shooting icon 305 is detected, in response to the operation, the electronic device acquires the image currently displayed in the display area 30 and saves it in the gallery middle. The gallery can be entered through a user operation (eg, touch operation, gesture operation, etc.) on the gallery icon 304 .

The switch icon 306 can be used to switch between the front camera and the rear camera. The shooting direction of the front camera is the same as the display direction of the screen of the electronic device used by the user, and the shooting direction of the rear camera is opposite to the display direction of the screen of the electronic device used by the user. If the display area 30 currently displays the image captured by the rear camera, when a user operation acting on the switch icon 306 is detected, the display area 30 displays the image captured by the front camera in response to the operation. If the display area 30 currently displays the image captured by the front camera, when a user operation acting on the switch icon 306 is detected, the display area 30 displays the image captured by the rear camera in response to the operation.

As shown in FIG. 5c, FIG. 5c exemplarily shows an application interface corresponding to the annotation mode 303A. The icon of the smart label 303A in the mode selection area 303 is marked, indicating that the current mode is the label mode.

Optionally, as shown in the display area 30 of FIG. 5c, prompt information may also be displayed, which is used to prompt the user that the current electronic device is connected to the wearable device. For example, the text "connected to a wearable device" is displayed in the prompt area 308, indicating that the current electronic device has been connected to a wearable device, and the wearable device may be an earphone, a watch, a bracelet, glasses, or the like. For another example, the text "not connected to the wearable device" is displayed in the prompt area 308, prompting the user that the current electronic device is not connected to the wearable device. The connection method may be a short-distance connection method such as a Bluetooth connection or a WiFi connection.

Optionally, in some embodiments, the content in the prompt area 308 may prompt the user using the electronic device, so that the user wearing the wearable device is within the shooting area. For example, the text "Please confirm that the user wearing the wearable device is within the shooting range of the lens" is displayed in the prompt area 308 .

In this application, user operations include but are not limited to operations such as clicks, shortcut keys, gestures, floating touch, and voice commands.

Optionally, referring to Fig. 5d, Fig. 5d shows yet another possible user interface provided by the camera application. The application interface of the camera 205 is shown in FIG. 5d . The difference from FIG. 5b is that the application interface 31 includes a label icon 310 . When the electronic device detects a user operation acting on the annotation icon 310, in response to the operation, the electronic device activates the annotation function. In this way, the labeling function can be enabled in any shooting mode in the mode selection area 303 .

When the electronic device is in the photographing mode 303C, a user operation acting on the labeling icon 310 is detected, and in response to the operation, the electronic device starts the labeling function in the photographing mode 303C. When the electronic device is in the night scene mode 303B, a user operation acting on the label icon 310 is detected, and in response to the operation, the electronic device starts the label function in the night scene mode 303B.

As shown in FIG. 5e, FIG. 5e exemplarily shows the application interface after the labeling function is activated. The labeling icon 310 is marked, indicating that the labeling function is currently activated.

In the embodiment of the present application, it is not limited to start the camera through the application icon of the camera 205 . For example, you can also activate the camera through text messages, the shooting function of social software, video calls, etc., and further enable the labeling function. When the annotation function is enabled, the electronic device obtains pictures/videos with biometric information, and shares the pictures/videos with biometric information through short messages, social software, video calls, etc.

In the second method, the camera is activated through the first application icon, and the shooting mode of the camera is the labeling mode by default.

Among them, "smart labeling" is only an example, and other names are also possible. As shown in FIG. 6a, the display interface 202 of FIG. 6a displays a plurality of application icons. The display interface 202 includes the application icon of the smart label 212 . If the user wants to activate the annotation mode, the application icon of the smart annotation 212 is triggered by the user operation. The electronic device displays the application interface of the smart annotation 212 in response to the user operation.

In some embodiments, if the electronic device does not have Bluetooth enabled, when the electronic device detects a user operation acting on the smart label 212, in response to the user operation, the electronic device automatically enables Bluetooth, automatically searches for connectable Bluetooth devices, and selects a Bluetooth device according to the user's choice. A connection is established, or the electronic device automatically establishes a connection with a Bluetooth device that was previously connected.

Referring to FIG. 6b , FIG. 6b exemplarily shows a possible application interface provided by the smart annotation 212 . The application interface may include: a display area 40 , a flash icon 401 , a setting icon 402 , a gallery icon 403 , a shooting icon 404 , a switching icon 405 , and a prompt area 406 .

It should be noted that, based on the same inventive concept, the flash icon 401, the setting icon 402, the gallery icon 403, the shooting icon 404, the switching icon 405, and the prompt area 406 provided by the embodiment shown in FIG. The embodiments are similar, so the implementation of the flash icon 401, the setting icon 402, the gallery icon 403, the shooting icon 404, the switching icon 405, and the prompt area 406 in FIG. The corresponding descriptions corresponding to the implementation of the prompt area 308 of the shooting icon 305 and the switching icon 306 will not be repeated here.

Optionally, in some embodiments, the labeling mode can also be started through the wearable device APP in the electronic device. For example, the application icon in the display interface 202 of FIG. 6a, the smart wearable 214. The smart wearable 214 is an application for managing and interacting with one, one or more types of wearable devices, including function management, authority management, and the like. The application interface of the smart wearable 214 may include function controls of multiple wearable devices. The electronic device and the wearable device are paired and connected, and the user selects to enter the user interface of the corresponding wearable device. In the user interface of the wearable device, the electronic device detects the user operation for starting the labeling mode, and realizes the starting labeling mode. The electronic device can obtain to the biometric information corresponding to the sensor data of the wearable device.

The above two methods describe different methods for enabling the labeling mode of the electronic device and the corresponding display interface. Both FIG. 5c and FIG. 6b exemplarily show the application interface in the annotation mode. Optionally, Figure 7a also provides a possible application interface.

As shown in Fig. 7a, compared to Fig. 6b, the display area 40 may further include a preview icon 407, where the preview icon 407 is used to trigger the electronic device to acquire biometric information. When the electronic device detects a user operation on the preview icon 407, the electronic device sends a request message to the wearable device. After the wearable device receives the request message, it sends sensor data or biometric information to the electronic device. Or biometric information, displaying at least part of the biometric information on the display screen in real time. Specifically, after the wearable device receives the request message, the wearable device sends sensor data to the electronic device, and the electronic device determines the biometric information based on the received sensor data, and displays at least part of the biometric information on the display screen in real time; or, After the wearable device receives the request message, the wearable device determines the biometric information based on the sensor data, sends the biometric information to the electronic device, and the electronic device displays at least part of the biometric information on the display screen in real time based on the received biometric information.

Optionally, in some embodiments, when the electronic device detects a user operation for the smart annotation 212 in FIG. 6a, the electronic device displays the interface shown in FIG. 7a, and sends a request message to the wearable device, and the wearable device receives the request. After the message, sensor data or biometric information is sent to the electronic device. The preview icon 407 is used to trigger the electronic device to display at least part of the biometric information on the display screen in real time based on the acquired sensor data or biometric information.

For example, as shown in FIG. 7b, when the electronic device detects a user operation on the preview icon 407, the electronic device displays the interface shown in FIG. 7b, the display area 40 includes a preview area 408, and the preview area 408 displays the user widget A The health status of Xiao A, for example, the heart rate of Xiao A is normal; it also shows the exercise status of Xiao A, such as Xiao A is running. The preview area 408 may also include information such as Little A's blood pressure, blood sugar, whether the exercise posture is standardized, emotions (eg, happy, nervous, sad).

Optionally, in a possible embodiment, in response to a user operation, the electronic device may directly display the application interface of FIG. 7b without being triggered by the preview icon 407 in FIG. 7a. When the electronic device detects the user operation that activates the annotation mode, the electronic device sends a request message to the wearable device. After the wearable device receives the request message, it sends sensor data or biometric information to the electronic device. feature information, at least part of the biometric information is displayed on the display screen in real time, as shown in Figure 7b.

The above-mentioned embodiment provides a possible application interface of the smart annotation, and the electronic device displays the application interface of the smart annotation 303A in response to the user operation 307 (as shown in FIG. 5c ); or the electronic device displays the application interface of the intelligent annotation 212 in response to the user operation (as shown in Figure 6b); or display the application interface as shown in Figure 7a or Figure 7b, and so on.

Optionally, in the labeling mode or before entering the labeling mode, the user can configure the biometric information on the side of the electronic device, and select to obtain specific biometric information as required.

The user clicks the setting icon in the above application interface, and the electronic device displays the setting interface. Exemplarily, in the above application interface, the electronic device may display the setting interface 60 of the annotation mode as shown in FIG. 8a in response to the user operation on the setting icon 302 (or the setting icon 402).

As shown in FIG. 8a, the setting interface 60 includes functions such as setting resolution, taking pictures regularly, muting pictures, voice-activated pictures, and capturing smiley faces. In the setting interface 60 , the setting interface 60 includes a region bar of the wearable device 601 , and the wearable device can be specifically configured.

The electronic device displays the interface 70 of the wearable device as shown in FIG. 8b in response to the user operation for the option 602 . The setting interface 70 includes my device 701 and other devices 702, wherein, my device 701 includes the device that the electronic device is connecting to and the devices that have been connected, for example, the watch including small A is the device that the electronic device is connecting to, while the small B , indicating that the electronic device and Xiao B's watch have been connected before, but there is currently no connection. The other devices 702 refer to the connectable and unconnected devices searched by the electronic device through Bluetooth, for example, including the earphones of small A and the watch of small C.

The user can click on the area 703 to enter the specific configuration interface of the small A watch, and the electronic device can enter the specific configuration interface 80 of the small A watch as shown in FIG. 8c in response to the user operation on the area 703. As shown in FIG. 8c, the configuration interface 80 includes a plurality of configuration options, such as heart rate, blood pressure, blood sugar, exercise state, emotional state, and the like. The icon 801 can be described as a switch, when the circle in the icon 801 is on the left, it means off; when the circle in the icon 801 is on the right, it means on; wherein the off state and the on state can be switched by a single-click operation. For example, as shown in FIG. 8c, the circle of the icon 801 in the heart rate column is on the left at this time, indicating that the biometric information of the picture/video currently captured by the electronic device does not include heart rate information. 801, the circle of the icon 801 moves to the right, indicating that the biometric information of the picture/video currently captured by the electronic device includes heart rate information.

The configuration interface 80 also includes an icon 802 associated with a facial image. The electronic device, in response to a user operation on the icon 802, obtains the facial image information of the user wearing the small A watch, and combines the small A watch with the facial image information. The facial image information can be an image uploaded by the user. The user clicks on the icon 802 associated with the facial image to upload the facial image of Little A, or after clicking on the icon 802, the facial image of Little A is obtained through the camera of the electronic device, and the facial image of Little A is obtained by clicking on the icon 802. A's face image is associated with little A's watch.

As shown in FIG. 8d , after the electronic device successfully associates the small A's watch with the facial image, the facial image 7031 is displayed in the area 703 . The face image 7031 indicates the face image of the user wearing the small A watch. The electronic device can recognize the small A in the picture through the facial recognition technology.

It can be understood that when the small A's watch is connected to the electronic device again, the electronic device can automatically display the facial image 7031 associated with the small A's watch in FIG. After that, the interface as shown in Figure 8d is directly displayed. Optionally, before establishing a connection between the electronic device and the small A's watch again, when the electronic device searches for the small A's watch, the electronic device can automatically display the facial image associated with the small A's watch.

When the user clicks on the area 703 in Fig. 8d, the electronic device displays the interface 80 shown in Fig. 8c in response to the click operation. At this time, the icon 802 can be used to change the face image associated with the watch of Little A. The electronic device, in response to the user operation on the icon 802, combines the watch of Little A with the most recently received facial image information.

Optionally, in some embodiments, the icon 802 may also be used to add a face image associated with the watch of the little A. The electronic device, in response to the user operation on the icon 802, combines the small A's watch with the received facial image information. Without affecting the previously added associated facial image, the small A watch can be combined with multiple facial image information.

The above embodiments describe the configuration process of the electronic device for the annotation mode and the corresponding user interface. In another implementation manner of the present application, the labeling mode may also be configured through a wearable device. The wearable device is a wearable device that establishes a connection with an electronic device.

As shown in FIG. 9a, FIG. 9a exemplarily shows the main interface 90 of the wearable device, and the main interface 90 displays a plurality of application icons. The main interface 90 includes the application icon of the camera 901 . When the wearable device detects a user operation acting on the application icon of the camera 901, the wearable device displays an application interface provided by the camera application.

Referring to Fig. 9b, Fig. 9b shows an application interface 1000 provided by a camera application. The application interface 1000 of the camera 901 is shown in FIG. 9 b , and the application interface 1000 may include: a labeling icon 1001 , a gallery icon 1002 , a shooting icon 1003 , and a switching icon 1004 . Among them, the problem-solving principles of the gallery icon 1002, the shooting icon 1003, and the switching icon 1004 provided by the embodiment shown in FIG. 9b are similar to those of the embodiment shown in FIG. 5b. Therefore, in FIG. For the implementation, reference may be made to the corresponding descriptions of the implementation of the gallery icon 304 , the shooting icon 305 , and the switching icon 306 in FIG. 5 b , which will not be repeated here.

The display content of the display area of the application interface 1000 is the image captured by the camera currently used by the electronic device. The camera currently used by the electronic device may be the default camera set by the camera application, and the camera currently used by the electronic device may also be the camera used when the camera application was closed last time.

The annotation icon 1001 indicates that the user can start the annotation mode and configure the annotation mode. When the wearable device detects a user operation on the label icon 1001, a configuration interface as shown in FIG. 9c is displayed. Figure 9c is used to configure the labeling mode, and the configuration options include heart rate, blood pressure, blood sugar, exercise state, emotional state, and the like. Icon 1005 is used to identify whether the electronic device obtains heart rate information. As shown in Figure 9c, the circle in icon 1005 is on the left, indicating that the biometric information of the picture/video currently captured by the electronic device does not include heart rate information. The electronic device detects a click operation on the icon 1005, and the circle in the icon 1005 moves to the right, indicating that the biometric information of the picture/video currently captured by the electronic device includes heart rate information. Icon 1007 is used to identify whether the electronic device obtains blood pressure information. In FIG. 9c, the circle in icon 1007 is on the right, indicating that the biometric information of the picture/video currently captured by the electronic device includes blood pressure information.

When the user completes the configuration, the OK icon 1006 is clicked. The wearable device detects the user operation for determining the icon 1006, completes the configuration of the annotation mode, and enables the annotation mode. After the annotation mode is enabled, the user can trigger the shooting icon 1003 on the application interface 1000 to shoot a picture or video. The wearable device sends sensor data to the electronic device. In some embodiments, if the user triggers shooting on the wearable device side, the type of data sent is based on the configuration on the wearable device side; if the user triggers shooting on the electronic device side, the type of data sent is based on the configuration on the wearable device side. The configuration on the electronic device side shall prevail. Taking the configuration of FIG. 9c as an example, the wearable device sends the blood pressure data detected by the blood pressure sensor to the electronic device.

Optionally, in some embodiments, the wearable device detects sensor data through one or more sensors, and sends biometric information to the electronic device based on the configuration on the side of the wearable device. Taking the configuration of FIG. 9c as an example, the wearable device sends biometric information to the electronic device, and the biometric information corresponds to the blood pressure data, including blood pressure information (eg, normal blood pressure).

Optionally, in some embodiments, when the wearable device detects a click operation on the labeling icon 1001, the labeling mode is directly activated, and the wearable device sends sensor data or biometric information to the electronic device according to the configuration information; A long-press or double-click operation on the label icon 1001 is detected, and a configuration interface as shown in FIG. 9c is displayed.

The above-mentioned embodiment provides a possible setting interface of the labeling mode, and the electronic device displays the setting interface of the labeling mode (as shown in FIG. 8a ) in response to a user operation on the setting icon 302 (or the setting icon 402 ); The user operation of the annotation icon 1001 displays the setting interface of the annotation mode (as shown in FIG. 9c ); and so on.

After the configuration is completed, the electronic device takes a picture in the labeling mode. In the interface shown in FIG. 7b, the display content of the preview area 408 changes correspondingly with the different configurations. For example, if the user chooses to enable heart rate and exercise status in the configuration interface 80 shown in FIG. 8c, the display content in the preview area 408 may include heart rate information (such as heart rate data or normal heart rate) and exercise state information (such as running ).

When the electronic device detects the user operation that triggers the taking of the picture, the electronic device fuses and encodes the currently collected picture information with the biometric information to generate a picture with the biometric information. Corresponding sensor data such as heart rate, blood pressure, blood sugar, exercise state, emotional state, etc.

Unlike taking pictures, taking videos is a process over a period of time. When the electronic device detects a user operation that triggers video shooting, the present application provides an exemplary user interface. As shown in FIG. 10 , FIG. 10 shows a user interface 45 for capturing a video by an electronic device, wherein an icon 411 in the user interface 45 indicates that the current capturing mode is capturing a video. The display area of the user interface 45 may include a timing area 410 , a preview area 408 , and callout icons 412 . The timing area 410 counts the shooting length of the video in real time. The preview area 408 displays the user's health state, exercise state, emotional state and other information in real time. For example, it displays the health state of user A (the user of the wearable device connected to the electronic device), and the heart rate is normal; it also displays the exercise state of Xiao A, For example, A is running. The display content of the preview area 408 may be displayed according to the user's configuration on the electronic device side or the wearable device side, and the display content of the preview area 408 may be updated in real time along with the biometric information obtained by the electronic device.

The callout icon 412 is used to control the opening and closing of the callout mode, wherein the callout icon 412 may indicate the on state and the off state by displaying brightness or color.

For example, when the user is shooting a video, when the mark icon 412 is on, it means that the current shooting mode is the mark mode, the user interface 45 displays the preview area 408, and the electronic device compares the acquired biometric information with the captured image. The frame is fused and encoded; when the label icon 412 is in a dark state, it means that the current shooting mode is not the label mode (such as normal mode), the user interface 45 does not display the preview area 408, and the electronic device does not associate the acquired biometric information with the captured image frame. Fusion coding is performed, or the electronic device no longer captures biometric information. That is, the user can control the opening and closing of the annotation mode through user operations on the annotation icon 412 during the video recording process. When the annotation mode is turned on, the electronic device fuses and encodes continuously collected picture information and biometric information to generate image frames with biometric information, and a series of image frames are combined into a continuous video.

Optionally, in some embodiments, the callout icon 412 may be used to control the display and hiding of the preview area 408, wherein the callout icon 412 may indicate the display state and the hidden state in the form of display brightness or color. In this embodiment, regardless of whether the preview area 408 is displayed or hidden, the electronic device can acquire the biometric information, and based on the acquired biometric information, perform fusion coding with the captured image frame.

In the above labeling mode, the electronic device takes pictures/videos to generate pictures/videos with biometric information. The finished pictures/videos can be viewed in the gallery. When the electronic device detects a user operation on the gallery icon, the electronic device displays an application interface of the gallery.

As shown in FIG. 11a, FIG. 11a exemplarily shows a gallery interface 1100, and the gallery interface 1100 displays a plurality of pictures. The user clicks a picture 1101 taken in the annotation mode to enter the picture viewing interface. As shown in FIG. 11 b , FIG. 11 b shows a picture viewing interface 1200 , including a display area 1201 , a share icon 1202 , an edit icon 1203 , a delete icon 1204 , a more icon 1205 and an annotation area 1206 . in,

The display area 1201 displays the picture 1101 . The share icon 1202 can be used to trigger the start of a picture sharing function, and share the picture 1101 to other devices or application software. The editing icon 1203 can be used to trigger editing functions such as rotation, trimming, adding filters, and blurring to the picture 1101 . The delete icon 1204 can be used to trigger deletion of the picture 1101 . The more icon 1205 can be used to trigger opening of more functions related to the picture 1101 .

Optionally, in some embodiments, the display areas of the share icon 1202 , the edit icon 1203 , the delete icon 1204 and the more icons 1205 may be collectively referred to as a menu area. This menu area is optional. The menu area can be hidden in the picture viewing interface 1200. For example, the user can click the display area 1201 to hide the menu area, and click the display area 1201 again to display the menu area, which is not limited in this application.

Annotated area 1206 displays the biometric information of picture 1101 . The text content displayed in the labeling area 1206 includes, but is not limited to, descriptions related to heart rate, blood pressure, blood sugar, exercise state, emotional state, and the like. For example, the text content displayed in the labeling area 1206 may be "Little A is running" or "Little A's heart rate is normal, the exercise state is running, and the emotional state is happy". The text content displayed in the labeling area 1206 may also include the evaluation of the user's movement posture, such as posture specification, posture irregularity, etc.; may also include suggestions for the user, such as the user's mood index is low, and I hope to be happy every day; It can also push correspondingly according to the user's status, for example, the user's running posture is not standardized, the electronic device searches for pictures or videos of the correct running posture through the Internet, and recommends the user to watch related learning videos or picture materials, etc. This application does not limit this. It can be understood that the marked area 1206 can be displayed in the picture viewing interface 1200 in any position and in any form.

The user can view the detailed information of the picture 1101 through the more icon 1205. As shown in FIG. 11c, when the electronic device detects a user operation for more icons 1205, a user interface 1300 is displayed. The user interface 1300 in FIG. 11c exemplarily displays the detailed information of small A, including information such as normal blood pressure, normal blood sugar, normal heart rate, exercise state as running, emotional state as happy, and geographic state as a certain park, etc. Specific numerical information such as blood pressure data, blood sugar data, heart rate data, and mood index.

In some embodiments, FIG. 12 exemplarily shows yet another gallery interface 1400 . As shown in FIG. 12 , the gallery interface 1400 divides different picture sets for different types of pictures, and the division method can be divided according to the biometric information of the pictures. For example, Figure 12 includes a picture set with intelligent annotation, and the pictures/videos in this picture set all carry biometric information; including picture sets distinguished by characters, such as the picture set of small A and the picture set of small B, the picture set of small A The picture sets are all pictures/videos of small A, and the picture sets of small B are all pictures/videos of small B; including the picture sets distinguished by the motion state, such as the picture set of running and the picture set of playing badminton. Optionally, it may also include a set of pictures distinguished by emotional states, such as a set of happy pictures, a set of sad pictures, and the like. In this way, the user can use the biometrics as a distinction to select the set of pictures they want to view.

In some embodiments, FIG. 13a exemplarily shows yet another picture viewing interface 1201 . As shown in Figure 13a, the display area of the picture viewing interface 1201 displays the picture 1101. Compared with the picture viewing interface 1200 , the picture viewing interface 1201 does not include a marked area, but includes a cursor 1210 . The cursor 1210 prompts the user to select to view the biometric information of the picture. As shown in Fig. 13a and Fig. 13b, when the electronic device detects a user operation acting on the cursor 1210, a label area 1211 of the picture 1101 is displayed, and the text content in the label area 1211 may include at least part of biometric information, such as motion status , emotional state, health state, etc. In Figure 13a, "Little A is running" indicates the movement state of Little A.

The display position of the cursor 1210 is near the small A, indicating that the biometric information represented by the cursor 1210 is the information of the small A. The display position of the marked area 1211 is near the small A, indicating that the character described by the text content in the marked area 1211 is the small A. When two or more characters are included in the picture, this method can accurately display the specific characters described by the biometric information.

Optionally, the cursor 1210 can be hidden in the picture viewing interface 1201 . The user can click the display area of the small A in the picture 1101 to view the biometric information of the small A in the picture 1101, and click the display area of the small A again to hide the biometric information of the small A, which is not limited in this application.

As shown in FIG. 14a, the picture displayed in the display area 1201 in FIG. 14a includes two people, and the label area 1206 includes the biometric information of the two users (Little A is running, Little B is running), that is, That is, the electronic device can simultaneously receive sensor data or biometric information of the wearable devices of two users, and display the biometric information of the two users in one picture.

In some embodiments, biometric information may be displayed near the character. As shown in Figure 14b, the biometric information "Little A is running" is displayed near the person whose clothes are size 12, while the biometric information "Little B is running" is displayed near the person whose clothes are size 8. If the wearable device is associated with the facial image, the electronic device can identify the user in the image according to the facial image information through facial recognition technology, match the user corresponding to the wearable device with the person in the image, and display the biometric information in the image. the vicinity of the corresponding character.

The above embodiments describe the picture viewing interface in the electronic device. Next, the application interface for viewing videos with biometric information is introduced.

As shown in Figure 15a, Figure 15a exemplarily shows a video viewing interface 1500. One or more videos are included in the video viewing interface 1500, wherein the title of each video may be automatically named by the electronic device according to the biometric information of the video. For example, Xiao A running indicates that the user in the video is Xiao A, and the movement state of Xiao A is running. The electronic device detects a user operation for the video "Xiao A running", and plays the video of "Xiao A running".

As shown in Fig. 15b, Fig. 15b exemplarily shows a video playing interface 1600. The video playing interface 1600 includes a progress bar 1601 for indicating the progress of the video playing. The title of the video "Xiao A running" indicates the motion state of the user in the video, and the title of the video may also include information such as the user's health state and emotional state.

In some embodiments, the video playback interface 1600 displays at least a portion of the biometric information in real time during the video playback. Since a video is composed of multiple image frames. Each image frame has biometric information, and according to the progress of the playback, the electronic device can display at least part of the biometric information of each image frame in real time. As shown in Fig. 16a and Fig. 16b, compared with Fig. 15b, Fig. 16a and Fig. 16b further include a labeling area 1603 and a labeling area 1604, and the text content in the labeling area 1603 and the labeling area 1604 may include the user's health status, exercise Relevant descriptions such as state, emotional state, etc.; it can also include the evaluation of the user's movement posture, such as posture specification, posture irregularity, etc.; it can also include the user's suggestion, such as the user's mood index is low, I hope to be happy every day; You can also make corresponding pushes according to the user's status. For example, if the user's running posture is not standardized, it is recommended to watch the video linked below for learning; and so on.

It can be seen in Fig. 16a and Fig. 16b that, in the same video, the content in the label area 1603 and the label area 1604 can be changed as the video playback progress changes. That is, the video playback interface 1600 can display at least part of the biometric information in real time.

In some embodiments, at least part of the biometric information displayed in real time in the video playback interface is part of the biometric information. If the user wants to view the complete biometric information, they can view the detailed information by pausing. As shown in FIG. 17a, in FIG. 17a, the progress bar 1605 currently indicates a pause state (pause at the moment of 1 minute and 41 seconds), when the electronic device detects a user operation for the more icons 1205, the current image frame ( The complete biometric information of the image frame at the time of 1 minute and 41 seconds) is shown in Fig. 17b. The detailed information in Fig. 17b is the complete biometric information of the image frame of the video in Fig. 17a at 1 minute and 41 seconds.

In some embodiments, the biometric information may be displayed in the vicinity of the character in real time. As shown in Fig. 18a, the video played in the video playing interface 1600 in Fig. 18a is a video of "Little A and Little B running", including two characters and biometric information of two users. Among them, the biometric information "Xiao A pose specification" is displayed near the person whose clothes are size 12, while the biometric information "Xiao B pose specification" is displayed near the person whose clothing is size 8. If the wearable device is associated with the facial image, the electronic device can identify the user in the image according to the facial image information through facial recognition technology, match the user corresponding to the wearable device with the person in the image, and display the biometric information in the image. the vicinity of the corresponding character.

As can be seen in Figures 16a and 16b, in the same video, the displayed content in the area 1603 and the area 1604 can be changed as the video playback progress changes. That is, the video playback interface 1600 can display biometric information in real time in the case of multiple users.

The foregoing describes the display interface and method flow involved in this application. It can be understood that the aforementioned content related to the association of biometric information and pictures/messages all rely on the fusion coding of biometric information and picture/video information, Thereby, a corresponding new picture/video with biometric information is obtained. The specific technical implementation and principle for generating a picture/video file with biometric information by merging and encoding biometric information and picture/video information involved in the embodiments of the present application are described below.

(1) Image file format with biometric information.

The embodiment of the present application provides a picture file format with biometric information. The electronic device fuses and encodes the biometric information and the picture information to generate the picture file format. As shown in Fig. 19a, Fig. 19a exemplarily shows a picture file format with biometric information. The basic data structure of the picture file format includes two types: "segment" and compressed-encoded image data. "Segment" includes fields identifying the start of image (SOI), fields for image identification information (such as APP1, APP2, etc.), fields defining quantization tables (define quantization table, DQT), fields defining Huffman tables (define quantization table, DQT) huffman table, DHT) field, image frame start (start of frame, SOF) field, scan start (start of scan, SOS) field, identification image end (end of image, EDI) field, etc.

Wherein, the field of the image identification information may have one or more sections, such as APP1, APP2, etc., and each field of the image identification information defines the attribute information of the image. Among them, the attribute information of the image includes various information related to the shooting conditions at the time of shooting, such as aperture, shutter, date and time, such as camera brand model, color code, sound recorded when shooting, and global positioning system (GPS) and other information.

Specifically, the APP1 field includes segment identifier, segment character, segment length, and data of tag image file format (TIFF). The TIFF data includes the above attribute information. Among them, a TIFF file can contain multiple images, each image has its own image file directory (image file director, IFD) and a series of tags, and can use a variety of compression algorithms.

Taking IFD0 as an example, it includes multiple data fields (eg DE1, DE2, etc.), each data field has an identification tag, and different tags indicate different attribute information. For example, the field of DE1 may indicate the capture time of the image, the field of DE2 may indicate the camera model, the field of Tag=0x8825 may indicate the GPS information of the image, and the field of Tag=0x8769 may indicate the biometric information of the image (movement posture, heart rate, blood pressure, etc.).

Exemplarily, in the field of Tag=0x8769, the fields of 0x00-0x7F are defined to represent motion gestures, then the field can include up to 128 motion gestures, for example, 0x00 represents running, 0x01 represents walking, 0x02 represents swimming, etc.; 0x80 The field of –0x9F represents vital signs, and this field can include up to 32 vital signs, such as 0x80 for heart rate, 0x81 for blood pressure, 0x82 for blood sugar, etc.; 0xA0–0xAF field represents basic personal information, and the most It can include 16 kinds of personal basic information, such as 0xA0 for height, 0xA1 for age, 0xA2 for gender, and so on.

If the sensor data or biometric information obtained by the electronic device indicates that the user is running, write a 0x00 field in the field indicating the biometric information of the image; if the sensor data or biometric information obtained by the electronic device indicates the user's heart rate is 60 times/min, then in the field indicating the biometric information of the image, write 0x80 0x3C field (0x3C is 60 hexadecimal); and so on.

In some embodiments, the IFD in the image file format also includes fields such as identity information, scene information, etc., and the identity information includes the device name of the wearable device, the device account (such as a Huawei account), a custom user name, etc.; the scene The information includes that the electronic device comprehensively determines the scene in the picture by identifying the scene in the image and the geographic location information when the picture is taken, such as a park, a bar, a lakeside, a museum, etc.

It can be understood that the picture format shown in FIG. 19a is an exemplary picture format provided by the present application, wherein the position of the biometric information field in the picture format is not limited by the present application.

(2) Video frame format with biometric information.

The embodiment of the present application also provides a video frame format with biometric information. A complete video is composed of multiple video frames, and one video frame corresponds to one picture. The electronic device fuses and encodes the biometric information and the video frame information to generate the video frame format. As shown in FIG. 19b, FIG. 19b exemplarily shows a video frame format with biometric information. The video frame format includes supplemental enhancement information (SEI), sequence parameter set (sequence parameter sset, SPS), picture parameter set (picture parameter set, PPS) and compression-encoded video data sequence (VCL data). Among them, a set of global parameters of a coded video sequence (coded video sequence) are stored in the SPS. The coded video sequence is a sequence composed of a structure after the pixel data of one frame of the original video has been coded. The parameters on which the encoded data of each frame depends are stored in the PPS.

SEI belongs to the code stream category, which provides a method of adding extra information to the video code stream. During the generation and transmission of video content, SEI information can be inserted. This inserted information, along with other video content, travels through the transmission link to the electronic device. Among them, the SEI includes fields such as network abstraction layer unit (NAL) type, SEI type, and SEI length.

The SEI also includes fields indicating biometric information, and the biometric information includes exercise posture, heart rate, blood pressure, and the like. Exemplarily, the field defining 0x00-0x7F represents motion posture, and the field can include up to 128 motion postures, for example, 0x00 represents running, 0x01 represents walking, 0x02 represents swimming, etc.; the field of 0x80-0x9F represents vital signs, Then this field can include up to 32 vital signs, such as 0x80 for heart rate, 0x81 for blood pressure, 0x82 for blood sugar, etc.; the fields of 0xA0–0xAF indicate personal basic information, then this field can include up to 16 kinds of personal basic information, For example, 0xA0 is height, 0xA1 is age, 0xA2 is gender, and so on.

In some embodiments, the SEI also includes fields such as identity information and scene information, where the identity information includes the device name of the wearable device, the device account (such as a Huawei account), a user-defined user name, etc.; the scene information includes the identification image of the electronic device The scene in the picture, and the geographic location information when the picture was taken, comprehensively determine the scene in the picture, such as parks, bars, lakes, museums, etc.

It can be understood that the video frame format shown in FIG. 19b is an exemplary video frame format provided by the present application, wherein the position of the biometric information field in the video frame format is not limited by the present application.

It can be understood that the above-mentioned embodiments, method flows and related technical principles can be organically combined to obtain other new embodiments, which are not limited in this application.

Based on the above-mentioned technical principles and the feature labeling system shown in FIG. 3 , the following first introduces the method flow of the photographing method provided by the present application with reference to an example. Please refer to Fig. 20a, Fig. 20a shows a flowchart of a method for photographing a picture. The devices involved in the flow chart of the method include wearable devices and photographing devices. The photographing device includes an electronic device 100 having a photographing function, and the wearable device exemplarily includes a wearable device 201 and a wearable device 202, and may also include more devices. The method includes:

Step S101: Establish a connection between the wearable device and the photographing device.

The connection between the wearable device and the shooting device is not limited to Bluetooth (blue tooth, BT), near field communication (NFC), wireless fidelity (wireless fidelity, WiFi), WiFi direct connection, network and other wireless communication Way. In the embodiments of the present application, pairing using Bluetooth will be described as an example.

Optionally, in the process of establishing the connection, the wearable device and the photographing device obtain each other's connection information (such as hardware information, interface information, identity information, etc.) through Bluetooth. The shooting device can obtain the sensor data of the wearable device after the shooting function is turned on, and the wearable device can synchronize some functions of the shooting device. The device can actively trigger the shooting device to start the shooting function, and the wearable device can view the picture/video files in the shooting device, etc.

Optionally, in some embodiments, when the photographing device detects a user operation for enabling the labeling mode, the photographing device automatically turns on Bluetooth, and automatically establishes a Bluetooth connection with the wearable device.

Step S102: The photographing device detects a user operation that triggers the function of photographing a picture.

The photographing device detects the user operation that triggers the function of taking pictures, triggers the function of taking pictures, and obtains the picture information currently collected by the camera. The user operation may be a touch operation, a voice operation, a hovering gesture operation, etc., which is not limited herein. In the present application, the photographing device detects a user operation that triggers the function of photographing a picture in the annotation mode, and the specific content may refer to the aforementioned UI embodiment. For example, the user can trigger the photographing device to take a picture through a user operation on the icon 305 in FIG. 5c or the icon 404 in FIG. 6b, and the photographing device detects the user operation and triggers the function of taking a picture.

Optionally, in some embodiments, the user operation may be a user operation for a wearable device, the wearable device detects a user operation that triggers a function of taking pictures, and sends it to the photographing device through Bluetooth, and the photographing device detects the user operation, Trigger the capture picture function. Referring to Fig. 9a, the user starts the camera application by clicking the application icon 901 in Fig. 9a, and triggers the function of taking pictures by clicking the icon 1003 in Fig. 9b, and the wearable device sends a picture taking instruction to the photographing device through Bluetooth, so that the photographing device triggers the photographing, The specific content will not be repeated here. In this application, the user operation that triggers the function of taking pictures may also be referred to as the first operation.

Step S103: The photographing device sends a request message, where the request message is used to request acquisition of sensor data or biometric information.

After the photographing device detects a user operation that triggers the function of taking pictures, it sends a request message to the wearable device, where the request message is used to request acquisition of sensor data or biometric information.

Optionally, the request message includes the requested data type, data collection method, and data collection interval. The requested data type may be: health state, exercise state, emotional state, and the like. Among them, the health status includes heart rate, blood pressure, blood sugar, EEG, ECG, EMG, body temperature, etc.; exercise status includes walking, running, cycling, swimming, playing badminton, skating, surfing, dancing and other common sports postures. It can also include some more fine-grained motion gestures, such as: forehand, backhand, Latin dance, mechanical dance, etc.; emotional states include tension, anxiety, sadness, stress, excitement, joy, etc.

Optionally, the data collection methods can be divided into single collection and continuous collection. For taking pictures, the data collection method can generally be a single collection.

The data collection interval is related to the data collection method. When the data collection method is single collection, the data collection interval is invalid. That is, the shooting device only needs to obtain the data sent by the wearable device once. When the data collection method is continuous collection, the data collection interval is the preset interval. That is, the photographing device obtains the data sent by the wearable device at preset intervals.

The type of data requested to be acquired by the photographing device can be configured by the user on the side of the photographing device. The specific configuration method can refer to the aforementioned UI embodiment. For example, as shown in FIG. 8c, the configuration interface 80 includes a plurality of configuration options. The options all correspond to a data type, and the photographing device sends a request message to the wearable device according to the data type selected by the user. For example, motion posture, heart rate, blood pressure, blood sugar, and emotional state are each represented by one byte. If the shooting device wants to obtain the motion posture, set the byte representing the motion posture to 1, otherwise set it to 0; the same is true for other data types .

Optionally, in some embodiments, the user may trigger a picture-taking function on the wearable device, and the wearable device detects a user operation that triggers the picture-taking function, and sends a picture-taking instruction to the photographing device. The photographing device receives the picture-taking instruction, obtains picture information currently collected by the camera, and sends a request message to the wearable device. In this application, the wearable device detects a user operation that triggers the function of taking pictures, and the user operation may be referred to as a second operation.

Step S104: The wearable device sends sensor data or biometric information to the photographing device.

The wearable device parses the received request message, and obtains the data type, data collection method and data collection interval required by the shooting device. According to the request message, the wearable device sends sensor data or biometric information to the photographing device. Exemplarily, for taking pictures, the data collection method is single collection, the data collection interval is set to an invalid value, and the wearable device sends data to the shooting device once. If the data type in the request message includes motion posture and heart rate, the wearable device sends the sensor data or biometric information obtained by the motion sensor and the heart rate sensor to the photographing device.

Optionally, the sensor data is raw data, that is, data directly detected by the sensor; for example, the sensor data may include the user's heart rate data obtained by the wearable device through the heart rate sensor; the user's movement amplitude, angle, and speed obtained through the motion sensor. and other data; obtain the user's skin resistance, conductance and other data through the skin sensor; obtain the user's blood sugar, blood pressure, body temperature and other data through the biosensor.

Optionally, the biometric information is processed data based on sensor data, that is, further data obtained by calculation/calculation based on the original data. For example, the biometric information can be calculated by the wearable device based on the heart rate data that the user's current heart rate is within the normal range; the user's current exercise posture, such as walking, running, cycling, swimming, playing badminton, can be calculated based on the data obtained by the motion sensor. and so on; calculate the user's current mood index, stress index, etc. according to the data obtained by the galvanic sensor; calculate the user's current health state according to the data obtained by the biosensor.

Exemplarily, the fields of 0x00-0x7F represent motion gestures, and the field can include up to 128 motion gestures, for example, 0x00 represents running, 0x01 represents walking, 0x02 represents swimming, etc.; the fields 0x80-0x9F represent vital signs, this field It can include up to 32 kinds of vital signs, such as 0x80 for heart rate, 0x81 for blood pressure, 0x82 for blood sugar, etc.; the fields of 0xA0–0xAF indicate personal basic information, and this field can include up to 16 kinds of personal basic information, such as 0xA0 for height , 0xA1 for age, 0xA2 for gender, and so on. If the shooting device wants to obtain the motion posture, and the wearable device detects that the data obtained by the motion sensor indicates that the user is running, the wearable device writes 0x00 to the data packet to be sent, indicating that the user is running. If the shooting device wants to obtain the heart rate, and the wearable device detects that the data obtained by the heart rate sensor is 60 beats/min at this time, the wearable device writes the data bytes indicating "heart rate is 60 beats/min" into the data packet to be sent. For example, 0x80 0x3C (0x3C is 60 in hexadecimal); Optionally, if the heart rate is 60 beats/min within the normal range, the wearable device can write data bytes indicating "normal heart rate" into the data packet to be sent.

Optionally, in some embodiments, the wearable device parses the received request message, and obtains the data type, data collection method, and data collection interval required by the photographing device. According to the request message, the wearable device acquires sensor data before a preset time period, and sends the sensor data or biometric information obtained based on the sensor data to the photographing device. The preset time period may be 1 second or 0.5 seconds, which is not limited here. Since the time when the wearable device obtains the sensor data always lags behind the time when the electronic device detects the user operation that triggers the function of taking pictures, the delayed data value is obtained through experiments and set to a preset time period, then the wearable device can provide accurate information to the shooting device. sensor data or biometric information, reducing errors.

Optionally, in some embodiments, the wearable device parses the received request message, and according to the request message, the wearable device sends sensor data or biometric information to the photographing device according to the timestamp. The time stamp is the moment when the electronic device detects a user operation that triggers the function of taking pictures, and the wearable device acquires sensor data at this moment, and sends the sensor data or biometric information to the photographing device. In this way, the wearable device can provide more accurate information to the photographing device.

Step S105: Fusion coding the captured picture information and the biometric information to generate a picture with the biometric information.

After receiving the sensor data or biometric information, the photographing device fuses and encodes the captured picture information and biometric information to generate a picture with biometric information. The format of the picture can refer to the picture format shown in Figure 19a above, The biometric information corresponds to sensor data. The display content of the biometric information may be a simple sentence, or may be detailed and complete information. The display content, display position and display form of the biometric information in the picture are not limited in this application.

Optionally, the electronic device receives sensor data, determines biometric information based on the sensor data, and fuses the captured picture information with the biometric information; For the biometric information, the captured picture information and the biometric information are fused and encoded.

For example, if the sensor data obtained by the photographing device is the user's heart rate data, the biometric information can be whether the user's heart rate is within the normal range; the sensor data obtained by the photographing device is the user's current exercise posture, such as walking, running, riding car, swimming, playing badminton, etc., the biometric information can be whether the user's movement posture is standard; the sensor data obtained by the shooting device is the user's current mood index, stress index and other information, then the biometric information can be whether the user is in a happy mood ;etc.

The generated picture can be referred to as shown in Figure 11b above. When the user views a picture with biometric information, the biometric information can be displayed in a fixed position in the picture, such as the upper part of the picture. The biometric information in the picture can be hidden at the beginning of viewing, and then displayed by triggering the control. For example, in FIG. 13a, the biometric information is displayed by triggering the cursor 1210. As another example in Figure 11b, biometric information is viewed in the user interface 1300 in Figure 11c by triggering the icon 1205.

Optionally, in some embodiments, the electronic device can also obtain the identity information of the wearable device, such as the device name, device account (such as Huawei account), user-defined user name and other information; Fusion coding generates image files with identity information. The electronic device can also obtain the scene information when the picture is taken. The scene information includes the scene in the picture comprehensively determined by the electronic device by identifying the scene in the image and the geographic location information when taking the picture, such as parks, bars, lakes, etc. Museum etc. The scene information and the picture information are fused and encoded to generate a picture file with the scene information.

Optionally, in some embodiments, the biometric information is obtained by combining sensor data and picture information. According to the sensor data, the user's health status, movement status, emotional status and other information can be obtained. According to the picture information, the scene information, user movement posture and other information can be obtained. For example, the sensor data received by the photographing device includes the user's heart rate data of 60 beats/min, and the movement posture of running; the photographing device performs image analysis on the captured picture information, and concludes that the user's movement posture in the picture is running. , and the filming location is a park. Combining sensor data and picture information, the final biometric information obtained by the shooting device includes the user running in the park with a heart rate of 60 beats/min.

Optionally, in some embodiments, the information of one or more facial images is stored in the photographing device, the photographing device determines a preset facial image corresponding to the wearable device through the identity information of the wearable device, and associates the preset facial image with the picture. One or more of the characters are matched for similarity, and if the preset facial image is successfully matched with one of the characters, the photographing device displays at least part of the biometric information near the character.

The preset facial image is associated with the wearable device, wherein the preset facial image may be preset by the user in the shooting device, or uploaded to the shooting device in the form of an image or video, or may be preset by the user in the shooting device. In the wearable device, the wearable device is then provided to the photographing device, which is not limited in this application.

For example, the wearable device is Huawei Watch 1, and the user name bound to the Huawei Watch 1 is Xiao A, and the photographing device determines the facial information of Xiao A. Among them, the photographing device can determine the facial information of Xiao A by looking up the binding relationship between the user and the face in the address book; it can also determine the facial information of Xiao A through the binding relationship between the wearable device and the facial information (for example, in FIG. 8c by association The icon 802 of the face image uploads the face information of the little A, and the face information of the little A is the face image 7031); and so on. The photographing device performs image recognition on the picture, identifies one or more faces in the picture, and performs similarity matching between the face information of Little A and the one or more faces in the picture. If the similarity between the small A face information and one of the faces in the picture is greater than the threshold, the biometric information is displayed near the face. Referring to FIG. 13b, the biometric information of the picture in FIG. 13b is displayed near the person, indicating the person described by the biometric information.

Optionally, if the preset facial image is unsuccessfully matched with the person in the picture, that is, the photographing device detects that the preset facial image is not in the picture collected by the current photographing device, the photographing device may output a prompt message (first prompt), For example "there is no user A in the picture".

Optionally, in some embodiments, compared with the method flow shown in FIG. 20a above, the present application further shows a method flow for photographing a picture, as shown in FIG. 20b.

Step S101a: Establish a connection between the wearable device and the photographing device. For a specific description, reference may be made to the description of step S101, which will not be repeated here.

Step S102a: The photographing device sends a request message, where the request message is used to request acquisition of sensor data or biometric information.

After the wearable device and the photographing device are connected, the photographing device sends a request message to the wearable device, where the request message is used to request acquisition of sensor data or biometric information.

Optionally, when the photographing device detects a user operation for enabling the annotation mode, the photographing device sends a request message to the wearable device.

Optionally, in the labeling mode, the photographing device detects a user operation that triggers acquisition of sensor data or biometric information, and the photographing device sends a request message to the wearable device. Referring to Fig. 7a, in Fig. 7a, the photographing device detects a user operation for the icon 407, and the photographing device sends a request message to the wearable device.

For a specific description of the request message, reference may be made to the description of step S103, which will not be repeated here.

Step S103a: The wearable device sends sensor data or biometric information to the photographing device.

The specific description of this step can refer to the description of step S104. additional,

In some embodiments, after acquiring the sensor data or biometric information sent by the wearable device, the photographing device displays at least part of the biometric information on the photographing interface. Referring to FIG. 7b, the image captured by the camera in real time is displayed in the display area 40, and the preview area 408 is displayed in the preview area 408. The display content in the preview area 408 is at least part of the biometric information for the user to view in real time. The biometric information corresponds to sensor data. In this embodiment of the present application, the shooting interface may be referred to as a shooting preview interface, and the shooting preview interface includes a preview image collected by a camera. The photographing device displays at least part of the biometric information on the preview image, and the at least part of the biometric information displayed on the preview image may be referred to as second information, and the second information corresponds to the second sensor data.

In some embodiments, the information of one or more facial images and the corresponding relationship between each facial image and the wearable device are stored in the photographing device. The photographing device determines a preset facial image corresponding to the wearable device from one or more facial images through the identity information of the wearable device, and matches the preset facial image with one or more characters in the picture. If the image is successfully matched with one of the characters in the picture, the photographing device displays at least part of the biometric information near the character. For example, the preview area 408 in FIG. 7b may be displayed near the matched characters.

Optionally, if the target facial information is unsuccessfully matched with the person in the picture, that is, the photographing device detects that the preset facial image is not in the picture collected by the current photographing device, the photographing device can output a prompt message (the first prompt), such as "There is no user A in the picture, please point the camera at user A".

Step S104a: The photographing device detects a user operation that triggers the function of photographing a picture. For a specific description, reference may be made to the description of step S102, which will not be repeated here.

Step S105a: Fusion coding the captured picture information and biometric information to generate a picture with biometric information corresponding to sensor data. For a specific description, reference may be made to the description of step S105, which will not be repeated here.

Compared with the embodiment described in FIG. 20a, the photographing device obtains sensor data or biometric information from the wearable device after detecting a user operation that triggers the function of taking pictures; while in the method shown in FIG. 20b, the photographing device obtains sensor data or biometric information from the wearable device. The sensor data or biometric information can be obtained from the wearable device after the annotation mode is activated, and at least part of the biometric information can be displayed in real time on the preview interface of the shooting, so as to achieve the preview effect of the biometric information and improve the user experience. The shooting device can also obtain sensor data or biometric information from the wearable device after starting the annotation mode and after receiving the user's operation, and display at least part of the biometric information in real time on the preview interface of the shooting, and the user can freely control the display and display of the biometric information. Hidden to improve user experience.

Optionally, in some embodiments, the user may trigger a picture taking function on the wearable device, and the wearable device detects the user operation triggering the photographing function, and sends a picture taking instruction and sensor data (or biometric information) to the photographing device. After receiving the image capture instruction and sensor data (or biometric information), the photographing device obtains the image information collected by the current camera, fuses and encodes the captured image information and biometric information, and generates a picture with biometric information. Biometric information corresponds to sensor data. For a specific description of the sensor data and biometric information, reference may be made to the above step S104.

For example, referring to Fig. 9c, the user selects the desired data type in the configuration interface as shown in Fig. 9c, and after the configuration is completed, the wearable device starts the labeling mode. In response to the user's operation on the icon 1003 in the interface 1000, the wearable device sends a picture-taking instruction to the photographing device, and provides sensor data or biometric information to the photographing device according to the data type selected by the user in FIG. 9c. The photographing device receives an instruction to take a picture, obtains the picture information collected by the current camera, and fuses and encodes the photographed picture information with or biometric information to generate a picture with biometric information.

Optionally, the present application further provides a method flow for shooting video. Please refer to FIG. 21 , which shows a flowchart of a method for shooting video.

Step S201: Establish a connection between the wearable device and the photographing device. For a specific description, reference may be made to the description of step S101, which will not be repeated here.

Step S202: The photographing device detects a user operation that triggers the function of photographing a video.

The shooting device detects a user operation that triggers the shooting video function, triggers the shooting video function, and obtains the video frame information currently collected by the camera. The user operation may be a touch operation, a voice operation, a hovering gesture operation, etc., which is not limited herein. In the present application, the photographing device detects a user operation that triggers the function of photographing a video in the annotation mode, and the specific content may refer to the aforementioned UI embodiment.

Optionally, in some embodiments, the user operation may be a user operation for a wearable device, the wearable device detects a user operation that triggers a function of shooting a video, sends it to the shooting device through Bluetooth, and the shooting device detects the user operation, Trigger the capture video function.

Step S203: The photographing device sends a request message, where the request message is used to request acquisition of sensor data or biometric information.

After the photographing device detects a user operation that triggers the function of photographing a video, it sends a request message to the wearable device, where the request message is used to request acquisition of sensor data or biometric information.

The request message includes the data type supported by the photographing device, the data collection method, and the data collection interval. For the specific description of this part, please refer to the description of step S103 above. The difference from step S103 is that for shooting video, the data collection method is generally continuous collection, and the data collection interval can be set to 1 second. That is, the shooting device acquires sensor data or biometric information every 1 second during the process of shooting a video.

Optionally, in some embodiments, the user may trigger a video capture function on a wearable device, and the wearable device detects a user operation that triggers the video capture function, and sends a video capture instruction to the capture device. The shooting device receives the shooting video instruction, and sends a request message to the wearable device.

In some embodiments, the user triggers the video capture function on the wearable device, the wearable device detects the user operation that triggers the video capture function, wants to send the video capture command to the capture device, and sends sensor data or biometric information to the capture device. In this case, step S203 does not need to be performed, and the wearable device sends sensor data or biometric information to the photographing device based on the configuration on the side of the wearable device.

Step S204: The wearable device periodically sends sensor data or biometric information.

The wearable device parses the received request message, and obtains the data type, data collection method and data collection interval required by the shooting device. According to the request message, the wearable device periodically sends sensor data or biometric information to the photographing device. Exemplarily, for video shooting, the data collection method is continuous collection, and the data collection interval is set to 1 second. Then the wearable device sends sensor data or biometric information to the photographing device every 1 second. For a specific description of the sensor data and biometric information, reference may be made to the above step S104.

Step S205 : each time the photographing device receives sensor data or biometric information, it fuses and encodes the captured video information and or biometric information to generate an image frame with biometric information corresponding to the sensor data.

Each time the photographing device receives sensor data or biometric information, it fuses and encodes the captured video information with or biometric information. Since a video is composed of multiple image frames, each image frame generated by the photographing device has corresponding biometric information, and the biometric information of the image frame corresponds to the sensor data. Exemplarily, the photographing device receives sensor data every 1 second, and there are 24 image frames generated by the photographing device in one second, then the biometric information of the 24 image frames in the one second is based on the next one. seconds of sensor data. For example, the photographing device acquires sensor data once in the 5th second, and the photographing device fuses and encodes 24 frames of images from the 4th to 5th second in the video information with the sensor data to generate image frames with biometric information; The shooting device obtains sensor data once in the 6th second, and the shooting device fuses and encodes 24 frames of images in the video information between the 5th second and the 6th second with the sensor data to generate image frames with biometric information; and so on. .

Optionally, in some embodiments, the photographing device fuses and encodes the biometric information and the video information according to the timestamp. For example, the wearable device periodically sends sensor data or biometric information, and the sensor data or biometric information has a time stamp, indicating the time when the sensor data or biometric information corresponds to the video information. In this way, the wearable device can provide more accurate information to the photographing device, avoiding the mismatch between the biometric information and the video content due to information transmission delay.

Optionally, in some embodiments, the biometric information is obtained by combining sensor data and video information. According to the sensor data, the user's health status, motion status, emotional status and other information can be obtained. According to the video information, the scene information, user movement posture and other information can be obtained. For example, the sensor data received by the shooting device includes the user's heart rate data of 60 beats/min, and the exercise posture of running; the shooting device performs image analysis on the captured video information, and concludes that the user's exercise posture in the video is running. , and the filming location is a park. Combining the sensor data and video information, the shooting device obtains the final biometric information of the image frames in the video, including the user running in the park and the heart rate of 60 beats/min.

Step S206: The photographing device detects a user operation that triggers stopping of photographing the video.

The shooting device detects a user operation that triggers the stop of shooting the video, and stops shooting the video. The user operation may be a touch operation, a voice operation, a hovering gesture operation, etc., which is not limited herein. In this application, the shooting device detects a user operation that triggers the function of stopping video shooting in the annotation mode, and the specific content may refer to the aforementioned UI embodiment.

Optionally, in some embodiments, the user may trigger the wearable device to stop shooting video, and the wearable device detects a user operation that triggers the stop of shooting video, and sends an instruction to stop shooting video to the shooting device. The shooting device receives the instruction to stop shooting video, and stops shooting video.

Step S207: The photographing device sends a request message to the wearable device for stopping acquiring sensor data or biometric information.

After the photographing device detects a user operation that triggers the function of stopping video recording, it sends a request message to the wearable device, where the request message is used to stop acquiring sensor data or biometric information. The wearable device receives the request message and stops sending sensor data or biometric information to the photographing device.

Optionally, in some embodiments, the user may trigger the stop of shooting video on the wearable device, the wearable device detects the user operation that triggers the stop of shooting video, sends the instruction to stop shooting video to the shooting device, and stops sending sensors to the shooting device. data or biometric information.

Step S208: The photographing device generates and saves a video.

After the shooting device detects a user operation that triggers the function of stopping the shooting of the video, it generates and saves the shot video. For the format of the video, reference may be made to the video format shown in FIG. 19b. The video contains biometric information, and the biometric information corresponds to the sensor data.

The generated video can refer to the above-mentioned Figure 16a or Figure 16b. When the user views the video, the video also includes biometric information, and the display content of the biometric information can be a simple sentence, or it can be detailed and complete information. The display content, display position and display form of the biometric information are not limited in this application.

Optionally, in some embodiments, the electronic device can also obtain the identity information of the wearable device, such as the device name, device account (such as a Huawei account), custom user name and other information; Perform fusion encoding to generate video files with identity information. The electronic device can also obtain scene information when the video is shot. The scene information includes the scene in the video frame comprehensively determined by the electronic device by identifying the scene in the video frame and the geographic location information when shooting the video, such as parks, bars, lakes, etc. borders, museums, etc. The scene information and video frame information are fused and encoded to generate a video file with scene information.

Optionally, in some embodiments, the information of one or more facial images is stored in the photographing device, the photographing device determines the preset facial image corresponding to the wearable device through the identity information of the wearable device, and associates the preset facial image with the video. The similarity matching is performed on one or more characters in the image frames of the image frame, and if the preset facial image is successfully matched with one of the characters, the photographing device displays at least part of the biometric information near the character. Referring to Fig. 18a or Fig. 18b above, the biometric information of the video in Fig. 18a or Fig. 18b is displayed above the character, indicating the character described by the biometric information.

Optionally, the preset facial image is associated with the wearable device, wherein the preset facial image can be preset by the user in the shooting device, or uploaded to the shooting device in the form of an image or video, or the user can It is preset in the wearable device, and the wearable device is then provided to the photographing device, which is not limited in this application.

It should be noted that the photographing device can obtain sensor data or biometric information from the wearable device after starting the annotation mode, and display at least part of the biometric information in real time on the preview interface of the shooting, so as to achieve the preview effect of the biometric information and improve the user experience. . The shooting device can also obtain sensor data or biometric information from the wearable device after starting the annotation mode and after receiving the user's operation, and display at least part of the biometric information in real time on the preview interface of the shooting, and the user can freely control the display and display of the biometric information. Hidden to improve user experience.

Optionally, in some embodiments, during the process of shooting a video, the biometric information may be displayed on the shooting interface in real time for the user to view in real time.

The method for shooting pictures and videos described above is applicable to the system shown in FIG. 1 . Optionally, the present application also provides a shooting system, which can connect one shooting device with multiple wearable devices of the same type, and the shooting device obtains sensor data or biometric information of multiple wearable devices of the same type. Feature recognition for multiple users in captured pictures/videos. As shown in FIG. 22 , the photographing system shown in FIG. 22 includes a photographing device 101 , a plurality of wearable devices 201 of the same type, and a third device 301 . The photographing device 101 and multiple wearable devices 201 of the same type establish connections through the third device 301 . in,

The photographing device 101 is an electronic device with a camera function, such as a mobile phone, a tablet, a camera, and the like. The wearable device 201 includes wireless earphones, smart watches, smart bracelets, smart glasses, electronic clothing, electronic bracelets, electronic necklaces, electronic accessories, electronic tattoos, smart mirrors, and the like.

The third device 301 may be a relay device. For example, a Bluetooth repeater hub, the Bluetooth repeater is connected to the photographing device 101 and multiple wearable devices 201 of the same type through Bluetooth; the third device 301 can also be a cloud server, and the third device 301 communicates with the photographing device 101 through a mobile communication module. and multiple wearable devices 201 are connected. Through the third device 301 , the photographing device 101 can establish connections with multiple wearable devices 201 of the same type. Optionally, in a possible implementation manner, the wearable device and the photographing device may also establish a connection with the third device 301 through other wireless communication methods such as WiFi. At this time, the third device 301 may also be capable of data processing and computing. functional router.

Based on the foregoing technical principles and the photographing system shown in FIG. 22 , the following describes the method flow for photographing a picture provided by the present application with reference to an example. Please refer to FIG. 23 , which shows a flowchart of a method for photographing a picture. The devices involved in the flow chart of the method include n wearable devices, a photographing device and a third device, where n is a positive integer. The method includes:

Step S301 : the n wearable devices establish a connection with the third device, and the photographing device establishes a connection with the third device.

The connection method is not limited to Bluetooth (blue tooth, BT), near field communication (near field communication, NFC), wireless fidelity (wireless fidelity, WiFi), WiFi direct connection, network and other wireless communication methods. In the embodiments of the present application, pairing using Bluetooth will be described as an example.

In the process of establishing the connection, the n wearable devices and the photographing device obtain each other's connection information (such as hardware information, interface information, identity information, etc.) through the third device. The camera device can obtain sensor data or biometric information of the wearable device, and the wearable device can synchronize some functions of the camera device. Actively trigger the shooting device to turn on the shooting function, and the wearable device can view the pictures/video files in the shooting device, etc.

Step S302: The photographing device detects a user operation that triggers the function of photographing a picture. For a specific description, reference may be made to the description of step S102, which will not be repeated here.

Step S303: The photographing device sends a request message to the third device, where the request message is used to request acquisition of sensor data or biometric information.

For a specific description of this step, reference may be made to the description of step S103 . The difference from step S103 is that the photographing device sends a request message to the third device.

Step S304: The third device forwards the request message to n wearable devices.

After receiving the request message sent by the photographing device, the third device forwards the request message to the n wearable devices.

Step S305: The wearable device sends sensor data or biometric information, where the sensor data or biometric information includes the identity information of the wearable device.

The specific description of this step can refer to the description of step S104. Different from step S104, the wearable device sends sensor data or biometric information to the third device, and the sensor data or biometric information also includes the identity information of the wearable device. Information can uniquely represent a wearable device.

Step S306: After the third device determines that the sensor data or biometric information of all connected wearable devices are received completely, it sends the data to the photographing device.

The third device receives the sensor data or biometric information sent by the wearable device, and after receiving the sensor data or biometric information of n wearable devices, the third device sends the sensor data or biometric information of the n wearable devices to the shooting device . The sensor data or biometric information of the n wearable devices includes the identity information of the wearable devices, and the third device determines whether to receive the sensor data or biometric information of all connected wearable devices according to the acquired identity information.

Step S307: The photographing device fuses and encodes the photographed picture information and the biometric information to generate a picture with the biometric information, and the biometric information corresponds to the sensor data.

The specific description of this step can refer to the description of step S105. The difference from step S105 is that since the sensor data or biometric information received by the photographing device is for n wearable devices, the biometric information also includes the corresponding information of n users. Biometric Information.

The generated pictures can be referred to as shown in the above-mentioned Fig. 14a. Fig. 14a shows a picture viewing interface, and the viewed picture includes two characters. When the user views the picture, the picture also includes biometric information, and the biometric information includes biometric information of two characters, namely, little A is running and little B is running. The displayed content of the biometric information can be a simple sentence or a detailed and complete information. The display content, display position and display form of the biometric information are not limited in this application.

Optionally, in some embodiments, the information of one or more facial images is stored in the photographing device, the photographing device determines a preset facial image corresponding to the wearable device through the identity information of the wearable device, and associates the preset facial image with the picture. One or more of the characters are matched for similarity, and if the preset facial image is successfully matched with one of the characters, the photographing device displays at least part of the biometric information near the character. The photographing device obtains the identity information of n wearable devices, and can perform the above matching process n times at most. Referring to Figure 14b, Figure 14b includes two biometric information and two characters, and each biometric information is displayed near a different character (small A is displayed above the number 12 clothes when running, and small B is displayed at 8 when running. (above the person in the number clothes), the person described by the biometric information is indicated according to the display position of the biometric information.

In the embodiment of the present application, the photographing device and n wearable devices are connected through a third device, and the photographing device detects a user operation that triggers the function of taking pictures in the labeling mode, and at the same time obtains the picture information, the third device sends the information to the wearable device through the third device. Send a request message. The photographing device receives sensor data or biometric information of n wearable devices, and fuses and encodes the n or biometric information with the picture information to generate a picture with biometric information, and the biometric information corresponds to the n sensor data. It can be seen that the method provided by this application is to complete the labeling of the picture during the process of generating the picture, without the need for subsequent feature extraction of the saved picture, which saves hardware resources; and can be combined with the sensor data of the wearable device. Images are marked with more accurate and rich features.

Next, the present application also provides a method flow for shooting video. Please refer to FIG. 24, which shows a flowchart of a method for shooting video.

Step S401 : the n wearable devices establish connections with the third device, and the photographing device establishes the connection with the third device. For a specific description, reference may be made to the description of step S301, which will not be repeated here.

Step S402: The photographing device detects a user operation that triggers the function of photographing a video. For a specific description, reference may be made to the description of step S202, which will not be repeated here.

Step S403: The photographing device sends a request message to the third device, where the request message is used to request acquisition of sensor data or biometric information.

For a specific description of this step, reference may be made to the description of step S203. Different from step S203, the photographing device sends a request message to the third device.

Step S404: The third device forwards the request message to the n wearable devices respectively.

Step S405: The wearable device periodically sends sensor data or biometric information, and the sensor data or biometric information includes the identity information of the wearable device.

The specific description of this step can refer to the description of step S204. The difference from step S204 is that n wearable devices send sensor data or biometric information to the third device, and the sensor data or biometric information also includes the identity information of the wearable device. The identity information can uniquely represent a wearable device.

Step S406: The third device determines that the sensor data or biometric information of all connected wearable devices in a cycle is completely received, and then sends the data to the photographing device.

Since n wearable devices periodically send sensor data or biometric information to the third device, the third device will receive n sensor data or biometric information in one period. The sensor data or biometric information of the n wearable devices includes the identity information of the wearable device, and the third device determines whether to receive the sensor data or biometric information of all connected wearable devices according to the acquired identity information. If the sensor data or biometric information of n wearable devices is received, the sensor data or biometric information of the n wearable devices is sent to the photographing device.

Step S407: Each time the photographing device receives sensor data or biometric information, it performs fusion coding on the captured video information and biometric information to generate an image frame with biometric information corresponding to the sensor data.

For a specific description, reference may be made to the description of step S205, which will not be repeated here.

Step S408: The photographing device detects a user operation that triggers stopping of photographing the video. For a specific description, reference may be made to the description of step S206, which is not repeated here.

Step S409: The photographing device sends a request message to the third device for stopping acquiring sensor data or biometric information. For a specific description, reference may be made to the description of step S207. Different from step S207, the photographing device sends a request message to the third device.

Step S410: The third device forwards the request message to the n wearable devices respectively.

After receiving the request message sent by the photographing device, the third device forwards the request message to the n wearable devices. The request message is used to stop acquiring sensor data or biometric information, and the wearable device receives the request message and stops sending sensor data or biometric information.

Step S411: The photographing device generates and saves a video.

The specific description of this step can refer to the description of step S208. Different from step S208, the generated video can refer to the above-mentioned Figures 18a and 18b. When the user views the video, the biometric information is displayed along with the playback progress of the video. , the display content of the biometric information can be a simple sentence or a detailed and complete information. The display content, display position and display form of the biometric information are not limited in this application.

In the embodiment of the present application, the photographing device and n wearable devices are connected through a third device, and the photographing device detects a user operation that triggers the function of shooting video in the smart labeling mode, and at the same time acquires video information, sends the video information to the wearable device through the third device. The device sends a request message. The shooting device receives sensor data or biometric information of n wearable devices through a third device, fuses and encodes the n biometric information and image frames, and generates image frames with biometric information along with the process of video shooting. Biometric information corresponds to sensor data. It can be seen that the method provided by this application is to complete the feature recognition of image frames in the video in the process of video generation, without the need for subsequent feature recognition and extraction of the saved video, which saves hardware resources; and can be combined with The sensor data of the wearable device can perform more accurate and rich feature recognition on the video.

The user operations mentioned in the embodiments of the present application include, but are not limited to, operations such as clicking, double-clicking, long-pressing, sliding, hovering gestures, and voice commands.

The first operation mentioned in the embodiment of the present application may be a user operation that triggers the function of capturing a picture, or a user operation that triggers the function of capturing a video. For example, the user operation described in the foregoing steps S102 and S302 to trigger the function of capturing a picture, and the user operation described in the foregoing steps S202 and S402 to trigger the function of capturing a video.

The first information mentioned in the embodiment of the present application may be biometric information that is fused and encoded with a picture/video file.

The second information mentioned in the embodiment of the present application may be biometric information displayed on the preview interface.

With regard to the above method flow, three application scenarios to which the embodiments of the present application are applicable are briefly introduced exemplarily below.

Scenario 1, home environment, shooting dance pictures/videos, yoga pictures/videos of specific users, etc.

If the first user wants to use the mobile phone to shoot a dance video for the second user, he can select the labeling mode to shoot for the second user. Wear the smart watch on the user's second-hand, and connect the mobile phone with the user's second-hand smart watch. After the connection is successful, user 1 selects the labeling mode and uses the mobile phone to shoot for user 2, and the pictures/videos captured at this time have biometric information. The biometric information can indicate the health status of the second user during the dance, and further determine the degree of fatigue and physical stress of the second user; the biometric information can also indicate the movement posture of the second user during the dance, and further determine the Whether the movement posture is standard; etc.

Before taking a photo, the user can configure the mobile phone or smart watch according to their needs, and the configuration content determines the content of the biometric information. For example, as soon as the user configures options such as heart rate information, exercise state information, and emotional state information on the mobile phone, the biometric information of the captured dance video includes information such as the heart rate information, exercise state information, and emotional state information of the second user.

Since the home scene is relatively simple, there is a high probability that only user 2 will be included in the video. The biometric information indicates the relevant information of user 2, and the biometric information of the video can be displayed in a fixed position on the display screen of the mobile phone.

Scene 2, outdoor activity scene, shooting pictures/videos of specific users in multiple people.

If the user wants to use the mobile phone to take a picture of the play in the park for the second user, he can select the mark mode to take the picture for the second user. Wear the smart watch on the user's second-hand, and connect the mobile phone with the user's second-hand smart watch. After the connection is successful, the first user selects the labeling mode and uses the mobile phone to shoot for the second user. The pictures/videos captured at this time have biometric information. The biometric information may indicate the movement posture, emotional state (eg, happy, excited, etc.) of the second user, and may also indicate the current environmental status (eg, air quality, air temperature and humidity, etc.).

Since there are many tourists in the park and the scene is more complicated, other tourists may appear in the picture taken by User 1 for User 2. The biometric information indicates the relevant information of User 2, and the biometric information of the picture can be displayed on User 2's Nearby, match biometric information with user two on the picture.

Scenario 3, professional training venues, fitness venues, shooting training pictures/videos of multiple people and multiple users.

In scenarios such as fitness venues and training rooms, multiple students are usually involved in exercising together, and the coach needs to grasp the exercise status of each student. Then the photographing device needs to be connected with multiple wearable devices. For example, in the system architecture shown in FIG. 22 , the photographing device and multiple wearable devices are connected through a third device.

The multiple smart watches are respectively worn on the hands of multiple users, the multiple smart watches are connected with the third device, and the shooting device is connected with the third device. After the connection is successful, use the shooting device to shoot for multiple users, and the pictures/videos obtained at this time have biometric information. The biometric information can indicate the user's heart rate, blood pressure, blood sugar, exercise posture and other information, and further determine whether the exercise posture is standard and whether the user is suitable for increasing the training intensity. It can be used for the coach's action guidance for the sports body and the monitoring of training fatigue.

Since scenario 3 is for multiple users, and the biometric information indicates the relevant information of multiple users, in the picture/video, the biometric information of different students can be displayed near the corresponding users respectively, and the biometric information of different users can be displayed in the picture/video. The information is matched with the user himself on the picture. This facilitates targeted viewing of biometric information of different users.

Embodiments of the present application also provide a computer-readable storage medium. The methods described in the foregoing method embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the functions may be stored on or transmitted over as one or more instructions or code on a computer-readable medium. Computer-readable media can include both computer storage media and communication media and also include any medium that can transfer a computer program from one place to another. A storage medium can be any available medium that can be accessed by a computer.

The embodiments of the present application also provide a computer program product. The methods described in the foregoing method embodiments may be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, it may be implemented in whole or in part in the form of a computer program product. A computer program product includes one or more computer instructions. When the above-mentioned computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the above-mentioned method embodiments are generated. The aforementioned computer may be a general purpose computer, a special purpose computer, a computer network, network equipment, user equipment, or other programmable devices.

In the above-mentioned embodiments, it may be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented in software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present application are generated. The computer may be a general purpose computer, special purpose computer, computer network, or other programmable device. The computer instructions may be stored in or transmitted over a computer-readable storage medium. The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server, data center, etc. that includes an integration of one or more available media. The usable media may be magnetic media (eg, floppy disks, hard disks, magnetic tapes), optical media (eg, DVDs), or semiconductor media (eg, solid state disks (SSDs)), and the like.

The steps in the method of the embodiment of the present application may be adjusted, combined and deleted in sequence according to actual needs.

As mentioned above, the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand: The technical solutions described in the embodiments are modified, or some technical features thereof are equivalently replaced; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application.

Claims

A shooting system, comprising: an electronic device and a first wearable device, the electronic device comprising a camera; wherein,

the electronic device for establishing a connection with the first wearable device;

The electronic device is further configured to receive the first operation;

The electronic device is further configured to acquire the multimedia file collected by the camera in response to the first operation;

the first wearable device for detecting first sensor data through at least one sensor;

The electronic device is further configured to acquire first information, where the first information corresponds to the first sensor data;

The electronic device is further configured to save the multimedia file, where the multimedia file is associated with the first information.
The system according to claim 1, wherein the first wearable device is further configured to:

determining the first information according to the first sensor data;

The first information is sent to the electronic device.
The system according to claim 1, wherein the first wearable device is further configured to send the first sensor data to the electronic device;

The electronic device is further configured to determine the first information according to the first sensor data.
The system according to claim 1, wherein the electronic device is specifically configured to acquire the first information in response to the first operation.
The system according to claim 2, wherein the electronic device is further configured to: in response to the first operation, send a first request message to the first wearable device;

The first wearable device is specifically configured to: in response to the first request message, send the first information to the electronic device.
The system according to claim 3, wherein the electronic device is further configured to: in response to the first operation, send a second request message to the first wearable device;

The first wearable device is specifically configured to: in response to the second request message, send the first sensor data to the electronic device.
The system according to claim 1, wherein the electronic device is further configured to: display a shooting preview interface, wherein the shooting preview interface includes a shooting button, and the first operation includes an action acting on the shooting button. Enter an action.
The system according to claim 1, wherein the attribute information of the multimedia file includes the first information.
The system according to claim 1, wherein the electronic device, for establishing a connection with the first wearable device comprises: in response to the electronic device entering a preset shooting mode, the electronic device is connected to the first wearable device. The first wearable device establishes a connection.
The system according to claim 1, wherein the first wearable device is further configured to: receive a second operation;

The first wearable device is further configured to instruct the electronic device to turn on the camera in response to the second operation;

The first electronic device is further configured to display a shooting preview interface, where the shooting preview interface displays a preview image collected by the camera; the first operation includes an operation acting on the shooting preview interface.
The system according to claim 1, wherein the electronic device is further configured to display the multimedia file and at least part of the first information.
The system according to claim 11, wherein the electronic device is further configured to display the multimedia file and at least part of the first information, comprising:

The electronic device is further configured to display the multimedia file and at least part of the first information in response to the multimedia file including a preset facial image; the preset facial image corresponds to the first wearable device.
The system according to claim 11, wherein the electronic device is further configured to display the multimedia file and at least part of the first information, comprising:

The electronic device is further configured to respond that the multimedia file includes a first facial image and a second facial image, and the first facial image matches a preset facial image, in the first facial image of the multimedia file. area displays at least part of the first information; wherein the preset facial image corresponds to the first wearable device; the distance between the first area and the display area of the first facial image is smaller than the first The distance of an area from the display area of the second facial image.
The system according to claim 1, wherein the electronic device is further configured to: before receiving the first operation, display a shooting preview interface, wherein the shooting preview interface includes a preview image collected by a camera;

The electronic device is further configured to display at least part of second information on the preview image, where the second information corresponds to second sensor data detected by the first wearable device.
The system according to claim 14, wherein the electronic device is further configured to display at least part of the second information on the preview image, comprising:

The electronic device is further configured to display the preview image and at least part of the second information in response to the preview image including a preset facial image, the preset facial image corresponding to the first wearable device.
The system according to claim 14, wherein the electronic device is further configured to display at least part of the second information on the preview image, comprising:

The electronic device is further configured to, in response to that the preview image includes a third facial image and a fourth facial image, and the third facial image matches the preset facial image, in the second part of the preview image; an area displaying at least part of the second information; wherein the preset facial image corresponds to the first wearable device; the distance between the second area and the display area of the third facial image is smaller than the second The distance of the area from the display area of the fourth face image.
The system according to claim 15, wherein the electronic device is further configured to: in response to the preview image not including the preset facial image, output a first prompt, wherein the first prompt is used for Prompt the user to aim at the face.
The system according to any one of claims 1-17, wherein the first information includes at least one of the following:

Health state information, exercise state information, or emotional state information.
The system according to any one of claims 1-17, wherein the first sensor data includes data detected by at least one sensor, and the at least one sensor includes at least one of the following:

Acceleration sensor, gyroscope sensor, geomagnetic sensor, atmospheric pressure sensor, heart rate sensor, blood pressure sensor, ECG sensor, myoelectric sensor, body temperature sensor, skin electric sensor, air temperature and humidity sensor, light sensor, or bone conduction sensor.
The system according to claim 1, wherein the system further comprises a second wearable device;

The electronic device is further configured to establish a connection with the second wearable device;

the second wearable device for detecting fourth sensor data through at least one sensor;

Wherein, the first information also corresponds to the fourth sensor data.
A shooting method, applied to an electronic device including a camera, characterized in that the method comprises:

establishing a connection between the electronic device and the first wearable device;

the electronic device receives the first operation;

The electronic device, in response to the first operation, acquires the multimedia file collected by the camera;

obtaining, by the electronic device, first information corresponding to first sensor data detected by at least one sensor of the first wearable device;

The electronic device saves the multimedia file, and the multimedia file is associated with the first information.
The method according to claim 21, wherein the obtaining, by the electronic device, the first information comprises:

acquiring, by the electronic device, the first sensor data;

The electronic device determines the first information based on the first sensor data.
The method according to claim 21, wherein the obtaining, by the electronic device, the first information comprises:

The electronic device acquires first information determined by the first wearable device based on the first sensor data.
The method according to claim 21, wherein the obtaining, by the electronic device, the first information comprises:

In response to the first operation, the electronic device acquires first information.
The method according to claim 22, wherein in response to the first operation, the electronic device obtains the first information, comprising:

In response to the first operation, the electronic device sends a first request message to the first wearable device, where the first request message is used to request to acquire the first wearable device detected by at least one sensor of the first wearable device. a sensor data;

The electronic device acquires first information, and the first information corresponds to the first sensor data.
The method of claim 23, wherein, in response to the first operation, the electronic device obtains first information, comprising:

In response to the first operation, the electronic device sends a second request message to the first wearable device, where the second request message is used to request to obtain the information determined by the first wearable device based on the first sensor data. first information;

The electronic device acquires first information, and the first information corresponds to the first sensor data.
The method of claim 21, wherein the electronic device receiving the first operation comprises:

The electronic device displays a shooting preview interface, and the shooting preview interface includes a shooting button;

The electronic device receives a first operation, and the first operation includes an input operation acting on the shooting button.
The method according to claim 21, wherein the attribute information of the multimedia file includes the first information.
The method according to claim 21, wherein establishing a connection between the electronic device and the first wearable device comprises:

In response to the electronic device entering the preset shooting mode, the electronic device establishes a connection with the first wearable device.
The method of claim 21, wherein the method further comprises:

The electronic device displays the multimedia file and at least part of the first information.
The method according to claim 30, wherein the electronic device displays the multimedia file and at least part of the first information, specifically comprising:

In response to the multimedia file including a preset facial image, the electronic device displays the multimedia file and at least part of the first information; the preset facial image corresponds to the first wearable device.
The method according to claim 30, wherein the electronic device displays the multimedia file and at least part of the first information, specifically comprising:

In response to the multimedia file including a first facial image and a second facial image, and the first facial image matches a preset facial image, the electronic device displays at least a portion of the first area of the multimedia file the first information; wherein the preset facial image corresponds to the first wearable device; the distance between the first area and the display area of the first facial image is smaller than the distance between the first area and the the distance of the display area of the second facial image.
The method according to claim 21, wherein before the electronic device receives the first operation, the method further comprises:

The electronic device displays a shooting preview interface, and the shooting preview interface includes a preview image collected by a camera;

The electronic device displays at least part of second information on the preview image, where the second information corresponds to second sensor data detected by the first wearable device.
The method according to claim 33, wherein the electronic device displays at least part of the second information on the preview image, which specifically includes:

In response to the preview image including a preset facial image, the electronic device displays the preview image and at least part of the second information, the preset facial image corresponding to the first wearable device.
The method according to claim 33, wherein the electronic device displays at least part of the second information on the preview interface, which specifically includes:

In response to the preview image including a third facial image and a fourth facial image, and the third facial image matches the preset facial image, the electronic device displays at least a portion in the second area of the preview image the second information; wherein the preset facial image corresponds to the first wearable device; the distance between the second area and the display area of the third facial image is smaller than the second area and the The distance of the display area of the fourth face image.
The method of claim 34, wherein the method further comprises:

In response to the preview image not including the preset face image, the electronic device outputs a first prompt for prompting the user to align the face.
The method according to any one of claims 20-36, wherein the first information includes at least one of the following:

Health state information, exercise state information, or emotional state information.
The method according to any one of claims 20-36, wherein the first sensor data includes data detected by at least one sensor, and the at least one sensor includes at least one of the following:

Acceleration sensor, gyroscope sensor, geomagnetic sensor, atmospheric pressure sensor, heart rate sensor, blood pressure sensor, electrocardiogram sensor, myoelectric sensor, body temperature sensor, skin electrical sensor, air temperature and humidity sensor, light sensor, or bone conduction sensor.
The method of claim 21, wherein the method further comprises:

The electronic device establishes a connection with a second wearable device; the second wearable device is configured to detect fourth sensor data through at least one sensor, and the first information also corresponds to the fourth sensor data.
An electronic device, comprising: one or more processors, one or more memories, and at least one camera; the one or more memories are coupled with the one or more processors; the one or more memories A plurality of memories are used to store computer program code, the computer program code includes computer instructions; when the computer instructions are executed on the processor, the electronic device is caused to perform as claimed in any one of claims 21-39. the shooting method described.
A computer-readable storage medium, characterized by comprising computer instructions, which, when the computer instructions are executed on an electronic device, cause the electronic device to execute the photographing method according to any one of claims 21-39.