WO2022228068A1

WO2022228068A1 - Image acquisition method, apparatus, and system for electronic device, and electronic device

Info

Publication number: WO2022228068A1
Application number: PCT/CN2022/085407
Authority: WO
Inventors: 裴璇
Original assignee: Oppo广东移动通信有限公司
Priority date: 2021-04-27
Filing date: 2022-04-06
Publication date: 2022-11-03
Also published as: CN113138669A

Abstract

The present application provides an image acquisition method, apparatus, and system for an electronic device, and an electronic device. The electronic device is in communicational connection with a wearable device. The image acquisition method for an electronic device comprises: sending a first message when an image acquisition request is detected, wherein the first message is used for notifying the wearable device to collect head action information; receiving the head action information, and performing action recognition according to the head action information to generate an action recognition result; and if the action recognition result conforms to a preset action, controlling an electronic device to execute a corresponding image acquisition operation. In the image acquisition method, apparatus, and system for an electronic device and the electronic device of the present application, the electronic device obtains head action information of a user by means of a wearable device, and recognizes the head action information of the user so as to control the electronic device to execute different image acquisition operations, such that the user can remotely perform the image acquisition operations such as photographing and video photographing, thereby improving the degree of freedom of photographing and enriching human-machine interaction modes.

Description

Image acquisition method, device, system and electronic device for electronic equipment

This application claims the priority of the Chinese patent application filed on April 27, 2021 with the application number 202110462283.1 and the title of the invention is "image acquisition method, device, system and electronic equipment for electronic equipment", the entire content of which is approved by Reference is incorporated in this application.

technical field

The present application relates to the field of electronic technologies, and more particularly, to an image acquisition method, apparatus, system and electronic device of an electronic device.

Background technique

In recent years, with the development of technology and economy, electronic devices such as mobile phones and tablet computers have become popular rapidly. When most users use electronic devices, they need to touch and control the electronic devices to perform corresponding operations. For example, the mobile phone can play music, videos, etc. according to the user's operations. However, these operations all require the user to directly touch the display screen, and the human-computer interaction method is very limited.

SUMMARY OF THE INVENTION

The present application provides an image acquisition method, device, system and electronic device of an electronic device, which enriches the interaction mode of human-computer interaction.

An aspect of the present application provides an image acquisition method for an electronic device, the electronic device is communicatively connected to a wearable device, and the image acquisition method includes: when an image acquisition request is detected, sending a first message, where the first message is used to notify The wearable device collects head motion information; receives the head motion information, performs motion recognition according to the head motion information, and generates a motion recognition result; if the motion recognition result conforms to a preset motion, controls the electronic device to perform a corresponding image acquisition operation.

In another aspect, the present application also provides an image acquisition device, the image acquisition device is communicatively connected to a wearable device, and the image acquisition device includes: a message sending module, configured to send a first message when an image acquisition request is detected, The first message is used to notify the wearable device to collect head motion information; the motion recognition module is used to receive the head motion information, perform motion recognition according to the head motion information, and generate a motion recognition result; the operation execution module is used for When the action recognition result conforms to the preset action, the electronic device is controlled to perform a corresponding image acquisition operation.

In another aspect, the present application also provides an electronic device, the electronic device is communicatively connected with a wearable device, the electronic device includes one or more processors and a memory; one or more programs are stored in the memory and configured to Executed by one or more processors to implement the above-described method.

In yet another aspect, the present application also provides an image acquisition system, including an electronic device and a wearable device; the electronic device and the wearable device are communicatively connected; the electronic device includes a processor, and the processor is configured to: when an image acquisition request is detected , send a first message, where the first message is used to notify the wearable device to collect head motion information; receive head motion information, perform motion recognition according to the head motion information, and generate motion recognition results; When setting the action, the electronic device is controlled to perform the corresponding image acquisition operation.

The image acquisition method, device, system and electronic equipment of electronic equipment provided by this application, the electronic equipment acquires the user's head motion information through the wearable device, and recognizes the user's head motion information, and different head motion information corresponds to different operation, thereby controlling the electronic device to perform different image acquisition operations. Through the above method, the user can remotely perform image acquisition operations such as taking pictures and taking pictures without holding the electronic device, which greatly expands the interactive mode of human-computer interaction and improves the degree of freedom and quality of taking pictures.

Additional aspects and advantages of embodiments of the present application will be set forth, in part, in the following description, and in part will be apparent from the following description, or learned by practice of embodiments of the present application.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without creative effort.

FIG. 1 is a schematic diagram of an image acquisition system proposed by an embodiment of the present application;

FIG. 2 is a flowchart of an image acquisition method of an electronic device proposed by an embodiment of the present application;

3 is a flowchart of an image acquisition method of an electronic device proposed by another embodiment of the present application;

FIG. 4 is a flowchart of an image acquisition method of an electronic device proposed by another embodiment of the present application;

Fig. 5 is a kind of structural schematic diagram of the image acquisition method of the electronic device proposed in Fig. 4;

6 is another schematic structural diagram of the image acquisition method of the electronic device proposed in FIG. 4;

FIG. 7 is a flowchart of an image acquisition method of an electronic device proposed by yet another embodiment of the present application;

Fig. 8 is a kind of structural schematic diagram of the image acquisition method proposed in Fig. 7;

FIG. 9 is a structural block diagram of an image acquisition apparatus proposed by an embodiment of the present application;

FIG. 10 is a structural block diagram of an electronic device proposed by an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

References in this specification to "one embodiment" or "some embodiments" and the like mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically emphasized otherwise. The terms "including", "including", "having" and their variants mean "including but not limited to" unless specifically emphasized otherwise.

In order to facilitate the understanding of the solutions shown in the embodiments of the present application, several terms appearing in the embodiments of the present application are introduced below.

Human-computer interaction: Human-computer interaction refers to the process of information exchange between people and computers using a certain dialogue language between people and computers to complete certain tasks in a certain interactive way. The human-computer interaction function is mainly completed by external devices that can input and output and corresponding software. The devices that can be used for human-computer interaction mainly include keyboard display, mouse, and various pattern recognition devices. The software corresponding to these devices is the part of the operating system that provides the human-computer interaction function. The main function of the human-computer interaction part is to control the operation of the relevant equipment and to understand and execute the relevant commands and requirements transmitted through the human-computer interaction equipment. An early human-computer interaction facility was the keyboard monitor. The operator enters the command through the keyboard, and the operating system executes the command immediately after receiving the command and displays the result through the display. There are different ways to enter the command, but the explanation of each command is clear and unique.

Face Recognition: Face recognition is a biometric identification technology based on human facial feature information. Capture images or video streams containing faces with a camera, and automatically detect and track faces in the images.

Pose Recognition: Human pose recognition is defined in this application as the problem of locating key points of the human body, namely the detection of human skeleton key points, also known as pose estimation, which has always been an important concern in the field of computer vision. Human skeleton key point detection mainly detects the key point information of the human body, such as joints, facial features, etc., and describes the human skeleton information through key points.

With the popularity of wireless Bluetooth headsets, new human-computer interaction methods have begun to appear. How to develop more human-computer interaction methods based on wireless Bluetooth headsets to adapt to various scenarios and various needs of different users is a problem that needs to be considered at present. The new interaction method based on head movements provides better privacy protection while being convenient, and is a way forward.

Please refer to FIG. 1 , which is a schematic diagram of an image acquisition system 100 proposed by an embodiment of the present application. The image acquisition system 100 may include an electronic device 110 and a wearable device 120 . As shown in FIG. 1 , the electronic device 110 may be a smart phone, a tablet computer, a smart wearable device (such as a smart bracelet, a smart watch, etc.), a smart large screen, a gateway, a vehicle-mounted device, a notebook computer, etc. Device 120 may include at least one of smart glasses, smart headphones, smart earrings, or smart collars. When the wearable device 120 is a smart earphone, the smart earphone may be a TWS (True Wireless Stereo, true wireless stereo) earphone.

Optionally, the wearable device 120 may be a single component, and the wearable device 120 may also include a first component and a second component. In a possible manner, the first component and the second component may be independent components, the first component is an independent device in physical form, and the second component is also an independent device in physical form. For example, when the wearable device is a TWS earphone, the first component is the left earphone in the TWS earphone, and the second component is the right earphone in the TWS earphone. Alternatively, when the wearable device is a smart earring, the first component is the left earring in the smart earring, and the second component is the right earring in the smart earring.

In another possible implementation manner, the first component and the second component are components disposed in different positions in the same wearable device 120 . For example, when the wearable device 120 is smart glasses, the first component is arranged in the left temple, and the second component is arranged in the right temple.

The electronic device 110 and the wearable device 120 are communicatively connected, wherein the electronic device 110 and the wearable device 120 may be connected wirelessly or connected through a physical connection line. The electronic device 110 and the wearable device 120 may establish a communication link through a wireless communication protocol, wherein the wireless communication protocol may include a Wlan protocol, a Bluetooth protocol, or a ZigBee protocol.

The electronic device 110 includes a processor 1120 . The electronic device 110 may further include a display screen and a camera. When detecting an image acquisition request, the processor 1120 sends a first message, where the first message is used to notify the wearable device 120 to collect head motion information. Subsequently, the processor 1120 receives the head motion information, performs motion recognition according to the head motion information, and generates a motion recognition result. When the action recognition result conforms to the preset action, the processor 1120 controls the electronic device 110 to perform a corresponding image capturing operation. The image acquisition operation here may be photographing and recording, or may be live video broadcasting, video conferencing, or the like. The electronic device 110 obtains the user's head motion information through the wearable device 120 and performs corresponding operations. When taking pictures or videos, there is no need to click on the display screen or manually control the wearable device 120, which eliminates the need for the Selfie to hold the mobile phone and use a selfie stick. Or the trouble of touching the camera buttons in certain positions of the earphone with your hand, which greatly improves the freedom of taking pictures and the quality of the film.

The wearable device 120 may include an acceleration sensor or a magnetic sensor, or an acceleration sensor and a magnetic sensor. The accelerometer can detect and sense sudden acceleration or deceleration to obtain the acceleration data of the head. Magnetic sensors can acquire magnetic field data and detect the linear or angular velocity of the head. Optionally, an acceleration sensor and a magnetic sensor are used to collect acceleration data and magnetic field data, and the electronic device calculates the head attitude angle according to the acceleration data and the magnetic field data, and then can judge the head attitude according to the attitude angle.

The electronic device 110 may include a motion recognition module for performing motion recognition on sensor data collected by the wearable device 120 , and then the electronic device 110 may be controlled through the motion recognition result.

In one embodiment, the image acquisition system 100 includes an electronic device 110 and a wearable device 120 , and the electronic device 110 is communicatively connected to the wearable device 120 . When detecting an image acquisition request, the processor 1120 of the electronic device 110 sends a first message, where the first message is used to notify the wearable device 120 to collect head motion information. The wearable device 120 detects head motion information through an acceleration sensor and a magnetic sensor, and sends the information to the electronic device 110 . The processor 1120 of the electronic device 110 receives the head motion information, performs motion recognition according to the head motion information, and generates the motion recognition result; when the motion recognition result conforms to a preset motion, the processor 1120 controls The electronic device performs a corresponding image acquisition operation.

Referring to FIG. 2, an embodiment of the present application provides an image acquisition method of an electronic device, where the electronic device is communicatively connected to a wearable device, and the method includes:

01: When an image acquisition request is detected, a first message is sent, wherein the first message is used to notify the wearable device to collect head motion information;

In this embodiment of the present application, the detection of an image acquisition request is the detection of a user's image acquisition triggering operation in the electronic device, and the triggering method may be triggered by the user clicking on the display screen, or may be triggered by voice, or may be triggered by a button. The app can take pictures or videos from the camera, as well as video calls, video conferences or live broadcasts. When the electronic device detects the user's image acquisition request, the electronic device sends a first message to the wearable device, where the first message is used to notify the wearable device to collect head motion information.

As an implementation manner, the electronic device sends the first message to the wearable device, which may send a continuous signal or a discrete signal. Specifically, when sending discrete signals, it can stop after sending one or more signals per unit time, such as sending notification messages three or five times per second; information. This information may be head motion information sent by the wearable device, or may simply be the feedback information of the wearable device indicating that the head motion information has been received.

Optionally, the wearable device may include an acceleration sensor and a magnetic sensor. The acceleration sensor can be a three-axis sensor. When the wearable device receives the first message sent by the electronic device, the built-in three-axis acceleration sensor of the wearable device starts to collect sensor data from the three dimensions of X, Y, and Z, while the magnetic sensor collects the magnetic field. data. The wearable device sends two kinds of data to the electronic device to identify the head movement information.

Optionally, the head motion information includes but is not limited to: shaking his head, nodding, raising his head and shaking his head. Shaking is the movement of rotating the head with the neck as the base axis, and the mandibular angle is almost kept at the same level during the rotation; nodding is the movement of turning the head towards the chest; looking up is the movement of raising the head, and the angle of the mandible is moved away from the chest Directional movement; head swing is a movement of turning the neck towards the shoulders left and right, with the left mandibular angle close to one shoulder and away from the other shoulder when the head is deflected, the head and the back are almost kept on the same level .

In this embodiment of the present application, before sending the first message, the electronic device can also detect whether the electronic device is in a connected state with the wearable device, and if the electronic device is in a connected state with the wearable device, the electronic device sends the first message again.

Specifically, the connection state between the electronic device and the wearable device may include a connected state and a non-connected state, wherein the non-connected state includes a non-connected state and a connection interrupted state.

As one of the methods, the connection state between the electronic device and the wearable device can be determined by checking the state value of the electronic device. Specifically, two different state values can be set for the electronic device in advance. When the electronic device is connected When the wearable device is used, the first state value is returned, and when the electronic device is not connected to the wearable device, the second state value is returned, so that whether the electronic device is compatible with the wearable device can be determined by detecting the first state value and the second state value. The device is connected. Exemplarily, the first state value of the electronic device is set to 1 in advance, and the second state value of the electronic device is set to 0. If the state value of the electronic device is detected to be 1, it is determined that the electronic device is connected to the wearable device. If it is detected that the state value returned by the electronic device is 0, it is determined that the electronic device and the wearable device are in a disconnected state. Optionally, if it is detected in time sequence that the state value returned by the electronic device changes from 1 to 0 at adjacent moments, it is determined that the connection between the electronic device and the wearable device is in a disconnected state.

As another method, the electronic device will send a broadcast when the wearable device is connected and disconnected, so the electronic device can determine whether the electronic device is in a connected state with the wearable device by monitoring the broadcast.

04: Receive the head motion information, perform motion recognition according to the head motion information, and generate the motion recognition result;

In the embodiment of the present application, the head motion information is the data of the acceleration sensor and the magnetic sensor, and the electronic device receives the data of the acceleration sensor and the magnetic sensor of the wearable device, and imports them into the action recognition model to obtain the actions output by the action recognition model. Classification results. Specifically, according to the received head motion information, the electronic device obtains the average value of the values of the X, Y, and Z channels of the three-axis acceleration sensor data, and inputs the average value together with the magnetic sensor data into the motion recognition model action classification.

Specifically, the electronic device calculates the attitude angle data according to the average value of the acceleration sensor data together with the magnetic force sensor data, and performs action classification and recognition according to the attitude angle data.

In some embodiments, the motion recognition occurs on the wearable device side. The motion recognition result is generated after the wearable device performs motion recognition according to the head motion information. The electronic device receives the motion recognition result, and then proceeds to step 07 .

07: If the action recognition result conforms to a preset action, control the electronic device to perform a corresponding image acquisition operation.

In an embodiment of the present application, if the action recognition result is a preset head action (such as nodding, shaking, shaking, raising, turning, etc.), the electronic device is controlled to perform an image acquisition operation corresponding to the head action . The above method does not need to click on the display screen or manually control the wearable device during the image acquisition process, and the user can remotely and freely control the image acquisition operation, which greatly improves the degree of freedom of photographing and the quality of the film.

In some embodiments, the motion recognition result conforming to the preset motion further includes that the motion recognition result conforms to the preset motion magnitude. Optionally, when the action recognition result conforms to a preset action, the attitude angle data is compared with a preset angle threshold, and when the attitude angle data is greater than the preset angle threshold, the electronic device is controlled to perform a corresponding image acquisition operation.

In an implementation manner, the angle threshold of the nodding action is set to face the ground and move downwards by 20 degrees. If it is detected that the user's head moves downwards towards the ground by more than 20 degrees, the electronic device is controlled to execute the image corresponding to the nodding. Acquisition operation; if it is detected that the downward movement of the user's head facing the ground does not exceed 20 degrees, the corresponding image acquisition operation is not performed. Correspondingly, it can be detected whether the user's head rotates with the neck as the center, and whether the deflection amplitude towards the left shoulder or the right shoulder exceeds the preset amplitude (at this time, the head and the back are on the same level), when the user's head is detected. When the deflection of the user's head toward the left shoulder or the right shoulder exceeds the preset angle, the electronic device is controlled to perform the image acquisition operation corresponding to the head swing; if it is detected that the deflection of the user's head toward the left shoulder or the right shoulder does not exceed the preset angle angle, the image acquisition operation is not performed. Exemplarily, the preset angle of the head swing action may be set as a deflection of 45 degrees toward the left shoulder or toward the right shoulder in advance, and when it is detected that the user's head is deflected toward the left shoulder or toward the right shoulder by more than 45 degrees, the electronic device is controlled to perform the swing. The image acquisition operation corresponding to the header. Setting the angle threshold increases the steps of judging the action recognition result and increases the action triggering threshold, thereby reducing the probability of false touches caused by the user's posture adjustment or random actions during image acquisition.

Referring to FIG. 3, in some embodiments, 01: when an image acquisition request is detected, a first message is sent, wherein the first message is used to notify the wearable device to collect head motion information, which may include :

012: when the image acquisition request is detected, determine whether the specified event exists;

014: if the specified event exists, send a first message, wherein the first message is used to notify the wearable device to collect head motion information;

Specifically, when the electronic device detects an image acquisition request, it needs to identify whether the specified event exists. If the specified event exists, the electronic device sends a first message to the wearable device, notifying the wearable device to collect the user's head motion information. Afterwards, the electronic device receives the head motion information, performs motion recognition according to the head motion information, and generates the motion recognition result. If the action recognition result conforms to a preset action, the electronic device is controlled to perform a corresponding image acquisition operation. The addition of a judging step of identifying whether a specified event exists, improves the judgment conditions, and can prevent misoperations caused by moving when the user is not ready.

Please refer to FIG. 4 , which is a flowchart of an image acquisition method of an electronic device provided by another embodiment of the present application. In the image acquisition method provided in FIG. 4 , the above-mentioned identifying whether the specified event exists is identifying whether the face exists or whether the gesture matching is successful. 07: If the action recognition result conforms to a preset action, control the electronic device to perform a corresponding image acquisition operation, including:

072: when the action recognition result is a first action, control the electronic device to perform a shooting operation and store the shooting result after a first preset time;

And/or, when the action recognition result is the second action, the electronic device is controlled to perform a switching operation.

4, in some embodiments, when the electronic device detects that the user applies for shooting and other operations, it will first perform a face recognition or gesture recognition operation, that is, detect whether a face is recorded within the shooting range of the camera or whether the user presses a specified posture. station. The camera here can be a front camera or a rear camera. When the electronic device recognizes the face or recognizes that the gesture is successfully matched, it will send a notification to the wearable device to collect the user's head information. After the wearable device receives the notification, it will continuously collect the user's head information and send it to the electronic device. The electronic device receives the head motion information, performs motion recognition according to the head motion information, and generates the motion recognition result. When the action recognition result is a first action, control the electronic device to perform a shooting operation and store the shooting result after a first preset time; and/or, when the action recognition result is a second action, The electronic device is controlled to perform a switching operation. When the action recognition result is the second action, the electronic device is controlled to perform a switching operation. After the image acquisition operation is performed, the electronic device will enter the next recognition operation according to the head motion information sent by the wearable device. In the above image acquisition method, the monitoring of the user's actions by the earphone does not start after turning on the camera, but is triggered by detecting that the face is in the mirror or posing the correct posture, avoiding the process of walking to the photographing position and adjusting the posture. The movement of the middle head is mistakenly touched.

Specifically, the first action may be any one or other combinations of nodding, raising your head, turning your head, shaking your head or swinging your head, and the second action may be different from the first action. Any one or a combination of these actions of turning your head, shaking your head or swinging your head, or other head actions, is not limited here. For example, the first action may be nodding, and the second action may be shaking head. The switching operation may be switching filters or switching gesture prompt boxes, and can be divided into left switching and right switching. Shaking left can correspond to switching to the left, and shaking right can correspond to switching to the right. Similarly, the first action can be shaking left and right, and the second action can be nodding and looking up, wherein nodding can correspond to switching to the left (or switching to the previous one), and looking up can correspond to switching to the right (or switching to the next); You can nod your head to switch to the right (or switch to the previous one), and shake your head to switch to the left (or switch to the next one).

Specifically, the first preset time may be 3 seconds or 5 seconds or any other time, and the first preset time can be set by the user at will, which is not limited here.

Please refer to FIG. 4 and FIG. 5 , in an implementation example, the user clicks on the camera selfie mode, and the deep learning face recognition model built in the mobile phone starts to work. When a human face is detected, the selfie assistant system of human-computer interaction is activated, the mobile phone sends a notification to the Bluetooth headset, and the Bluetooth headset starts to collect the user's head motion information and send it to the mobile phone. The mobile phone performs motion recognition and judgment. When it recognizes a nodding motion, the camera starts to count down for 3 seconds. After the timer ends, it automatically takes a picture, and then continues to monitor the motion feedback of the headset. When the phone recognizes a shaking motion, the camera switches the filters in a preset order, from the left to the previous one, and the right to the next.

Please refer to FIG. 4 and FIG. 6 , in another embodiment, the user clicks on the gesture prompting photo mode, the camera interface generates a translucent photo taking gesture prompt box, and prompts the photographer to pose in the form of a silhouette. At this time, the mobile phone The built-in deep learning attitude detection module starts to work. When the six nodes of the human limbs, head and torso are detected to be basically consistent with the prompt box, the system is activated, the mobile phone sends a notification to the Bluetooth headset, and the Bluetooth headset starts to collect the user's head. Ministry action information and sent to the mobile phone. The mobile phone performs motion recognition and judgment in real time. When a nodding motion is recognized, the camera starts to count down for 5 seconds, and automatically takes a picture after the timer ends, and then continues to monitor the action feedback of the headset. When the phone recognizes a shaking motion, the camera switches the gesture prompt boxes according to the preset order, from the left to the previous one, and the right to the next one. At this time, the camera interface will refresh to display a new prompt box.

In yet another embodiment, the image acquisition operation may be a live video, the user enters the live video page, and when the user nods, the mobile phone recognizes the nodding operation to turn on the microphone; when the user shakes his head, shaking his head to the left corresponds to switching Go to the filter to the left of the current filter, and shake your head to the right corresponds to switching to the filter to the right of the current filter.

Please refer to FIG. 7 , which is a flowchart of a method for acquiring an image of an electronic device according to yet another embodiment of the present application. The image acquisition method provided in FIG. 7, 07: if the action recognition result conforms to a preset action, controlling the electronic device to perform a corresponding image acquisition operation may include:

073: When the action recognition result is the first action, control the electronic device to perform the recording operation or stop the recording operation; and/or, when the action recognition result is the second action, control the electronic device to perform the end of the action An operation is recorded and the recording result is stored, wherein the second action is different from the first action.

In some embodiments, when the electronic device detects that the user applies for photography and other operations, it sends a first message for notifying the wearable device to collect head motion information. After the wearable device receives the notification, it will continuously collect the user's head movement information and send it to the electronic device. The electronic device receives the head motion information, performs motion recognition according to the head motion information, and generates the motion recognition result. When the action recognition result is the first action, the electronic device is controlled to perform the recording operation or stop the recording operation. Specifically, when the electronic device is in the recording state and the action recognition result is the first action, a stop recording operation is performed; when the electronic device is in the recording stop state and the action recognition result is the first action, the operation is performed recording operation; and/or, when the action recognition result is a second action, controlling the electronic device to perform an end recording operation and store the recording result, wherein the second action is different from the first action. After the image acquisition operation is performed, the electronic device will enter the next recognition operation according to the head motion information sent by the wearable device.

Specifically, the first action may be any one or other combinations of nodding, raising your head, turning your head, shaking your head or swinging your head, and the second action may be different from the first action. Any one or a combination of these actions of turning your head, shaking your head or swinging your head, or other head actions, is not limited here.

Please refer to FIG. 7 and FIG. 8 , in an embodiment, an auxiliary camera function in a Selfie assistance system is provided. As shown in the figure, the user clicks the recording mode, and the system is activated. At this time, the built-in sensor of the wireless Bluetooth headset will continuously send the collected signal data to the mobile terminal, and the mobile terminal will perform motion recognition and judgment. When the mobile terminal recognizes a nodding motion, if the camera is recording, it will automatically pause the recording; if it is in the paused state, it will automatically continue recording. When the mobile terminal recognizes a shaking motion, whether the camera is recording or paused, it will automatically end the recording. This video recording method can free hands, meet the use of disabled people, and provide a new way of human-computer interaction.

In another embodiment, the image acquisition request is a video conference request. When the mobile phone detects that the user has entered the video conference page, it sends a notification to the headset to start collecting head motion information. After the headset receives the information, it starts to collect head motion information and continues to send it to the mobile phone. The mobile phone identifies and makes judgments based on the received head motion information. If it is recognized that the user has performed a head-turning operation, the head-turning operation is an operation in which the head wraps around the neck at least once (that is, 360 degrees), and if the current microphone and/or camera are off, turn on the microphone and/or camera; if If the current microphone and/or camera is on, turn off the microphone and/or camera. If it is recognized that the user performs a force continuous head swing operation (the left and right head swings are repeated at least three times), the current video conference is exited.

In some embodiments, the performing action recognition according to the head motion information in 04 and generating the motion recognition result may include: performing the action recognition based on a convolutional neural network model according to the head motion information Action recognition, outputting the action recognition result.

Specifically, an embodiment of the present application provides an image acquisition method, the method includes: when an image acquisition request is detected, sending a first message, wherein the first message is used to notify the wearable device to collect headers Action information; receive the head action information, and according to the head action information, perform the action recognition based on the convolutional neural network model, and output the action recognition result; if the action recognition result conforms to the preset action, The electronic device is controlled to perform a corresponding image acquisition operation. The model involved in the image acquisition method based on the convolutional neural network model has good generalization ability to different groups of people, and strong anti-interference ability to noise.

Optionally, the image acquisition method provided in the embodiment of the present application further includes: when the action recognition result is a third action, controlling the electronic device to perform a focus adjustment operation.

Specifically, the third action here may be any head action different from the first action and the second action. For example, if the third action is head swing, when the user is recording or taking pictures, the user swings his head to the left to increase the focal length to shoot a close-up view; the user swings his head to the right to reduce the focal length to capture a long-range view. Each head swing corresponds to a 5mm focal length change. Here, the focal length change and the corresponding relationship between the action and the focal length can be set manually, and there is no forcible restriction. In this embodiment, by setting different actions to correspond to different focal lengths, it is possible to flexibly shoot long-range and short-range scenes during the human-computer interaction process, thereby improving the flexibility of shooting.

Optionally, the image acquisition method provided in the embodiment of the present application further includes: when the action recognition result is a fourth action, controlling the electronic device to perform a continuous shooting operation; the continuous shooting operation is performed at a second preset time After that, take a preset number of images.

Specifically, the fourth action here may be any head action different from the first action, the second action and the third action. For example, if the fourth action is raising the head, the electronic device notifies the wearable device to detect the head action of the user when detecting that the user is in the photographing mode. The electronic device recognizes the head motion information after acquiring the head motion information. If it recognizes the head-up motion, it performs a continuous shooting operation. The electronic device captures a preset number of images after a second preset time, where the second preset time and the preset number There are no restrictions, the preset number can be 5 or 10, and the second preset time can be 3 seconds or 5 seconds later. More specifically, the continuous shooting operation may set a frame rate of shooting per second, for example, 10 frames per second. In this embodiment, the actions corresponding to the continuous shooting are set, so that the user can remotely realize the continuous shooting, which improves the user's human-computer interaction experience.

Referring to FIG. 9 , the present application provides an image acquisition device, which runs on an electronic device and is communicatively connected to a wearable device. The image acquisition device includes a message sending module 210 , an action recognition module 220 and an operation Module 230 is executed.

a message sending module 210, configured to send a first message when an image acquisition request is detected, wherein the first message is used to notify the wearable device to collect head motion information;

A motion recognition module 220, configured to receive the head motion information, perform motion recognition according to the head motion information, and generate the motion recognition result;

The operation execution module 230 is configured to control the electronic device to perform a corresponding image acquisition operation when the action recognition result conforms to a preset action.

The image acquisition device in this embodiment makes it possible to shoot with both hands free.

In a feasible implementation manner, the message sending module 210 is configured to determine whether a specified event exists when the image acquisition request is detected; if the specified event exists, send a first message, wherein the The first message is used to notify the wearable device to collect head motion information;

Specifically, the operation execution module 230 is configured to, when the action recognition result is a first action, control the electronic device to perform a shooting operation and store the shooting result after a first preset time; and/or, when all the When the action recognition result is the second action, the electronic device is controlled to perform a switching operation.

In another feasible implementation manner, the message sending module 210 is configured to send a first message when an image acquisition request is detected, wherein the first message is used to notify the wearable device to collect head movements information;

The operation execution module 230 is configured to control the electronic device to select to perform the recording operation or stop the recording operation based on the current state when the action recognition result is the first action; and/or, when the action recognition result is the second action , control the electronic device to perform an end recording operation and store the recording result.

In a feasible implementation manner, the operation execution module 230 is further configured to control the electronic device to perform a focus adjustment operation when the action recognition result is a third action.

In a feasible implementation manner, the operation execution module 230 is further configured to control the electronic device to perform a continuous shooting operation when the action recognition result is the fourth action; the continuous shooting operation is performed at a second preset time. After that, take a preset number of images.

Please refer to FIG. 10 , which is a schematic diagram of an internal structure of an electronic device in one embodiment. With reference to FIG. 10 and FIG. 1 , the electronic device 110 is connected to the wearable device for communication 120 . As shown in FIG. 10 , the electronic device includes one or more processors 1120 , a memory 1140 , a camera 1160 and a display 1180 connected through a system bus. One or more programs are stored in the memory 1140 and configured to perform, by the one or more processors 1120, steps corresponding to the image acquisition method.

The processor 1120 may include one or more processing cores. The processor 1120 uses various interfaces and lines to connect various parts of the entire electronic device 110, and executes by running or executing the instructions, programs, code sets or instruction sets stored in the memory 1140, and calling the data stored in the memory 1140. Various functions of the electronic device 110 and processing data. Optionally, the processor 1120 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), field-programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). implemented in a hardware form. The processor 1120 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface and application programs, etc.; the GPU is used for rendering and drawing of the display content; the modem is used to handle wireless communication. It can be understood that, the above-mentioned modem may not be integrated into the processor 1120, and is implemented by a communication chip alone.

The memory 1140 may include random access memory (Random Access Memory, RAM), or may include read-only memory (Read-Only Memory). Memory 1140 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 1140 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing the following method embodiments, and the like. The storage data area may also store data created by the electronic device 110 during use (eg, phone book, audio and video data, chat record data) and the like.

The camera 1160 is used for receiving image information of the user. In one possible implementation, the camera 1160 may be a separate component. In another possible manner, the camera 1160 is a camera module including a plurality of camera units. This embodiment of the present application does not limit the specific implementation of the camera 1160 .

The display screen 1180 is used to display the information of the electronic device 110 . In a possible implementation manner, the display screen 1180 may include a touch function for receiving touch information from the user.

In the description of this specification, reference to the description of the terms "certain embodiments," "in an example," "exemplarily," etc. means that a particular feature, structure, material, or characteristic described in connection with an embodiment or example is included in the present application at least one embodiment or example of . In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples. Furthermore, those skilled in the art may combine and combine the different embodiments or examples described in this specification, as well as the features of the different embodiments or examples, without conflicting each other.

Any description of a process or method in the flowcharts or otherwise described herein may be understood to represent a module, segment or portion of code comprising one or more executable instructions for implementing a specified logical function or step of the process , and the scope of the preferred embodiments of the present application includes alternative implementations in which the functions may be performed out of the order shown or discussed, including performing the functions substantially concurrently or in the reverse order depending upon the functions involved, which should It is understood by those skilled in the art to which the embodiments of the present application belong.

Although the embodiments of the present application have been shown and described above, it should be understood that the above embodiments are exemplary and should not be construed as limitations to the present application. Embodiments are subject to variations, modifications, substitutions and alterations.

Claims

An image acquisition method of an electronic device, wherein the electronic device is communicatively connected to a wearable device, wherein the method comprises:

When an image acquisition request is detected, a first message is sent, wherein the first message is used to notify the wearable device to collect head motion information;

receiving the head motion information, and performing motion recognition according to the head motion information to generate the motion recognition result;

If the action recognition result conforms to a preset action, the electronic device is controlled to perform a corresponding image acquisition operation.
The image acquisition method according to claim 1, wherein the sending a first message when an image acquisition request is detected, comprising:

When the image acquisition request is detected, determine whether the specified event exists;

If the specified event exists, a first message is sent, where the first message is used to notify the wearable device to collect head motion information.
The image acquisition method according to claim 2, wherein the judging whether the specified event exists includes recognizing whether the face exists or recognizing whether the gesture matching is successful, and if the action recognition result conforms to a preset action, control all The electronic device performs corresponding image acquisition operations, including:

When the action recognition result is a first action, control the electronic device to perform a shooting operation and store the shooting result after a first preset time;

And/or, when the action recognition result is the second action, the electronic device is controlled to perform a switching operation.
The image acquisition method according to claim 1, wherein if the motion recognition result conforms to a preset motion, controlling the electronic device to perform a corresponding image acquisition operation, comprising:

When the action recognition result is the first action, controlling the electronic device to select to perform the recording operation or stop the recording operation based on the current state;

And/or, when the action recognition result is the second action, the electronic device is controlled to perform an end recording operation and store the recording result.
The image acquisition method according to claim 1, wherein the performing motion recognition according to the head motion information to generate the motion recognition result comprises:

According to the head motion information, the motion recognition is performed based on a convolutional neural network model, and the motion recognition result is output.
The image acquisition method according to claim 3 or 4, wherein the method further comprises:

When the action recognition result is a third action, the electronic device is controlled to perform a focus adjustment operation.
The image acquisition method according to claim 3, wherein the method further comprises:

When the action recognition result is the fourth action, the electronic device is controlled to perform a continuous shooting operation; the continuous shooting operation is to shoot a preset number of images after a second preset time.
An image acquisition device, the image acquisition device is communicatively connected with a wearable device, characterized in that the image acquisition device comprises:

a message sending module, configured to send a first message when an image acquisition request is detected, wherein the first message is used to notify the wearable device to collect head motion information;

a motion recognition module, configured to receive the head motion information, perform motion recognition according to the head motion information, and generate the motion recognition result;

An operation execution module, configured to control the electronic device to perform a corresponding image acquisition operation when the action recognition result conforms to a preset action.
An electronic device, the electronic device is communicatively connected with a wearable device, characterized in that the electronic device comprises one or more processors and a memory; one or more programs are stored in the memory and configured to The method of any of claims 1-7 is performed by the one or more processors.
An image acquisition system, characterized in that the system includes an electronic device and a wearable device;

the electronic device is in communication connection with the wearable device;

The electronic device includes a processor for:

When an image acquisition request is detected, a first message is sent, wherein the first message is used to notify the wearable device to collect head motion information;

receiving the head motion information, and performing motion recognition according to the head motion information to generate the motion recognition result;

When the action recognition result conforms to a preset action, the electronic device is controlled to perform a corresponding image acquisition operation.