WO2019056236A1

WO2019056236A1 - Control method based on gestures, multimedia playback terminal, and storage medium

Info

Publication number: WO2019056236A1
Application number: PCT/CN2017/102537
Authority: WO
Inventors: 庞杰; 许逸君; 张玉磊; 桂明建
Original assignee: 深圳传音通讯有限公司
Priority date: 2017-09-21
Filing date: 2017-09-21
Publication date: 2019-03-28

Abstract

The present invention discloses a control method based on gestures, a multimedia playback terminal, and a storage medium. A gesture of a user is monitored, and an infrasonic sound wave signal generated by a preset gesture is acquired when the user performs the preset gesture, and then it is determined whether a sound wave frequency curve matching the infrasonic sound wave signal exists in a sound wave library. If the sound wave frequency curve matching the infrasonic sound wave signal exists in the sound wave library, a multimedia playback terminal is controlled to execute a corresponding playback operation according to a control instruction corresponding to the sound wave frequency curve. The invention avoids a situation in which noise affects others when using button or voice control, as well as inaccurate control of a touch screen caused by sweat or water on the hands, thereby improving the precision and accuracy of control.

Description

Gesture action-based control method, multimedia play terminal, and storage medium

Technical field

The present invention relates to the field of multimedia play terminals, and in particular, to a gesture action based control method, a multimedia play terminal, and a storage medium.

Background technique

Existing multimedia playback terminals need to be implemented by buttons, touch screens, or voice operations when performing multimedia playback, display zooming, or other control. However, when using a voice operation or a button operation to control a multimedia playback terminal in a quiet resting place, it will affect others; in the case where the hands cannot be liberated, the multimedia operation terminal cannot be controlled by using a button operation and a touch screen operation, and it is also in hand. When touched with sweat or water stains, the touch screen control is not accurate.

Summary of the invention

The main object of the present invention is to provide a control method based on gesture action, which aims to reduce noise when controlling a multimedia playing terminal, and improve control precision and accuracy.

To achieve the above object, the gesture-based control method proposed by the present invention is applied to a multimedia playing terminal, and the control method includes the following steps:

Monitor user gestures;

When the user makes a preset gesture action, acquiring an infrasound wave signal generated by the gesture action;

Determining whether there is a sound wave frequency curve matching the infrasound wave signal in the sound wave library;

If yes, controlling the multimedia playing terminal to perform a corresponding playing operation according to a control instruction corresponding to the sound wave frequency curve.

Further, before performing all the steps, the control method further includes:

The mapping relationship between the acoustic wave frequency curve generated by the preset gesture action and the control command is stored, and the acoustic wave frequency curve of the secondary acoustic wave signal generated by the different gesture actions is stored in the sound wave library.

Further, the step of the mapping relationship between the sound wave frequency curve generated by the preset gesture action and the control instruction includes:

Presetting the sound wave frequency curve generated by the single-handed action from left to right at a first speed to map a control instruction for playing the next multimedia file;

Presetting the sound wave frequency curve generated by the single-handed action from right to left at a first speed to map a control instruction for playing the previous multimedia file;

Presetting a control command for reducing the volume of the sound wave frequency curve generated by the single wave gesture from top to bottom at a first speed;

Presetting a control command for increasing the volume of the sound wave frequency curve generated by the single wave gesture from bottom to top at the first speed;

Presetting a sound wave frequency curve generated by a set of wave motions at a first speed to map a control instruction for playing the current multimedia file;

Presetting the sound wave frequency curve mapping generated by a set of wave motions at the second speed to close the currently playing control command;

The sound wave frequency curve mapping generated by the first half of the first speed and the second half of the second half of the wave motion is suspended to temporarily suspend the currently playing control command.

Further, the step of monitoring a gesture action of the user specifically includes:

When it is detected that the noise value of the current environment is less than a preset threshold, whether the output is switched to the prompt of the silent control mode;

If yes, or when no feedback information is received within the preset duration, the user's gesture action is monitored.

Further, the step of acquiring the infrasound wave signal generated by the gesture action when the user performs the preset gesture action includes:

Extracting an action parameter of a gesture action made by the user based on image feature extraction;

When the action parameter matches the action parameter of the preset gesture action, the secondary sound wave signal generated by the gesture action is acquired.

Further, when the gesture action made by the user does not match the preset gesture action, the display unit of the multimedia terminal displays an operation example corresponding to various control instructions.

Further, if yes, the step of controlling the multimedia play terminal to perform a corresponding play operation according to the control instruction corresponding to the sound wave frequency curve, specifically includes:

When a sound wave frequency curve matching the infrasound wave signal exists in the sound wave library, acquiring a control instruction corresponding to the sound wave frequency curve based on the mapping relationship;

Displaying the control instruction on the display unit of the multimedia playing terminal for confirmation by the user;

When receiving the confirmation instruction of the user, the multimedia playback terminal is controlled to perform a play, pause, switch, volume adjustment, or shutdown operation according to the control instruction.

Further, when there is no acoustic wave frequency curve matching the subsonic wave signal in the sound wave library, the output prompts the repetitive gesture action.

The present invention further provides a multimedia playing terminal, the multimedia playing terminal comprising a memory, a processor, and a control program stored in the memory and running on the processor, the control program being implemented by the processor The steps of the gesture-based control method.

The present invention also proposes a storage medium storing a control program, the control program being executed by the processor to implement the steps of the gesture-based control method as described above.

The gesture control-based control method of the embodiment of the present invention is used for the playback control of the multimedia play terminal, specifically: monitoring the gesture action of the user, and acquiring the infrasound generated by the preset gesture action when the user makes the preset gesture action a signal, and then determining whether there is an acoustic wave frequency curve matching the subsonic wave signal in the sound wave library, and when there is an acoustic wave frequency curve matching the subsonic wave signal in the acoustic wave library, controlling the sounding frequency curve according to a control instruction corresponding to the acoustic wave frequency curve The multimedia playing terminal performs a corresponding playing operation. The control method based on the gesture action, when the noise value of the current environment is less than the preset threshold, based on the sound wave frequency curve corresponding to the infrasound wave signal generated by the gesture action made by the user, generates a corresponding control instruction to play control on the multimedia playing terminal, avoiding The use of buttons or voice control causes noise effects on others, and also avoids inaccurate touch screen control caused by sweat or water stains on the hands, improving control accuracy and accuracy.

DRAWINGS

In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the embodiments or the description of the prior art will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present invention, and those skilled in the art can obtain other drawings according to the structures shown in the drawings without any creative work.

1 is a schematic structural diagram of hardware of an embodiment of a multimedia playing terminal according to the present invention;

2 is a flowchart of an embodiment of a gesture-based control method according to the present invention;

3 is a specific flowchart of an embodiment of step S20 in FIG. 2;

4 is a specific flowchart of an embodiment of step S30a of FIG. 2;

FIG. 5 is a specific flowchart of an embodiment of step S50 in FIG. 2.

Description of the reference numerals:

[Correct according to Rule 91 09.11.2017]

The implementation, functional features, and advantages of the present invention will be further described in conjunction with the embodiments.

Detailed ways

It is understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

In the following description, the use of suffixes such as "module", "component" or "unit" for indicating an element is merely an explanation for facilitating the present invention, and has no specific meaning per se. Therefore, "module", "component" or "unit" can be used in combination.

The terminal can be implemented in various forms. For example, the terminal described in the present invention may include, for example, a mobile phone, a tablet, a notebook computer, a palmtop computer, a personal digital assistant (Personal) Digital Assistant, PDA), Portable Media Player (Portable Media) Players, PMPs, navigation devices, wearable devices, smart bracelets, pedometers and other multimedia playback terminals, as well as fixed terminals such as digital TVs, desktop computers.

In the following description, a multimedia playback terminal will be described as an example, and those skilled in the art will understand that the configuration according to an embodiment of the present invention can be applied to a terminal of a fixed type in addition to an element particularly for mobile purposes.

FIG. 1 is a schematic structural diagram of hardware of a multimedia playing terminal according to various embodiments of the present invention. The multimedia playing terminal 100 may include: RF (Radio) Frequency, radio frequency unit 102, WiFi module 102, audio output unit 103, A/V (audio/video) input unit 104, sensor 105, display unit 106, user input unit 107, interface unit 108, memory 109, processor 110 And power supply 111 and other components. It can be understood by those skilled in the art that the structure of the multimedia playing terminal shown in FIG. 1 does not constitute a limitation of the multimedia playing terminal, and the multimedia playing terminal may include more or less components than the illustrated, or combine some components, or Different parts are arranged.

The following describes the various components of the multimedia playback terminal in conjunction with FIG. 1:

The radio frequency unit 101 can be used for receiving and transmitting signals during the transmission and reception of information or during a call. Specifically, after receiving the downlink information of the base station, the radio frequency unit 101 processes the data to the processor 110. In addition, the uplink data is sent to the base station. In general, radio frequency unit 101 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low noise amplifier, a duplexer, and the like. In addition, the radio unit 101 can also communicate with the network and other devices through wireless communication. The above wireless communication may use any communication standard or protocol, including but not limited to GSM (Global System of Mobile communication, Global System for Mobile Communications, GPRS (General Packet Radio) Service, General Packet Radio Service), CDMA2000 (Code Division Multiple) Access2000, Code Division Multiple Access 2000), WCDMA (Wideband Code Division Multiple) Access, Wideband Code Division Multiple Access), TD-SCDMA (Time Division-Synchronous Code Division Multiple) Access, Time Division Synchronous Code Division Multiple Access), FDD-LTE (Frequency Division Duplexing-Long Term) Evolution, Frequency Division Duplex Long Term Evolution) and TDD-LTE (Time Division Duplexing-Long Term) Evolution, time-sharing duplex long-term evolution, etc.

WiFi is a short-range wireless transmission technology, and the multimedia playback terminal can help users to send and receive emails, browse web pages, and access streaming media through the WiFi module 102, which provides wireless broadband Internet access for users. Although FIG. 1 shows the WiFi module 102, it can be understood that it does not belong to the essential configuration of the multimedia playback terminal, and can be omitted as needed within the scope of not changing the essence of the invention.

The audio output unit 103 may receive the radio frequency unit 101 or the WiFi module 102 or in the memory 109 when the multimedia playback terminal 100 is in a call signal receiving mode, a call mode, a recording mode, a voice recognition mode, a broadcast receiving mode, and the like. The stored audio data is converted into an audio signal and output as sound. Moreover, the audio output unit 103 can also provide audio output (eg, call signal reception sound, message reception sound, etc.) related to a specific function performed by the multimedia playback terminal 100. The audio output unit 103 may include a speaker, a buzzer, and the like.

The A/V input unit 104 is for receiving an audio or video signal. The A/V input unit 104 can include a graphics processor (Graphics) Processing The unit, GPU 1041 and the microphone 1042, the graphics processor 1041 processes the image data of the still picture or video obtained by the image capturing device (such as a camera) in the video capturing mode or the image capturing mode. The processed image frame can be displayed on the display unit 106. The image frames processed by the graphics processor 1041 may be stored in the memory 109 (or other storage medium) or transmitted via the radio unit 101 or the WiFi module 102. The microphone 1042 can receive sound (audio data) via the microphone 1042 in an operation mode of a telephone call mode, a recording mode, a voice recognition mode, and the like, and can process such sound as audio data. The processed audio (voice) data can be converted to a format output that can be transmitted to the mobile communication base station via the radio unit 101 in the case of a telephone call mode. Microphone 1042 can implement various types of noise cancellation (or suppression) algorithms to cancel (or suppress) noise or interference generated during the process of receiving and transmitting audio signals.

The multimedia playback terminal 100 also includes at least one type of sensor 105, such as a light sensor, motion sensor, and other sensors. Specifically, the light sensor includes an ambient light sensor and a proximity sensor, wherein the ambient light sensor can adjust the brightness of the display interface 1061 according to the brightness of the ambient light, and the proximity sensor can close the display interface 1061 when the multimedia play terminal 100 moves to the ear. And / or backlight. As a kind of motion sensor, the accelerometer sensor can detect the magnitude of acceleration in all directions (usually three axes). When it is stationary, it can detect the magnitude and direction of gravity. It can be used to identify the gesture of the mobile phone (such as horizontal and vertical screen switching, related Game, magnetometer attitude calibration), vibration recognition related functions (such as pedometer, tapping), etc.; as well as fingerprint sensors, pressure sensors, iris sensors, molecular sensors, gyroscopes, barometers, hygrometers, Other sensors such as thermometers and infrared sensors will not be described here.

The display unit 106 is for displaying information input by the user or information provided to the user. The display unit 106 can include a display interface 1061, which can employ a liquid crystal display (Liquid Crystal Display, LCD), Organic Light-Emitting The display interface 1061 is configured in the form of a Diode, OLED, or the like.

The user input unit 107 can be configured to receive input numeric or character information and to generate key signal inputs related to user settings and function control of the multimedia playback terminal. Specifically, the user input unit 107 may include a touch panel 1071 and other input devices 1072. The touch panel 1071, also referred to as a touch screen, can collect touch operations on or near the user (such as the user using a finger, a stylus, or the like on the touch panel 1071 or near the touch panel 1071. Operation), and drive the corresponding connecting device according to a preset program. The touch panel 1071 may include two parts of a touch detection device and a touch controller. Wherein, the touch detection device detects the touch orientation of the user, and detects a signal brought by the touch operation, and transmits the signal to the touch controller; the touch controller receives the touch information from the touch detection device, converts the touch information into contact coordinates, and sends the touch information. The processor 110 is provided and can receive commands from the processor 110 and execute them. In addition, the touch panel 1071 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic waves. In addition to the touch panel 1071, the user input unit 107 may also include other input devices 1072. Specifically, the other input device 1072 may include, but is not limited to, one or more of a physical keyboard, a function key (such as a volume control button, a switch button, and the like), a trackball, a mouse, a joystick, and the like. .

Further, the touch panel 1071 can cover the display interface 1061. After the touch panel 1071 detects a touch operation thereon or nearby, the touch panel 1071 transmits to the processor 110 to determine the type of the touch event, and then the processor 110 according to the touch event. The type provides a corresponding visual output on the display interface 1061. Although the touch panel 1071 and the display interface 1061 are used as two independent components to implement the input and output functions of the multimedia playback terminal, in some embodiments, the touch panel 1071 and the display interface 1061 may be used. Integration and implementation of the input and output functions of the multimedia playback terminal, which is not limited herein.

The interface unit 108 serves as an interface through which at least one external device can connect with the multimedia playback terminal 100. For example, the external device may include a wired or wireless headset port, an external power (or battery charger) port, a wired or wireless data port, a memory card port, a port for connecting a device having an identification module, and an audio input/output. (I/O) port, video I/O port, headphone port, and more. The interface unit 108 can be configured to receive input from an external device (eg, data information, power, etc.) and transmit the received input to one or more components within the multimedia play terminal 100 or can be used at the multimedia play terminal 100 Transfer data to and from external devices.

Memory 109 can be used to store software programs as well as various data. The memory 109 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application required for at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may be stored according to Data created by the use of the mobile phone (such as audio data, phone book, etc.). Further, the memory 109 may include a high speed random access memory, and may also include a nonvolatile memory such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The processor 110 is a control center of the multimedia playing terminal, and connects various parts of the entire multimedia playing terminal by using various interfaces and lines, by running or executing software programs and/or modules stored in the memory 109, and calling them in the memory 109. The data, performing various functions and processing data of the multimedia playing terminal, thereby performing overall monitoring on the multimedia playing terminal. The processor 110 may include one or more processing units; preferably, the processor 110 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, an application, etc., and performs modulation and demodulation. The processor primarily handles wireless communications. It can be understood that the above modem processor may not be integrated into the processor 110.

The multimedia playing terminal 100 may further include a power source 111 (such as a battery) for supplying power to various components. Preferably, the power source 111 may be logically connected to the processor 110 through a power management system to manage charging, discharging, and power consumption through the power management system. Management and other functions.

Although not shown in FIG. 1, the multimedia playing terminal 100 may further include a Bluetooth module and the like, and details are not described herein again.

The multimedia playing terminal of the embodiment of the present invention monitors the gesture action of the user, and acquires the secondary sound wave signal generated by the preset gesture action, and then determines the sound wave library, when the user makes a preset gesture action, based on the hardware structure of the multimedia play terminal. Whether there is a sound wave frequency curve matching the infrasound wave signal, and when there is a sound wave frequency curve matching the infrasound wave signal in the sound wave library, controlling the multimedia playing terminal to perform corresponding according to a control instruction corresponding to the sound wave frequency curve The playback operation avoids the noise influence on others when using the button or voice control, and also avoids the inaccurate touch screen control caused by sweat or water stain on the hand, and improves the control precision and accuracy.

As shown in FIG. 1, an operating system and a control program may be included in the memory 109 as a computer storage medium.

In the multimedia playing terminal 100 shown in FIG. 1, the WiFi module 102 is mainly used to connect to a background server or a big data cloud, and performs data communication with a background server or a big data cloud; the processor 110 can be used to call the control stored in the memory 109. Program and do the following:

Monitor user gestures;

Further, the processor 110 can also call the control program stored in the memory 109 to perform the following operations:

When the gesture action made by the user does not match the preset gesture action, an operation example corresponding to various control instructions is displayed on the display unit of the multimedia terminal.

When there is no acoustic wave frequency curve matching the infrasound wave signal in the sound wave library, the output prompts the repetitive gesture action.

The invention further provides a control method based on gesture action, which is applied to a multimedia playing terminal.

Referring to FIG. 2, FIG. 2 is a flowchart of an embodiment of a gesture-based control method according to the present invention.

In this embodiment, the gesture-based control method includes the following steps:

S10: a mapping relationship between a sound wave frequency curve generated by a preset gesture action and a control instruction, and storing a sound wave frequency curve of the secondary sound wave signal generated by different gesture actions in the sound wave library;

S20: monitoring user gestures;

S30a: when the user performs a preset gesture action, acquiring an infrasound wave signal generated by the gesture action;

S40: determining whether there is a sound wave frequency curve matching the infrasound wave signal in the sound wave library;

If yes, proceed to step S50;

S50: Control, according to the control instruction corresponding to the sound wave frequency curve, the multimedia playing terminal to perform a corresponding playing operation;

If not, proceed to step S60;

S60: Output a prompt to re-create the preset gesture action.

In this embodiment, the gesture-based control method is mainly used for playing control of a multimedia play terminal in a resting place or a quiet environment, and the multimedia play terminal may be a mobile phone, an audio player, or a video player in this embodiment. The control method is actually a silent control based on a gesture action, and uses a user's gesture action, such as a single wave gesture or two wave gestures to generate different infrasound signals, and corresponding to the acoustic wave frequency curve corresponding to different infrasound signals. The corresponding playback control command is preset to complete the corresponding playback operation. The infrasound wave comes from the air vibration caused by the gesture action, and the vibration is generally low in frequency and amplitude, and is not easily heard by the human ear, thereby achieving silent control. When the wave is single, the difference in the wave speed will produce the infrasound wave of different frequencies. When the wave is waved twice, the sound wave time map will have a more consistent curve at both ends. According to these features, different gesture actions can be distinguished to match the corresponding control commands.

In order to accurately recognize the gesture action of the user and thereby perform the accuracy of the playback control of the multimedia player terminal, it is necessary to preset the mapping relationship between the sound wave frequency curve generated by the gesture action and the control instruction, that is, what kind of gesture action will trigger what kind of gesture action Controlling operation, the acoustic wave frequency curve is a mechanical wave generated by a gesture action, and the mechanical wave is expressed in the form of an electrical signal, for example, a left-to-right single-handed action performed at the first speed is mapped to play the next multimedia file. The control command, of course, can also map the right-to-left single-wave gesture completed at the first speed to the control command for playing the next multimedia file according to the user's custom setting, and then the respective control commands corresponding to the multimedia playback terminal. The acoustic frequency curve of the infrasound wave signal generated by the gesture action is stored in the sound wave library, and the sound wave library may be a local database or a cloud database.

When the user carries the multimedia playing terminal into a resting place or a quiet environment, the original button or voice control mode may generate noise that affects others. Therefore, when the noise value of the current environment is detected to be less than a preset threshold, the user may start. The silent control mode monitors the behavior of the user, especially the gesture action of the user. The preset threshold is 15 dB. When the gesture action made by the user matches the preset gesture action, the user intention can be determined to control the multimedia playback terminal. Performing a certain playback operation, thereby controlling the secondary acoustic wave receiving device of the multimedia playing terminal to acquire the secondary acoustic wave signal generated by the gesture action, and the secondary acoustic wave receiving device converts the received secondary acoustic wave signal into an infrasonic digital electrical signal by performing A/D conversion. The sound frequency curve is processed by the processor of the multimedia playback terminal.

After converting the infrasound wave signal generated by the gesture action made by the user into the acoustic wave frequency curve, the processor of the multimedia playing terminal can continue to retrieve the acoustic wave frequency curve corresponding to various control commands in the sound wave library, and the sound wave library can be The local sound wave library existing in the local memory may also be a cloud sound wave library based on the cloud database, and when the sound wave frequency curve matching the sound wave frequency curve generated by the gesture action exists in the sound wave library, based on the sound wave frequency curve And a mapping relationship between the control instruction, generating a corresponding control instruction, and then controlling, according to the control instruction, the multimedia playing terminal to perform a playing operation in the control instruction; when the sound wave library is not generated with the gesture action When the acoustic frequency curve matched by the acoustic frequency curve is used, it can be inferred that there is a deviation between the gesture action made by the user and the preset gesture action, and the user needs to re-preset the preset gesture action, so as to perform the playback control of the multimedia playback terminal after performing the matching again. .

The gesture control-based control method of the embodiment of the present invention is used for the playback control of the multimedia play terminal, specifically: monitoring the gesture action of the user, and acquiring the infrasound generated by the preset gesture action when the user makes the preset gesture action a signal, and then determining whether there is an acoustic wave frequency curve matching the subsonic wave signal in the sound wave library, and when there is an acoustic wave frequency curve matching the subsonic wave signal in the acoustic wave library, controlling the sounding frequency curve according to a control instruction corresponding to the acoustic wave frequency curve The multimedia playing terminal performs a corresponding playing operation. The control method based on the gesture action, when the noise value of the current environment is less than the preset threshold, based on the sound wave frequency curve corresponding to the infrasound wave signal generated by the gesture action made by the user, generates a corresponding control instruction to play control on the multimedia playing terminal, avoiding The use of buttons or voice control has a noise effect on others, and also avoids inaccurate touch screen control caused by sweat or water stains on the hands, and also solves the problem that when the user cannot liberate his hands, the multimedia playback terminal cannot be used. The problem of playing control improves the control accuracy, accuracy, and convenience of controlling the multimedia playback terminal.

Presetting the sound wave frequency curve generated by the single-handed action from left to right at a first speed to map a control command for playing the next multimedia file; preset to complete the sound wave generated by the single-handed action from right to left at the first speed The frequency curve maps the control command of the previous multimedia file; presets the sound wave frequency curve mapping generated by the single wave motion from top to bottom to reduce the volume control command at the first speed; the preset is completed at the first speed The sound wave frequency curve generated by the single wave action on the top maps the control command for increasing the volume; presets the sound wave frequency curve generated by the set of wave motions at the first speed to map the control command of the current multimedia file; preset to the second Speed completes a set of sound wave frequency curve mapping generated by the wave action to close the currently playing control command; presets the first half group to complete the first half, and the second half completes the second half group wave action to generate the sound wave frequency curve map to pause the current play control instruction.

Wherein, the set of wave gestures may be a combination of a single wave gesture from left to right and a single wave gesture from right to left, or a single wave from right to left and a single from left to right. a waved combination of actions; the first half of the wave action may be a single wave gesture from left to right, or a single wave gesture from right to left; the second half of the wave gesture may be from right to left The single wave action may also be a single wave action from left to right; the first speed may be greater than the second speed or less than the second speed.

In addition, the mapping relationship between the acoustic frequency curve generated by the gesture operation and the control command in the present embodiment is only used as an example, and is not used to limit the mapping relationship. In other embodiments, the user may perform self-expression according to his own meaning. Define settings.

Further, referring to FIG. 3, based on the gesture action-based control method of the foregoing embodiment, step S20 specifically includes:

S21: prompting whether the output is switched to the silent control mode when detecting that the noise value of the current environment is less than a preset threshold;

S22: If yes, or does not receive feedback information within a preset duration, monitor the gesture action of the user.

In this embodiment, when the user carries the multimedia playing terminal into a resting place or a quiet environment, the noise of the current environment may be generated based on the original key or voice control mode, so that the noise value of the current environment is detected to be less than When the threshold is set, when the display unit of the multimedia play terminal is in the wake-up state, the prompt of whether to switch to the silent control mode is directly displayed on the display unit, and when the display unit of the multimedia play terminal is in the standby state, alignment is required. Wake-up, wake-up mode can be set according to the control property of the multimedia play terminal itself, such as by shaking the mobile phone or flipping the mobile phone to generate a wake-up command, waking up the display unit of the multimedia play terminal, and the user receiving the multimedia play terminal output on the display unit After the prompt of switching, you can choose to accept the control mode switching suggestion of the playback control terminal, or do not make any feedback. According to the built-in control program of the multimedia playback terminal, when no feedback information is received within the preset duration, the default user Accept The pre-play control mode is switched to the silent control mode, and corresponding control commands are generated, and the gestures of the user in the camera of the multimedia playback terminal are called to monitor, so that the infrasound receiving device of the multimedia playing terminal acquires other infrasound signals in the current environment, and plays the multimedia The terminal causes error control, which improves the accuracy and accuracy of the multimedia playback terminal control.

In another embodiment, when the user does not accept to switch the current play control mode to the silent control mode, the prompt for reducing the noise is further output, or the sensitivity of the current play control mode is increased, so that the user can quickly complete the corresponding play control operation. Or restricting the current play control authority until the noise value of the current environment is greater than the preset threshold, and the permission limit is released, so as to reduce the user's control operation and reduce noise.

Further, referring to FIG. 4, based on the gesture action-based control method of the foregoing embodiment, step S30a specifically includes:

S31a: acquiring an action parameter of a gesture action made by the user based on image feature extraction;

S32a: Acquire an infrasound wave signal generated by the gesture action when the action parameter matches an action parameter of the preset gesture action.

In this embodiment, in order to prevent the secondary acoustic wave receiving device of the multimedia playing terminal from acquiring other secondary acoustic signals of the current environment, and thus causing error control on the multimedia playing terminal, when monitoring the gesture action of the user, mainly relying on the multimedia playing terminal. The camera unit, for example, monitors the gesture action of the user by using a camera, the camera captures a multi-frame image of the gesture action of the user, and then processes the multi-frame image based on the image feature extraction to obtain the gesture action made by the user user. An action parameter, such as a speed, a direction, etc., and then matching the acquired action parameter of the gesture action with the action parameter of the preset gesture action, and when the action parameter of the gesture action matches the action parameter of the preset gesture action, generating The infrasound wave signal acquisition command acquires the infrasound wave signal generated by the gesture action. In other embodiments, after the camera is started, the infrasound wave receiving device may also be activated to monitor the gesture action while the gesture action is performed. Infrared wave letter Synchronization acquisition, in order to improve control efficiency.

Further, referring to FIG. 2, based on the gesture action-based control method of the above embodiment, the control method further includes step S30b,

S30b: When the gesture action made by the user does not match the preset gesture action, the display unit of the multimedia terminal displays an operation example corresponding to various control instructions.

In this embodiment, when the action parameter of the gesture action made by the user does not match the action parameter of the preset gesture action, it is determined that the gesture action made by the user does not conform to the preset specification, and the user needs to re-create the preset gesture action. a matching gesture action, in order to facilitate the user to quickly and accurately make a corresponding preset gesture action, display an operation example corresponding to various control instructions on the display unit of the multimedia terminal, so that the user can make corresponding according to the operation example according to the intention of the user. The gesture action matched with the preset gesture action, thereby improving the efficiency and accuracy of the playback control.

Further, referring to FIG. 5, based on the gesture action-based control method of the foregoing embodiment, step S50 includes:

S51: When there is a sound wave frequency curve matching the infrasound wave signal in the sound wave library, acquiring a control instruction corresponding to the sound wave frequency curve based on the mapping relationship;

S52: Display the control instruction on a display unit of the multimedia playing terminal, for the user to confirm;

S53: When receiving the confirmation instruction of the user, controlling, by the control instruction, the multimedia playing terminal to perform a play, pause, switch, volume adjustment, or shutdown operation.

In this embodiment, after converting the infrasound wave signal generated by the gesture action made by the user into the acoustic wave frequency curve, the processor of the multimedia playing terminal can continue to retrieve the acoustic wave frequency curve corresponding to various control commands in the sound wave library for matching. When there is a sound wave frequency curve matching the sound wave frequency curve generated by the gesture action in the sound wave library, generating a corresponding control instruction based on a mapping relationship between the sound wave frequency curve and the control instruction, and then the control command The display unit displayed on the multimedia playing terminal is provided for confirmation by the user, so as to improve the interaction between the multimedia playing terminal and the user, and also avoids the erroneous operation against the user's intention, and improves the accuracy of the playing control, and the user receives the When the control instruction is received, the control instruction may be confirmed or canceled according to the intention of the user. When receiving the confirmation instruction of the user, the multimedia playback terminal is controlled to perform playback, pause, switch, volume adjustment, according to the control instruction. Or turn off the currently playing control operation.

In addition, the embodiment of the present invention further provides a storage medium storing a control program, and the control program is executed by the processor to implement the steps of the gesture action-based control method as described above.

For the method that is implemented when the control program is executed, reference may be made to various embodiments of the control method based on the gesture action of the present invention, and details are not described herein again.

It is to be understood that the term "comprises", "comprising", or any other variants thereof, is intended to encompass a non-exclusive inclusion, such that a process, method, article, or It also includes other elements that are not explicitly listed, or elements that are inherent to such a process, method, item, or system. An element defined by the phrase "comprising a ..." does not exclude the presence of additional equivalent elements in a process, method, article, or system that includes the element, without further limitation.

The serial numbers of the embodiments of the present invention are merely for the description, and do not represent the advantages and disadvantages of the embodiments.

Through the description of the above embodiments, those skilled in the art can clearly understand that the foregoing embodiment method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be through hardware, but in many cases, the former is better. Implementation. Based on such understanding, portions of the technical solution of the present invention that contribute substantially or to the prior art may be embodied in the form of a software product stored in a storage medium (such as a ROM/RAM as described above). , a disk, an optical disk, including a number of instructions for causing a terminal device (which may be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to perform the methods described in various embodiments of the present invention.

The above are only the preferred embodiments of the present invention, and are not intended to limit the scope of the invention, and the equivalent structure or equivalent process transformations made by the description of the present invention and the drawings are directly or indirectly applied to other related technical fields. The same is included in the scope of patent protection of the present invention.

Claims

A control method based on gesture action is applied to a multimedia playing terminal, and the control method comprises the following steps:

Monitor user gestures;

When the user makes a preset gesture action, acquiring an infrasound wave signal generated by the gesture action;

Determining whether there is a sound wave frequency curve matching the infrasound wave signal in the sound wave library;

If yes, controlling the multimedia playing terminal to perform a corresponding playing operation according to a control instruction corresponding to the sound wave frequency curve.
The control method according to claim 1, wherein the control method further comprises: before performing all the steps:

The mapping relationship between the acoustic wave frequency curve generated by the preset gesture action and the control command is stored, and the acoustic wave frequency curve of the secondary acoustic wave signal generated by the different gesture actions is stored in the sound wave library.
The control method according to claim 2, wherein the step of mapping the sound wave frequency curve generated by the preset gesture action with the control instruction comprises:

Presetting the sound wave frequency curve generated by the single-handed action from left to right at a first speed to map a control instruction for playing the next multimedia file;

Presetting the sound wave frequency curve generated by the single-handed action from right to left at a first speed to map a control instruction for playing the previous multimedia file;

Presetting a control command for reducing the volume of the sound wave frequency curve generated by the single wave gesture from top to bottom at a first speed;

Presetting a control command for increasing the volume of the sound wave frequency curve generated by the single wave gesture from bottom to top at the first speed;

Presetting a sound wave frequency curve generated by a set of wave motions at a first speed to map a control instruction for playing the current multimedia file;

Presetting the sound wave frequency curve mapping generated by a set of wave motions at the second speed to close the currently playing control command;

The sound wave frequency curve mapping generated by the first half of the first speed and the second half of the second half of the wave motion is suspended to temporarily suspend the currently playing control command.
The control method according to claim 1 or 2, wherein the step of monitoring a gesture action of the user specifically includes:

When it is detected that the noise value of the current environment is less than a preset threshold, whether the output is switched to the prompt of the silent control mode;

If yes, or when no feedback information is received within the preset duration, the user's gesture action is monitored.
The control method according to claim 4, wherein the step of acquiring the secondary sound wave signal generated by the gesture action when the user performs the preset gesture action comprises:

Extracting an action parameter of a gesture action made by the user based on image feature extraction;

When the action parameter matches the action parameter of the preset gesture action, the secondary sound wave signal generated by the gesture action is acquired.
The control method according to claim 4, wherein when the gesture action made by the user does not match the preset gesture action, the display unit of the multimedia terminal displays an operation example corresponding to various control instructions.
The control method according to claim 2, wherein the step of controlling the multimedia playback terminal to perform a corresponding playback operation according to the control command corresponding to the sound wave frequency curve comprises:

When a sound wave frequency curve matching the infrasound wave signal exists in the sound wave library, acquiring a control instruction corresponding to the sound wave frequency curve based on the mapping relationship;

Displaying the control instruction on the display unit of the multimedia playing terminal for confirmation by the user;

When receiving the confirmation instruction of the user, the multimedia playback terminal is controlled to perform a play, pause, switch, volume adjustment, or shutdown operation according to the control instruction.
The control method according to claim 1 or 2, wherein when there is no acoustic wave frequency curve matching the infrasound wave signal in the sound wave library, the prompt to re-create the preset gesture action is output.
A multimedia playing terminal, comprising: a memory, a processor, and a control program stored in the memory and running on the processor, wherein the control program is implemented by the processor, such as The steps of the gesture-based control method according to any one of claims 1-8.
A storage medium, characterized in that the storage medium stores a control program, and when the control program is executed by the processor, the steps of the gesture-based control method according to any one of claims 1-8 are implemented.