CN113542606B - Shooting control method, shooting control device, electronic equipment and storage medium - Google Patents
Shooting control method, shooting control device, electronic equipment and storage medium Download PDFInfo
- Publication number
- CN113542606B CN113542606B CN202110795794.5A CN202110795794A CN113542606B CN 113542606 B CN113542606 B CN 113542606B CN 202110795794 A CN202110795794 A CN 202110795794A CN 113542606 B CN113542606 B CN 113542606B
- Authority
- CN
- China
- Prior art keywords
- sound source
- spatial position
- preset
- change
- shooting
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000000034 method Methods 0.000 title claims abstract description 56
- 238000003860 storage Methods 0.000 title claims abstract description 23
- 230000008859 change Effects 0.000 claims abstract description 96
- 238000001228 spectrum Methods 0.000 claims description 16
- 230000003595 spectral effect Effects 0.000 claims description 12
- 238000004458 analytical method Methods 0.000 claims description 8
- 238000012544 monitoring process Methods 0.000 description 29
- 230000006870 function Effects 0.000 description 21
- 230000005236 sound signal Effects 0.000 description 11
- 230000033001 locomotion Effects 0.000 description 10
- 238000010586 diagram Methods 0.000 description 8
- 230000000694 effects Effects 0.000 description 7
- 238000012545 processing Methods 0.000 description 7
- 230000001960 triggered effect Effects 0.000 description 7
- 238000005265 energy consumption Methods 0.000 description 6
- 230000008569 process Effects 0.000 description 6
- 230000009471 action Effects 0.000 description 5
- 238000012360 testing method Methods 0.000 description 5
- 238000013145 classification model Methods 0.000 description 4
- 210000005069 ears Anatomy 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 238000009826 distribution Methods 0.000 description 2
- 238000005315 distribution function Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 230000009466 transformation Effects 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004590 computer program Methods 0.000 description 1
- 230000008878 coupling Effects 0.000 description 1
- 238000010168 coupling process Methods 0.000 description 1
- 238000005859 coupling reaction Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 238000003384 imaging method Methods 0.000 description 1
- 230000000977 initiatory effect Effects 0.000 description 1
- 230000004807 localization Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 230000000873 masking effect Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 238000009877 rendering Methods 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 239000002699 waste material Substances 0.000 description 1
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
- H04N23/60—Control of cameras or camera modules
- H04N23/667—Camera operation mode switching, e.g. between still and video, sport and normal or high- and low-resolution modes
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/03—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 characterised by the type of extracted parameters
-
- G—PHYSICS
- G10—MUSICAL INSTRUMENTS; ACOUSTICS
- G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
- G10L25/00—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00
- G10L25/48—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use
- G10L25/51—Speech or voice analysis techniques not restricted to a single one of groups G10L15/00 - G10L21/00 specially adapted for particular use for comparison or discrimination
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Computational Linguistics (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- Human Computer Interaction (AREA)
- Physics & Mathematics (AREA)
- Acoustics & Sound (AREA)
- Circuit For Audible Band Transducer (AREA)
- Studio Devices (AREA)
Abstract
The application discloses a shooting control method, a shooting control device, electronic equipment and a storage medium, which are applied to the electronic equipment. The photographing control method includes: acquiring change information of the spatial position of at least one sound source in a recording signal based on the recording signal acquired by a microphone; and when the change information of the space position meets a preset change condition, entering a shooting mode. According to the method, the self-adaptive control of shooting is realized by tracking the spatial position change of external sound, so that shooting equipment is prevented from continuously shooting and recording, and the storage space is saved.
Description
Technical Field
The present application relates to the field of shooting technologies, and in particular, to a shooting control method, a device, an electronic apparatus, and a storage medium.
Background
At present, with the wide use of cameras in daily life, people have more and more demands for image and video shooting. For example, in a monitoring scene, the state of a certain area, the activities of persons, and the like are recorded/monitored by using a camera. Currently, in order to improve the monitoring effect, the camera is usually required to be always in an on state to continuously shoot the monitoring area, but the storage space is too occupied, the cost is too high, and the energy consumption is too high.
Disclosure of Invention
In view of the above problems, the present application provides a shooting control method, a shooting control device, an electronic apparatus, and a storage medium.
In a first aspect, an embodiment of the present application provides a photographing control method, including: acquiring change information of the spatial position of at least one sound source in a recording signal based on the recording signal acquired by a microphone; and when the change information of the space position meets a preset change condition, entering a shooting mode.
In a second aspect, an embodiment of the present application provides a photographing control apparatus, including: the position analysis module is used for acquiring the change information of the spatial position of at least one sound source in the recording signals based on the recording signals acquired by the microphone; and the shooting starting module is used for entering a shooting mode when the change information of the space position meets a preset change condition.
In a third aspect, an embodiment of the present application provides an electronic device, including: a microphone; one or more processors; a memory; one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the photographing control method provided in the first aspect.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having stored therein program code that is callable by a processor to perform the photographing control method provided in the first aspect described above.
According to the scheme provided by the application, based on the recorded signals collected by the microphone, the change information of the spatial position of at least one sound source in the recorded signals can be obtained, and when the change information of the spatial position meets the preset change condition, the shooting mode can be controlled. Therefore, the self-adaptive control of shooting is realized by tracking the spatial position change of external sound, continuous shooting and recording are avoided, the storage space is saved, and the shooting with low cost and low energy consumption is realized.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
Fig. 1 shows a flowchart of a photographing control method according to an embodiment of the present application.
Fig. 2 shows a flowchart of a photographing control method according to another embodiment of the present application.
Fig. 3 is a schematic diagram illustrating an arrangement of microphones in a photographing control method according to an embodiment of the present application.
Fig. 4 shows a flowchart of step S230 in a photographing control method according to another embodiment of the present application.
Fig. 5 illustrates a schematic diagram of sound modeling in a photographing control method according to an embodiment of the present application.
Fig. 6 shows an overall flowchart in a photographing control method according to another embodiment of the present application.
Fig. 7 shows a block diagram of a photographing control apparatus according to an embodiment of the present application.
Fig. 8 is a block diagram of an electronic apparatus for performing a photographing control method according to an embodiment of the present application.
Fig. 9 is a storage unit for storing or carrying program codes for implementing a photographing control method according to an embodiment of the present application.
Detailed Description
In order to enable those skilled in the art to better understand the present application, the following description will make clear and complete descriptions of the technical solutions according to the embodiments of the present application with reference to the accompanying drawings.
Referring to fig. 1, fig. 1 is a flowchart illustrating a shooting control method according to an embodiment of the application. In a specific embodiment, the photographing control method is applicable to a photographing control apparatus 700 as shown in fig. 7 and an electronic device (fig. 8) provided with the photographing control apparatus 700. The following will describe the flowchart shown in fig. 1 in detail, and the photographing control method may specifically include the following steps:
step S110: based on the recorded signals collected by the microphone, the change information of the spatial position of at least one sound source in the recorded signals is obtained.
In the embodiment of the application, the electronic equipment can be terminal equipment with a recording function, namely, the electronic equipment can carry a microphone with an audio acquisition function. The microphone may be one microphone or may be a microphone array formed by a plurality of microphones. The microphone array may include N microphones, where N is an integer greater than or equal to 2. Alternatively, the microphone array in the electronic device may be an array formed by arranging a group of omnidirectional microphones located at different positions in space according to a certain shape rule. Therefore, when the recording function of the electronic device is started, the microphone array (i.e., the N microphones) in the electronic device can collect the audio signals in the recording environment space, so as to obtain the recording signals. The sound signal may include sound signals emitted from one or more sound sources in the environment space. Alternatively, the sound source may be a person speaking, or may be other physical devices capable of speaking, such as an automobile for sounding a whistle.
It will be appreciated that in embodiments of the present application, at least one sound source may be present in the monitored environment. When a plurality of sound sources exist in the environment, sound emitted by each sound source is integrated to form a recording signal collected by the microphone, that is, the recording signal collected by the microphone is the sum of audio signals corresponding to the sound emitted by the plurality of sound sources.
In the embodiment of the application, before the current monitoring environment needs to be subjected to image shooting or video recording, a microphone monitoring state can be firstly entered so as to track at least one sound source in the current monitoring environment. Specifically, when the monitoring function of the electronic device is started, the camera can not be directly started to shoot images or record videos, and the recording function of the electronic device can be started first, so that the microphone in the electronic device can acquire and record audio signals in the current environment space, and the electronic device can acquire the recording signals acquired by the microphone.
In some embodiments, in order to accurately track the sound source in the current monitoring environment, the electronic device may acquire the sound recording signal in the current monitoring environment by using an external microphone, where the external microphone may be an independent sound recording device or may be a microphone in another terminal device, which is not limited herein. It can be appreciated that in some application scenarios, the microphone built-in the electronic device may have an undesirable microphone recording effect due to a manufacturing process or cost saving, so that accuracy of tracking the sound source in the current environment can be improved through the external microphone with a better recording effect.
In some embodiments, it may be first determined whether the monitoring function of the electronic device is started, and if the monitoring function of the electronic device is started, only the recording function of the electronic device may be started first, and the related operation of S110 is performed. Alternatively, it may be determined that the monitoring function of the electronic device has been started after receiving the monitoring function call instruction input by the user. The monitoring function calling instruction may be triggered after detecting a virtual key in a display screen of the electronic device touched by a user, for example, when the electronic device is a mobile terminal such as a mobile phone or a smart watch, the user may trigger the virtual key of the monitoring function displayed on the touch setting interface, or may be triggered after detecting a preset track or multi-finger sliding of the user on the display screen, for example, the user may call out the monitoring function through a preset four-finger sliding gesture. As a further way, the monitoring function call instruction may also be triggered after detecting that the user presses a physical key on the electronic device for starting recording, for example, when the electronic device is a stand-alone camera device, the user may trigger the monitoring function by pressing a switch button of the camera device.
In the embodiment of the application, after the monitoring function of the electronic equipment is started, the microphone in the electronic equipment can be controlled to collect and record the audio signal in the current environment space, so that the electronic equipment can acquire the change information of the space position of at least one sound source in the recording signal based on the recording signal collected by the microphone. The spatial position of the sound source may include information such as distance, azimuth, and orientation of the sound source, and the fluctuation information of the spatial position may include information such as a fluctuation direction and a fluctuation speed of the spatial position.
It will be appreciated that the loudness of sound emitted by the sound sources gradually decreases with increasing distance and gradually increases with increasing distance, and thus, in some embodiments, the electronic device may determine, after acquiring the recording signal acquired by the microphone, change information of the spatial position of at least one sound source in the recording signal according to the change of the loudness of sound of at least one sound source in the recording signal.
In some embodiments, when the electronic device includes two microphones, the binaural effect may be used to determine the spatial position of at least one sound source in the recording signal, and then the recording signal is obtained by continuous recording, and the front-back variation of the spatial position of at least one sound source in the recording signal may be determined, so as to obtain the variation information of the spatial position of at least one sound source in the recording signal. The binaural effect is a spatial localization technique, for example, by people, where the ears of the person are symmetrically distributed on both sides of the head, and the auricles and the head of the person play an effective role in masking the sound. When the time and frequency intensity distribution of the sound source sent to the ears of the human are different, the same sound source can be sent to the ears with obvious time difference and intensity difference, and the phenomenon can clearly and accurately judge the position of the sound source, namely 'binaural effect'.
In some embodiments, when the electronic device includes three microphones or a microphone array formed by more than three microphones, based on the recorded signals collected by the microphones, a time difference of arrival of the sound signal of each sound source at the plurality of microphones may be obtained, so as to obtain a time difference of arrival (TimeDifferenceofArrival, TODA) of each sound signal, and a spatial position of the corresponding sound source relative to the electronic device is obtained through the time difference of arrival. And then continuously recording to obtain a recording signal, and determining the front-back variation of the spatial position of at least one sound source in the recording signal, thereby obtaining the variation information of the spatial position of at least one sound source in the recording signal. Alternatively, the phase difference and the amplitude difference of the sound signal of each sound source reaching the plurality of microphones may be obtained to locate the spatial position of at least one sound source in the recording signal, so as to obtain the change information of the spatial position of at least one sound source. The specific manner is not limited in the embodiments of the present application.
Step S120: and when the change information of the space position meets a preset change condition, entering a shooting mode.
In the embodiment of the application, when the change information of the spatial position of at least one sound source in the recording signal is obtained, whether the change information of the spatial position of at least one sound source meets the preset change condition can be judged, and when the change information of the spatial position of at least one sound source meets the preset change condition, a shooting mode can be formally entered, namely a camera is started to shoot images or record videos. Therefore, when the audio source with the preset motion attribute appears in the current monitoring environment, the shooting action is automatically triggered, so that video and pictures of the audio source with the preset motion attribute can be shot, the storage space and energy waste caused by continuous shooting and recording of shooting equipment are avoided, and the self-adaptive control of shooting is realized.
In some embodiments, when the change information of the spatial position of each sound source in the recording signal does not meet the preset change condition, the current monitored scene may be considered to have no sound source with the preset motion attribute, and the shooting action may not be triggered, i.e. the closed state of the camera may be maintained. The sound source with the preset motion attribute can be an object to be monitored in the current monitoring scene, such as a person, a pet, a vehicle and the like, so as to realize safety monitoring.
In some embodiments, the preset change condition may be one or more. When the preset change conditions are plural, the shooting mode may be entered when the change information of the spatial position satisfies all the preset change conditions, or the shooting mode may be entered when the change information of the spatial position satisfies any or any preset number of preset change conditions, which is not limited herein.
According to the shooting control method provided by the embodiment of the application, based on the recording signals collected by the microphone, the change information of the spatial position of at least one sound source in the recording signals can be obtained, and when the change information of the spatial position meets the preset change condition, the shooting mode can be controlled. Therefore, the self-adaptive control of shooting is realized by tracking the spatial position change of external sound, continuous shooting and recording are avoided, the storage space is saved, and the shooting with low cost and low energy consumption is realized.
Referring to fig. 2, fig. 2 is a flowchart illustrating a shooting control method according to another embodiment of the present application. The following will describe the flowchart shown in fig. 2 in detail, and the photographing control method may specifically include the following steps:
step S210: based on the recorded signals collected by the microphone, the spectrum characteristics of the recorded signals are obtained.
In some embodiments, after acquiring the recording signal acquired by the microphone, the electronic device may acquire a spectral feature of the recording signal, so as to obtain a spatial position of at least one sound source in the recording signal according to the spectral feature analysis. As one way, fourier transformation may be performed on the recorded signal collected by the microphone, so as to convert the recorded signal from the time domain to the frequency domain, and obtain the spectral feature of the recorded signal. Wherein the spectral characteristics include frequency, amplitude, envelope, energy, etc., and are not limited herein.
In some embodiments, the electronic device may include at least three microphones, which may be distributed on a non-identical plane of the electronic device, i.e., spatially distributed, so that phase and amplitude differences between the same sound source to the respective microphones may be perceived. Specifically, the spectral characteristics of the recording signals of each of the at least three microphones may be obtained based on the recording signals collected by the at least three microphones, so as to analyze and locate the spatial position of the same sound source according to the spectral characteristic difference of the recording signals of the same sound source corresponding to different microphones. Fig. 3 is a schematic layout diagram of microphones in an electronic device according to an embodiment of the present application, and as shown in fig. 3, three microphones in three-dimensional distribution are provided on the electronic device, which are denoted as a, b, and c respectively.
Step S220: and according to the frequency spectrum characteristics, determining the audio characteristics of at least one sound source in the recording signal, wherein the audio characteristics are used for representing the sound source in the recording signal.
In some embodiments, after the spectral feature of the recording signal is obtained, the audio feature of at least one sound source in the recording signal may be extracted according to the spectral feature, where the audio feature may be used to characterize the sound source in the recording signal, so that the sound source may be uniquely identified by the audio feature, and further, in the continuous recording process, spatial position variation of the sound source with the same audio feature may be continuously compared.
In some embodiments, since noise signals may exist in the current monitoring environment or sound signals with weak loudness that do not have analytical significance, after the recording signal is converted from the time domain to the frequency domain, useful frequency signals can be screened for analysis based on the energy of each frequency of the frequency domain signal. Alternatively, only frequency signals higher than the average energy can be analyzed, and the more the frequency signals exceed the average energy, the higher the weight is, so that the analysis difficulty and the workload can be reduced.
In some embodiments, the frequency, amplitude, envelope and other features of the sound signals of multiple sound sources can be learned in advance through a machine learning technology to obtain a classification model, so that the learned classification model can be utilized to classify the features of the sound. Specifically, after the spectrum features of the recording signal are obtained, the spectrum features are input into a classification model, and the audio features of at least one sound source in the recording signal can be extracted by using the classification model.
Step S230: and acquiring the change information of the spatial position of the at least one sound source in the recording signal based on the audio characteristics.
In some embodiments, after extracting the audio feature of at least one sound source in the recording signal, spatial variation information of the at least one sound source in the recording signal may be obtained based on the audio feature. Specifically, based on the audio feature, the spatial position of the audio source with the audio feature can be located from the recording signal, so that the difference of the spatial positions before and after agreeing to the audio source can be compared by continuously recording (for example, the interval of 0.125ms is 8KHz sampling rate) and continuously locating the spatial position, and the variation information of the spatial positions of the audio sources in the recording signal can be obtained.
In some embodiments, at least three microphones may be included on the electronic device such that the spatial distance and angle of a sound source of the same audio feature from the electronic device may be determined by comparing the phase and amplitude differences of the sound source of the same audio feature to each microphone. Specifically, referring to fig. 4, step S230 may also include:
step S231: based on the audio characteristics, a difference in corresponding amplitude and/or a difference in phase of the same sound source in the spectral characteristics corresponding to each two microphones of the at least three microphones is calculated.
Step S232: and determining the change information of the spatial position of the at least one sound source in the recording signal according to the amplitude difference and/or the phase difference corresponding to the at least one sound source.
In some embodiments, where at least three microphones are included on an electronic device, the spatial location of the sound source may be subsequently located according to a pre-established sound space model. Specifically, a fixed sound source for testing can be placed in one direction relative to the electronic device in the recording test environment, sound sources are not placed in other directions, recording signals of the fixed sound source are obtained through at least three microphones on the electronic device, and then fourier transformation is carried out on the recording signals corresponding to each of the at least three microphones respectively to obtain corresponding frequency spectrum data. Likewise, a fixed sound source for testing is also placed in another direction relative to the electronic device, and no sound source is placed in the other direction, and then the spectral data of the recorded signal corresponding to each of the at least three microphones is acquired. And repeating the process, and placing a fixed sound source for testing in a plurality of directions relative to the electronic equipment to obtain frequency spectrum data of the recording signals corresponding to each microphone under each sound source point, so that according to the frequency spectrum data, the difference between the amplitude values and/or the phase differences of at least three microphones from different sound source points can be obtained, and a space distribution function of the sound source can be constructed, so that a sound space model of at least three microphones on the electronic equipment can be obtained. Alternatively, if the electronic device is located at the origin of the X, Y, Z axes, the single point stationary sound source may be placed at different points of X, Y, Z, at least 8 positions, to ensure that there is one test sound source point for each axis, both positive and negative. For example, referring to fig. 5, the electronic device is located at the X, Y, Z axis origin, and the single point sound source is placed at a different point X, Y, Z.
It can be understood that after the spatial distribution function of the sound source is obtained, the phase difference and the amplitude difference of the same point sound source at each microphone can be compared, so that the sound source can be positioned in which direction, the phase difference between the same characteristic sound source and each microphone can be also compared, and the spatial distance between the point sound source and the electronic equipment can be obtained by combining the wavelength of the frequency. The spatial position of the same characteristic sound source can be continuously compared, and the fluctuation information such as the fluctuation direction, the fluctuation speed and the like of the spatial position of the sound source can be further positioned.
Based on the above, when the electronic device includes at least three microphones, the sound space model can be used to locate at least one sound source in the recording signal, so that the spatial position information of each sound source in the recording signal relative to the electronic device can be obtained, and the sound source tracking of the current monitoring environment is realized. It can be understood that when the spatial position of the audio source changes, the recorded recording signal recorded by the microphone also changes, so that the change information of the spatial position of each audio source relative to the electronic device can be determined by continuously recording the recorded recording signal. Specifically, if the position of the sound source is changed, the change information of the spatial position of the sound source can be obtained from the spatial position information before and after the change.
Specifically, the recording signals obtained by recording by at least three microphones can be obtained, then the fourier transform is performed on the recording signals corresponding to each microphone in the at least three microphones respectively to obtain the frequency spectrum characteristics of the recording signals corresponding to each microphone, then the same sound source, namely the same audio characteristic, is calculated based on the audio characteristics, and the difference of the corresponding amplitude and/or the difference of the phase in the frequency spectrum characteristics corresponding to each two microphones in the at least three microphones can be combined with the sound space model to obtain the space distance and the angle between the sound source of the same audio characteristic and the electronic equipment, so that the space position of the sound source of the same audio characteristic can be positioned. Thus, by continuously recording and locating the sound sources of the same audio feature, the change information of the spatial position of the at least one sound source in the recording signal can be analyzed.
Step S240: and when the change information of the space position meets a preset change condition, entering a shooting mode.
In some embodiments, the spatial position change information may include a spatial position change direction. Therefore, when the direction of the change in the spatial position of at least one sound source in the recording signal is a preset direction, the shooting mode may be entered. Thereby realizing automatic shooting of the object in a specific motion direction.
It can be understood that, because the electronic device does not know which audio feature source will move in a specific direction, in the embodiment of the present application, by recording the recording signal of the current monitoring environment, the sound in the current monitoring environment is categorized, and the audio feature source is tracked, so that once a sound source with a specific movement direction is detected, the shooting action can be triggered.
In some embodiments, the preset direction may be a specific direction angle, or may be a direction angle interval, that is, there is a tolerable deviation. For example, the left to right is a preset direction (horizontal angle 0 degrees), the tolerable deviation is horizontal ±10 degrees, and when the sound variation direction of a certain sound source satisfies the left to right, but the horizontal direction is 3 degrees, the sound variation direction of the sound source may be determined to be the preset direction. It will be appreciated that angles exceeding 10 are not considered a preset direction.
In other embodiments, the spatial position change information may also include a spatial position change speed. Therefore, the shooting mode may be entered when the speed of fluctuation of the spatial position of at least one sound source in the recording signal is a preset speed. Thereby realizing automatic shooting of the object with specific movement speed. The preset speed may be a fixed speed value, or may be a speed interval or a speed level, for example, a sound source greater than a certain speed threshold may be considered overspeed.
It can be understood that, because the electronic device does not know which audio feature source will move along a specific speed, in the embodiment of the present application, by recording the recording signal of the current monitoring environment, the sound in the current monitoring environment is categorized, and the audio feature source is tracked, so that once a sound source with a specific movement speed is detected, the shooting action can be triggered.
In some embodiments, after the electronic device enters the shooting mode, it may be detected whether a shooting closing instruction is received. The electronic device may automatically generate a shooting closing instruction, and as a way, may generate a shooting closing instruction when the shooting time length reaches a preset time length, so that the electronic device may respond to the shooting closing instruction and exit the shooting mode, that is, close the camera to stop shooting images or record videos. Therefore, shooting is realized to exceed the preset time length, the current shooting is automatically exited and the next trigger is waited.
The preset duration may be pre-stored in the electronic device, and may be set reasonably according to practical application, which is not limited herein. For example, 15 seconds, 1 minute, or the like may be used. In some embodiments, the preset duration may also be determined in real time according to the motion characteristics of the sound source. As one method, the predetermined time period may be determined based on the speed of fluctuation of the spatial position of the sound source. Alternatively, the faster the speed of variation, the shorter the preset time period may be. It can be understood that if the moving speed of the sound source is too fast, the preset time period is too long, the electronic device is likely to capture a large number of useless images, and if the moving speed of the sound source is too slow, the preset time period is too short, and the electronic device is likely to miss capturing some available images. Therefore, the preset shooting duration can be adaptively adjusted according to the change speed of the sound source, so that unnecessary energy consumption is avoided, and enough useful images can be shot.
Alternatively, when the loudness of the recording signal is less than the preset threshold, a shooting closing instruction may be generated, so that the electronic device may respond to the shooting closing instruction and exit the shooting mode. The loudness may be understood as the volume, which may be related to the amplitude of vibration of the sound source, the greater the loudness of the recording signal, and may be related to the distance of the sound source from the electronic device, the greater the distance, the lesser the loudness of the recording signal. Thus, the recording signal is too weak (the sound is too small, such as in a quiet environment), the current shooting is automatically exited and the next trigger is waited for. The preset threshold may be pre-stored in the electronic device, and may be set reasonably according to practical application, which is not limited herein. It will be appreciated that when the loudness of the recorded signal is too small, the sound source may be considered to be far from the electronic device, and the electronic device may not be able to capture a clear and useful image, so that the camera may be directly turned off to stop capturing, so as to save storage space and energy consumption.
Referring to fig. 6, fig. 6 shows an overall flowchart of a photographing control method provided by the present application. Specifically, after the recording device is started, the camera is not directly opened, but is in a microphone monitoring state, so that frequency domain feature analysis is performed on recorded recording signals, the spatial position of at least one sound source in the recording signals is positioned, then the change of the spatial position of the sound source is analyzed, when the sound source signals with specific movement directions appear, the camera can be opened to execute shooting actions, and once the recording signals are lower than a threshold value or the shooting duration exceeds a set time, the current shooting mode is exited and the next triggering is waited.
In some embodiments, the camera on the electronic device may be a normal camera or a rotatable camera with a wider shooting area. In one mode, when the camera on the electronic device is a rotatable camera, the electronic device can adjust the rotation angle of the camera according to the current spatial position of the sound source after tracking the sound source with the preset variation characteristic and entering a shooting mode, so that the shooting area of the camera can be aligned to the current spatial position of the sound source, and the sound source with the preset variation characteristic can be accurately shot. Alternatively, the method may also be a tracked sound source with preset variation characteristics, and the variation track of the spatial position of the sound source is predicted, so that the rotation angle of the camera can be synchronously adjusted according to the variation track, so that the camera can continuously track the sound source with the preset variation characteristics.
As one mode, the rotation angle of the camera may be adjusted to align the imaging area of the camera only when there is one sound source with a preset variation characteristic, or the spatial position variation trajectory of the sound source may be predicted only when there is one sound source with a preset variation characteristic. When there are a plurality of sound sources conforming to the preset variation characteristics, the rotation angle of the camera can not be adjusted so as to track and shoot the plurality of sound sources. Optionally, if the spatial positions of the audio sources with the preset variation characteristics are adjacent, and the predicted variation tracks are also consistent, the rotation angle of the camera may be synchronously adjusted according to the variation tracks. The method is not limited herein, and it is only necessary to ensure that all sound sources meeting preset variation characteristics of the current environment can be shot, and then determine whether to perform subsequent tracking shooting according to actual conditions.
According to the shooting control method provided by the embodiment of the application, based on the recording signals collected by the microphone, the change information of the spatial position of at least one sound source in the recording signals can be obtained, and when the change information of the spatial position meets the preset change condition, the shooting mode can be controlled. Therefore, the self-adaptive control of shooting is realized by tracking the spatial position change of external sound, continuous shooting and recording are avoided, the storage space is saved, and the shooting with low cost and low energy consumption is realized. Image capturing or video recording of specific scenes and objects is achieved.
Referring to fig. 7, a block diagram of a shooting control apparatus 700 according to an embodiment of the application is shown and applied to an electronic device. The photographing control apparatus 700 includes: the position analysis module 710 is configured to obtain, based on a recording signal collected by a microphone, variation information of a spatial position of at least one sound source in the recording signal; the shooting start module 720 is configured to enter a shooting mode when the change information of the spatial position meets a preset change condition.
In some embodiments, the location analysis module 710 may include: the frequency domain conversion unit is used for acquiring the frequency spectrum characteristics of the recording signal based on the recording signal acquired by the microphone; the characteristic extraction unit is used for determining the audio characteristic of at least one sound source in the recording signal according to the frequency spectrum characteristic, and the audio characteristic is used for representing the sound source in the recording signal; and the information acquisition unit is used for acquiring the change information of the spatial position of the at least one sound source in the recording signal based on the audio characteristics.
In some embodiments, the electronic device comprises at least three microphones, and the frequency domain conversion unit may be specifically configured to: and acquiring the frequency spectrum characteristics of the recording signals of each of the at least three microphones based on the recording signals acquired by the at least three microphones. The information acquisition unit may be specifically configured to: calculating the difference of the corresponding amplitude values and/or the difference of the phases of the same sound source in the frequency spectrum characteristics corresponding to each two microphones in the at least three microphones based on the audio characteristics; and determining the change information of the spatial position of the at least one sound source in the recording signal according to the amplitude difference and/or the phase difference corresponding to the at least one sound source.
In some embodiments, the change information of the spatial position includes a change direction of the spatial position, and the shooting initiation module 720 may specifically be configured to: and when the change direction of the space position is a preset direction, entering a shooting mode.
In some embodiments, the change information of the spatial position includes a change speed of the spatial position, and the shooting start module 720 may specifically be configured to: and when the fluctuation speed of the space position is a preset speed, entering a shooting mode.
In some embodiments, the photographing control apparatus 700 may further include: and the first closing module is used for exiting the shooting mode when the shooting time length reaches the preset time length.
In other embodiments, the photographing control apparatus 700 may further include: and the second closing module is used for exiting the shooting mode when the loudness of the sound recording signal is smaller than a preset threshold value.
It will be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the apparatus and modules described above may refer to the corresponding process in the foregoing method embodiment, which is not repeated herein.
In several embodiments provided by the present application, the coupling of the modules to each other may be electrical, mechanical, or other.
In addition, each functional module in each embodiment of the present application may be integrated into one processing module, or each module may exist alone physically, or two or more modules may be integrated into one module. The integrated modules may be implemented in hardware or in software functional modules.
In summary, the photographing control apparatus provided in the embodiment of the present application is configured to implement the corresponding photographing control method in the foregoing method embodiment, and has the beneficial effects of the corresponding method embodiment, which is not described herein again.
Referring to fig. 8, a block diagram of an electronic device according to an embodiment of the present application is shown. The electronic device 100 may refer to a mobile phone, a tablet computer, a wearable device, a camera device, and other terminal devices capable of running an application program. The electronic apparatus 100 may also be a third party device for controlling the start-up or shut-down of the image pickup apparatus. The electronic device 100 of the present application may include one or more of the following components: processor 110, memory 120, microphone 130, and one or more applications, wherein the one or more applications may be stored in memory 120 and configured to be executed by the one or more processors 110, the one or more applications configured to perform the method as described in the foregoing method embodiments. The microphone 130 may be one or more microphones, and in the case of multiple microphones, may be an array of microphones.
Processor 110 may include one or more processing cores. The processor 110 utilizes various interfaces and lines to connect various portions of the overall electronic device 100, perform various functions of the electronic device 100, and process data by executing or executing instructions, programs, code sets, or instruction sets stored in the memory 120, and invoking data stored in the memory 120. Alternatively, the processor 110 may be implemented in hardware in at least one of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), programmable logic array (Programmable Logic Array, PLA). The processor 110 may integrate one or a combination of several of a central processing unit (Central Processing Unit, CPU), a photographing controller (Graphics Processing Unit, GPU), and a modem, etc. The CPU mainly processes an operating system, a user interface, an application program and the like; the GPU is used for being responsible for rendering and drawing of display content; the modem is used to handle wireless communications. It will be appreciated that the modem may not be integrated into the processor 110 and may be implemented solely by a single communication chip.
The Memory 120 may include a random access Memory (Random Access Memory, RAM) or a Read-Only Memory (Read-Only Memory). Memory 120 may be used to store instructions, programs, code, sets of codes, or sets of instructions. The memory 120 may include a stored program area and a stored data area, wherein the stored program area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playing function, an image playing function, etc.), instructions for implementing the various method embodiments described below, etc. The storage data area may also store data created by the electronic device 100 in use (e.g., phonebook, audiovisual data, chat log data), and the like.
It is understood that the configuration shown in fig. 8 is merely an example, and that electronic device 100 may also include more or fewer components than shown in fig. 8, or have a completely different configuration than shown in fig. 8. The embodiment of the present application is not limited thereto.
Referring to fig. 9, a block diagram of a computer readable storage medium according to an embodiment of the present application is shown. The computer readable medium 800 has stored therein program code which can be invoked by a processor to perform the methods described in the method embodiments described above.
The computer readable storage medium 800 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 800 comprises a non-volatile computer readable medium (non-transitory computer-readable storage medium). The computer readable storage medium 800 has storage space for program code 810 that performs any of the method steps described above. The program code can be read from or written to one or more computer program products. Program code 810 may be compressed, for example, in a suitable form.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present application, and are not limiting; although the application has been described in detail with reference to the foregoing embodiments, it will be appreciated by those of ordinary skill in the art that: the technical scheme described in the foregoing embodiments can be modified or some technical features thereof can be replaced by equivalents; such modifications and substitutions do not drive the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims (9)
1. A photographing control method, characterized in that the method comprises:
acquiring change information of the spatial position of at least one sound source in a recording signal based on the recording signal acquired by a microphone;
when the change information of the space position meets a preset change condition, entering a shooting mode, wherein the change information meets the preset change condition and comprises a change direction being a preset direction or a change speed being a preset speed;
and when the shooting time length reaches a preset time length, exiting the shooting mode, wherein the preset time length is determined in real time based on the change speed of the spatial position of the sound source.
2. The method of claim 1, wherein the acquiring, based on the recorded signals collected by the microphone, variation information of the spatial position of at least one sound source in the recorded signals includes:
acquiring the frequency spectrum characteristics of a recording signal based on the recording signal acquired by a microphone;
determining an audio feature of at least one sound source in the recorded signal according to the spectral feature, the audio feature being used to characterize the sound source in the recorded signal;
and acquiring the change information of the spatial position of the at least one sound source in the recording signal based on the audio characteristics.
3. The method of claim 2, wherein the electronic device comprises at least three microphones, the acquiring spectral features of the recorded signals based on the recorded signals acquired by the microphones, comprising:
acquiring the frequency spectrum characteristics of the recording signals of each of the at least three microphones based on the recording signals acquired by the at least three microphones;
the obtaining, based on the audio feature, change information of a spatial position of the at least one sound source in the recording signal includes:
calculating the difference of the corresponding amplitude values and/or the difference of the phases of the same sound source in the frequency spectrum characteristics corresponding to each two microphones in the at least three microphones based on the audio characteristics;
and determining the change information of the spatial position of the at least one sound source in the recording signal according to the amplitude difference and/or the phase difference corresponding to the at least one sound source.
4. The method according to claim 1, wherein the change information of the spatial position includes a change direction of the spatial position, and the entering into the photographing mode when the change information of the spatial position satisfies a preset change condition includes:
and when the change direction of the space position is a preset direction, entering a shooting mode.
5. The method according to claim 1, wherein the change information of the spatial position includes a change speed of the spatial position, and the entering into the photographing mode when the change information of the spatial position satisfies a preset change condition includes:
and when the fluctuation speed of the space position is a preset speed, entering a shooting mode.
6. The method according to any one of claims 1 to 5, wherein after entering a shooting mode when the change information of the spatial position satisfies a preset change condition, the method further comprises:
and when the loudness of the sound recording signal is smaller than a preset threshold value, exiting the shooting mode.
7. A photographing control apparatus, characterized in that the apparatus comprises:
the position analysis module is used for acquiring the change information of the spatial position of at least one sound source in the recording signals based on the recording signals acquired by the microphone;
the shooting starting module is used for entering a shooting mode when the change information of the space position meets a preset change condition, wherein the change information meets the preset change condition and comprises a change direction being a preset direction or a change speed being a preset speed;
and the first closing module is used for exiting the shooting mode when the shooting time length reaches a preset time length, wherein the preset time length is determined in real time based on the change speed of the spatial position of the sound source.
8. An electronic device, comprising:
a microphone;
one or more processors;
a memory;
one or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, the one or more applications configured to perform the method of any of claims 1-6.
9. A computer readable storage medium, characterized in that the computer readable storage medium has stored therein a program code, which is callable by a processor for executing the method according to any one of claims 1-6.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110795794.5A CN113542606B (en) | 2021-07-14 | 2021-07-14 | Shooting control method, shooting control device, electronic equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110795794.5A CN113542606B (en) | 2021-07-14 | 2021-07-14 | Shooting control method, shooting control device, electronic equipment and storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN113542606A CN113542606A (en) | 2021-10-22 |
| CN113542606B true CN113542606B (en) | 2023-10-10 |
Family
ID=78127916
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110795794.5A Active CN113542606B (en) | 2021-07-14 | 2021-07-14 | Shooting control method, shooting control device, electronic equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113542606B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115942017B (en) * | 2022-10-27 | 2024-02-02 | 深圳市台电实业有限公司 | Recording and broadcasting system, recording and broadcasting method and device and electronic equipment |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104821068A (en) * | 2015-03-27 | 2015-08-05 | 努比亚技术有限公司 | Mobile terminal real-time antitheft alarm method and apparatus |
| CN106060378A (en) * | 2015-04-10 | 2016-10-26 | 三星电子株式会社 | Apparatus and method for setting camera |
| CN107677992A (en) * | 2017-09-30 | 2018-02-09 | 深圳市沃特沃德股份有限公司 | Mobile detection method, device and monitoring device |
| WO2019158839A1 (en) * | 2018-02-13 | 2019-08-22 | Observatoire Regional Du Bruit En Idf | System for indicating the crossing of a loudness threshold |
| JP2020017904A (en) * | 2018-07-27 | 2020-01-30 | キヤノン株式会社 | Control arrangement, method and program of imaging apparatus |
-
2021
- 2021-07-14 CN CN202110795794.5A patent/CN113542606B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104821068A (en) * | 2015-03-27 | 2015-08-05 | 努比亚技术有限公司 | Mobile terminal real-time antitheft alarm method and apparatus |
| CN106060378A (en) * | 2015-04-10 | 2016-10-26 | 三星电子株式会社 | Apparatus and method for setting camera |
| CN107677992A (en) * | 2017-09-30 | 2018-02-09 | 深圳市沃特沃德股份有限公司 | Mobile detection method, device and monitoring device |
| WO2019158839A1 (en) * | 2018-02-13 | 2019-08-22 | Observatoire Regional Du Bruit En Idf | System for indicating the crossing of a loudness threshold |
| JP2020017904A (en) * | 2018-07-27 | 2020-01-30 | キヤノン株式会社 | Control arrangement, method and program of imaging apparatus |
Also Published As
| Publication number | Publication date |
|---|---|
| CN113542606A (en) | 2021-10-22 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11074466B2 (en) | Anti-counterfeiting processing method and related products | |
| CN108566516B (en) | Image processing method, device, storage medium and mobile terminal | |
| CN108241434B (en) | Man-machine interaction method, device and medium based on depth of field information and mobile terminal | |
| CN105956518A (en) | Face identification method, device and system | |
| CN113014983B (en) | Video playing method and device, storage medium and electronic equipment | |
| CN107613368A (en) | Video pause method and apparatus, computer installation and computer-readable recording medium | |
| KR101677607B1 (en) | Method, device, program and recording medium for video browsing | |
| CN109034052B (en) | Face detection method and device | |
| CN108668080A (en) | Prompt method and device, the electronic equipment of camera lens degree of fouling | |
| CN113542606B (en) | Shooting control method, shooting control device, electronic equipment and storage medium | |
| CN114816610A (en) | Page classification method, page classification device and terminal equipment | |
| CN111416996B (en) | Multimedia file detection method, multimedia file playing device, multimedia file equipment and storage medium | |
| CN110248401B (en) | WiFi scanning control method and device, storage medium and mobile terminal | |
| CN112286364A (en) | Man-machine interaction method and device | |
| CN113778255B (en) | Touch recognition method and device | |
| CN110674728A (en) | Method, device, server and storage medium for playing mobile phone based on video image identification | |
| CN113542466A (en) | Audio processing method, electronic device and storage medium | |
| CN112395921B (en) | Abnormal behavior detection method, device and system | |
| JP2007114885A (en) | Classification method and device by similarity of image | |
| CN108829600B (en) | Method and device for testing algorithm library, storage medium and electronic equipment | |
| CN113438286B (en) | Information pushing method and device, electronic equipment and storage medium | |
| CN115086759A (en) | Video processing method, video processing device, computer equipment and medium | |
| CN106296722B (en) | Information processing method and electronic equipment | |
| CN116502203B (en) | User identity recognition method and electronic equipment | |
| US20240205239A1 (en) | Methods and systems for fraud detection using relative movement of facial features |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |