CN117278911A

CN117278911A - Audio real-time signal state acquisition system, method, equipment and terminal

Info

Publication number: CN117278911A
Application number: CN202311292925.3A
Authority: CN
Inventors: 常洪刚; 迟浩; 陈周伟; 邹力
Original assignee: Wuhan Zhihui Digital Technology Co ltd
Current assignee: Wuhan Zhihui Digital Technology Co ltd
Priority date: 2023-10-08
Filing date: 2023-10-08
Publication date: 2023-12-22

Abstract

The invention belongs to the technical field of signal processing, and particularly relates to an audio real-time signal state acquisition system, method, equipment and terminal, which comprise the following steps: the audio acquisition module is used for acquiring audio; the audio processing module is used for processing the collected audio; the equipment control module is used for controlling and adjusting equipment; the equipment monitoring module is used for monitoring the state of equipment in real time; and the display module is used for displaying the equipment state and the audio processing result. The system provided by the invention not only has the basic equipment management function, but also realizes the advanced functions of equipment state monitoring, equipment control, data storage and the like. Implementation of these functions depends on advanced software engineering and computer science and technology.

Description

Audio real-time signal state acquisition system, method, equipment and terminal

Technical Field

The invention belongs to the technical field of signal processing, and particularly relates to an audio real-time signal state acquisition system, method, equipment and terminal.

Background

Audio processing is an audio processing device that we often use when using many large electronic devices, and it can help us control music or soundtrack, so that it can produce different sound effects in different scenes, increase the impact of music or soundtrack, and at the same time can control many audio functions on site.

The audio processor is also called a digital processor, is used for processing digital signals, and the internal structure of the audio processor is generally composed of an input part and an output part. The functions inside the system are more complete, and some system are provided with processing modules capable of being dragged and programmed, so that the system can be freely built by a user.

An audio processor is generally a series of audio devices that modify, process, and process audio signals.

Common audio processors can be divided into 4 architectures:

1. simple sound box processor: for example, 2 in and 4 out, 2 in and 6 out, 2 in and 8 out, 4 in and 6 out, 4 in and 8 out of DA series, and the inside has simple fixed processing modules such as parameter equalization, frequency division, delay, mixing, etc. The device is used for connecting the sound console to the power amplifier to replace analog peripheral equipment for signal processing.

2. Multifunctional digital audio processor: typically 8 in 8 out, or more; the input channels are all provided with phantom power supply, and can be directly connected with a conference gooseneck microphone. The functions inside the system are more complete, and some system are provided with processing modules capable of being dragged and programmed, so that the system can be freely built by a user. Such processors can typically replace analog systems consisting of small tuning tables and peripherals in conference systems. Often with a network interface, programmed and online real-time controlled by an ethernet access computer.

3. Digital audio processor with network audio transmission function: they are similar to the above 2 functions, but add an audio transmission function (generally supporting CobraNet) of the network, so that audio data can be mutually transmitted in a local area network, thereby facilitating interconnection and interworking of multiple conference rooms. The audio network also supports the control function and can realize the flexible operation of the centralized control or the decentralized control of the network.

4. Large centrally processed digital audio matrix: the system is a host with extremely strong processing capability, audio in each room is packed into network data through an interface box, the network data is sent to a processing host of a master control room, and the network data is sent to each room for replay after the processing of the host is finished. Such audio networks are typically gigabit ethernet based CobraNet or other protocols, while supporting real-time transmission and control. The method is mainly applied to large-scale broadcasting systems or conference centers and other places. In contrast to the third item above, the small network audio processor is a decentralized system, each room has a separate small host, which can be used alone or in combination for interworking; the large processing matrix is intensively placed in a certain machine room, and the processing control of all rooms is completed by the machine of the total machine room, so that the processor of the total machine room must be started at any time no matter 1 or more rooms are used.

In the prior art at present:

the hardware aspect: the real-time signal is obtained by selecting an audio input device that is suitable for the requirements.

Driver and interface: the real-time signals are acquired through the driver and the interface, so that the hardware equipment can be connected with a computer or an embedded system, and the corresponding driver is installed. Common interfaces include USB, firewire, thunderbolt, etc.

Buffer management: data read from the audio device is stored by setting a buffer of a suitable size. These buffers can help you avoid losing data when doing signal processing.

Through the above analysis, the problems and defects existing in the prior art are as follows:

the hardware aspect: the hardware problem is used for acquiring the real-time signal, so that the result is stable, but the field system is required to be modified, related equipment is upgraded, the use limitation is strong, and the method cannot be widely applied in a large range.

Driver and interface: the acquisition of real-time signals through the driver and the interface is more strict on the version and the interface of the driver. Although the audio real-time signal is acquired, the realized result is relatively stable. However, the version and interface of the device are better, and once the version and interface of the device are not matched with the driver and interface, the audio real-time signal cannot be acquired.

Buffer management: the real-time audio signal is obtained through buffer area management, the obtained result is relatively stable, and the state of the real-time audio signal can be obtained; however, the acquisition of the real-time audio signal is not real-time when the system is replayed through the buffer area, and continuous refreshing acquisition is required.

Disclosure of Invention

Aiming at the problems existing in the prior art, the invention provides a method, a system, equipment and a terminal for acquiring the state of an audio real-time signal.

The invention is realized in that an audio real-time signal state acquisition system comprises:

device management module (Device Management): the module is responsible for processing device access requests of developers, including operations of adding, deleting, modifying, querying and the like of devices. When the operations are performed, the device management module interacts with the persistence layer, requests the read-write operation of the database, and receives feedback of the read-write result.

Device status monitoring module (Device Monitoring): the module is responsible for receiving a developer's device status query request, verifying the device information in the request, and sending a device list query to the device control module. After receiving the device list, the device status monitoring module may feed back device status information to the developer.

Device Control module (Device Control): the module is responsible for receiving device control instructions of a developer, verifying device information and instruction information in the instructions, and sending the control instructions to the device. After the device control module executes the instruction, the execution result is fed back to the developer.

Data save module (Data Storage): the module is responsible for receiving a data saving instruction of a developer, inquiring a device list from the device control module and receiving a device list retrieval result. And then, the data storage module sends a data storage instruction to the persistent layer, and after the data storage is completed, the execution result is fed back to the developer.

Further, the device management module includes:

interface and input/output unit: receiving requests from a developer or User Interface (UI), the requests being RESTful API calls, webSocket messages, or other forms;

returning the operation result and the state information to a developer or a user interface, usually through the same interface;

request verification unit: when a request is received, verification is first made to ensure that the request is from a valid or authorized developer;

interaction unit with persistence layer: when the equipment information needs to be inquired, the equipment management module sends a read request to the persistent layer; the persistence layer retrieves information from the database and returns the results to the device management module;

For the add, delete or modify operations of the device, the device management module generates and sends corresponding database write operations to the persistence layer; the persistent layer executes the writing operation and returns successful or failed state information;

an operation result processing unit: the device management module receives an operation result from the persistent layer; this includes a status code of success or failure, as well as any associated data or error information;

feedback to the developer unit: the device management module encapsulates the operation result into a response message and returns the response message to the developer through the interface defined before;

log and monitor unit: all operations and results are recorded in the log to facilitate subsequent analysis and troubleshooting;

an abnormality processing unit: if errors or anomalies occur at any stage, the device management module captures the anomalies and includes the corresponding error code and description in the response message.

Further, the device status monitoring module includes:

interface and input/output unit: accepting a device state query request from a developer, the request being implemented through a RESTful API, webSocket, MQTT or other communication protocol; feeding back device status information to the developer, typically via the same communication protocol;

Request verification unit: after receiving the equipment state query request, the equipment state monitoring module firstly needs to verify the validity of the request, including checking equipment information and authority;

interaction unit with device control module: the equipment state monitoring module sends an equipment list query to the equipment control module; after receiving the inquiry, the device control module extracts corresponding device state information from a device list managed by the device control module;

a state information processing unit: the equipment state monitoring module receives the equipment state information returned by the equipment control module and further processes or formats the equipment state information;

feedback to the developer unit: after the steps are finished, the equipment state monitoring module packages the equipment state information and feeds the equipment state information back to a developer through a predefined interface or protocol;

log and monitor unit: all details of the operation, including received requests, sent queries, received device status, are logged for problem diagnosis and performance monitoring;

an abnormality processing unit: if any errors or anomalies occur in the series of operations, the device status monitoring module captures the anomalies and feeds error information or error codes back to the developer.

The audio real-time signal state acquisition system of claim 1, wherein the device control module comprises:

interface and input/output unit: receiving device control instructions from a developer or device status monitoring module, which are implemented through RESTful APIs, webSocket, MQTT or other communication protocols; returning instruction execution results, typically via the same interface or communication protocol;

an instruction verification unit: when receiving a device control instruction, firstly verifying the legitimacy of the instruction; this includes checking the instruction format, device information, and whether there is authority to execute the instruction;

instruction forwarding and execution unit: after the verification is passed, the device control module sends a control instruction to the target device; this is achieved by communicating directly with the device, or by a middleware;

an execution result acquisition unit: after the device executes the instruction, the device control module acquires an execution result from the device; this is a status code, also including more operational data;

and a result feedback unit: the device control module encapsulates the acquired execution result and sends the execution result to a developer or a device state monitoring module through a predefined interface or protocol;

Log and monitor unit: all received instructions, sent control information and received execution results are recorded in a log for subsequent problem diagnosis and system monitoring;

an abnormality processing unit: in the whole process, the equipment control module can capture various abnormal conditions and feed back abnormal information to a developer or the equipment state monitoring module.

Further, the data saving module includes:

interface and input/output unit: receive data save instructions from a developer or other module, typically through a RESTful API, webSocket, MQTT, or other communication protocol; feeding back the result of the execution of the data saving to the developer, which is also typically via the same interface or communication protocol;

an instruction verification unit: when the module receives the data saving instruction, firstly verifying to ensure that the instruction is from an authorized developer and the format is correct;

querying a device list unit: after verification is successful, the data storage module sends an equipment list query request to the equipment control module; the device control module returns the queried device list;

a data saving instruction generation unit: the data storage module generates a specific data storage instruction according to the received equipment list and the storage instruction of the developer;

Interaction unit with persistence layer: the data saving module sends the generated data saving instruction to the persistent layer;

the persistence layer is responsible for saving the data to a database or other storage device and returning an operation result;

and a result feedback unit: the data storage module receives an operation result returned by the persistence layer and feeds the result back to the developer;

log and monitor unit: the whole data storage process, including the received instructions, interactions with the device control module and the persistence layer, and the final operation results, are recorded in the log;

an abnormality processing unit: anomalies that occur at any stage are captured and anomaly information or error codes are fed back to the developer.

Further, the device control module includes:

the Abstract equipment is used for identifying the identification, the name, the state, the type and the channel type of the equipment, judging whether the equipment is central control equipment or not and performing switching operation on the equipment;

the device controller is used for calling the device function, monitoring the states of the device and the channel, identifying the controller identifier, judging whether to retry the strategy, carrying out log tracking, and checking the device list and the channel list;

and the Abstract control channel is used for realizing a control mode, connecting the source end and the destination end, executing disconnection operation, and receiving and sending messages.

Further, the equipment control module is an internal module and is only used for being called by a developer to provide coding realization of all equipment; when a new device is added, the corresponding type device is extended, and when no type exists, the new device type is added.

Further, in the Abstract equipment, part of the equipment can be independently controlled, and also can be controlled through central control, and part of the equipment is intelligently controlled through central control; the extended central control interface can be used for determining whether to directly control or to control by a developer; when the central control is selected, the controller transfers the instruction to the central control equipment for execution.

Another object of the present invention is to provide a method for acquiring audio real-time signal status, including:

s1, equipment management: the developer performs addition, deletion and verification on the access device, the access device requests a database from the persistent layer, reads and writes in the persistent layer, returns the read and write result to the access device and feeds back to the developer;

s2, checking the state of equipment: after a developer requests the equipment state from the equipment monitoring module, the equipment monitoring module verifies the equipment information, the refrigerator equipment control module inquires an equipment list, and the equipment control module feeds back the equipment state after searching the equipment list;

s3, equipment control: the developer sends a control instruction to the equipment monitoring module, after the equipment monitoring module verifies the equipment information and the instruction information, the developer sends an equipment instruction to the equipment control module, and after the equipment control module calls the equipment instruction to complete the control operation, the execution result is fed back;

S4, saving equipment data: the developer sends a data saving instruction to the equipment monitoring module, the equipment monitoring module inquires an equipment list to the equipment control module, and the equipment control module retrieves the equipment list and returns a retrieval result to the equipment monitoring module; the device monitoring module sends a data storage instruction to the persistent layer, the persistent layer stores the data, and the execution result is fed back after the storage is completed.

It is a further object of the present invention to provide a computer device comprising a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the audio real-time signal state acquisition method.

It is a further object of the present invention to provide a computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the audio real-time signal state acquisition method.

Another object of the present invention is to provide an information data processing terminal, where the information data processing terminal is configured to implement the audio real-time signal status acquisition system.

In combination with the technical scheme and the technical problems to be solved, the technical scheme to be protected has the following advantages and positive effects:

First, audio real-time signal state acquisition may address a number of issues, including but not limited to the following:

sound quality monitoring problem: by monitoring the real-time audio signal quality, a communication service provider can quickly identify and resolve problems in an audio call, providing a higher quality communication service. The music streaming media platform can monitor the quality of the audio stream in real time, and ensure that a user obtains high-quality audio experience.

And (3) safety monitoring: the audio real-time signal state acquisition can be used for detecting emergency events such as fire alarms, explosion sounds, thief invasion and the like, and helps to improve safety and rapidly respond to emergency situations.

Traffic management: in city traffic management, audio sensors may be used to detect traffic accidents, traffic jams, and emergencies to improve traffic flow and safety.

Customer service optimization: by analyzing the voice and emotion of the customer, the enterprise can improve customer support and custom services, and better meet customer needs.

Identity authentication and security: the audio signal may be used to identify the voice biometric of the individual for authentication and security measures such as unlocking the device or accessing a restricted area. In the field of financial services, voice biometric recognition can be used to enhance customer authentication to prevent fraud.

Smart home: through discernment voice command, intelligent house equipment can realize more convenient user experience, provides home automation and control function.

Manufacturing industry: in factory production, audio sensors can be used to monitor equipment status and operating conditions, helping to predict equipment failure.

In summary, audio real-time signal state acquisition helps to address various sound-related problems, including improving quality of service, improving security, optimizing operation, providing personalized services, and providing a better user experience. These applications have a wide range of uses in various industries, contributing to increased efficiency, reduced costs, and enhanced decision support.

Second, the audio real-time signal status acquisition requires a series of technical schemes to protect the privacy and data security of the user. These technical solutions aim at achieving the following technical effects and advantages:

data desensitization and encryption: for stored and transmitted audio data, data desensitization and anonymization techniques may be employed to prevent the identity of individuals from being compromised. A strong encryption algorithm is used to protect the transmission and storage of audio data to ensure that the data is only accessible to authorized users.

Sound feature privacy: the sound features are separated from the identity of the individual to prevent the sound features from being used to identify a particular individual. Differential privacy techniques are applied to preserve the privacy of sound features by adding noise, making it difficult to re-identify individuals.

Authentication and authorization: multi-factor authentication is used to ensure that only authorized personnel have access to the audio data and analysis results.

And (3) authority control: strict rights control is implemented, limiting the user to only access the data and functions they need.

Data use audit: the user's access to and use of the audio data is tracked and recorded in order to supervise and audit the data processing activities.

De-identification technology: by adopting the technical means, the identification information which causes the identification of the individual in the sound is removed. The audio data is prevented from being associated with other available data sets to prevent individual identification.

Interpretability and transparency: the data processing process is made traceable and understandable using an interpretable machine learning algorithm. An explicit privacy policy is formulated to provide detailed information to the user explaining the data collection and usage patterns.

Compliance with data privacy regulations: ensuring that the system design and operation complies with applicable data privacy regulations, such as GDPR, CCPA, etc.

The goal of these solutions and measures is to ensure data privacy and security during the audio real-time signal state acquisition while still providing accurate and valuable audio analysis results. They can help reduce potential privacy disclosure and abuse risk and improve user trust and satisfaction. At the same time, compliance is also a key factor in ensuring operation within legal frameworks.

Thirdly, the expected benefits and commercial value after the technical scheme of the invention is converted are as follows:

audio real-time signal state acquisition is of great commercial value in the business arts because it can provide real-time information and insight, helping to optimize operation, improve customer experience, and enhance security. The following are examples of some commercial value aspects related to audio real-time signal state acquisition:

communication industry: telephone and video conference service providers can improve communication services by monitoring real-time audio signal quality. This can help provide clearer voice and video calls, improving customer satisfaction.

Music streaming media: the music streaming media platform can monitor the audio quality in real time, and ensure that a user obtains high-quality audio streams.

Safety and monitoring: the audio real-time signal status acquisition can be used to detect emergency events such as fire alarms, explosion sounds, thief intrusion, etc. This may help to improve safety and reduce risk.

Traffic management: in city traffic management, audio sensors may be used to detect traffic accidents, traffic jams, and emergencies.

Customer service: by analyzing the voice and emotion of the customer, customer support and customized services can be improved. For example, an anger emotion of a customer is identified and measures are taken in time.

And (3) safety authentication: the voice biometric of the individual is identified by means of the audio signal acquired in real time, so that it is possible to realize a method for authentication, for example unlocking a mobile phone or accessing a secure area.

Financial services: banks and financial institutions may use voice biometrics to enhance customer authentication and to prevent fraud.

Smart home: through discernment voice command, realize speech control intelligent household equipment, high-efficient convenient user experience. Such as controlling lights, controlling curtains, controlling household appliances, controlling home security systems, etc.

Manufacturing industry: in the manufacturing industry, the production equipment of a factory can monitor the running state of the equipment and detect faults through the audio sensor, and the monitoring are normalized and sustainable. Ensure the stability and reliability of production equipment.

These examples demonstrate that audio real-time signal state acquisition has tremendous commercial value in various business areas, increasing efficiency, improving security, enhancing user experience, and providing data insight to enterprises. As technology continues to advance, the commercial value of this area will continue to increase.

(2) The technical solution of the present invention may help to overcome some technical bias (bias), the following are examples of some overcome and introduced technical bias:

Sound diversity: the audio real-time signal state acquisition system may be trained to recognize various speech types, including different accents, dialects, and languages. This helps overcome regional and cultural prejudices in speech recognition.

Sex equality: to avoid gender bias, a large amount of speech data of different sexes is required to be used when training a speech recognition model to ensure that the model has good recognition performance for both male and female speech.

Emotion and accent recognition: the audio processor can overcome the prejudice introduced by cultural and social backgrounds by analyzing emotion and accent features to more accurately understand the emotion and accent of the speaker.

In summary, audio real-time signal state acquisition may help overcome some technical prejudice, but requires careful handling to ensure that the system can provide fair, accurate and inclusive service in different people and scenarios.

Fourth, 1. Device management module (Device Management): the module can realize the functions of automatic registration, configuration, maintenance and the like of the equipment, greatly simplifies the process of equipment access and improves the equipment management efficiency.

2. Device status monitoring module (Device Monitoring): the module can monitor the running state of the equipment in real time, supports functions such as historical state inquiry and the like, helps developers discover abnormal conditions of the equipment in time, and avoids equipment faults.

3. Device Control module (Device Control): the module supports various device control instructions and can dynamically adjust the control strategy according to different states of the device. This flexible control scheme can maximize the performance of the device.

4. Data save module (Data Storage): the module adopts advanced data storage and retrieval technology, and ensures the safety and the integrity of data. In addition, the module also supports quick reading and writing of data, and improves the response speed of the system.

The system provided by the invention not only has the basic equipment management function, but also realizes the advanced functions of equipment state monitoring, equipment control, data storage and the like. Implementation of these functions depends on advanced software engineering and computer science and technology.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings that are needed in the embodiments of the present invention will be briefly described below, and it is obvious that the drawings described below are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

Fig. 1 is a schematic diagram of an audio real-time signal status acquisition system according to an embodiment of the present invention;

fig. 2 is a flowchart of a method for acquiring audio real-time signal status according to an embodiment of the present invention;

FIG. 3 is a diagram of an MRX7-D architecture provided by an embodiment of the present invention;

FIG. 4 is an audio processor effect diagram provided by an embodiment of the present invention;

FIG. 5 is a diagram of a physical connection architecture of an audio processor provided by an embodiment of the present invention;

FIG. 6 is a diagram of an overall object model of a device control module provided by an embodiment of the present invention;

fig. 7 is a schematic diagram of the specific application and effect principle of the embodiment of the present invention and the prior art scheme.

Detailed Description

The present invention will be described in further detail with reference to the following examples in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.

Aiming at the problems existing in the prior art, the invention provides an audio real-time signal state acquisition system, an audio real-time signal state acquisition method, audio real-time signal state acquisition equipment and an audio real-time signal state acquisition terminal, and the invention is described in detail below with reference to the accompanying drawings.

As shown in fig. 1, an audio real-time signal status acquisition system provided by an embodiment of the present invention includes:

The audio acquisition module is used for acquiring audio;

the audio processing module is used for processing the collected audio;

the equipment control module is used for controlling and adjusting equipment;

the equipment monitoring module is used for monitoring the state of equipment in real time;

and the display module is used for displaying the equipment state and the audio processing result.

The specific design and workflow of the audio real-time signal state acquisition system provided by the embodiment of the invention are as follows:

audio acquisition module

1. And a hardware interface, wherein a microphone, a sound card or other audio input devices are used for audio signal acquisition.

2. Sample rate and bit depth, e.g., 44.1kHz sample rate, 16 bits or 24 bits.

3. Buffer design, data caching for subsequent processing.

Audio processing module

1. Digital filtering, namely noise elimination or frequency domain filtering is carried out on the signal.

2. Signal analysis-FFT (fast fourier transform) or other means to obtain information on frequency, amplitude etc.

3. And the characteristic extraction includes MFCC (Mel frequency cepstral coefficient), rhythm, pitch and the like.

Equipment control module

1. And volume control, namely automatically or manually adjusting the volume according to the processing result.

2. Device selection different audio input or output devices may be selected.

3. Preset configuration, storing and calling preset audio processing setting.

Equipment monitoring module

1. And (3) monitoring in real time, namely acquiring the working states of the equipment, such as CPU load, memory use, network state and the like.

2. Alarm mechanism, which is to send out alarm or repair automatically in abnormal state.

Display module

1. And a visual interface for displaying audio waveforms, spectrograms and the like in real time.

2. And the equipment state is the real-time working state of the display equipment.

3. User interaction, namely user interaction is performed through buttons, sliding bars and the like.

Signal and data processing procedure

1. Initializing, namely starting all modules and hardware interfaces.

2. And acquiring audio frequency, namely acquiring original audio frequency data through an audio frequency acquisition module.

3. Preprocessing-the necessary preprocessing, such as denoising, is performed in the audio processing module.

4. And (3) analyzing the characteristics, namely further analyzing the audio signals and extracting key characteristics.

5. And the state monitoring is to acquire the state of the equipment through the equipment monitoring module.

6. And controlling and adjusting, namely correspondingly adjusting in the equipment control module according to the analysis result and the equipment state.

7. And displaying the result, namely displaying all the information and the processing result in the display module.

8. And (5) looping or ending, namely returning to the step 2 or ending the processing according to the requirement.

The embodiment of the invention can construct a device management system or a system for interacting with devices aiming at four steps of the method. The module of the system provided by the invention comprises the following parts:

1. Device management module (Device Management): the module is responsible for processing device access requests of developers, including operations of adding, deleting, modifying, querying and the like of devices. When the operations are performed, the device management module interacts with the persistence layer, requests the read-write operation of the database, and receives feedback of the read-write result.

2. Device status monitoring module (Device Monitoring): the module is responsible for receiving a developer's device status query request, verifying the device information in the request, and sending a device list query to the device control module. After receiving the device list, the device status monitoring module may feed back device status information to the developer.

3. Device Control module (Device Control): the module is responsible for receiving device control instructions of a developer, verifying device information and instruction information in the instructions, and sending the control instructions to the device. After the device control module executes the instruction, the execution result is fed back to the developer.

4. Data save module (Data Storage): the module is responsible for receiving a data saving instruction of a developer, inquiring a device list from the device control module and receiving a device list retrieval result. And then, the data storage module sends a data storage instruction to the persistent layer, and after the data storage is completed, the execution result is fed back to the developer.

As a specific optimization scheme of the invention, the working principle of the equipment management module is specifically as follows:

1. interface and input/output

Input to receive requests from a developer or User Interface (UI) in the form of RESTful API calls, webSocket messages, or other forms.

And outputting, namely returning the operation result and the state information to a developer or a user interface, and usually passing through the same interface.

2. Request verification

When a request is received, verification is first made to ensure that the request is from a valid or authorized developer.

3. Interaction with persistence layer

And (3) reading, namely when the device information needs to be queried, the device management module sends a read request to the persistent layer. The persistence layer retrieves information from the database and returns the results to the device management module.

Write operations for add, delete, or modify operations to the device, the device management module generates and sends corresponding database write operations to the persistence layer. The persistence layer performs the write operation and returns status information of success or failure.

4. Operation result processing

The device management module receives the operation result from the persistence layer. This includes a status code of success or failure, as well as any associated data or error information.

5. Feedback to developer

The device management module encapsulates the operation result into a response message and returns the response message to the developer through the interface defined before.

6. Log and monitoring

All operations and results are logged for subsequent analysis and troubleshooting.

7. Exception handling

If errors or anomalies occur at any stage, the device management module captures the anomalies and includes the corresponding error code and description in the response message.

In this way, the device management module effectively handles aspects of device management, including interactions with the persistence layer and communications with the developer, as a critical part of the overall system. By means of strict request processing and feedback mechanisms, stability and reliability of the system are ensured.

As a specific optimization scheme of the invention, the working principle of the state monitoring module of the # # equipment is specifically as follows:

1. interface and input/output

Input accepting device state query requests from a developer, which may be implemented through RESTful APIs, webSocket, MQTT or other communication protocols.

The output is that device status information is fed back to the developer, typically via the same communication protocol.

2. Request verification

After receiving the device state query request, the device state monitoring module first needs to verify the validity of the request, including checking device information, rights, and the like.

3. Interaction with device control module

Device list query the device status monitoring module may send a device list query to the device control module.

And acquiring the equipment state, namely extracting corresponding equipment state information from the managed equipment list by the equipment control module after the equipment control module receives the inquiry.

4. State information processing

The equipment state monitoring module receives the equipment state information returned by the equipment control module and performs further processing or formatting.

5. Feedback to developer

After the above steps are completed, the device status monitoring module encapsulates the device status information and feeds back to the developer through a predefined interface or protocol.

6. Log and monitoring

All details of the operation, including received requests, sent queries, received device status, etc., are logged for problem diagnosis and performance monitoring.

7. Exception handling

If any errors or anomalies occur in the series of operations, the device status monitoring module captures the anomalies and feeds error information or error codes back to the developer.

The design ensures that the equipment state monitoring module can effectively process the state query request, and ensures the accuracy and timeliness of the equipment state information returned to the developer through interaction with the equipment control module. Meanwhile, the robustness of the system is also increased through the log and an exception handling mechanism.

As a specific optimization scheme of the invention, the working principle of the control module of the # # device is specifically as follows:

1. interface and input/output

Input device control instructions are received from a developer or device status monitoring module, which are implemented through RESTful APIs, webSocket, MQTT or other communication protocols.

The output is to return the instruction execution results, typically via the same interface or communication protocol.

2. Instruction verification

When a device control command is received, the legitimacy of the command is first verified. This includes checking the instruction format, device information, and whether there is authority to execute the instruction.

3. Instruction forwarding and execution

After the verification is passed, the device control module sends a control instruction to the target device. This is achieved by communicating directly with the device, or by a middleware.

4. Execution result acquisition

After the device executes the instruction, the device control module obtains the execution result from the device. This is a status code and also includes more operational data.

5. Result feedback

The device control module encapsulates the obtained execution result and sends the encapsulated execution result to a developer or a device state monitoring module through a predefined interface or protocol.

6. Log and monitoring

All received instructions, sent control information, and received execution results are recorded in a log for subsequent problem diagnosis and system monitoring.

7. Exception handling

In the whole process, the equipment control module can capture various abnormal conditions and feed back abnormal information to a developer or the equipment state monitoring module.

Through the steps, the equipment control module effectively realizes the equipment control function, and ensures the accurate transmission and execution of control instructions. The log and exception handling mechanisms also enhance the robustness of the system.

As a specific optimization scheme of the invention, the working principle of the # # data storage module is specifically as follows:

1. interface and input/output

Input-data save instructions are received from a developer or other module, typically through a RESTful API, webSocket, MQTT or other communication protocol.

Output, feedback to the developer of the result of the execution of the data save, which is also typically via the same interface or communication protocol.

2. Instruction verification

When the module receives the data save instruction, it first verifies to ensure that the instruction is from an authorized developer and is in the correct format.

3. Querying a list of devices

After the verification is successful, the data storage module sends a device list query request to the device control module.

The device control module returns the queried device list.

4. Data save instruction generation

And the data storage module generates a specific data storage instruction according to the received equipment list and the storage instruction of the developer.

5. Interaction with persistence layer

The data saving module may send the generated data saving instruction to the persistence layer.

The persistence layer is responsible for saving data to a database or other storage device and returning a result of the operation.

6. Result feedback

The data saving module receives the operation result returned by the persistence layer and feeds the result back to the developer.

7. Log and monitoring

The entire data retention process, including the received instructions, interactions with the device control module and persistence layers, and the final results of the operation, are recorded in a log.

8. Exception handling

Anomalies that occur at any stage are captured and anomaly information or error codes are fed back to the developer.

The working principle ensures that the data storage module can accurately and effectively process the data storage request of a developer, and ensures the accuracy and consistency of data through interaction with the equipment control module and the persistent layer. The logging and exception handling mechanisms further enhance the robustness of the system.

As shown in fig. 2, the method for acquiring audio real-time signal status provided by the embodiment of the invention includes:

As shown in table 1: audio matrix action: for switching and distributing audio signals (unbalanced stereo audio signals), multiple signals can be switched from an input channel to any one of output channels, and the output channels are independent from each other. The audio matrix video matrix is generally used for various monitoring occasions, and the audio matrix video matrix switcher is specially used for switching and distributing the video signal and the audio signal unbalanced stereo audio signal, so that multiple paths of signals can be switched and transmitted from an input channel to any one of output channels and among the output channels. The invention employs an MRX7-D audio processor, with an MRX7-D architecture as shown in FIG. 3.

Common network communication modes of the audio processor include:

1、TCP

typically the audio processor will provide a TCP service address, and the external system will send instructions to the other, the audio processor will receive the instructions and execute the last return execution status code. TCP is connection-based and UDP is non-connection-based.

2、UDP

Firstly, the UDP protocol is based on non-connection, the data packets are simply encapsulated, and then sent out from the network card, so that the data packets have no state connection, and because of the simple data processing mode of UDP, the performance loss of the data packets is very little, the occupation of the CPU memory resources is far less than TCP, but the UDP protocol cannot be guaranteed for packet loss generated in the network transmission process, so that the UDP is weaker than TCP in transmission stability.

FADE: and the clippers on the sound console are used for controlling the volume.

MUTE: mute and non-mute control.

METER: representing the real-time volume of a certain path in the audio processor.

A control table is maintained in the audio processor, a corresponding control number is found out, and a corresponding instruction is sent to obtain the state of the control number. As shown in table 2, each control number represents a different meaning, and three control numbers represent: the push state, mute state and real-time volume of the main sound box.

Name of the name	FADER	MUTE	METER
				Main sound-amplifying left front	1151	1251	1351

1. Acquisition of FADE

Such as: GSB 1151

2. Acquisition of MUTE

Such as: GSB 1251

3. Acquisition of METER

GSB 1351

For FADE and MUTE, the processing modes of different audio processors are mostly the same. Whereas for the acquisition of METER there is a relatively large difference between different audio processors. This is also a problem to be solved with the present invention.

The symmetrix family processor will give some volume to each client in the form of a TCP broadcast.

YAMAHA MTX7D will let the currently connected client specify how long to accept the continuous push of the server next, every interval.

In addition, the number of different audio processors for client connection is severe, and is generally below 10. Preventing strings of codes for multiple concurrent requests with very few connections is also a great challenge.

In addition, no corresponding cases can be referred to in the open source community and the market, and the final effect is shown in fig. 4.

Fig. 5 shows a physical connection architecture of an audio processor according to an embodiment of the present invention, where:

an input device: notebook, sound console, desktop, high definition camera, acquisition card;

an output device: splice screen, display screen, projecting apparatus, outdoor LED, stereo set.

As shown in fig. 6, the device control module includes:

The equipment control module is an internal module and is only used for being called by a developer to provide coding realization of all equipment; when a new device is added, the corresponding type device is extended, and when no type exists, the new device type is added.

In the Abstract equipment, part of the equipment can be independently controlled or controlled by the central control, and part of the equipment is intelligently controlled by the central control; the extended central control interface can be used for determining whether to directly control or to control by a developer; when the central control is selected, the controller transfers the instruction to the central control equipment for execution.

An application embodiment of the present invention provides a computer device including a memory and a processor, the memory storing a computer program which, when executed by the processor, causes the processor to perform the steps of the audio real-time signal state acquisition method.

An application embodiment of the present invention provides a computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of an audio real-time signal state acquisition method.

The embodiment of the invention provides an information data processing terminal which is used for realizing an audio real-time signal state acquisition system.

The technical scheme of the invention is widely applied in the conference industry, for example: in a major secure video conference, conference security personnel are inconvenient to stay in a conference room and monitor audio of the conference room when performing conference security. At this time, meeting support personnel only need acquire in real time through looking over audio signal, judge whether the audio signal of pickup passageway and the public address passageway of meeting is normal, both can ensure the normal clear of meeting, also can keep secret to meeting content and information.

When one meeting guarantee personnel guarantees a plurality of meetings, the meeting guarantee personnel are very tedious. In terms of space, each meeting is held in different meeting rooms, and floor positions are high and low, so meeting guarantee staff cannot effectively consider guaranteeing each meeting place. After the scheme of acquiring the audio signals in real time is adopted, meeting security personnel can check the audio states of all meeting rooms in real time only by checking an application interface of the scheme of acquiring the audio signals in real time before a computer, and once the audio problems occur, the problems can be found out at the first time to be secured.

An application example of the audio signal real-time acquisition scheme shows that the audio real-time signal state acquisition can improve efficiency, improve safety and enhance user experience in practical application.

As shown in fig. 7, the present invention has been put to practical use in an item (hereinafter, will be simply referred to as "a item"). The project A is shared by the rest of conference rooms 30, is distributed in 1-16 buildings of the building, and a conference management center is arranged in a 4-building conference room and is provided with 1 conference guarantee person.

As can be seen from fig. 7, in the situation of limited personnel, once the audio frequency has a problem, the relevant personnel in the conference room are required to feed back information, and the conference support personnel upstairs and downstairs again to solve the problem. Therefore, conference security personnel are tired and alive, and security efficiency is not high.

After the method and the system are applied, conference security personnel can quickly locate the problem by detecting the audio signal of the conference room in real time through the terminal, and the problem of faults is solved remotely by combining with a field system. Therefore, the response speed and the working efficiency of meeting guarantee personnel are greatly improved, and the meeting guarantee can be efficiently carried out.

It should be noted that the embodiments of the present invention can be realized in hardware, software, or a combination of software and hardware. The hardware portion may be implemented using dedicated logic; the software portions may be stored in a memory and executed by a suitable instruction execution system, such as a microprocessor or special purpose design hardware. Those of ordinary skill in the art will appreciate that the apparatus and methods described above may be implemented using computer executable instructions and/or embodied in processor control code, such as provided on a carrier medium such as a magnetic disk, CD or DVD-ROM, a programmable memory such as read only memory (firmware), or a data carrier such as an optical or electronic signal carrier. The device of the present invention and its modules may be implemented by hardware circuitry, such as very large scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., as well as software executed by various types of processors, or by a combination of the above hardware circuitry and software, such as firmware.

The foregoing is merely illustrative of specific embodiments of the present invention, and the scope of the invention is not limited thereto, but any modifications, equivalents, improvements and alternatives falling within the spirit and principles of the present invention will be apparent to those skilled in the art within the scope of the present invention.

Claims

1. An audio real-time signal state acquisition system, comprising:

and the equipment management module: the module is responsible for processing the device access request of the developer, including the operations of adding, deleting, modifying and inquiring the device; when the operations are performed, the equipment management module interacts with the persistent layer, requests the read-write operation of the database to the equipment management module, and receives feedback of the read-write result;

the equipment state monitoring module: the module is responsible for receiving a device state query request of a developer, verifying device information in the request, and sending a device list query to the device control module; after receiving the equipment list, the equipment state monitoring module feeds equipment state information back to a developer;

and the equipment control module is used for: the module is in charge of receiving equipment control instructions of the developer, verifying equipment information and instruction information in the instructions and sending the control instructions to the equipment; after the device control module executes the instruction, the execution result is fed back to the developer;

And a data storage module: the module is responsible for receiving a data storage instruction of a developer, inquiring a device list from the device control module and receiving a device list retrieval result; the data storage module sends a data storage instruction to the persistent layer, and after the data storage is completed, the execution result is fed back to the developer.

2. The audio real-time signal state acquisition system of claim 1, wherein the device management module comprises:

3. The audio real-time signal status acquisition system of claim 1 wherein the device status monitoring module comprises:

4. The audio real-time signal state acquisition system of claim 1, wherein the device control module comprises:

5. The audio real-time signal state acquisition system of claim 1, wherein the data storage module comprises:

interface and input/output unit: receive data save instructions from a developer or other module, typically through RESTfulAPI, webSocket, MQTT or other communication protocol; feeding back the result of the execution of the data saving to the developer, which is also typically via the same interface or communication protocol;

interaction unit with persistence layer: the data saving module sends the generated data saving instruction to the persistent layer; the persistence layer is responsible for saving the data to a database or other storage device and returning an operation result;

6. The audio real-time signal state acquisition system of claim 1, wherein the device control module comprises:

7. The audio real-time signal state acquisition system of claim 1, wherein the device control module provides a coded implementation of all devices; when a new device is added, the corresponding type device is extended, and when no type exists, the new device type is added.

8. The audio real-time signal state acquisition system according to claim 2, wherein in the Abstract equipment, part of the equipment can be controlled independently or by central control, and part of the equipment is controlled intelligently by central control; the extended central control interface can be used for determining whether to directly control or to control by a developer; when the central control is selected, the controller transfers the instruction to the central control equipment for execution.

9. An audio real-time signal state acquisition method is characterized by comprising the following steps:

10. A computer readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the steps of the audio real-time signal state acquisition method as claimed in claim 5.