CN114661942A

CN114661942A - Method and device for processing streaming tone data, electronic equipment and computer readable medium

Info

Publication number: CN114661942A
Application number: CN202210337305.6A
Authority: CN
Inventors: 赖昆; 蔡康宁; 刘琴
Original assignee: Yidu Cloud Beijing Technology Co Ltd
Current assignee: Yidu Cloud Beijing Technology Co Ltd
Priority date: 2022-03-31
Filing date: 2022-03-31
Publication date: 2022-06-24

Abstract

The disclosure relates to a method and a device for processing stream modulation data, electronic equipment and a computer readable medium, and belongs to the technical field of data processing. The method comprises the following steps: acquiring audio stream modulation data of a target object from a mobile terminal with a recording function, and storing the audio stream modulation data for stream modulation; performing voice recognition on the audio stream modulation data through voice recognition service to obtain a voice text data stream modulation corresponding to the audio stream modulation data; carrying out structuralization processing on the voice text data to obtain structuralization streaming information of the target object; and acquiring a preset flow modulation report template, and generating a flow modulation report of the target object according to the flow modulation report template and the structured flow modulation information. The present disclosure can improve the efficiency of data flow adjustment and study and judgment by performing the processing of flow adjustment data and the generation of flow adjustment reports in a digital and automatic form.

Description

Method and device for processing streaming tone data, electronic equipment and computer readable medium

Technical Field

The present disclosure relates to the field of data processing technologies, and in particular, to a method and an apparatus for processing stream modulation data, an electronic device, and a computer-readable medium.

Background

The related information of confirmed cases of infectious diseases can be acquired by means of epidemiological regulation, and corresponding epidemiological survey reports are generated.

At present, epidemiological investigation does not have a digital or informatization scheme, a flow regulation report mainly depends on manual field recording and postscript writing, and analysis and study and judgment of the flow regulation report are mainly carried out manually, so that the overall efficiency and accuracy are poor.

In view of the above, there is a need in the art for a method that can improve the efficiency and accuracy of data flow and data research.

It is to be noted that the information disclosed in the above background section is only for enhancement of understanding of the background of the present disclosure, and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

The present disclosure is directed to a method for processing streaming data, a device for processing streaming data, an electronic device, and a computer readable medium, so as to improve efficiency of studying and judging streaming data and data to at least a certain extent.

According to a first aspect of the present disclosure, there is provided a method for processing streaming data, including:

acquiring audio stream modulation data of a target object from a mobile terminal with a recording function, and storing the audio stream modulation data;

performing voice recognition on the audio tone streaming data through voice recognition service to obtain voice text data corresponding to the audio tone streaming data;

carrying out structuralization processing on the voice text data to obtain structuralization streaming information of the target object;

and acquiring a preset flow modulation report template, and generating a flow modulation report of the target object according to the flow modulation report template and the structured flow modulation information.

In an exemplary embodiment of the present disclosure, after the obtaining of the structured flow modulation information of the target object, the method further includes:

sending the structured flow modulation information of the target object to a display terminal for displaying, and judging whether the structured flow modulation information is complete or not;

if the structured stream modulation information is incomplete, determining target information needing to be supplemented in the structured stream modulation information, determining a target audio clip in the audio stream modulation data according to the target information, and obtaining the complete structured stream modulation information according to the target audio clip;

and if the structured flow adjustment information is complete, continuing to execute the subsequent steps to generate a flow adjustment report of the target object.

In an exemplary embodiment of the present disclosure, the determining a target audio segment in the audio stream data according to the target information includes:

determining a target voice text segment in the voice text data according to the target information, and acquiring a text generation timestamp corresponding to the target voice text segment;

and determining an audio time stamp corresponding to the target audio fragment according to the text generation time stamp, and acquiring the target audio fragment in the audio stream modulation data according to the audio time stamp.

In an exemplary embodiment of the present disclosure, the performing a structuring process on the speech text data to obtain the structured flow information of the target object includes:

and carrying out structural processing on the voice text data through a regular expression to obtain structural flow modulation information of the target object.

and carrying out structural processing on the voice text data through a pre-trained language representation model to obtain structural flow modulation information of the target object.

In an exemplary embodiment of the disclosure, after the generating of the style report of the target object according to the style report template and the structured style information, the method further includes:

and determining a target coping strategy related to the flow regulation report according to the flow regulation report of the target object.

In an exemplary embodiment of the present disclosure, the determining, according to the retuning report of the target object, an objective coping strategy related to the retuning report includes:

acquiring track data of the target object from the flow regulation report of the target object, and acquiring site data of a target site related to the track data according to the track data of the target object;

and determining a risk level of the target place according to the place data of the target place, and determining an objective coping strategy aiming at the target place according to the risk level of the target place.

In an exemplary embodiment of the present disclosure, the determining a target coping strategy related to the flow regulation report according to the flow regulation report of the target object includes:

acquiring track data and state data of the target object from the flow regulation report of the target object;

determining the object type of the target object according to the trajectory data and the state data of the target object, and determining a target coping strategy aiming at the target object according to the object type of the target object.

acquiring contact object information of the target object from the flow regulation report of the target object, and obtaining a risk level corresponding to the contact object of the target object according to the contact object information;

and determining a target coping strategy aiming at the contact object according to the risk level corresponding to the contact object.

According to a second aspect of the present disclosure, there is provided a method for processing streaming data, including:

the method comprises the steps that a mobile terminal with a recording function obtains audio stream tone data of a target object and transmits the audio stream tone data to a server side from the mobile terminal;

the server side stores the audio frequency stream modulation data, and carries out voice recognition on the audio frequency stream modulation data in the server side through a voice recognition service to obtain voice text data corresponding to the audio frequency stream modulation data;

the server carries out structuralization processing on the voice text data to obtain structuralization streaming information of the target object;

the server side sends the structured stream modulation information of the target object to a display terminal for displaying, and adjusts the structured stream modulation information when the structured stream modulation information is incomplete to obtain the complete structured stream modulation information;

and the server side acquires a preset flow regulation report template and generates a flow regulation report of the target object according to the flow regulation report template and the complete structured flow regulation information.

According to a third aspect of the present disclosure, there is provided a processing apparatus of streaming data, comprising:

the audio data acquisition module is used for acquiring audio stream modulation data of a target object from a mobile terminal with a recording function and storing the audio stream modulation data;

the voice text recognition module is used for carrying out voice recognition on the audio frequency flow modulation data through voice recognition service to obtain voice text data corresponding to the audio frequency flow modulation data;

the voice text structuring module is used for carrying out structuring processing on the voice text data to obtain the structured stream modulation information of the target object;

and the flow modulation report generating module is used for acquiring a pre-configured flow modulation report template and generating a flow modulation report of the target object according to the flow modulation report template and the structured flow modulation information.

According to a fourth aspect of the present disclosure, there is provided a processing apparatus of streaming data, comprising:

the audio data acquisition module is used for acquiring audio stream modulation data of a target object through a mobile terminal with a recording function and transmitting the audio stream modulation data from the mobile terminal to a server;

the voice text recognition module is used for storing the audio frequency flow modulation data through the server side and carrying out voice recognition on the audio frequency flow modulation data in the server side through a voice recognition service to obtain voice text data corresponding to the audio frequency flow modulation data;

the voice text structuring module is used for performing structuring processing on the voice text data through the server end to obtain the structured flow modulation information of the target object;

the information integrity processing module is used for sending the structured flow modulation information of the target object to a display terminal through the server terminal for displaying, and adjusting the structured flow modulation information when the structured flow modulation information is incomplete to obtain the complete structured flow modulation information;

and the flow regulation report generation module is used for acquiring a pre-configured flow regulation report template through the server side and generating a flow regulation report of the target object according to the flow regulation report template and the complete structured flow regulation information.

According to a fifth aspect of the present disclosure, there is provided an electronic device comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to execute the processing method of the stream modulation data according to any one of the above items through executing the executable instruction.

According to a sixth aspect of the present disclosure, there is provided a computer readable medium, on which a computer program is stored, the computer program, when executed by a processor, implementing the method for processing caption data as described in any one of the above.

The exemplary embodiments of the present disclosure may have the following advantageous effects:

in the method for processing the streaming tone data according to the exemplary embodiment of the disclosure, the audio streaming tone data of the target object is acquired from the mobile terminal with the recording function, the audio streaming tone data is transmitted from the mobile terminal to the server, then the voice recognition is performed on the audio streaming tone data in the server through the voice recognition service, the voice text data is subjected to the structuring processing, and finally the streaming tone report of the target object is generated according to the structured streaming tone information of the target object. In the flow regulation data processing method in the exemplary embodiment of the disclosure, the whole data flow regulation and processing process is performed in a digital manner by introducing a computer-aided manner, on one hand, the efficiency and the accuracy of generating a flow regulation report can be improved, on the other hand, the speed is improved for subsequently carrying out infectious disease research and judgment according to the flow regulation report, and more time is strived for infectious disease prevention and control.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure. It is to be understood that the drawings in the following description are merely exemplary of the disclosure, and that other drawings may be derived from those drawings by one of ordinary skill in the art without the exercise of inventive faculty.

Fig. 1 is a schematic diagram illustrating an exemplary system architecture of a method and an apparatus for processing streaming data to which embodiments of the present invention may be applied;

FIG. 2 shows a flow diagram of a method of processing streamed tone data in an example embodiment of the disclosure;

FIG. 3 illustrates a flow diagram of file integrity checking and voice playback for an example embodiment of the present disclosure;

FIG. 4 illustrates a flow diagram for determining an objective coping strategy for a target site in an example embodiment of the present disclosure;

FIG. 5 shows a flowchart illustration of determining a target coping strategy for a target object according to an example embodiment of the present disclosure;

FIG. 6 illustrates a flow diagram for determining a target coping strategy for a contact object according to an example embodiment of the present disclosure;

FIG. 7 shows a flow diagram of a method of processing tone data according to another example embodiment of the present disclosure;

FIG. 8 schematically illustrates an architecture diagram of a processing system that streams tone data in accordance with a particular embodiment of the present disclosure;

fig. 9 shows a block diagram of a processing apparatus of streaming tone data of an example embodiment of the present disclosure;

fig. 10 shows a block diagram of a processing device of streaming tone data of another example embodiment of the present disclosure;

FIG. 11 illustrates a schematic structural diagram of a computer system suitable for use in implementing the electronic device of an embodiment of the present disclosure.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. Example embodiments may, however, be embodied in many different forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a thorough understanding of embodiments of the disclosure. One skilled in the relevant art will recognize, however, that the subject matter of the present disclosure can be practiced without one or more of the specific details, or with other methods, components, devices, steps, and the like. In other instances, well-known technical solutions have not been shown or described in detail to avoid obscuring aspects of the present disclosure.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus their repetitive description will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in the form of software, or in one or more hardware modules or integrated circuits, or in different networks and/or processor devices and/or microcontroller devices.

Fig. 1 is a schematic diagram illustrating a system architecture of an exemplary application environment to which a method and an apparatus for processing streaming data according to an embodiment of the present invention may be applied.

As shown in fig. 1, the system architecture 100 may include multiple ones of the

mobile terminals

101, 102, 103, a network 104, and a server 105. The network 104 is used to provide a medium for communication links between the

mobile terminals

101, 102, 103 and the server 105. The network 104 may include various connection types, such as wireless communication links and the like.

It should be understood that the number of mobile terminals, networks, and servers in fig. 1 is merely illustrative. There may be any number of mobile terminals, networks, and servers, as desired for implementation. For example, server 105 may be a server cluster comprised of multiple servers, or the like.

The

mobile terminals

101, 102, 103 may be various electronic devices having a processor including, but not limited to, smart phones, tablet computers, portable computers, and the like. The server 105 may be a server that provides various services. For example, the

mobile terminals

101, 102, 103 may obtain audio streaming data of the target object through the processor and transmit the audio streaming data to the server 105. The server 105 may perform voice recognition on the audio tone streaming data in the server through the voice recognition service to obtain voice text data corresponding to the audio tone streaming data, then perform structural processing on the voice text data to obtain structural tone streaming information of the target object, obtain a preset tone streaming report template, and generate a tone streaming report of the target object according to the tone streaming report template and the structural tone streaming information.

The epidemiological survey report is an abbreviation of epidemiological survey report, and is generally written by a disease control center of a government health department, a worker is sent by the disease control department to carry out on-the-spot survey (or telephone survey) on a confirmed case of an infectious disease, and information such as basic information condition, family condition, working condition, close contact person condition (abbreviated as close contact), places (track condition) which have been visited for the past several days (generally 7 days, 14 days or longer) and the like is recorded and is arranged into the epidemiological survey report.

Currently, epidemiological investigation has no digital or informatization scheme, and a flow regulation report mainly depends on manual field recording and postwriting, so that the efficiency is very low. Analysis and research and judgment of the flow regulation report are mainly carried out manually, computer assistance is rarely introduced, and the overall efficiency and accuracy are poor.

The present exemplary embodiment first provides a method for processing streaming data, which is applied to a server. Referring to fig. 2, the method for processing the tone modulation data may include the following steps:

and S210, acquiring audio stream tone data of the target object from the mobile terminal with the recording function, and storing the audio stream tone data.

And S220, carrying out voice recognition on the audio frequency stream tone data through the voice recognition service to obtain voice text data corresponding to the audio frequency stream tone data.

And step S230, carrying out structural processing on the voice text data to obtain structural flow modulation information of the target object.

And S240, acquiring a preset flow modulation report template, and generating a flow modulation report of the target object according to the flow modulation report template and the structured flow modulation information.

In the method for processing the tone streaming data according to the embodiment of the disclosure, the audio tone streaming data of the target object is acquired from the mobile terminal with the recording function, the audio tone streaming data is transmitted from the mobile terminal to the server, then the voice recognition is performed on the audio tone streaming data in the server through the voice recognition service, the voice text data is subjected to the structuring processing, and finally, the tone streaming report of the target object is generated according to the structured tone streaming information of the target object. In the flow regulation data processing method in the exemplary embodiment of the disclosure, the whole data flow regulation and processing process is performed in a digital manner by introducing a computer-aided manner, on one hand, the efficiency and the accuracy of generating a flow regulation report can be improved, on the other hand, the speed is improved for subsequently carrying out infectious disease research and judgment according to the flow regulation report, and more time is strived for infectious disease prevention and control.

The above steps of the present exemplary embodiment will be described in more detail with reference to fig. 3 to 6.

In step S210, audio tone data of the target object is acquired from the mobile terminal having the sound recording function, and the audio tone data is stored.

In this exemplary embodiment, a mobile terminal with a sound recording function, such as a mobile phone or a tablet App, may be used to record sound and video, or wirelessly control through a far-field sound pickup device to obtain audio stream modulation data of a target object, where the target object is a target stream modulation object, such as a diagnosed infectious disease patient. And after the audio stream modulation data of the target object is acquired through the mobile terminal, the audio stream modulation data is transmitted from the mobile terminal to the server side for further storage and processing.

The recording of the audio stream data may be performed through various media, such as a mobile phone, a tablet, a computer, or other devices with a microphone.

In practical application, taking a flow regulation report as an example, when a confirmed case of an infectious disease occurs in a certain place, flow regulation group staff firstly log in a flow regulation system after receiving a notice, create a new flow regulation task in the flow regulation system, and input basic flow regulation information, including time, place, confirmed person basic information, processing person, collaborating person and the like. The streaming system is a streaming system composed of a mobile terminal with a recording function, a server and a display terminal according to a processing method of streaming data in the exemplary embodiment of the present disclosure.

After receiving the task, the task handler immediately carries out the dispatching work. Firstly, a patient with confirmed diagnosis is transported to a safe place through a safe transportation means, and after the protective equipment is worn, a flow adjustment worker enters a flow adjustment room to perform flow adjustment work. The dispatching staff can carry the mobile terminal in this system, like cell-phone and far-field recording equipment, when carrying out the dispatching with the confirmed patient, record to whole conversation with cell-phone App or far-field recording equipment, when the recording, can return the recording data in real time to the server of rear end.

In step S220, voice recognition is performed on the audio tone data through a voice recognition service, so as to obtain voice text data corresponding to the audio tone data.

In this exemplary embodiment, when receiving the audio tone streaming data, the server at the back end stores the audio tone streaming data and sends the audio tone streaming data to the speech recognition service, where the speech recognition service performs speech recognition in real time, receives a text result of the speech recognition, obtains speech text data corresponding to the audio tone streaming data, and sends a result of the speech recognition to a queue inside the server at the back end.

In this exemplary embodiment, if the display terminal is connected to the server at the back end in real time, the user may also receive the audio data and the corresponding voice recognition result in the streaming engineering at the display terminal in real time. The real-time data connection and transmission protocol may use WebSocket and other protocols, wherein the WebSocket protocol is a network communication protocol for transmitting data in real time on the internet.

In step S230, the speech text data is structured to obtain structured tone information of the target object.

After receiving the text data of the voice recognition, the user can perform structural processing on the voice text data to obtain the structural flow modulation information of the target object. The structuring refers to a process of extracting valuable information from a manually written report text and arranging the valuable information into a formatted text which can be read, identified and processed by a computer.

The structured process can be completed in a server at the back end, taking a flow regulation report as an example, the structured content includes, but is not limited to, basic information (name, sex, age, living address, work unit, vaccination condition, etc.) of a confirmed patient, information of medical treatment (disease onset time, confirmed time, symptom, presence or absence of basic disease, inspection condition, etc.), information of activity track (date, time, place, trip mode, whether wearing a mask, stay time, contact person, etc.), close contact information (date, contact person name, contact person relationship, contact person mobile phone number, contact person identification card, etc.), and the structured flow regulation information obtained by the above information through the structured processing can be sent to a display terminal of a user for display.

In this exemplary embodiment, the voice text data may be structured through a regular expression, so as to obtain structured tone information of the target object.

For example, for information extraction for clinical typing of infectious diseases, the regular expression can be adopted as follows: "(infectious disease a | infectious disease a is totally called)? {0,2} (diagnosed cases | clinical typing)? {0,2} ((mild | normal | severe | critical) type | asymptomatic) ". And matching the regular expression serving as a template with the character strings in the searched voice text data, and if the regular expression is successfully matched with the character strings in the voice text data, acquiring the 3 rd capture group, and extracting specific clinical typing.

In this example embodiment, the voice text data may be further subjected to a structured processing by a pre-trained language representation model, so as to obtain structured flow information of the target object.

For example, for the information extraction of the address and name types, a deep learning-based model can be adopted. Among them, an open source bert (bidirectional Encoder replication from transformer) pre-training model can be used, and based on NER (Named Entity Recognition) task made by FastHan (chinese natural language processing tool) open source library of Python. For the location names extracted by the NER, if the location name positions are connected with each other, or the distance between the location name positions is short (for example, not more than 5 characters), it can be treated as an address. In addition, some public place names can be supplemented in a dictionary mode if the public place names are not identified, and a dictionary can be built for filtering and eliminating the place names with wrong model identification.

After the structured stream key information of the target object is obtained, the processing method of stream key data in this exemplary embodiment may further perform integrity check and voice playback of a file, so as to ensure accuracy and integrity of the stream key information. As shown in fig. 3, the integrity check and speech echo method may specifically include the following steps:

and S310, sending the structured flow modulation information of the target object to a display terminal for displaying, and judging whether the structured flow modulation information is complete.

Firstly, the structured flow information of the target object is sent to a display terminal for displaying, and a user judges whether the structured flow information is complete and accurate.

And S320, if the structured stream tone information is incomplete, determining target information needing to be supplemented in the structured stream tone information, determining a target audio clip in the audio stream tone data according to the target information, and obtaining complete structured stream tone information according to the target audio clip.

In this example embodiment, when the target audio segment in the audio stream tone data is determined according to the target information, the target speech text segment in the speech text data may be determined according to the target information, and a text generation timestamp corresponding to the target speech text segment is obtained, and then an audio timestamp corresponding to the target audio segment is determined according to the text generation timestamp, and the target audio segment in the audio stream tone data is obtained according to the audio timestamp.

The user can reacquire partial target information needing to be supplemented in the structured stream-modulation information by performing voice echo on the target audio clip, so that complete structured stream-modulation information is obtained.

And S330, if the structured flow adjustment information is complete, continuing to execute the subsequent steps to generate a flow adjustment report of the target object.

In addition, the user can manually supplement the missing information in the process or require on-site flow adjustment workers to make questions based on the integrity of the structured information at the display terminal, so that the complete and accurate structured flow adjustment information is obtained.

In step S240, a preset scheduling report template is obtained, and a scheduling report of the target object is generated according to the scheduling report template and the structured scheduling information.

After the structured flow regulation information is complete, a user can generate a flow regulation report with a uniform specification at any time according to a pre-configured flow regulation report template, and the generated report can be directly issued on a platform. The published flow modulation report supports modifications that can be reissued after completion of the supplemental information. The whole process is automatically completed with the help of a computer, and manual recording and report writing are basically not needed.

In the present exemplary embodiment, after the flow adjustment report of the target object is generated, the target coping strategy related to the flow adjustment report may be determined according to the flow adjustment report of the target object. Specifically, the coping strategies can be determined according to the following methods.

In the present exemplary embodiment, as shown in fig. 4, determining a target coping strategy related to a flow regulation report according to the flow regulation report of a target object may specifically include the following steps:

and S410, acquiring the track data of the target object from the flow regulation report of the target object, and acquiring the site data of the target site related to the track data according to the track data of the target object.

And S420, determining the risk level of the target place according to the place data of the target place, and determining a target coping strategy aiming at the target place according to the risk level of the target place.

By combining the trajectory data of the target object and relevant place data of the target place related to the trajectory, such as place type, average people flow condition, ventilation condition, protection condition, visit time and the like, the risk level of the target place can be calculated, and then the target coping strategy for the target place is determined according to the risk level of the target place. If the risk of the target site is large, the high-risk site can be closed according to the judgment result, or other necessary administrative measures can be adopted for intervention and control.

In the present exemplary embodiment, as shown in fig. 5, determining a target coping strategy related to a flow regulation report according to the flow regulation report of a target object may further include the following steps:

and step S510, acquiring the track data and the state data of the target object from the flow regulation report of the target object.

And S520, determining the object type of the target object according to the track data and the state data of the target object, and determining a target coping strategy aiming at the target object according to the object type of the target object.

Based on the information of time, space track, attack time and the like of the target object, the source tracing can be automatically carried out, the probability that the patient confirmed to be the zero patient is calculated, the definite operation direction is guided for disease control personnel, and corresponding measures are executed according to the type of the patient.

In the present exemplary embodiment, as shown in fig. 6, determining a target coping strategy related to a flow regulation report according to the flow regulation report of a target object may further include the following steps:

and S610, obtaining contact object information of the target object from the flow regulation report of the target object, and obtaining the risk level corresponding to the contact object of the target object according to the contact object information.

And S620, determining a target coping strategy for the contact object according to the risk level corresponding to the contact object.

The system can automatically deduce the tight-joint relation according to the contact object information of the target object, and remind disease control personnel of paying attention to which people are high-risk people, so that the prevention, control and interception efficiency is improved.

Because the flow scheduling task has different specific execution measures in each city, the service flow related in the system generally needs to be flexibly configurable, a BPMN (Business Process Modeling Notation) flow engine support (contained in a back-end server) is generally adopted, the system designs the BPMN flow according to different flows, a design result is submitted to the flow engine, and the flow engine flexibly schedules the workflow according to the design, so as to meet the complex service logic flow condition. The BPMN is a general and standard language for process modeling, and is used to draw a business process diagram so as to better understand business processes and relationships among various departments.

In addition, the present exemplary embodiment further provides a method for processing the stream modulation data, which is applied to the whole stream modulation system. Referring to fig. 7, the method for processing the tone modulation data may include the following steps:

and S710, the mobile terminal with the recording function acquires the audio stream tone data of the target object and transmits the audio stream tone data to the server side from the mobile terminal.

And S720, the server stores the audio frequency modulation data, and performs voice recognition on the audio frequency modulation data in the server through a voice recognition service to obtain voice text data corresponding to the audio frequency modulation data.

And S730, the server carries out structural processing on the voice text data to obtain structural flow modulation information of the target object.

And step S740, the server side sends the structured flow modulation information of the target object to the display terminal for displaying, and adjusts the structured flow modulation information when the structured flow modulation information is incomplete, so as to obtain complete structured flow modulation information.

And S750, the server side acquires a preset flow modulation report template and generates a flow modulation report of the target object according to the flow modulation report template and the complete structured flow modulation information.

The specific steps of the method for processing the flow modulation data applied to the flow modulation system are similar to those in fig. 2, and are not described again here.

Fig. 8 schematically shows an architecture diagram of a processing system for streaming data according to an embodiment of the present disclosure, which is an illustration of the above steps in the present exemplary embodiment, and the architecture diagram may specifically include the following parts:

1. mobile phone or tablet terminal App: the remote control system is mainly used for recording and recording, wirelessly controlling far-field pickup equipment, transmitting audio and video data with a server at the rear end and the like. The audio and video data can also be synchronously stored in the mobile phone or the tablet terminal, and if the data received by the server is incomplete due to network transmission problems, the complete audio and video data can be acquired from the mobile phone or the tablet terminal again.

2. A server: the central control component of the system is mainly used for receiving, processing and storing audio and video data, communicating with a voice recognition service, receiving a voice recognition result and sending the voice recognition result to a browser end of a user; the service logic flow processing, data storage and the like of system operation are also completed by the back-end server; the structuralization of the text speech recognition result, and the calculation and study and judgment aiming at the structured data are mainly completed in a back-end server.

3. Voice recognition service: for recognizing words from the speech data provided by the server and for communicating the result back to the server.

4. User browser side: and the display terminal is used for providing an interface used by the user, acquiring data from the server and displaying the data to the user.

5. Far-field sound pickup equipment: the portable equipment for enhancing the recording effect comprises a microphone array, and a mobile phone terminal App carries out remote control through wireless technologies such as Bluetooth or Wifi direct connection and the like, so that high-quality voice data can be transmitted back to the mobile phone; the device is optional, and the recording function can be replaced by a mobile phone.

The scheme only comprises a core link, and the general function requirements are more under the condition of falling to the ground in a real scene. For example, the above solution may further include user management, authority management, organization structure management, workflow configuration management, style report template configuration, text voice answering, voice role-based recognition, style report publishing, case information base, style material information base, search function, statistical analysis, cockpit management, and the like.

It should be noted that although the various steps of the methods of the present disclosure are depicted in the drawings in a particular order, this does not require or imply that these steps must be performed in this particular order, or that all of the depicted steps must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions, etc.

Furthermore, the disclosure also provides a device for processing the stream modulation data. Referring to fig. 9, the apparatus for processing the caption data may include an audio data acquisition module 910, a speech text recognition module 920, a speech text structuring module 930, and a caption report generating module 940. Wherein:

the audio data acquiring module 910 may be configured to acquire audio tone data of a target object from a mobile terminal having a recording function, and store the audio tone data;

the voice text recognition module 920 may be configured to perform voice recognition on the audio tone streaming data through a voice recognition service to obtain voice text data corresponding to the audio tone streaming data;

the speech text structuring module 930 may be configured to perform a structuring process on the speech text data to obtain structured tone information of the target object;

the flow modulation report generating module 940 may be configured to obtain a pre-configured flow modulation report template, and generate a flow modulation report of the target object according to the flow modulation report template and the structured flow modulation information.

In some exemplary embodiments of the present disclosure, a processing apparatus of stream tone data provided by the present disclosure may further include an information integrity processing module, and the information integrity processing module may include a stream tone information display unit, a target section determination unit, an audio section hearing back unit, and an information integrity determination unit. Wherein:

the flow modulation information display unit can be used for sending the structured flow modulation information of the target object to the display terminal for displaying and judging whether the structured flow modulation information is complete or not;

the target segment determining unit may be configured to determine target information to be supplemented in the structured stream key information if the structured stream key information is incomplete, determine a target audio segment in the audio stream key data according to the target information, and obtain complete structured stream key information according to the target audio segment;

the information integrity determination unit may be configured to continue to execute subsequent steps to generate a flow regulation report of the target object if the structured flow regulation information is intact.

In some exemplary embodiments of the present disclosure, the target section determining unit may include a text time stamp determining unit and an audio time stamp determining unit. Wherein:

the text timestamp determining unit may be configured to determine a target speech text segment in the speech text data according to the target information, and acquire a text generation timestamp corresponding to the target speech text segment;

the audio time stamp determining unit may be configured to determine an audio time stamp corresponding to the target audio segment according to the text generation time stamp, and acquire the target audio segment in the audio stream modulation data according to the audio time stamp.

In some exemplary embodiments of the present disclosure, the speech text structuring module 930 may include a regular expression structuring unit, and may be configured to perform a structuring process on the speech text data through a regular expression, so as to obtain the structured tone information of the target object.

In some exemplary embodiments of the present disclosure, the speech text structuring module 930 may further include a language model structuring unit, which may be configured to perform a structuring process on the speech text data through a pre-trained language representation model to obtain the structured flow information of the target object.

In some exemplary embodiments of the present disclosure, a processing apparatus of tone flow data provided by the present disclosure may further include a coping strategy determining module, which may be configured to determine a target coping strategy related to the tone flow report according to the tone flow report of the target object.

In some exemplary embodiments of the present disclosure, the coping strategy determining module may include a venue data acquiring unit and a venue coping strategy determining unit. Wherein:

the location data acquiring unit may be configured to acquire trajectory data of the target object from the streaming report of the target object, and acquire location data of a target location related to the trajectory data according to the trajectory data of the target object;

the location coping strategy determining unit may be configured to determine a risk level of the target location from the location data of the target location, and determine the objective coping strategy for the target location from the risk level of the target location.

In some exemplary embodiments of the present disclosure, the coping strategy determining module may further include a status data acquiring unit and a target object coping strategy determining unit. Wherein:

the state data acquisition unit can be used for acquiring the track data and the state data of the target object from the flow regulation report of the target object;

the target object coping strategy determining unit may be configured to determine an object type of the target object from the trajectory data and the state data of the target object, and determine the target coping strategy for the target object from the object type of the target object.

In some exemplary embodiments of the present disclosure, the coping strategy determining module may further include a contact object risk level determining unit and a contact object coping strategy determining unit. Wherein:

the contact object risk level determining unit may be configured to acquire contact object information of the target object from the flow regulation report of the target object, and obtain a risk level corresponding to the contact object of the target object according to the contact object information;

the contact object coping strategy determining unit may be configured to determine a target coping strategy for the contact object according to a risk level corresponding to the contact object.

Furthermore, the disclosure also provides a device for processing the stream modulation data. Referring to fig. 10, the processing device of the caption data may include an audio data acquisition module 1010, a speech text recognition module 1020, a speech text structuring module 1030, an information integrity processing module 1040, and a caption report generation module 1050. Wherein:

the audio data obtaining module 1010 may be configured to obtain audio streaming data of a target object through a mobile terminal having a recording function, and transmit the audio streaming data from the mobile terminal to a server;

the voice text recognition module 1020 may be configured to store the audio tone streaming data through the server, and perform voice recognition on the audio tone streaming data in the server through a voice recognition service to obtain voice text data corresponding to the audio tone streaming data;

the voice text structuring module 1030 may be configured to perform structuring processing on the voice text data through the server side to obtain structured streaming information of the target object;

the information integrity processing module 1040 may be configured to send the structured streaming information of the target object to the display terminal through the server for display, and adjust the structured streaming information when the structured streaming information is incomplete, to obtain complete structured streaming information;

the flow modulation report generating module 1050 may be configured to obtain a pre-configured flow modulation report template through the server, and generate a flow modulation report of the target object according to the flow modulation report template and the complete structured flow modulation information.

The details of each module/unit in the above-mentioned processing apparatus for streaming data have been described in detail in the corresponding method embodiment section, and are not described herein again.

FIG. 11 illustrates a schematic structural diagram of a computer system suitable for use with the electronic device to implement an embodiment of the invention.

It should be noted that the computer system 1100 of the electronic device shown in fig. 11 is only an example, and should not bring any limitation to the functions and the scope of the application of the embodiment of the present invention.

As shown in fig. 11, the computer system 1100 includes a Central Processing Unit (CPU)1101, which can perform various appropriate actions and processes in accordance with a program stored in a Read Only Memory (ROM)1102 or a program loaded from a storage section 1108 into a Random Access Memory (RAM) 1103. In the RAM 1103, various programs and data necessary for system operation are also stored. The CPU 1101, ROM 1102, and RAM 1103 are connected to each other by a bus 1104. An input/output (I/O) interface 1105 is also connected to bus 1104.

The following components are connected to the I/O interface 1105: an input portion 1106 including a keyboard, mouse, and the like; an output portion 1107 including a signal such as a Cathode Ray Tube (CRT), a Liquid Crystal Display (LCD), and the like, and a speaker; a storage section 1108 including a hard disk and the like; and a communication section 1109 including a network interface card such as a LAN card, a modem, or the like. The communication section 1109 performs communication processing via a network such as the internet. A driver 1110 is also connected to the I/O interface 1105 as necessary. A removable medium 1111 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is mounted on the drive 1110 as necessary, so that a computer program read out therefrom is mounted into the storage section 1108 as necessary.

In particular, according to an embodiment of the present invention, the processes described above with reference to the flowcharts may be implemented as computer software programs. For example, embodiments of the invention include a computer program product comprising a computer program embodied on a computer-readable medium, the computer program comprising program code for performing the method illustrated in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network through the communication portion 1109 and/or installed from the removable medium 1111. When the computer program is executed by a Central Processing Unit (CPU)1101, various functions defined in the system of the present application are executed.

It should be noted that the computer readable media shown in the present disclosure may be computer readable signal media or computer readable storage media or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the foregoing. More specific examples of the computer readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In contrast, in the present disclosure, a computer-readable signal medium may include a propagated data signal with computer-readable program code embodied therein, for example, in baseband or as part of a carrier wave. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination thereof. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams or flowchart illustration, and combinations of blocks in the block diagrams or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.

As another aspect, the present application also provides a computer-readable medium, which may be contained in the electronic device described in the above embodiment; or may exist separately without being assembled into the electronic device. The computer readable medium carries one or more programs which, when executed by an electronic device, cause the electronic device to implement the method as described in the embodiments above.

It should be noted that although in the above detailed description several modules of the device for action execution are mentioned, this division is not mandatory. Indeed, the features and functionality of two or more of the modules described above may be embodied in one module, in accordance with embodiments of the present disclosure. Conversely, the features and functions of one module described above may be further divided into embodiments by a plurality of modules.

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains.

It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A method for processing stream modulation data is characterized by comprising the following steps:

2. The method for processing streaming data according to claim 1, wherein after the obtaining of the structured streaming information of the target object, the method further comprises:

if the structured stream modulation information is incomplete, determining target information needing to be supplemented in the structured stream modulation information, determining a target audio fragment in the audio stream modulation data according to the target information, and obtaining the complete structured stream modulation information according to the target audio fragment;

3. The method for processing the tone data according to claim 2, wherein the determining the target audio segment in the audio tone data according to the target information comprises:

and determining an audio time stamp corresponding to the target audio clip according to the text generation time stamp, and acquiring the target audio clip in the audio stream modulation data according to the audio time stamp.

4. The method for processing the caption data according to claim 1, wherein the performing the structuring process on the speech text data to obtain the structured caption information of the target object includes:

5. The method for processing the caption data according to claim 1, wherein the performing the structuring process on the speech text data to obtain the structured caption information of the target object includes:

6. The method of processing the flow modulation data according to claim 1, wherein after the generating the flow modulation report of the target object according to the flow modulation report template and the structured flow modulation information, the method further comprises:

7. The method for processing flow modulation data according to claim 6, wherein the determining a target coping strategy related to the flow modulation report according to the flow modulation report of the target object comprises:

8. The method for processing flow modulation data according to claim 6, wherein the determining a target coping strategy related to the flow modulation report according to the flow modulation report of the target object comprises:

9. The method for processing flow modulation data according to claim 6, wherein the determining a target coping strategy related to the flow modulation report according to the flow modulation report of the target object comprises:

10. A method for processing stream modulation data is characterized by comprising the following steps:

11. An apparatus for processing streaming data, comprising:

the voice text structuring module is used for carrying out structuring processing on the voice text data to obtain the structured flow modulation information of the target object;

12. An apparatus for processing streaming data, comprising:

the audio data acquisition module is used for acquiring audio stream modulation data of a target object through a mobile terminal with a recording function and transmitting the audio stream modulation data to a server side from the mobile terminal;

the voice text structuring module is used for carrying out structuring processing on the voice text data through the server end to obtain the structured flow regulation information of the target object;

and the flow regulation report generation module is used for acquiring a pre-configured flow regulation report template through the server side and generating a flow regulation report of the target object according to the flow regulation report template and the complete structural flow regulation information.

13. An electronic device, comprising:

a processor; and

a memory for storing one or more programs which, when executed by the processor, cause the processor to implement the method of processing of streaming data according to any of claims 1 to 10.

14. A computer-readable medium, on which a computer program is stored, which program, when being executed by a processor, is characterized by carrying out the method of processing the streamed tone data as claimed in any one of claims 1 to 10.