JP7160454B2 - Method, apparatus and system, electronic device, computer readable storage medium and computer program for outputting information - Google Patents

Description

TECHNICAL FIELD Embodiments of the present disclosure relate to the field of automotive network technology, and specifically to methods, apparatus and systems, electronic devices, computer-readable storage media and computer programs for outputting information.

At present, online dispatch safety accidents are frequent, the number of online dispatch is large, illegal accidents occur without warning, the safety technical measures of online dispatch platforms are limited, and there are serious delays, causing accidents. As a result, even if the victim is cautious and reports to the police, there is a problem that the reception of the report is delayed and it is difficult to respond.

At present, the accident abnormality handling mechanism of the online dispatch platform is that when the victim feels threatened in advance, he/she can connect the phone or send a message to the contact platform through the emergency contact or distress call on the mobile APP side. is to send Based on this mechanism, when an illegal or criminal act occurs, the victim is constrained by the environment at the time and cannot normally use the mobile phone to rescue or call the police, while the emergency contact After receiving the rescue call and information, contact the police, and contact the online reservation platform to obtain the location information of the accident vehicle. Vehicle characteristics should be provided.

In the existing in-vehicle intelligent alarm plan, by turning on the camera of the in-vehicle system, the image information of the person in the front row is captured, and the hidden dangerous scene is detected by image analysis to realize intelligent alarm countermeasures.

However, the existing intelligent alarm system relies on the trip computer camera, which can first turn on the trip computer camera and then capture the person image information, but in the danger scene, it is impossible to turn on the camera immediately. is. After turning on the camera, the entire interface of the trip computer becomes the camera screen, easy to be detected and turned off, and after turning off the camera, the alarm system is immediately disabled.

Embodiments of the present disclosure propose methods, apparatus and systems, electronic devices, computer-readable storage media and computer programs for outputting information.

In a first aspect, embodiments of the present disclosure include collecting audio data by at least two audio collecting devices at different locations in the vehicle, respectively, and performing echo cancellation on the collected at least two-way audio data. obtaining text information of at least two users by inputting the echo-removed data into at least two speech recognition engines, respectively, and performing speech recognition; Upload the location of at least two audio collecting devices and the corresponding text information of at least two users to the service side to analyze the text information through the learning model, and if the analysis result meets the alarm conditions, the alarm information. and a method for outputting information including.

In some embodiments, the method includes inputting the echo-cancelled data into at least two emotion recognition engines, respectively, and performing emotion recognition to obtain emotion information of at least two users; The location of at least two audio collecting devices, the corresponding text information of at least two users, and the emotion information of two users for the service side to analyze the text information and sentiment through a pre-trained deep learning model. uploading to the service side, and outputting alarm information if the analysis result satisfies alarm conditions.

In a second aspect, embodiments of the present disclosure include receiving at least two audio gathering device locations and corresponding textual information of at least two users uploaded from a vehicle; , obtaining the probability that the conversation is abnormal by composing the textual information of the two users into a conversation stream, inputting the conversation stream into a pre-trained deep learning model, and obtaining the probability that the conversation is anomalous; above a first threshold of, triggering an alarm.

In some embodiments, the method comprises: obtaining attribute information and location information of the vehicle; searching for the closest traffic police to the vehicle based on the location information; transmitting to the traffic police;

In some embodiments, the method includes receiving the emotional information of at least two users, the probability that the conversation is abnormal is less than a predetermined second threshold, and the passenger's emotional information includes a feeling of fear. and triggering an alarm if so.

In some embodiments, the method further includes triggering an alarm in response to detecting a loss of circuit connection in the vehicle's audio collection device.

In some embodiments, the method includes periodically sending test questions to the vehicle for passengers to answer and receiving response information determined to be successful by the deep learning model within a predetermined time. if not, triggering an alarm.

In a third aspect, embodiments of the present disclosure provide an audio collection unit arranged to collect audio data respectively by at least two audio collection devices at different locations in the vehicle; an echo canceling unit arranged to perform echo canceling on data; A voice recognition unit arranged to obtain text information, and at least two audio collection device locations and corresponding at least two users for the service side to perform analysis of the text information through pre-trained deep learning models. an information uploading unit arranged to upload text information to a service side and output alarm information if the analysis result satisfies an alarm condition.

In some embodiments, the apparatus is arranged to obtain the emotional information of at least two users by inputting the echo-cancelled data into at least two emotion recognition engines, respectively, and performing emotion recognition. and the information uploading unit further comprises a position of at least two audio collecting devices, corresponding at least two audio collecting devices, for the service side to perform analysis of text information and emotion through pre-trained deep learning models. It is arranged to upload the text information of one user and the emotion information of two users to the service side, and output the alarm information when the analysis result meets the alarm conditions.

In a fourth aspect, embodiments of the present disclosure provide a receiving unit arranged to receive at least two audio collection device locations and corresponding at least two user textual information uploaded from a vehicle; A text stitching unit arranged to compose the textual information of the two users into a dialogue stream based on the positions of the two audio collection devices, and by inputting the dialogue stream into a pre-trained deep learning model, for outputting information including a speech recognition unit arranged to obtain a probability that the speech is abnormal and an alarm unit arranged to trigger an alarm if the probability is higher than a predetermined first threshold; Regarding the device.

In some embodiments, the alarm unit further obtains the attribute information and location information of the vehicle, searches for the closest traffic police to the vehicle based on the location information, and sends the attribute information and location information of the vehicle to the traffic police. to send to, arranged as follows.

In some embodiments, the receiving unit is further arranged to receive the emotional information of at least two users, and the alarm unit is further configured that the probability that the speech is abnormal is less than a predetermined second threshold. , is arranged to trigger an alarm if the passenger's emotional information includes a sense of fear.

In some embodiments, the alarm unit is further arranged to trigger an alarm in response to detecting a loss of circuit connection in the vehicle's audio collection device.

In some embodiments, the alarm unit also periodically sends test questions to the vehicle for passengers to answer, and receives response information determined to be normal by the deep learning model within a predetermined period of time. If not, it is arranged to trigger an alarm.

In a fifth aspect, an embodiment of the present disclosure collects audio data respectively by at least two audio collection devices at different locations in the vehicle, performs echo cancellation on the collected at least two-way audio data, The echo-removed data are respectively input to at least two speech recognition engines and speech recognition is performed to obtain text information of at least two users, and to obtain the positions of at least two audio collecting devices and the corresponding at least a vehicle arranged to upload textual information of two users to a service side, and receiving locations of at least two audio collection devices and corresponding textual information of at least two users uploaded from the vehicle; Based on the positions of the two audio collection devices, the textual information of the two users is composed into a conversation stream, and the conversation stream is input to a pre-trained deep learning model to obtain the probability that the conversation is abnormal. , a service side arranged to trigger an alarm if the probability is higher than a predetermined first threshold.

In a sixth aspect, embodiments of the present disclosure include one or more processors and a storage device storing one or more programs, the one or more programs being executed by the one or more processors. When it relates to an electronic device for outputting information that causes one or more processors to implement the method according to any of the first and second aspects.

In a seventh aspect, an embodiment of the present disclosure is a computer readable medium having stored thereon a computer program which, when executed by a processor, performs the method of any of the first and second aspects. A computer-readable storage medium for implementation.

In an eighth aspect, an embodiment of the present disclosure relates to a computer program for implementing the method according to any of the first and second aspects when the computer program is executed by a processor.

The present invention is mainly for personal safety issues in in-vehicle scenes, by performing effective real-time voice monitoring, and by monitoring the user's voice conversations in in-vehicle scenes, the user's conversation content can be obtained and danger can be predicted in advance. , and take preventive measures in a timely manner, solving many potential safety problems in the car-moving scene.

To make other features, objects and advantages of the present disclosure more apparent, reference is made to the following detailed description of non-limiting examples made with reference to the drawings.
1 is an exemplary system architecture diagram to which an embodiment of the present disclosure is applicable; FIG. 4 is a flowchart of one embodiment of a method for outputting information according to the present disclosure; 4 is a flowchart of another embodiment of a method for outputting information according to the present disclosure; 1 is a schematic diagram illustrating one application scene of a method for outputting information according to the present disclosure; FIG. 1 is a schematic block diagram of one embodiment of an apparatus for outputting information according to the present disclosure; FIG. Fig. 2 is a schematic block diagram of another embodiment of an apparatus for outputting information according to the present disclosure; 1 is a schematic configuration diagram of a computer system of an electronic device applied to implement an embodiment of the present disclosure; FIG.

The present disclosure will now be described in more detail with reference to the drawings and examples. It can be understood that the specific embodiments described herein are only illustrative of the relevant invention and are not intended to limit the invention. For ease of explanation, only the parts related to the related inventions are shown in the drawings.

It should be noted that embodiments of the present disclosure and features in embodiments can be combined with each other where not inconsistent. Hereinafter, the present disclosure will be described in detail according to embodiments with reference to the drawings.

FIG. 1 shows an exemplary system architecture 100 of an embodiment of a method or apparatus for outputting information to which the present disclosure may be applied.

As shown in FIG. 1, system architecture 100 may include vehicle 101, network 102 and server 103, where vehicle 101 is equipped with audio collection devices 1011, 1012, 1013, 1014, and controller 1015. may The controller 1015 can perform steps 201-205. Network 102 is used to provide a medium for communication links between vehicle 101 and server 103 . Network 1012 may include various connection types such as wired, wireless communication links, or fiber optic cables, for example.

Passengers can use vehicle 101 to interact with server 103 over network 102 to send and receive messages and the like. Vehicle 101 may be fitted with multiple audio collection devices. Vehicle 101 can perform voice recognition locally after voice is detected. Each audio gathering device can receive one way of audio data, and each way of audio data is speech recognized by one speech recognition engine, thus each audio gathering device can be used by one user. corresponding to the text information of Based on the position of the audio collection device, the identity of the user corresponding to the audio data of the way can be determined, for example, the text information obtained by speech recognition of the sound collected near the driving position belongs to the driver.

The number of audio collection devices attached to vehicle 101 is not limited to two. It may be three or more. The purpose of installing multiple audio collection devices is primarily to determine if the driver poses a threat to the passenger by recognizing the location of the speaker. The number of audio collection devices can match the maximum passenger capacity.

The server 103 may be a server that provides various services, for example an alarm server that provides text analysis on speech recognition results uploaded from the vehicle 101 . The alarm server is installed with a neural network model for stitching the received text information into the conversation stream based on the user's identity, and then determining whether the conversation satisfies the alarm conditions by the neural network model. can be done. If the alarm conditions are met, it will trigger an alarm and notify the traffic police near the vehicle of the vehicle's location information and attribute information (license plate, vehicle type, vehicle owner identity information, telephone, etc.).

Note that the server may be hardware or software. Where the server is hardware, it may be implemented as a distributed server cluster of multiple servers, or it may be implemented as a single server. If the server is software, it may be implemented as multiple software or software modules (e.g., multiple software or software modules for providing distributed services) or as a single software or software module. . Here, it is not specifically limited.

It should be noted that the method for outputting information according to embodiments of the present disclosure may generally be performed by vehicle 101 and server 103, and correspondingly, the device for outputting information is generally It is arranged in the vehicle 101 and the server 103 . One server can serve multiple cars, which constitute an alarm system.

It should be understood that the numbers of terminals, vehicles, networks, and servers in FIG. 1 are merely exemplary. Any number of vehicles, networks, and servers can be provided as desired.

With continued reference to FIG. 2, a flow 200 of one embodiment of a method for outputting information in accordance with the present invention is shown. A method for outputting such information is applied to a vehicle and includes the following steps.

At step 201, audio data is collected by at least two audio collection devices at different locations in the vehicle, respectively.

In this embodiment, the subject performing the method for outputting information (eg the vehicle shown in FIG. 1) collects audio data respectively by at least two audio collection devices located at different locations in the vehicle. Audio collection devices may include devices for collecting audio such as microphones, pickups, tape recorders, and the like. Each audio collection device collects one way of data. The audio collection device is attached to the side of the seat and can collect the voice of the user seated in the seat. From the location of the audio collecting device, it can be determined to which user the collected audio data belongs. One of the two audio gathering devices is the audio gathering device used by the driver. Other audio collection devices are used by passengers. Generally, four audio collection devices can be deployed as shown in FIG.

At step 202, echo cancellation is performed on the collected at least two-way audio data.

In this embodiment, for the Acoustic Echo Cancellation (AEC) problem, the most popular algorithm today is the adaptive filter-based echo cancellation algorithm. By adjusting the filter weight vector using different adaptive filter algorithms, estimating the approximate echo path and approximating the actual echo path, an estimated echo signal is obtained and the pure speech and echo are separated. Echo cancellation was achieved by removing this signal from the mixed signal.

In step 203, the echo-removed data are respectively input to at least two speech recognition engines for speech recognition to obtain text information of at least two users.

In this example, each audio collection device corresponds to one speech recognition engine. A speech recognition engine employs speech recognition technology, also called Automatic Speech Recognition (ASR), whose goal is to translate the lexical content in human speech into computer to convert it into a readable input. In the solution of the present disclosure, speech recognition is started without wakeup.

In step 204, the locations of the at least two audio collecting devices and the corresponding text information of the at least two users are uploaded to the service side for analysis, so that the service side performs analysis of the text information through a pre-trained deep learning model; If the result satisfies the alarm condition, output the alarm information.

In this embodiment, after the end point of the speech is detected, the recognition of the text information can be started and uploaded to the service side, or the speech recognition can be performed in real time and the recognition result can be uploaded in real time. good. The speech recognition results of the audio data of each user may be uploaded by different transmission devices, or one transmission device may be shared to upload the text information of a plurality of users in one package. To save resources, different users' text information can also be reported by time. The reported text information carries the location indicator of the audio collection device, which the service can use to determine whose story the received text information is about. If there is a resource conflict in reporting the text information by time, the passenger's text information may be reported first, followed by the driver's text information. When reporting by time, it is necessary to distinguish chronological sequences of conversations by adding timestamps.

After the text information is reported to the service side, the service side analyzes the text information through a pre-trained deep learning model, and outputs alarm information if the analysis result meets the alarm conditions.

In some optional embodiments of the present example, the method inputs the echo-cancelled data into at least two emotion recognition engines, respectively, to perform emotion recognition, thereby providing at least two users with In order to obtain emotional information and the service side to analyze the textual information and sentiment through pre-trained deep learning models, the locations of at least two audio collecting devices, the corresponding textual information of at least two users, and two uploading the human user's emotion information to the service side, and outputting the alarm information if the analysis result meets the alarm condition. The emotion recognition engine, which may be a neural network classifier, extracts speech features to determine the user's emotion, such as tension, fear, joy, sadness, etc., and typically identifies the user's feeling of fear. Voice trembles. To simplify the training process, a secondary classifier may be used to recognize the probability that the user's emotion is fear. During training, the voice of the user who felt fear is used as a positive sample for training.

If the passenger is blackmailed into speaking normally according to the driver's request, it cannot judge whether the text information recognized by voice is abnormal. Recognizing emotional information is to compensate for the inability of voice-recognized text information to express the user's mental state. If the text information is normal, but the passenger's emotions are abnormal, the driver is likely to intimidate the passenger into a normal conversation, thus misleading the voice monitoring.

Still referring to FIG. 3, another example flow 300 of a method for outputting information is shown. The method flow 300 for outputting the information is applied on the service side and includes the following steps.

At step 301, the locations of at least two audio collection devices uploaded from a vehicle and the corresponding textual information of at least two users are received.

In this embodiment, the executing entity of the method for outputting information (for example, the service side shown in FIG. 1) receives the location of at least two audio collection devices uploaded from the vehicle via a wireless connection and the corresponding at least Receive text information for two users.

At step 302, the textual information of the two users is organized into a conversation stream based on the locations of the at least two audio collection devices.

In this embodiment, the user to whom the text information belongs can be distinguished based on the location corresponding to the received text information. Then, stitch different user stories into a complete dialogue based on the time the text information was received. When the vehicle reports textual information by time, it gives a time stamp, and the service side stitches the conversation stream based on the time stamp.

At step 303, the probability that the dialogue is abnormal is obtained by inputting the dialogue stream into a pre-trained deep learning model.

In this embodiment, the deep learning model can be trained using conversational information from past incidents as training samples. Conversation records between the driver and the victim (records of interrogations after arresting the driver) are recorded in past incidents, and this is used as a positive sample for supervisory training. The deep learning model obtained by training can obtain the probability that the conversation is abnormal based on the input conversation stream.

At step 304, if the probability is higher than a predetermined first threshold, trigger an alarm.

In this embodiment, if the probability that the conversation is abnormal is higher than a predetermined first threshold, an alarm is triggered. The process of triggering an alarm involves dialing 911 and reporting the location and attribute information (model, color, owner information, etc.) of the suspected vehicle. Based on the location information, the closest traffic police to the vehicle can be searched, and the attribute information and location information of the vehicle can be sent to the traffic police.

In some optional embodiments of this example, the method comprises receiving the emotional information of at least two users, the probability that the speech is abnormal is less than a predetermined second threshold, and the passenger's and triggering an alarm if the emotional information includes a feeling of fear. Depending on the position of the audio gathering device, it can be established whether the received speech is that of the driver or that of the passenger, so that which of the at least two users' emotional information can be determined. It is possible to recognize whether the information belongs to the passenger. Emotional information can include fear, tension, exhilaration, and the like. Affective information may further validate the received textual information. There is no problem with the text information, but the possibility that the passenger was threatened by the driver cannot be ruled out, so it is necessary to judge whether the passenger's emotions are normal or not based on the voice characteristics. If you can do it perfectly, it is very suspicious and you should report it to the police. The second threshold may be less than or equal to the first threshold.

In some optional embodiments of this example, the method further includes triggering an alarm in response to detecting a disconnection of the circuit connection of the vehicle's audio collection device. Disconnecting the circuit of the audio gathering device means that the audio gathering device is removed. A prerequisite for the successful operation of the present disclosure is the successful use of the audio collection device. Therefore, it is necessary to perform some checks on the audio collection device to prevent it from being removed by the driver. If you remove the audio collection device without permission from the operator, call the police.

In some optional embodiments of this example, the method includes periodically sending test questions to the vehicle for passengers to answer and answering the questions as normal by the deep learning model within a predetermined time. and triggering an alarm if the determined response information is not received. If you haven't listened to the conversation in the car all along, you can't tell if the passenger doesn't want to talk or if the driver is making it difficult for the passenger to talk, so send test questions periodically to get the passenger to answer. , it is necessary to check whether the passengers are safe. An alarm is triggered when a passenger does not respond within a certain period of time, or when the deep learning model determines that the results of speech recognition analysis after responding are abnormal.

Continuing to refer to FIG. 4, FIG. 4 is a schematic diagram showing the application scene of the method for outputting information according to this embodiment. In the application scene of FIG. 4, the passenger sits behind the driver after boarding and collects the passenger's voice by the audio collecting device closest to him. The driver's voice is collected by an audio collection device next to the driver. Then the voices of the two people are respectively recognized by the voice recognition engine to obtain two pieces of text information. The vehicle transmits these two pieces of text information to the service side. The service determines which textual information was spoken by the driver and which textual information was spoken by the passenger based on the location of the audio collecting device. After this, the two pieces of text information are stitched into the conversation based on the received time. Finally, the speech is fed into a pre-trained deep learning model to determine the probability that the speech is anomalous. If higher than a predetermined first threshold, trigger an alarm.

The method according to the above embodiments of the present disclosure has the following advantages.

1. Increase the safety attributes of vehicles and enrich safety technology solutions for car manufacturers.

2. For example, in the automotive-derived transport services industry, such as taxis and online ride-hailing, vehicles equipped with such solutions can improve passenger security.

3. By detecting the user's conversation content, the related recommendation service of the trip computer is optimized, and more interesting news is recommended to the user, and better content articles or products are recommended to the user.

Further referring to FIG. 5, as an implementation of the methods shown in the above figures, the present invention provides an embodiment of a device for outputting information, an embodiment of the device is shown in FIG. Corresponding to the method embodiments, the apparatus can be specifically applied to various electronic devices.

As shown in FIG. 5, the device 500 for outputting information according to this embodiment includes an audio collecting unit 501, an echo cancellation unit 502, a speech recognition unit 503 and an information uploading unit 504. FIG. Here, the audio collecting unit 501 is arranged to collect audio data respectively by at least two audio collecting devices at different positions in the vehicle. The echo cancellation unit 502 is arranged to perform echo cancellation on the collected at least two-way audio data. The speech recognition unit 503 is arranged to input the echo-cancelled data respectively to at least two speech recognition engines and perform speech recognition to obtain text information of at least two users. The information uploading unit 504 uploads the locations of the at least two audio collecting devices and the corresponding text information of the at least two users to the service side for the service side to analyze the text information through a pre-trained deep learning model. , is arranged to output alarm information if the analysis result satisfies the alarm condition.

In this embodiment, for the specific processing of the audio collection unit 501, echo cancellation unit 502, speech recognition unit 503 and information upload unit 504 of the device 500 for outputting information, see step 201 in the corresponding embodiment of FIG. , steps 202 , 203 and 204 .

In some optional embodiments of the present example, the device 500 inputs the echo-cancelled data to at least two emotion recognition engines, respectively, to perform emotion recognition, thereby providing a sense of emotion to at least two users. further comprising an emotion recognition unit arranged to obtain emotion information, the information uploading unit further including at least two audio collectors for the service side to analyze the text information and emotion through pre-trained deep learning models; , the corresponding text information of at least two users, and the emotion information of the two users are uploaded to the service side, and arranged to output alarm information when the analysis result meets the alarm conditions.

Further referring to FIG. 6, as an implementation of the methods shown in the above figures, the present invention provides an embodiment of a device for outputting information, an embodiment of the device is shown in FIG. Corresponding to the method embodiments, the apparatus can be specifically applied to various electronic devices.

As shown in FIG. 6, the device 600 for outputting information according to this embodiment includes a receiving unit 601 , a text stitching unit 602 , a speech recognition unit 603 and an alarm unit 604 . Here, the receiving unit 601 is arranged to receive the positions of at least two audio collecting devices and the corresponding textual information of at least two users uploaded from the vehicle. A text stitching unit 602 is arranged to compose the textual information of the two users into a speech stream based on the positions of the at least two audio gathering devices. The speech recognition unit 603 is arranged to obtain the probability that the speech is anomalous by inputting the speech stream into a pre-trained deep learning model. Alarm unit 604 is arranged to trigger an alarm if the probability is higher than a predetermined first threshold.

In this embodiment, for the specific processing of the receiving unit 601, the text stitching unit 602, the speech recognition unit 603 and the alarm unit 604 of the device 600 for outputting information, the steps 201 in the corresponding embodiment of FIG. Steps 202, 203 and 204 can be referred to.

In some optional embodiments of the present example, the alarm unit 604 further obtains the attribute information and location information of the vehicle, searches the closest traffic police to the vehicle based on the location information, and It is arranged to transmit attribute information and location information to the traffic police.

In some optional embodiments of this example, the receiving unit 601 is further arranged to receive the emotional information of at least two users, and the alarm unit 604 is further arranged to receive the probability that the conversation is abnormal. is less than a second predetermined threshold and the passenger's emotional information includes a sense of fear, the alarm is arranged to trigger.

In some optional embodiments of this example, the alarm unit 604 is further arranged to trigger an alarm in response to a detected disconnection of the circuit connection of the vehicle's audio collection device. .

In some optional embodiments of the present example, the alarm unit 604 also periodically sends test questions to the vehicle for passengers to answer, and within a predetermined time the deep learning model If no response information is received determined to trigger an alarm.

Referring now to FIG. 7, there is shown a schematic configuration diagram of an electronic device (eg, server or vehicle controller in FIG. 1) 700 applied to implement an embodiment of the present disclosure. The vehicle controller/server illustrated in FIG. 7 is merely an example and does not limit the functionality and scope of use of the embodiments of this disclosure.

As shown in FIG. 7, electronic device 700 performs various appropriate operations and processes according to programs stored in read only memory (ROM) 702 or programs loaded from storage unit 708 into random access memory (RAM) 703. A processing unit (eg, central processing unit, graphics processor, etc.) 701 can be included. The RAM 703 stores various programs and data necessary for operating the electronic device 700 . Processing unit 701 , ROM 702 and RAM 703 are interconnected by bus 704 . Input/output (I/O) interface 705 is also connected to bus 704 .

I/O interfaces 705 typically include input devices 706 including, for example, touch screens, touch panels, keyboards, mice, cameras, audio collection devices, accelerometers, gyros, etc.; , a storage device 708 including, for example, a magnetic tape or a hard disk, and a communication device 709 are connected. Communications device 709 may permit electronic device 700 to communicate wirelessly or by wire with other devices to exchange data. Although FIG. 7 illustrates electronic device 700 with various devices, it should be understood that you are not required to implement or include all of the devices shown. Alternatively, more or fewer devices may be implemented or provided. Each block shown in FIG. 7 may represent one device, or multiple devices, if desired.

In particular, according to embodiments of the present disclosure, the processes described with reference to the flowcharts above may be implemented as computer software programs. For example, an embodiment of the present disclosure includes a computer program product comprising a computer program carried on a computer readable medium, the computer program including program code for performing the methods illustrated in the flowcharts. In these examples, the computer program may be downloaded and installed from a network via communication device 709 , installed from storage device 708 , or installed from ROM 702 . When this computer program is executed by the processing unit 701, it performs the above-described functions limited in the manner of the embodiments of the present disclosure. It is noted that the computer-readable media described in the embodiments of the present disclosure may be computer-readable signal media or computer-readable storage media or any combination of the above. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus or device, or any combination of the foregoing. More specific examples of computer-readable storage media include electrical connections through one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, compact disc read only memory (CD-ROM), optical storage, magnetic storage, or any combination of the above. In the embodiments of the present disclosure, a computer-readable storage medium can be any tangible medium that contains or stores a program that is used by or in combination with an instruction execution system, apparatus or device. can be used In an embodiment of the present disclosure, a computer readable signal medium may include a data signal carried in baseband or as part of a carrier wave carrying computer readable program code. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer readable signal medium can be any computer readable medium other than a computer readable storage medium for use by or in combination with an instruction execution system, apparatus or device. You may send, propagate or transmit a program for Program code embodied in a computer readable medium may be transmitted over any suitable medium including wires, fiber optic cables, RF (radio frequency), etc., or any suitable combination of the above, including but not limited to Not exclusively.

The computer-readable medium may be included in the electronic device or may exist separately and not be incorporated into the electronic device. The computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device communicates with at least two audio collection devices at different locations in the vehicle. collect audio data respectively, perform echo removal on the collected at least two-way audio data, input the echo-removed data to at least two speech recognition engines, and perform speech recognition. obtains the text information of at least two users, and the location of at least two audio collection devices and the corresponding data of at least two users for the service side to analyze the text information through pre-trained deep learning models. Upload the text information to the service side, and output the alarm information when the analysis result satisfies the alarm conditions. Alternatively, the electronic device receives at least two audio gathering device locations and corresponding textual information of at least two users uploaded from the vehicle, and based on the locations of the at least two audio gathering devices, the two By composing the user's textual information into a conversation stream and inputting the conversation stream into a pre-trained deep learning model, a probability that the conversation is abnormal is obtained, and if the probability is higher than a predetermined first threshold, an alarm is generated. trigger.

Computer program code for carrying out operations of the embodiments of the present disclosure can be written in one or more programming languages, or combinations thereof, including Java®, Smalltalk®, It includes object-oriented programming languages such as C++, as well as traditional procedural programming languages such as the "C" language or similar programming languages. The program code may reside entirely on the user's computer, partially on the user's computer, as one single software package, partially on the user's computer and partially on a remote computer, or entirely on the remote computer or Can be run on a server. When referring to a remote computer, the remote computer can be connected to the user's computer via any kind of network, including a local area network (LAN) or wide area network (WAN), or can be connected to an external computer. Can be connected (eg, via the Internet using an Internet service provider).

The flowcharts and block diagrams in the figures illustrate possible architectures, functionality, and operation of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block of a flowchart or block diagram can represent a module, segment of programming, or portion of code, which module, segment of programming, or portion of code implements a given logic function. contains one or more executable instructions for It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may actually be executed in parallel, or they may sometimes be executed in the reverse order, depending on the functionality involved. Each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, may be implemented by a system of dedicated hardware that performs the specified functions or operations, or by dedicated hardware and a computer system. Note also that it may be implemented by a combination of commands.

A unit according to an embodiment of the present disclosure may be implemented by software or by hardware. The described units may be located in a processor, eg, may be described as "a processor including an audio acquisition unit, an echo cancellation unit, a speech recognition unit, and an information upload unit". Here, the names of these units do not in some cases limit the units themselves, e.g. an audio collection unit "collects audio data by at least two audio collection devices located at different locations in the vehicle, respectively. It may be written as a unit.

The above description is merely that of the preferred embodiment of the present disclosure and the principles of the applied technology. The scope of the invention according to the embodiments of the present disclosure is not limited to technical solutions obtained by combining the above technical features, and at the same time, if the concept of the above invention is not deviated from, the above technical features or Those skilled in the art should understand that other technical solutions obtained by arbitrarily combining equivalent technical features should be included. For example, a technical solution is formed by replacing the features described above with technical features having similar functions to those disclosed in the present disclosure (but not limited thereto).

Claims (18)

collecting audio data by at least two audio collection devices at different locations in the vehicle, respectively;
performing echo cancellation on the collected at least two-way audio data;
obtaining text information of at least two users by respectively inputting the echo-removed data to at least two speech recognition engines and performing speech recognition;
uploading the location of the at least two audio collecting devices and the corresponding text information of at least two users to the service side, for the service side to analyze the text information through a pre-trained deep learning model, and the analysis result alarms; outputting alarm information when the conditions are met;
A method for outputting information containing obtaining emotion information of at least two users by respectively inputting the echo-removed data to at least two emotion recognition engines and performing emotion recognition;
Locations of the at least two audio collecting devices, corresponding at least two users' text information, and two users' sentiment information, for the service side to analyze the text information and sentiment through a pre-trained deep learning model. is uploaded to the service side, and if the analysis result satisfies the alarm conditions, alarm information is output;
2. The method of claim 1, further comprising: receiving at least two audio collection device locations and corresponding at least two user textual information uploaded from a vehicle;
composing textual information of two users into a conversation stream based on the positions of the at least two audio collection devices;
obtaining a probability that the speech is anomalous by inputting the speech stream into a pre-trained deep learning model;
triggering an alarm if the probability is higher than a first predetermined threshold;
A method for outputting information containing obtaining attribute information and location information of the vehicle;
retrieving the closest traffic police to the vehicle based on the location information;
transmitting attribute information and location information of the vehicle to the traffic police;
4. The method of claim 3, further comprising: receiving affective information of at least two users;
triggering an alarm if the probability that the speech is abnormal is less than a second predetermined threshold and the passenger's emotional information includes fear;
4. The method of claim 3, further comprising: triggering an alarm in response to detecting a loss of circuit connection in the vehicle's audio collection device;
4. The method of claim 3, further comprising: periodically sending test questions to the vehicle for passengers to answer;
triggering an alarm if response information determined to be normal by the deep learning model is not received within a predetermined time;
4. The method of claim 3, further comprising: an audio collection unit arranged to collect audio data respectively by at least two audio collection devices at different locations in the vehicle;
an echo cancellation unit arranged to perform echo cancellation on the collected at least two-way audio data;
a speech recognition unit arranged to obtain textual information of at least two users by respectively inputting echo-cancelled data to at least two speech recognition engines and performing speech recognition;
uploading the location of the at least two audio collecting devices and the corresponding text information of at least two users to the service side, for the service side to analyze the text information through a pre-trained deep learning model, and the analysis result alarms; an information upload unit arranged to output alarm information if a condition is met;
A device for outputting information containing further comprising an emotion recognition unit arranged to obtain emotion information of at least two users by respectively inputting the echo-cancelled data into at least two emotion recognition engines and performing emotion recognition;
The information uploading unit further comprises the locations of the at least two audio collecting devices, the corresponding text information of at least two users, for the service side to conduct text information and sentiment analysis through pre-trained deep learning models; and uploading the emotional information of two users to the service side, and arranged to output alarm information when the analysis results meet the alarm conditions;
9. Apparatus according to claim 8. a receiving unit arranged to receive the location of the at least two audio collecting devices and the corresponding textual information of the at least two users uploaded from the vehicle;
a text stitching unit arranged to compose textual information of two users into a dialogue stream based on the positions of the at least two audio gathering devices;
a speech recognition unit arranged to obtain a probability that the speech is anomalous by inputting the speech stream into a pre-trained deep learning model;
an alarm unit arranged to trigger an alarm if the probability is higher than a first predetermined threshold;
A device for outputting information containing The alarm unit further
Acquiring attribute information and location information of the vehicle;
searching for the closest traffic police to the vehicle based on the location information;
transmitting attribute information and location information of the vehicle to the traffic police;
are arranged as
11. Apparatus according to claim 10. the receiving unit is further arranged to receive emotional information of at least two users;
The alarm unit is further arranged to trigger an alarm if the probability that the speech is abnormal is below a predetermined second threshold and the passenger's emotional information includes a sense of fear.
11. Apparatus according to claim 10. the alarm unit is further arranged to trigger an alarm in response to a detected disconnection of a circuit connection of an audio collection device of the vehicle;
11. Apparatus according to claim 10. The alarm unit further
periodically sending test questions to said vehicle for passengers to answer;
triggering an alarm if response information determined to be normal by the deep learning model is not received within a predetermined time;
are arranged as
11. Apparatus according to claim 10. Audio data is collected by at least two audio collecting devices located at different positions in the vehicle, echo cancellation is performed on the collected at least two-way audio data, and echo-cancelled data is collected by at least two audio data collection devices. Obtaining text information of at least two users by inputting to a speech recognition engine and performing speech recognition, and providing the locations of the at least two audio collecting devices and the corresponding text information of at least two users to the service side. a vehicle arranged to upload;
Receiving locations of at least two audio gathering devices and corresponding textual information of at least two users uploaded from a vehicle, and speaking textual information of the two users based on the locations of the at least two audio gathering devices. obtain a probability that the dialogue is abnormal by structuring it into a stream and inputting said dialogue stream into a pre-trained deep learning model, and triggering an alarm if said probability is higher than a predetermined first threshold a deployed service side;
A system for outputting information containing one or more processors;
a storage device storing one or more programs;
including
causing said one or more processors to implement the method of any one of claims 1 to 7 when said one or more programs are executed by said one or more processors;
An electronic device for outputting information. A computer readable medium having a computer program stored thereon,
realizing the method of any one of claims 1 to 7 when said computer program is executed by a processor;
computer readable medium. A computer program,
The computer program, when executed by a processor, implements the method of any one of claims 1 to 7,
computer program.

Images