CN109388368B - Human-computer interaction method and device, unmanned vehicle and storage medium thereof - Google Patents
Human-computer interaction method and device, unmanned vehicle and storage medium thereof Download PDFInfo
- Publication number
- CN109388368B CN109388368B CN201811034447.5A CN201811034447A CN109388368B CN 109388368 B CN109388368 B CN 109388368B CN 201811034447 A CN201811034447 A CN 201811034447A CN 109388368 B CN109388368 B CN 109388368B
- Authority
- CN
- China
- Prior art keywords
- law enforcement
- data
- certificate
- unmanned vehicle
- interaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- Human Computer Interaction (AREA)
- General Engineering & Computer Science (AREA)
- Management, Administration, Business Operations System, And Electronic Commerce (AREA)
- Burglar Alarm Systems (AREA)
Abstract
The application discloses a human-computer interaction method and device, an unmanned vehicle and a storage medium thereof. The method is applied to an unmanned automobile and comprises the following steps: sending first voice data for prompting law enforcement personnel to present certificates for law enforcement; receiving data relating to credentials for law enforcement; judging the validity of the certificate based on the related data; and if the certificate is legal, starting the intelligent voice interaction system to realize the interaction between the unmanned automobile and law enforcement personnel. According to the technical scheme of the embodiment of the application, the authenticity of the certificate of the law enforcement officer is detected, the safety of man-machine interaction is improved, and after the certificate is identified to be legal, the problem of interaction between the existing unmanned vehicle and the law enforcement officer is solved through the intelligent voice interaction system.
Description
Technical Field
The present application relates generally to the field of unmanned vehicles, and more particularly, to a human-computer interaction method and apparatus, an unmanned vehicle, and a storage medium thereof.
Background
An unmanned vehicle (hereinafter referred to as an unmanned vehicle) is one of intelligent vehicles, is also called a wheeled mobile robot, and mainly depends on an intelligent driver mainly comprising a computer system in the vehicle to achieve the unmanned aim. The intelligent automobile senses the road environment through a vehicle-mounted sensing system, automatically plans a driving route and controls the automobile to reach a preset target.
At present, the interaction between an unmanned automobile and law enforcement personnel cannot be avoided in the driving process.
When the unmanned vehicle has illegal driving behaviors, law enforcement personnel cannot effectively communicate with the unmanned vehicle. Leading to the problem of interaction which is difficult to avoid in practical application of unmanned vehicles.
Disclosure of Invention
In view of the above-mentioned defects or shortcomings in the prior art, embodiments of the present invention provide a human-computer interaction method, device and unmanned vehicle, which are used to solve the problem that the unmanned vehicle is not smooth to communicate with law enforcement officers.
In a first aspect, an embodiment of the present application provides a human-computer interaction method, which is applied to an unmanned automobile, and includes:
sending first voice data for prompting law enforcement personnel to present certificates for law enforcement;
receiving data relating to credentials for law enforcement;
judging the validity of the certificate based on the related data;
and if the certificate is legal, starting the intelligent voice interaction system to realize the interaction between the unmanned automobile and law enforcement personnel.
In a second aspect, an embodiment of the present application provides a human-computer interaction device, where the human-computer interaction device is applied to an unmanned vehicle, and the human-computer interaction device includes:
the voice data sending unit is used for sending first voice data and prompting law enforcement personnel to show certificates for law enforcement;
the certificate data receiving unit is used for receiving related data of the certificate for law enforcement;
the certificate data judging unit is used for judging the legality of the certificate based on the related data;
and the intelligent interaction starting unit is used for starting the intelligent voice interaction system to realize the interaction between the unmanned automobile and the law enforcement officers if the certificate is legal.
In a third aspect, embodiments of the present application provide an unmanned vehicle comprising at least one image capture device for capturing first image data of a law enforcement officer and/or image data of a document;
the vehicle-mounted control device is used for receiving and sending an action instruction so as to control the unmanned automobile to respond to the gesture action of the law enforcement officer; and a human-computer interaction processing device as described in the embodiments of the present application.
In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored, the computer program being configured to:
which when executed by a processor performs the method as described in embodiments of the present application.
According to the man-machine interaction method applied to the unmanned vehicle, the authenticity of the certificate of the law enforcement officer is detected, the safety of man-machine interaction is improved, and after the certificate is identified as a legal certificate, the problem of interaction between the unmanned vehicle and the law enforcement officer is solved through the intelligent voice interaction system.
Further, the voice recognition technology is used for recognizing the voice data of the unmanned vehicle and the law enforcement officer for prompting the unmanned vehicle to provide information for representing legal driving of the unmanned vehicle, and the information is presented through a related display device or a voice playing device of the unmanned vehicle, so that free interaction and response of the unmanned vehicle and the law enforcement officer in violation scenes are realized.
Furthermore, the driving process of the unmanned vehicle is effectively managed for the cloud platform through storage and uploading of the interactive data, and controllability of the unmanned vehicle is improved.
Furthermore, the interestingness of the unmanned vehicle communication process is enriched through diversified display settings.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the following detailed description of non-limiting embodiments thereof, made with reference to the accompanying drawings in which:
FIG. 1 is a flow chart diagram illustrating a human-computer interaction method provided by an embodiment of the present application;
FIG. 2 is a flow chart illustrating a human-computer interaction method according to another embodiment of the present disclosure;
FIG. 3 is a block diagram schematically illustrating a structure of a human-computer interaction device 300 provided by an embodiment of the present application;
FIG. 4 is a block diagram schematically illustrating a structure of a human-computer interaction device 400 according to another embodiment of the present application;
fig. 5 shows a schematic structural diagram of an unmanned vehicle 500 provided in an embodiment of the present application.
Detailed Description
The present application will be described in further detail with reference to the following drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the relevant invention and not restrictive of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.
Referring to fig. 1, fig. 1 is a schematic flowchart illustrating a human-computer interaction method according to an embodiment of the present disclosure.
As shown in fig. 1, the method is applied to an unmanned automobile, and comprises the following steps:
at step 110, first voice data is sent for prompting law enforcement personnel to present credentials for law enforcement.
In the embodiment of the application, after the unmanned automobile executes the parking instruction or parks beside. The unmanned vehicle judges whether first voice data need to be sent or not according to the preset distance between law enforcement officers and the unmanned vehicle or action recognition. Wherein the predetermined distance may be a distance of less than 20 cm. The action recognition can be that the image acquisition device acquires first voice data for prompting law enforcement personnel to show certificates for law enforcement, and the first voice data can be a standard voice segment stored in advance, such as 'please show legal certificates', or a voice segment set by a user in advance, such as 'please show your law enforcement certificates', and the like.
At step 120, relevant data for credentials used in law enforcement is received.
In the embodiment of the application, after the unmanned vehicle sends the first voice data to the law enforcement officer, the unmanned vehicle waits for a preset time, and acquires the related data of the certificate through the image acquisition device or the voice acquisition device, for example, the image acquisition device acquires the image of the certificate, or acquires the number of the certificate dictated by the law enforcement officer through the voice acquisition device, and extracts the information for identifying the legal identity of the law enforcement officer in the certificate through image detection or voiceprint recognition technology, wherein the information belongs to the related data of the certificate.
And step 130, judging the legality of the certificate based on the related data.
After acquiring the relevant data of the certificate, the relevant data is transmitted to a relevant device for identifying and judging the certificate, such as a processing device of an unmanned vehicle and/or a processing device of a cloud platform. After the related data is acquired, the related data, such as image data or voice data of the certificate, can be subjected to legality identification judgment in a processing device of the unmanned vehicle, or the related data can be transmitted to a cloud platform in an encryption mode, and the content of the related data is subjected to authenticity identification through the cloud platform. The characteristic data can be extracted from the relevant data through the processing device, the characteristic data are transmitted to the cloud platform for authenticity identification, the authenticity identification result is fed back to the unmanned vehicle as the legality judgment result, and relevant operation of the unmanned vehicle is controlled based on the legality judgment result, for example, if the certificate is legal, the unmanned vehicle can control the window glass to move towards the ground direction, so that the window is opened, and audio data in the unmanned vehicle can be conveniently transmitted out. If the certificate is judged to be illegal, the identity of the law enforcement officer is possibly suspicious, and at the moment, the unmanned vehicle sends the image data of the suspicious officer to the cloud platform so as to improve the safety of the system.
And step 140, if the certificate belongs to a legal certificate, starting an intelligent voice interaction system to realize the interaction between the unmanned vehicle and law enforcement officers.
In the embodiment of the application, when the certificate is judged to belong to a legal certificate, namely a real and effective certificate for law enforcement. The unmanned vehicle control intelligent voice interaction system realizes dialogue communication with law enforcement officers. The intelligent voice interaction system can intelligently analyze the collected audio data of the law enforcement officers. For example, law enforcement personnel say "please show the pass permit", the intelligent voice interaction system extracts the keyword "pass permit" through voice recognition technology, and sends the keyword "pass permit" to the relevant processing device, the processing device acquires the image data or audio data of the "pass permit" from the storage device, and controls the image data or audio data to be presented, so that the law enforcement personnel can check conveniently.
Optionally, on the basis of the above embodiment, the method may further include
And 150, if the certificate is illegal, sending an alarm request and/or sending the collected first image data of the law enforcement officer to the cloud platform, and waiting for indication information returned by the cloud platform.
Under the condition that the unmanned vehicle judges that the certificate is illegal, the unmanned vehicle sends an alarm request to the cloud platform, or sends image data of law enforcement officers when the alarm request is sent, so that the cloud platform can further verify and judge the identities of the law enforcement officers. And the unmanned vehicle waits for the indication information returned by the cloud platform within the preset time range, executes related operation according to the indication information, and generates an alarm sound or a light flashing alarm if the indication information exceeds the preset time range.
According to the method and the device, the identity of the law enforcement officer is judged through the unmanned vehicle, the interaction safety of the unmanned vehicle and the law enforcement officer is improved, and the interaction problem of the law enforcement officer and the unmanned vehicle is solved through the intelligent voice interaction system.
Referring to fig. 2, fig. 2 is a schematic flowchart illustrating a human-computer interaction method according to another embodiment of the present application.
As shown in fig. 2, the method is applied to an unmanned vehicle, and includes:
optionally, step 210, first image data of a law enforcement officer collected by an image collection device is received.
According to the embodiment of the application, the unmanned vehicle is driven illegally in the driving process, and is warned by law enforcement personnel through traffic police gestures, and the unmanned vehicle and the law enforcement personnel have problems in interaction under the scene needing parking inspection. To solve this problem, the image capturing device of the unmanned vehicle captures first image data of the law enforcement officer, which may for example contain image data of the law enforcement officer, and the target area may for example be a gesture movement of the law enforcement officer.
Optionally, a gesture action of the law enforcement officer is detected based on the first image data, step 220.
A gesture action of the law enforcement officer, such as a parking side gesture, is detected from the first image data.
Optionally, in step 230, an action instruction is sent to the vehicle-mounted control device to control the unmanned vehicle to respond to the gesture action.
According to the recognized gesture action, the processing device of the unmanned vehicle sends an action command to the vehicle-mounted control device, for example, a command of parking at the side is executed, so that the unmanned vehicle is controlled to respond to the gesture action.
At step 240, first voice data is sent for prompting law enforcement personnel to present credentials for law enforcement.
After executing the action instruction, the unmanned vehicle transmits first voice data for prompting law enforcement personnel to present credentials for law enforcement. For example, a preset standard voice segment or a voice segment pre-recorded by the user is sent, and the content of the voice segment may be "please show law enforcement certificate" or "please show legal certificate", for example.
At step 250, relevant data of the certificate for law enforcement is received, the relevant data being image data of the certificate captured by the image capture device.
After the first voice data are sent, waiting for a preset time, and starting an image acquisition device or an audio acquisition device for acquiring information of certificates shown by law enforcement officers. Image data of the certificate of a law enforcement officer can be acquired, for example, by an image acquisition device.
And extracting identity information representing law enforcement officers in the certificate based on the acquired image data of the certificate. The certificate may be, for example, a police certificate or the like. The identity information may be, for example, the bearer's name, gender, date of birth, job title, warnings, and images of the bearer.
After the identity information of the certificate is acquired, the authenticity of the identity information is verified, and the identity information can be processed by an unmanned vehicle and/or a cloud platform. For example, the processing device of the unmanned vehicle performs legality identification judgment on the identity information, or transmits the identity information to the cloud platform in an encryption manner, and performs authenticity identification on the identity information through the cloud platform. The identity information can be extracted through the processing device, the identity information is transmitted to the cloud platform as feature data to be subjected to authenticity identification, the authenticity identification result is fed back to the unmanned vehicle as a legality judgment result, and relevant operation of the unmanned vehicle is controlled based on the legality judgment result, for example, if the certificate is legal, the unmanned vehicle can control the window glass to move towards the ground direction, so that the window is opened, and audio data in the unmanned vehicle can be transmitted conveniently. If the certificate is judged to be illegal, the identity of the law enforcement officer is possibly suspicious, and at the moment, the unmanned vehicle sends the image data of the suspicious officer to the cloud platform so as to improve the safety of the system.
If the credential is legitimate, second voice data is received, step 270a, which is input by law enforcement personnel and which includes prompting the unmanned vehicle to provide information indicative of its legitimate driving.
And after the certificate is judged to be legal and is waited for a period of time or simply exchanged, the unmanned vehicle voice acquisition device acquires second voice data and performs voice recognition on the second voice data to obtain keywords. And controlling the operation of the unmanned vehicle according to the keywords. For example, the keyword may be "pass license", and the processing device acquires image data or audio data of the "pass license" from the storage device of the unmanned vehicle.
And 270b, if the certificate is illegal, sending an alarm request and/or sending the collected first image data of the law enforcement officer to the cloud platform, and waiting for indication information returned by the cloud platform.
When the certificate is judged to be illegal, an alarm request is sent to the cloud platform, or when the alarm request is sent, image data of law enforcement personnel is also sent, so that the cloud platform can further verify and judge the identity of the law enforcement personnel. And the unmanned vehicle waits for the indication information returned by the cloud platform within the preset time range, executes related operation according to the indication information, and generates an alarm sound or a light flashing alarm if the indication information exceeds the preset time range.
Optionally, step 280, information indicating its legal driving is sent to a voice player or display device in a voice data format or an image data format for presentation.
After information indicating its legitimate driving is acquired, the information is presented via the voice player or display device of the unmanned vehicle in a voice data format or an image data format. The information indicating its legitimate driving may be, for example, a certificate such as "pass permit" indicating the driving qualification of the unmanned vehicle.
The display device may be, for example, an in-vehicle display device of an unmanned vehicle, a front windshield, a window glass, a navigation display device, or the like.
Optionally, step 290a, automatically storing the data of the interaction in a log form;
and 290b, after the interaction is finished, uploading the interactive data to the cloud platform.
In order to further manage the travel behavior of the unmanned vehicle, the interaction information of the unmanned vehicle and law enforcement officers can be automatically stored in a log form in a corresponding storage device, such as the storage device of the unmanned vehicle, or the storage device of a cloud platform, or other movable storage devices, and the like. After the unmanned vehicle and the law enforcement officers finish interaction, all the content of the unmanned vehicle and the law enforcement officers interaction is packaged and uploaded to the cloud platform in an encryption or non-encryption mode.
It should be noted that while the operations of the method of the present invention are depicted in the drawings in a particular order, this does not require or imply that the operations must be performed in this particular order, or that all of the illustrated operations must be performed, to achieve desirable results. Additionally or alternatively, certain steps may be omitted, multiple steps combined into one step execution, and/or one step broken down into multiple step executions.
With further reference to fig. 3, fig. 3 is a block diagram schematically illustrating a structure of a human-computer interaction device 300 according to an embodiment of the present disclosure.
As shown in fig. 3, the apparatus is applied to an unmanned vehicle, and the apparatus 300 includes:
a voice data transmitting unit 310 for transmitting the first voice data for prompting law enforcement officers to present certificates for law enforcement.
In the embodiment of the application, after the unmanned automobile executes the parking instruction or parks beside. The unmanned vehicle judges whether first voice data need to be sent or not according to the preset distance between law enforcement officers and the unmanned vehicle or action recognition. Wherein the predetermined distance may be a distance of less than 20 cm. The action recognition can be that the image acquisition device acquires first voice data for prompting law enforcement personnel to show certificates for law enforcement, and the first voice data can be a standard voice segment stored in advance, such as 'please show legal certificates', or a voice segment set by a user in advance, such as 'please show your law enforcement certificates', and the like.
A certificate data receiving unit 320 for receiving data related to a certificate for law enforcement.
In the embodiment of the application, after the unmanned vehicle sends the first voice data to the law enforcement officer, the unmanned vehicle waits for a preset time, and acquires the related data of the certificate through the image acquisition device or the voice acquisition device, for example, the image acquisition device acquires the image of the certificate, or acquires the number of the certificate dictated by the law enforcement officer through the voice acquisition device, and extracts the information for identifying the legal identity of the law enforcement officer in the certificate through image detection or voiceprint recognition technology, wherein the information belongs to the related data of the certificate.
And a certificate data judging unit 330 for judging the validity of the certificate based on the related data.
After acquiring the relevant data of the certificate, the relevant data is transmitted to a relevant device for identifying and judging the certificate, such as a processing device of an unmanned vehicle and/or a processing device of a cloud platform. After the related data is acquired, the related data, such as image data or voice data of the certificate, can be subjected to legality identification judgment in a processing device of the unmanned vehicle, or the related data can be transmitted to a cloud platform in an encryption mode, and the content of the related data is subjected to authenticity identification through the cloud platform. The characteristic data can be extracted from the relevant data through the processing device, the characteristic data are transmitted to the cloud platform for authenticity identification, the authenticity identification result is fed back to the unmanned vehicle as the legality judgment result, and relevant operation of the unmanned vehicle is controlled based on the legality judgment result, for example, if the certificate is legal, the unmanned vehicle can control the window glass to move towards the ground direction, so that the window is opened, and audio data in the unmanned vehicle can be conveniently transmitted out. If the certificate is judged to be illegal, the identity of the law enforcement officer is possibly suspicious, and at the moment, the unmanned vehicle sends the image data of the suspicious officer to the cloud platform so as to improve the safety of the system.
And the intelligent interaction starting unit 340 is used for starting the intelligent voice interaction system to realize the interaction between the unmanned vehicle and the law enforcement officer if the unmanned vehicle belongs to the legal certificate.
In the embodiment of the application, when the certificate is judged to belong to a legal certificate, namely a real and effective certificate for law enforcement. The unmanned vehicle control intelligent voice interaction system realizes dialogue communication with law enforcement officers. The intelligent voice interaction system can intelligently analyze the collected audio data of the law enforcement officers. For example, the law enforcement officer speaks "ask for the pass permit", the intelligent voice interaction system extracts the keyword "pass permit" through the voice recognition technology, and sends the key word "pass permit" to the relevant processing device, and the processing device acquires the image data or the audio data of the "pass permit" from the storage device, and controls the image data or the audio data to be presented, so that the law enforcement officer can check the image data or the audio data.
Optionally, on the basis of the above embodiment, the apparatus may further include
And the alarm sending unit 350 is used for sending an alarm request and/or sending the collected first image data of the law enforcement officer to the cloud platform if the certificate belongs to illegal certificates, and waiting for indication information returned by the cloud platform.
Under the condition that the unmanned vehicle judges that the certificate is illegal, the unmanned vehicle sends an alarm request to the cloud platform, or sends image data of law enforcement officers when the alarm request is sent, so that the cloud platform can further verify and judge the identities of the law enforcement officers. And the unmanned vehicle waits for the indication information returned by the cloud platform within the preset time range, executes related operation according to the indication information, and generates an alarm sound or a light flashing alarm if the indication information exceeds the preset time range.
According to the embodiment of the application, the identity of the law enforcement officer is judged through the unmanned vehicle, the interaction safety of the unmanned vehicle and the law enforcement officer is improved, and the interaction problem of the law enforcement officer and the unmanned vehicle is solved through the intelligent voice interaction system.
Referring to fig. 4, fig. 4 is a block diagram illustrating a schematic structure of a human-computer interaction device according to another embodiment of the present application.
As shown in fig. 4, the apparatus is applied to an unmanned vehicle, and the apparatus 400 includes:
optionally, a law enforcement image receiving unit 410 for receiving first image data of law enforcement officers collected by the image collecting device.
According to the embodiment of the application, the unmanned vehicle is driven illegally in the driving process, and is warned by law enforcement personnel through traffic police gestures, and the unmanned vehicle and the law enforcement personnel have problems in interaction under the scene needing parking inspection. To solve this problem, the image capture device of the unmanned vehicle captures first image data of the law enforcement officer, which may for example comprise image data of the law enforcement officer, and the target area may for example be a gesture action of the law enforcement officer.
Optionally, a law enforcement image detection unit 420 for detecting a gesture action of a law enforcement officer based on the first image data.
A gesture action of the law enforcement officer, such as a parking side gesture, is detected from the first image data.
Optionally, the action instruction sending unit 430 is configured to send an action instruction to the vehicle-mounted control device to control the unmanned vehicle to respond to the gesture action.
According to the recognized gesture action, the processing device of the unmanned vehicle sends an action command to the vehicle-mounted control device, for example, a command of parking at the side is executed, so that the unmanned vehicle is controlled to respond to the gesture action.
And a voice data transmitting unit 440 for transmitting the first voice data for prompting a law enforcement officer to present a certificate for law enforcement.
After executing the action instruction, the unmanned vehicle transmits first voice data for prompting law enforcement personnel to present credentials for law enforcement. For example, a preset standard voice segment or a voice segment pre-recorded by the user is sent, and the content of the voice segment may be "please show law enforcement certificate" or "please show legal certificate", for example.
The certificate data receiving unit 450 is used for receiving related data of the certificate for law enforcement, wherein the related data is image data of the certificate acquired by the image acquisition device.
After the first voice data are sent, waiting for a preset time, and starting an image acquisition device or an audio acquisition device for acquiring information of certificates shown by law enforcement officers. Image data of the certificate of a law enforcement officer can be acquired, for example, by an image acquisition device.
An identity extraction subunit 460, configured to extract identity information in the image data of the document.
And extracting identity information representing law enforcement officers in the certificate based on the acquired image data of the certificate. The certificate may be, for example, a police certificate or the like. The identity information may be, for example, the bearer's name, gender, date of birth, job title, warnings, and images of the bearer.
And a legal judging subunit 470, configured to judge whether the certificate is a legal certificate based on the identity information.
After the identity information of the certificate is acquired, the authenticity of the identity information is verified, and the identity information can be processed by an unmanned vehicle and/or a cloud platform. For example, the processing device of the unmanned vehicle performs legality identification judgment on the identity information, or transmits the identity information to the cloud platform in an encryption manner, and performs authenticity identification on the identity information through the cloud platform. The identity information can be extracted through the processing device, the identity information is transmitted to the cloud platform as feature data to be subjected to authenticity identification, the authenticity identification result is fed back to the unmanned vehicle as a legality judgment result, and relevant operation of the unmanned vehicle is controlled based on the legality judgment result, for example, if the certificate is legal, the unmanned vehicle can control the window glass to move towards the ground direction, so that the window is opened, and audio data in the unmanned vehicle can be transmitted conveniently. If the certificate is judged to be illegal, the identity of the law enforcement officer is possibly suspicious, and at the moment, the unmanned vehicle sends the image data of the suspicious officer to the cloud platform so as to improve the safety of the system.
A voice receiving subunit 470a for receiving second voice data, which is input by law enforcement officers, including prompting the unmanned vehicle to provide information indicating its legal driving.
And after the certificate is judged to be legal and is waited for a period of time or simply exchanged, the unmanned vehicle voice acquisition device acquires second voice data and performs voice recognition on the second voice data to obtain keywords. And controlling the operation of the unmanned vehicle according to the keywords. For example, the keyword may be "pass license", and the processing device acquires image data or audio data of the "pass license" from the storage device of the unmanned vehicle.
And the alarm sending unit 470b is used for sending an alarm request to the cloud platform and/or sending the collected first image data of the law enforcement officer if the certificate belongs to illegal, and waiting for indication information returned by the cloud platform.
When the certificate is judged to be illegal, an alarm request is sent to the cloud platform, or when the alarm request is sent, image data of law enforcement personnel is also sent, so that the cloud platform can further verify and judge the identity of the law enforcement personnel. And the unmanned vehicle waits for the indication information returned by the cloud platform within the preset time range, executes related operation according to the indication information, and generates an alarm sound or a light flashing alarm if the indication information exceeds the preset time range.
Optionally, the information presenting subunit 480 is configured to send information indicating legal driving thereof to a voice player or a display device in a voice data format or an image data format for presentation.
After information indicating its legitimate driving is acquired, the information is presented via the voice player or display device of the unmanned vehicle in a voice data format or an image data format. The information indicating its legitimate driving may be, for example, a certificate such as "pass permit" indicating the driving qualification of the unmanned vehicle.
The display device may be, for example, an in-vehicle display device of an unmanned vehicle, a front windshield, a window glass, a navigation display device, or the like.
Optionally, an interaction data storage unit 490a, configured to automatically store data of interactions in a log form;
the interaction data sending unit 490b is configured to upload the interacted data to the cloud platform after the interaction is finished.
In order to further manage the journey behavior of the unmanned vehicle, the interaction information of the unmanned vehicle and law enforcement officers can be automatically stored in a log form in a corresponding storage device, such as the storage device of the unmanned vehicle, or the storage device of a cloud platform, or other movable storage devices, and the like. After the unmanned vehicle and the law enforcement officers finish interaction, all the content of the unmanned vehicle and the law enforcement officers interaction is packaged and uploaded to the cloud platform in an encryption or non-encryption mode.
It should be understood that the units or modules described in the apparatus 300-400 correspond to the various steps in the method described with reference to fig. 1-2. Thus, the operations and features described above with respect to the method are also applicable to the apparatus 300-400 and the units included therein, and are not described herein again. The apparatus 300-400 may be implemented in a browser or other security applications of the electronic device in advance, or may be loaded into the browser or other security applications of the electronic device by downloading or the like. The corresponding units in the apparatus 300-400 can cooperate with units in the electronic device to implement the solution of the embodiment of the present application.
Referring now to fig. 5, fig. 5 illustrates a schematic structural diagram of an unmanned vehicle 500 provided in an embodiment of the present application.
As shown in fig. 5, the unmanned vehicle 500 includes: the human-computer interaction device 501 described in the above embodiments; the description related to the man-machine interaction device 501 takes part in the contents of the above-described embodiments.
An image capture device 502 for capturing first image data of a law enforcement officer and/or image data of a document.
And the vehicle-mounted control device 503 is used for receiving and sending action instructions to control the unmanned vehicle to respond to the gesture actions of law enforcement officers.
The unmanned vehicle 500 may further include: a display device 504 for presenting image data representing information of its legal driving. The display device may be, for example, an in-vehicle display device, a front windshield, a window glass, a navigation display device, or the like.
And the voice playing device 505 is used for playing information for representing legal driving and/or preset first voice data/second voice data and other interactive voice data.
The drone vehicle 500 may also include at least one storage device 506, which may include removable media such as magnetic disks, optical disks, magneto-optical disks, semiconductor memory, and so forth. A computer program installed on a drive as needed so as to be read therefrom is installed in the storage device 506 as needed.
In particular, the processes described above with reference to fig. 1-2 may be implemented as computer software programs, according to embodiments of the present disclosure. For example, embodiments of the present disclosure include a computer program product comprising a computer program tangibly embodied on a machine-readable medium, the computer program comprising program code for performing the method of fig. 1-2. In such an embodiment, the computer program may be downloaded and installed from a network through the communication section, and/or installed from the removable medium 506.
The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
The units or modules described in the embodiments of the present application may be implemented by software or hardware. The described units or modules may also be provided in a processor. The names of these units or modules do not in some cases constitute a limitation of the unit or module itself.
As another aspect, the present application also provides a computer-readable storage medium, which may be the computer-readable storage medium included in the foregoing device in the foregoing embodiment; or it may be a separate computer readable storage medium not incorporated into the device. The computer-readable storage medium stores one or more programs for use by one or more processors in performing the human-computer interaction methods described herein.
The above description is only a preferred embodiment of the application and is illustrative of the principles of the technology employed. It will be appreciated by those skilled in the art that the scope of the invention herein disclosed is not limited to the particular combination of features described above, but also encompasses other arrangements formed by any combination of the above features or their equivalents without departing from the spirit of the invention as defined above. For example, the above features may be replaced with (but not limited to) features having similar functions disclosed in the present application.
Claims (13)
1. A human-computer interaction method is applied to an unmanned automobile and comprises the following steps:
receiving first image data of law enforcement officers collected by an image collecting device;
detecting a gesture action of the law enforcement officer based on the first image data;
sending an action instruction to a vehicle-mounted control device to control the unmanned automobile to respond to the gesture action;
sending first voice data for prompting law enforcement personnel to present certificates for law enforcement;
receiving data relating to the credential for law enforcement;
judging the validity of the certificate based on the related data;
receiving second voice data if the credential is legitimate, the second voice data being input by the law enforcement officer including prompting the unmanned vehicle to provide information indicative of its legitimate driving;
and sending the information for representing the legal driving to a voice player or a display device for presentation in a voice data format or an image data format.
2. The method of claim 1, further comprising:
and if the certificate is illegal, sending an alarm request to a cloud platform and/or sending the collected first image data of the law enforcement officer, and waiting for indication information returned by the cloud platform.
3. The method according to claim 1 or 2, wherein the related data is image data of the document acquired by an image acquisition device, and the determining the validity of the document based on the related data comprises:
extracting identity information in the image data of the certificate;
and judging whether the certificate is a legal certificate or not based on the identity information.
4. The method of claim 1, wherein upon initiating an intelligent voice interaction system to effect interaction of the unmanned vehicle with the law enforcement officer, the method further comprises:
automatically storing data of the interaction in a log form;
and after the interaction is finished, uploading the data of the interaction to a cloud platform.
5. The method of claim 3, wherein the display device comprises any one of:
the vehicle-mounted display device, the front windshield, the window glass and the navigation display device of the unmanned automobile.
6. A human-computer interaction device is characterized in that the device is applied to an unmanned automobile and comprises:
the law enforcement image receiving unit is used for receiving first image data of law enforcement personnel collected by the image collecting device;
a law enforcement image detection unit for detecting a gesture action of the law enforcement officer based on the first image data;
the action instruction sending unit is used for sending an action instruction to a vehicle-mounted control device so as to control the unmanned vehicle to respond to the gesture action;
the voice data sending unit is used for sending first voice data and prompting law enforcement personnel to show certificates for law enforcement;
the certificate data receiving unit is used for receiving the related data of the certificate for law enforcement;
the certificate data judging unit is used for judging the legality of the certificate based on the related data;
the intelligent interaction starting unit is used for receiving second voice data if the certificate is legal, wherein the second voice data is input by the law enforcement officer and comprises information for prompting the unmanned automobile to provide information for representing legal driving of the unmanned automobile; and sending the information for representing the legal driving to a voice player or a display device for presentation in a voice data format or an image data format.
7. The apparatus of claim 6, further comprising:
and the alarm sending unit is used for sending an alarm request and/or sending the collected first image data of the law enforcement officer to a cloud platform and waiting for the indication information returned by the cloud platform if the certificate is illegal.
8. The apparatus according to claim 6 or 7, wherein when the related data is image data of the document captured by the image capturing device, the determining unit further comprises:
the identity extraction subunit is used for extracting identity information in the image data of the certificate;
and the legal judging subunit is used for judging whether the certificate is a legal certificate or not based on the identity information.
9. The apparatus of claim 6, further comprising:
the interaction data storage unit is used for automatically storing the interaction data in a log form;
and the interactive data sending unit is used for uploading the interactive data to a cloud platform after the interaction is finished.
10. The apparatus of claim 6, wherein the display device comprises any one of:
the vehicle-mounted display device, the front windshield, the window glass and the navigation display device of the unmanned automobile.
11. An unmanned vehicle comprising at least one image capture device for capturing first image data of law enforcement officers and/or image data of documents;
the vehicle-mounted control device is used for receiving and sending action instructions to control the unmanned automobile to respond to the gesture actions of the law enforcement officers; and
a human-computer interaction device according to any of claims 6-10.
12. The unmanned vehicle of claim 11, further comprising at least one of:
vehicle-mounted display device, front windshield, window glass and navigation display device.
13. A computer-readable storage medium having stored thereon a computer program for:
the computer program, when executed by a processor, implementing the method as claimed in any one of claims 1-5.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811034447.5A CN109388368B (en) | 2018-09-04 | 2018-09-04 | Human-computer interaction method and device, unmanned vehicle and storage medium thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811034447.5A CN109388368B (en) | 2018-09-04 | 2018-09-04 | Human-computer interaction method and device, unmanned vehicle and storage medium thereof |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109388368A CN109388368A (en) | 2019-02-26 |
| CN109388368B true CN109388368B (en) | 2022-08-12 |
Family
ID=65418632
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811034447.5A Active CN109388368B (en) | 2018-09-04 | 2018-09-04 | Human-computer interaction method and device, unmanned vehicle and storage medium thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109388368B (en) |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111083242A (en) * | 2019-12-31 | 2020-04-28 | 神龙汽车有限公司 | Intelligent networked automobile cloud platform control operation system |
| CN111857638A (en) * | 2020-06-01 | 2020-10-30 | 江西江铃集团新能源汽车有限公司 | Voice interaction method and system based on face recognition and automobile |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103287362A (en) * | 2013-06-25 | 2013-09-11 | 安科智慧城市技术（中国）有限公司 | Vehicle-mounted voice control system and method and electric automobile |
| CN204821476U (en) * | 2015-08-20 | 2015-12-02 | 绍兴市敏智传感网技术研究院有限公司 | Information identification system is driven to motor vehicle |
| CN105893951A (en) * | 2016-03-29 | 2016-08-24 | 华东师范大学 | Multidimensional non-wearable type traffic police gesture identification method and system for driverless vehicles |
| CN106297353A (en) * | 2016-10-18 | 2017-01-04 | 百度在线网络技术（北京）有限公司 | The automatic Pilot method and device of the vehicles |
| CN106994958A (en) * | 2017-03-09 | 2017-08-01 | 百度在线网络技术（北京）有限公司 | Unmanned vehicle security control method and device |
| CN107590763A (en) * | 2017-09-10 | 2018-01-16 | 昆明理工大学 | A kind of motor vehicle driving qualification verifying system and method |
-
2018
- 2018-09-04 CN CN201811034447.5A patent/CN109388368B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103287362A (en) * | 2013-06-25 | 2013-09-11 | 安科智慧城市技术（中国）有限公司 | Vehicle-mounted voice control system and method and electric automobile |
| CN204821476U (en) * | 2015-08-20 | 2015-12-02 | 绍兴市敏智传感网技术研究院有限公司 | Information identification system is driven to motor vehicle |
| CN105893951A (en) * | 2016-03-29 | 2016-08-24 | 华东师范大学 | Multidimensional non-wearable type traffic police gesture identification method and system for driverless vehicles |
| CN106297353A (en) * | 2016-10-18 | 2017-01-04 | 百度在线网络技术（北京）有限公司 | The automatic Pilot method and device of the vehicles |
| CN106994958A (en) * | 2017-03-09 | 2017-08-01 | 百度在线网络技术（北京）有限公司 | Unmanned vehicle security control method and device |
| CN107590763A (en) * | 2017-09-10 | 2018-01-16 | 昆明理工大学 | A kind of motor vehicle driving qualification verifying system and method |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109388368A (en) | 2019-02-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10296883B2 (en) | Systems and methods for driver authentication through embedded sensing | |
| CN107608388B (en) | Autonomous police vehicle | |
| US11042619B2 (en) | Vehicle occupant tracking and trust | |
| CN105590454B (en) | The method and its system that a kind of rule-breaking vehicle behavior is put to the proof | |
| US9235750B1 (en) | Using passive driver identification and other input for providing real-time alerts or actions | |
| EP3756966A1 (en) | Method and apparatus for controlling unmanned vehicle | |
| US10576934B2 (en) | Decentralized cloud-based authentication for autonomous vehicles | |
| US11567509B2 (en) | Control system, control method, and non-transitory storage medium | |
| CN112541980A (en) | In-vehicle device, in-vehicle terminal system, payment method, and non-transitory computer-readable storage medium | |
| CN109388368B (en) | Human-computer interaction method and device, unmanned vehicle and storage medium thereof | |
| CN110808053A (en) | Driver identity verification method and device and electronic equipment | |
| CN110570597A (en) | identity recognition method, system and storage medium | |
| CN107004351B (en) | Traffic violation management system and traffic violation management method | |
| CN113272177A (en) | Method, computer program product, mobile communication device and communication system for providing visual information about at least a part of an environment | |
| CN106910291A (en) | A kind of car rental system based on fingerprint recognition and alcohol detection | |
| JP7013776B2 (en) | Vehicle control device, vehicle, and automatic vehicle allocation method | |
| CN111199461B (en) | Unmanned vehicle passenger carrying method based on blockchain and unmanned vehicle | |
| CN114511315A (en) | Vehicle-mounted camera mobile payment method and computer storage medium | |
| CN109376665B (en) | Taxi driver's driving behavior evaluation system based on crowd sensing | |
| KR102433345B1 (en) | Method and apparatus for providing information using vehicle's camera | |
| CN108074395B (en) | Identity recognition method and device | |
| CN108674367A (en) | A kind of automobile driver's vehicle control method and the system for realizing this method | |
| CN112686076A (en) | Image processing method, system and computer readable storage medium | |
| CN111600839B (en) | Traffic accident handling method, equipment and storage medium | |
| CN112565352A (en) | Military vehicle driving remote authorization and monitoring system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |