CN110784523B - Target object information pushing method and device - Google Patents
Target object information pushing method and device Download PDFInfo
- Publication number
- CN110784523B CN110784523B CN201910964968.9A CN201910964968A CN110784523B CN 110784523 B CN110784523 B CN 110784523B CN 201910964968 A CN201910964968 A CN 201910964968A CN 110784523 B CN110784523 B CN 110784523B
- Authority
- CN
- China
- Prior art keywords
- target
- passenger
- object information
- voice signal
- determining
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04L—TRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
- H04L67/00—Network arrangements or protocols for supporting network services or applications
- H04L67/50—Network services
- H04L67/55—Push-based network services
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V2201/00—Indexing scheme relating to image or video recognition or understanding
- G06V2201/07—Target detection
Abstract
The method and the device for pushing the target object information are disclosed, and comprise the following steps: determining a target passenger corresponding to the received voice signal; judging whether the voice signal meets a preset condition or not; when the voice signal meets a preset condition, the sight line direction of the target passenger is obtained; determining target object information according to the sight line direction of the target passenger; and pushing the target object information to the target passenger. Whether a target passenger sends an inquiry statement or not is obtained by determining the target passenger corresponding to the received voice signal and judging whether the voice signal meets a preset condition or not; determining a target object corresponding to the inquiry according to the sight line direction of the target passenger; further determining that the target object information is pushed to the target passenger; the method and the device realize automatic response to the inquiry of the target passenger for the target object, and improve the user experience.
Description
Technical Field
The present disclosure relates to the field of image analysis technologies, and in particular, to a method and an apparatus for pushing target object information.
Background
The current intelligent technology is more and more applied to the fields of vehicles and driving. Among them, the driving assistance technology is just one of the hot spots. The so-called driving assistance technology mainly uses a sensor to collect various sensing data in the driving process, and intelligently analyzes and calculates the sensing data to obtain specific information. The specific information can be used for prompting or early warning the driver, so that the driver can easily complete various decisions and obtain better driving experience.
In the prior art, the information provided by the auxiliary driving technology is almost directly related to the driving operation, and the requirements of the driver or passengers on other aspects cannot be further met.
Disclosure of Invention
The present disclosure is proposed to solve the above technical problems. The embodiment of the disclosure provides a method and a device for pushing target object information, which automatically determine a target object queried by a target passenger through voice analysis and image analysis, and push the target object information to the target passenger.
According to a first aspect of the present disclosure, there is provided a method for pushing target object information, including:
determining a target passenger corresponding to the received voice signal;
judging whether the voice signal meets a preset condition or not;
when the voice signal meets a preset condition, the sight line direction of the target passenger is obtained;
determining target object information according to the sight line direction of the target passenger;
and pushing the target object information to the target passenger.
According to a second aspect of the present disclosure, there is provided a target object information pushing apparatus, including:
the target passenger determining module is used for determining a target passenger corresponding to the received voice signal;
the voice signal judgment module is used for judging whether the voice signal meets a preset condition or not;
the sight line direction determining module is used for acquiring the sight line direction of the target passenger when the voice signal meets a preset condition;
the target object information determining module is used for determining target object information according to the sight direction of the target passenger;
and the pushing module is used for pushing the target object information to the target passenger.
According to a third aspect of the present disclosure, there is provided a computer-readable storage medium storing a computer program for executing the method for pushing object information described in the first aspect.
According to a fourth aspect of the present disclosure, there is provided an electronic apparatus comprising: a processor; a memory for storing the processor-executable instructions;
the processor is configured to read the executable instruction from the memory, and execute the executable instruction to implement the method for pushing the target object information in the first aspect.
Compared with the prior art, by adopting the target object information pushing method and device provided by the disclosure, whether the target passenger sends out an inquiry statement or not is known by determining the target passenger corresponding to the received voice signal and judging whether the voice signal meets the preset condition or not; determining a target object corresponding to the inquiry according to the sight line direction of the target passenger; further determining that the target object information is pushed to the target passenger; the method and the device realize automatic response to the inquiry of the target passenger for the target object, and improve the user experience.
Drawings
The above and other objects, features and advantages of the present disclosure will become more apparent by describing in more detail embodiments of the present disclosure with reference to the attached drawings. The accompanying drawings are included to provide a further understanding of the embodiments of the disclosure and are incorporated in and constitute a part of this specification, illustrate embodiments of the disclosure and together with the description serve to explain the principles of the disclosure and not to limit the disclosure. In the drawings, like reference numbers generally represent like parts or steps.
Fig. 1 is a schematic structural diagram of a system for pushing target object information according to an exemplary embodiment of the present disclosure;
fig. 2 is a schematic flowchart of a method for pushing target object information according to an exemplary embodiment of the present disclosure;
fig. 3 is a schematic flowchart of a method for pushing target object information according to an exemplary embodiment of the present disclosure;
fig. 4 is a flowchart illustrating a method for pushing target object information according to an exemplary embodiment of the present disclosure;
fig. 5 is a schematic view illustrating a direction of a line of sight in a pushing method of target object information according to an exemplary embodiment of the disclosure;
fig. 6 is a schematic structural diagram of a target object information pushing device according to an exemplary embodiment of the present disclosure;
fig. 7 is a schematic structural diagram of a target passenger determination module in a target object information pushing device according to an exemplary embodiment of the present disclosure;
fig. 8 is a schematic structural diagram of a voice signal determining module in a device for pushing target information according to an exemplary embodiment of the disclosure;
fig. 9 is a schematic structural diagram of a line-of-sight direction determining module in a pushing device for object information according to an exemplary embodiment of the present disclosure;
fig. 10 is a schematic structural diagram of a target object information determining module in a target object information pushing device according to an exemplary embodiment of the present disclosure;
fig. 11 is a schematic structural diagram of a push module in a push device for target object information according to an exemplary embodiment of the present disclosure;
fig. 12 is a block diagram of an electronic device provided in an exemplary embodiment of the present disclosure.
Detailed Description
Hereinafter, example embodiments according to the present disclosure will be described in detail with reference to the accompanying drawings. It is to be understood that the described embodiments are merely a subset of the embodiments of the present disclosure and not all embodiments of the present disclosure, with the understanding that the present disclosure is not limited to the example embodiments described herein.
Summary of the application
The current driving assistance technology is generally only capable of providing various information directly related to driving operation, such as providing navigation function or giving various prompts according to real-time road conditions and traffic rules. In practice, however, the potential needs of the driver or passenger are more than this during the course of the vehicle.
For example, in a driving trip, a business trip, etc., a driver and passengers travel in an unfamiliar environment, and are likely to be interested in the buildings, features, or various landscapes that they see. And then produces a similarity "what is this? "where this is? "etc. However, the current vehicle-mounted intelligent system and the driving assistance system cannot respond to such inquiry and cannot solve the requirements in similar scenes. The user experience is yet to be improved.
According to the method and the device for pushing the target object information, the target object inquired by the user is automatically determined in a similar scene, and the target object information is pushed to the user.
Exemplary System
As shown in fig. 1, in an exemplary push system of target object information, a combination of voice analysis, image analysis and positioning technology is implemented to automatically determine a target object for which a query is directed when a query is issued by a target passenger. And corresponding target object information is acquired to be pushed to the target passenger.
Specifically, it may be determined whether the passenger issues a query based on a voice analysis technique, and in the case where there are a plurality of passengers in the vehicle, a target passenger who issues the query is determined.
It is generally believed that the targeted passenger will look toward the target object to which it is directed when issuing an interrogation. It is possible to take an image of the target passenger and determine the direction of sight of the target passenger through the image using image analysis techniques.
The position of the target object in the map can be calculated by combining the sight line direction of the target passenger and the positioning of the automobile in the map when the inquiry is sent, and then the target object is determined in the map according to the position. And pushing the specific information of the target object to the target passenger, thereby realizing the response to the inquiry of the target passenger. So that the target passenger can conveniently know the corresponding target object of the inquiry.
Exemplary method
Fig. 2 is a flowchart illustrating a method for pushing target object information according to an exemplary embodiment of the present disclosure. The embodiment can be applied to an electronic device, as shown in fig. 2, and includes the following steps:
Speech signals are audio signals that are produced by a passenger speaking in a vehicle. Whereas the relevant queries that the passenger may issue are typically included in the speech signal. Therefore, in the present embodiment, in order to respond to the inquiry of the passenger, the voice signal is monitored in real time.
When a speech signal is received, its source can be determined according to the sound source position of the speech signal, i.e. which passenger in the vehicle has sent the speech signal, for example, by using voiceprint recognition or sound source localization. And the passenger who uttered the voice signal is determined as the target passenger. Or the video can be shot by combining the camera, and the passenger in the video is analyzed to have the mouth movement accompanied with the voice signal, so that the passenger with the mouth movement can be considered as the target passenger sending the voice signal.
In this step, it is determined whether the voice signal satisfies a predetermined condition, and actually, whether the content of the voice signal is a query statement of a target passenger for a target object in the driving environment. In the scenario related to this embodiment, the sentence pattern and content of the query sentence are usually more definite, for example, "what is this", "what is that", "where is this? "equal sentence" is a common query sentence.
From the semantic analysis point of view, the semantic features of such query sentences are obvious. Therefore, by combining with the conventional voice analysis technology, whether a specific voice signal meets a preset condition can be judged, and if the specific voice signal meets the preset condition, the voice signal is determined as an inquiry statement. In this embodiment, the specific manner of the voice analysis and the specific content of the preset condition are not limited. Any speech analysis means that can achieve the same or similar functionality can be incorporated into the overall solution of the present embodiment.
And step 203, when the voice signal meets a preset condition, acquiring the sight line direction of the target passenger.
And when the voice signal meets the preset condition, determining the voice signal as an inquiry statement. That is, it is considered that the target passenger corresponding to the voice signal issues an inquiry about a certain target object at this time. The method in this embodiment needs to further determine the relevant information of the target object.
Typically, a targeted passenger will issue a query while looking at the target to which the passenger is pointing. That is, at the time of the interrogation, the line of sight of the targeted occupant will be directed toward the corresponding target object. It is this principle that the position of the target object is determined in this embodiment.
Therefore, in this step, after the voice signal is determined as an inquiry statement, the sight line direction of the target passenger at the time of issuing the inquiry is further acquired as a basis for determining the position of the target object. Specifically, the vehicle-mounted camera can shoot or record a picture aiming at the target passenger, and the eye movement or the head posture and the head orientation angle of the passenger are analyzed through an image analysis technology, so that the sight line direction of the passenger is determined. The method for determining the sight line direction of the target passenger is not limited in this embodiment, and any method capable of achieving the same or similar functions may be combined in the overall technical solution of this embodiment.
And step 204, determining target object information according to the sight line direction of the target passenger.
According to the sight line direction of the target passenger, the approximate range of the visual field of the target passenger can be determined. The targeted passenger queries the targeted object, typically within this range. And in a large probability the target passenger asks that the target object in question will be a very obvious "scene" of a landmark building or a landmark landscape, etc. present in this area. Therefore, in this embodiment, one or more obvious "scenes" in the visual field of the target passenger can be obtained from the map as the target object pointed by the target passenger according to the sight line direction of the target passenger, and the target object information of the target object can be determined.
And step 205, pushing the target object information to the target passenger.
The determined target object information is pushed to the target passenger, namely, the response to the inquiry of the target passenger is realized, so that the target passenger obtains the corresponding 'answer' to the inquiry.
According to the technical scheme, the beneficial effects of the embodiment are as follows: whether a target passenger sends an inquiry statement or not is obtained by determining the target passenger corresponding to the received voice signal and judging whether the voice signal meets a preset condition or not; determining a target object corresponding to the inquiry according to the sight line direction of the target passenger; further determining that the target object information is pushed to the target passenger; the method and the device realize automatic response to the inquiry of the target passenger for the target object, and improve the user experience.
Fig. 2 shows only a basic embodiment of the method of the present disclosure, and based on this, certain optimization and expansion can be performed, and other preferred embodiments of the method can also be obtained.
Fig. 3 is a schematic flow chart of a method for pushing target object information according to another exemplary embodiment of the present disclosure. The embodiment can be applied to electronic equipment. This embodiment will specifically describe the processing procedure for a speech signal on the basis of the embodiment shown in fig. 2. In this embodiment, the method specifically includes the following steps:
In this embodiment, the sound source position may be determined using a microphone array. The microphone array includes a plurality of microphones therein and is distributed at different locations in the mobile device. Each microphone is at a different distance from the sound source location. Therefore, the position of the sound source can be estimated from the time difference between the voice signals received by the microphones.
In this embodiment, the source of the speech signal is determined according to its sound source position. That is, the sound source position of the voice signal is determined first, and the voice signal is considered to be emitted by a passenger located closest to the sound source position. The passenger is determined to be the target passenger.
In this step, the specific content of the speech signal is analyzed, and whether the content meets the preset condition is judged. The preset condition may be a specific keyword set including various keywords commonly referred to by the query sentence. If any keyword in the keyword set is included in the specific content of the voice signal, the voice signal can be considered to satisfy the preset condition, and the voice signal is determined as the query statement.
For example, keywords in a keyword set include "what this is", "what that is", "where this is", "where that is", "what was in front", and so on. If the specific content of the speech signal is "what sight is this", it is obvious that the keyword "what is this" is included in the speech signal. It may be determined as a query statement.
In this embodiment, the manner of analyzing the specific content of the speech signal may include any one of the following two manners, or may be applied in combination of the following two manners.
One way is that voice recognition is carried out on a voice signal to obtain recognized text information; and judging whether the text information contains preset keywords or not. That is, the specific content of the speech signal is processed into text words, and whether the specific content includes keywords is determined by combining text analysis technology, so as to determine the query sentence.
The other mode is that semantic recognition is carried out on the voice signal to obtain the intention of the user; it is determined whether the intent is a preset intent. That is, specific contents thereof are directly parsed from the voice signal, and it is judged whether the specific contents conform to the "intention" corresponding to the keyword, thereby determining the query sentence.
And step 304, when the voice signal is an inquiry statement, acquiring the sight line direction of the target passenger.
And step 305, determining target object information according to the sight line direction of the target passenger.
And step 306, pushing the target object information to the target passenger.
The steps 304 to 306 in this embodiment relate to the same steps as those in the embodiment shown in fig. 2, and are not repeated.
It should be noted that in the case where a plurality of passengers are present in the vehicle, other passengers may respond to the inquiry of the targeted passenger. There is no need to respond to the inquiry by the method of this embodiment, otherwise the target passenger gets redundant response, resulting in a bad experience. Therefore, in the present embodiment, it is preferable that after the query sentence is determined, the monitoring of the voice signal is continued for a period of time (for example, 5 to 10 seconds), and whether a reply sentence corresponding to the query sentence is issued by a passenger within the period of time is determined. And if no reply sentence exists, pushing the target object information to the target passenger. Otherwise, if a reply sentence exists, the target object information is not pushed.
Fig. 4 is a schematic flowchart of a method for pushing target object information according to another exemplary embodiment of the present disclosure. The embodiment can be applied to electronic equipment. In this embodiment, a process of determining the target object information will be specifically described on the basis of the embodiments shown in fig. 2 to 3. In this embodiment, the method specifically includes the following steps:
In the present embodiment, the steps 401 to 402 relate to the same steps as those in the embodiment shown in fig. 2, and the description is not repeated.
And step 403, acquiring at least one frame of target image when the voice signal meets a preset condition.
When the voice signal satisfies the predetermined condition, that is, the voice signal is an inquiry statement, the step may be executed. The target image in this step is an image taken for the target passenger, or an image frame in a video recorded for the target passenger. The number of target images may be one or more.
The target image is used to determine the direction of the target passenger's line of sight, so the target image needs to include the target passenger's eyes.
And step 404, performing sight tracking on the eyes of the target passenger to obtain the sight direction of the target passenger.
By combining with an image analysis technology, the sight tracking can be carried out on the eyes of the target passenger according to the target image so as to obtain the sight direction of the target passenger. The specific way of tracking the line of sight in this embodiment is not limited, and ways that can achieve the same or similar effects in the prior art can be combined with the overall technical solution of this embodiment.
Specifically, the position of the GPS module or any other positioning module of the mobile device at this time may be called up to obtain the position information while the target image is captured. Or a corresponding time stamp may be recorded for each frame of image in the video file as the video for the target passenger is recorded. And when one frame of image is determined as the target image, inquiring the driving track of the movable equipment according to the timestamp corresponding to the target image so as to acquire the position information of the movable equipment at the moment corresponding to the timestamp.
And step 406, determining target object information according to the position information and the sight line direction of the target passenger.
And calculating the approximate range of the gazing position of the target passenger in the map according to the geometric relationship by combining the position of the target passenger (namely the position of the movable equipment where the target passenger is located) and the sight direction of the target passenger. And the subject passenger will typically be within this approximate range to interrogate the subject matter of interest. Thus, the target object and the target object information can be determined in the map.
Specifically, at least two frames of target images can be determined, wherein the shooting time of one frame of target image is the time of receiving the voice signal; the shooting time of the other target images is a certain time after the voice signal is received and a certain time interval passes. And determining a position of the movable apparatus where the target passenger is located at a time of taking each target image, and a sight-line direction of the target passenger in each target image. Then determining the intersection point of the sight line direction corresponding to each target image according to the position information corresponding to each target image; and determining the intersection point of the sight line directions as the position of the target object.
As shown in fig. 5, a schematic view of the target passenger's direction of sight. In the case shown in fig. 5, two target images are determined, a first target image and a second target image, respectively. The moment when the first target image is shot is the moment when the voice signal is received. At this point in time the mobile device is at position a, the direction of the target passenger's line of sight is shown by the solid arrow. The time when the second target image is captured may be 0.5 seconds after the voice signal is received. At the moment of the second target image capture, the mobile device is in position B, the target passenger's direction of sight is shown as a dashed arrow. It can be inferred from the geometrical relationship that the intersection of the lines of sight of the target passengers in the two target images is located approximately in the region of the circular wire frame. It will of course be appreciated that the circular wire frame shown in figure 5 is merely an exemplary range.
This means that the line of sight of the target passenger is constantly looking at the area of the circular wire frame while the mobile device is moving from position a to position B, at the same time as the target passenger issues an inquiry. Then, with great probability, the object to which its query refers should be in the region where the circular wireframe is located. It can be determined that the position of the object in the map should be within the range of the circular wire frame. Further, map information may be queried to determine at least one object from within the circular wire frame.
And 407, pushing the information of the plurality of target objects to the target passenger according to a preset sequence.
In fact, there is often more than one "building" or "scene" within a corresponding range, so multiple object information can often be determined for the user to select. In the process of pushing the information to the user, the information of the plurality of target objects can be pushed to the user according to a certain sequence. For example, the pushing may be performed in order of distance from the above-described sight-line intersection. The pushing can also be carried out according to the sequence of the 'hot degree' or 'attention degree' of the target object information from high to low, and the hot 'buildings' or 'scenery' can be preferentially pushed to the target passenger.
In the process of pushing the target object information, whether the target passenger responds to corresponding instructions or not can be judged in real time. For example, if the pushed target object information corresponds to the target object pointed by the target passenger inquiry, the passenger may respond to a voice signal such as "yes" or "yes", and then the pushing may be stopped, or more related information corresponding to the target object may be continuously pushed. If the pushed target object information does not correspond to the target object pointed by the target passenger inquiry, the passenger may respond to the voice signals of 'not' or 'next', and the next target object information can be pushed continuously in sequence at the moment. Or in some scenarios the targeted passenger does not respond, the targeted passenger may be deemed to "default" the targeted object information.
The "degree of heat" or "degree of attention" regarding the target object information may be determined from data such as the number of searches and the amount of attention that have been made in the internet. Of course, the responses of the target passengers to the respective target object information may be further fed back to the "degree of heat" or "degree of attention" of the target object information, so as to update the "degree of heat" or "degree of attention", and thus, the responses may be more accurately pushed to other target passengers who subsequently make inquiries.
For example, if the target passenger responds to a voice signal such as "yes" or default, the "heat" or "attention" of the corresponding target object may be increased at this time. Otherwise, if the passenger responds to the voice signal such as "not" or "next", the "hotness" or "attention" of the corresponding object may be lowered. Objects interrogated by multiple targeted passengers in multiple mobile devices may have a higher "heat" or "focus".
In this embodiment, the specific form of pushing the target object information may be voice broadcast, text display, or directly pushing the information to an intelligent terminal (such as a mobile phone of a target passenger) connected to the electronic device executing the method. The specific form of pushing in this embodiment is not limited.
It should be noted that, for the voice signal (essentially, an audio file) or the target image (essentially, an image file or a video file) collected in the method according to this embodiment, after the flow of the method is finished, the voice signal or the target image may be deleted to release the storage space of the electronic device running the method.
As can be seen from the above technical method, the present embodiment further has the following beneficial effects on the basis of the foregoing embodiment: the position range of the target object is determined by combining the plurality of target images and the corresponding position information, so that the position information of the target object is more accurately acquired; by pushing the target object information to the target passenger in a specific order, the user experience is further improved.
Exemplary devices
Fig. 6 is a schematic structural diagram of a target object information pushing device according to an exemplary embodiment of the present disclosure. The apparatus of this embodiment is a physical apparatus for executing the methods of fig. 2 to 4. The technical solution is essentially the same as that in the above embodiment, and the corresponding description in the above embodiment is also applicable to this embodiment. The device in the embodiment comprises:
and a target passenger determining module 601, configured to determine a target passenger corresponding to the received voice signal.
The voice signal determining module 602 is configured to determine whether the voice signal satisfies a preset condition.
And a sight direction determining module 603, configured to obtain a sight direction of the target passenger when the voice signal meets a preset condition.
And the target object information determining module 604 is configured to determine the target object information according to the sight line direction of the target passenger.
And a pushing module 605, configured to push the target object information to the target passenger.
Fig. 7 is a schematic structural diagram of a targeted passenger determination module 601 in a target object information pushing device according to another exemplary embodiment of the present disclosure. As shown in fig. 7, in an exemplary embodiment, the targeted passenger determination module 601 includes:
the sound source positioning unit 711 is configured to perform sound source positioning on the voice signal to obtain a sound source position of the voice signal.
And a target passenger determining unit 712, configured to determine a target passenger corresponding to the voice signal according to the sound source position.
Fig. 8 is a schematic structural diagram of a voice signal determining module 602 in a target object information pushing apparatus according to another exemplary embodiment of the present disclosure. As shown in fig. 8, in an exemplary embodiment, the voice signal determination module 602 includes:
a text recognition unit 811 for performing speech recognition on the speech signal to obtain recognized text information; and judging whether the text information contains preset keywords or not.
A semantic recognition unit 812, configured to perform semantic recognition on the speech signal to obtain an intention of the user; it is determined whether the intent is a preset intent.
Fig. 9 is a schematic structural diagram of a line-of-sight direction determining module 603 in a pushing device for object information according to another exemplary embodiment of the present disclosure. As shown in fig. 9, in an exemplary embodiment, gaze direction determination module 603 includes:
and the target image acquiring unit 911 is configured to acquire at least one frame of target image, where the target image includes the eyes of the target passenger.
A sight line tracking unit 912, configured to perform sight line tracking on the eyes of the target passenger to obtain the sight line direction of the target passenger.
Fig. 10 is a schematic structural diagram of a target object information determining module 604 in a target object information pushing device according to another exemplary embodiment of the present disclosure. As shown in fig. 10, in an exemplary embodiment, the object information determination module 604 includes:
a position information acquiring unit 1011 for acquiring position information of the movable apparatus where the target passenger is located when the target image is captured.
A line-of-sight intersection determination unit 1012 is configured to determine an intersection of the line-of-sight directions corresponding to the target images, based on the position information corresponding to the target images.
Target information determining section 1013 determines the intersection of the visual line directions as the target position, and determines the target information based on the target position.
Fig. 11 is a schematic structural diagram of a pushing module 605 in a target object information pushing device according to another exemplary embodiment of the present disclosure. As shown in fig. 10, in an exemplary embodiment, the push module 605 includes:
the sorting unit 1111 is configured to sort the plurality of target object information according to a preset order.
A pushing unit 1112, configured to push the plurality of target object information to the target passenger in a preset order.
Exemplary electronic device
Next, an electronic apparatus according to an embodiment of the present disclosure is described with reference to fig. 12. The electronic device may be either or both of the first device 100 and the second device 200, or a stand-alone device separate from them that may communicate with the first device and the second device to receive the collected input signals therefrom.
FIG. 12 illustrates a block diagram of an electronic device in accordance with an embodiment of the disclosure.
As shown in fig. 12, the electronic device 10 includes one or more processors 11 and a memory 12.
The processor 11 may be a Central Processing Unit (CPU) or other form of processing unit having data processing capabilities and/or instruction execution capabilities, and may control other components in the electronic device 10 to perform desired functions.
Memory 12 may include one or more computer program products that may include various forms of computer-readable storage media, such as volatile memory and/or non-volatile memory. The volatile memory may include, for example, Random Access Memory (RAM), cache memory (cache), and/or the like. The non-volatile memory may include, for example, Read Only Memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium and executed by the processor 11 to implement the target object information pushing method and/or other desired functions of the various embodiments of the present disclosure described above. Various contents such as an input signal, a signal component, a noise component, etc. may also be stored in the computer-readable storage medium.
In one example, the electronic device 10 may further include: an input device 13 and an output device 14, which are interconnected by a bus system and/or other form of connection mechanism (not shown).
For example, when the electronic device is the first device 100 or the second device 200, the input device 13 may be a microphone or a microphone array as described above for capturing an input signal of a sound source. When the electronic device is a stand-alone device, the input means 13 may be a communication network connector for receiving the acquired input signals from the first device 100 and the second device 200.
The input device 13 may also include, for example, a keyboard, a mouse, and the like.
The output device 14 may output various information including the determined distance information, direction information, and the like to the outside. The output devices 14 may include, for example, a display, speakers, a printer, and a communication network and its connected remote output devices, among others.
Of course, for simplicity, only some of the components of the electronic device 10 relevant to the present disclosure are shown in fig. 12, omitting components such as buses, input/output interfaces, and the like. In addition, the electronic device 10 may include any other suitable components depending on the particular application.
Exemplary computer program product and computer-readable storage Medium
In addition to the above-described methods and apparatus, embodiments of the present disclosure may also be a computer program product comprising computer program instructions, the described methods and apparatusComputer programThe instructions, when executed by the processor, cause the processor to perform steps in a method of pushing object information according to various embodiments of the present disclosure as described in the "exemplary methods" section above in this specification.
The computer program product may write program code for carrying out operations for embodiments of the present disclosure in any combination of one or more programming languages, including an object oriented programming language such as Java, C + + or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device and partly on a remote computing device, or entirely on the remote computing device or server.
Furthermore, embodiments of the present disclosure may also be a computer-readable storage medium having stored thereon computer program instructions that, when executed by a processor, cause the processor to perform steps in a push method of object information according to various embodiments of the present disclosure described in the "exemplary methods" section above in this specification.
The computer-readable storage medium may take any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may include, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples (a non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
The foregoing describes the general principles of the present disclosure in conjunction with specific embodiments, however, it is noted that the advantages, effects, etc. mentioned in the present disclosure are merely examples and are not limiting, and they should not be considered essential to the various embodiments of the present disclosure. Furthermore, the foregoing disclosure of specific details is for the purpose of illustration and description and is not intended to be limiting, since the disclosure is not intended to be limited to the specific details so described.
The block diagrams of devices, apparatuses, systems referred to in this disclosure are only given as illustrative examples and are not intended to require or imply that the connections, arrangements, configurations, etc. must be made in the manner shown in the block diagrams. These devices, apparatuses, devices, systems may be connected, arranged, configured in any manner, as will be appreciated by those skilled in the art. Words such as "including," "comprising," "having," and the like are open-ended words that mean "including, but not limited to," and are used interchangeably therewith. The words "or" and "as used herein mean, and are used interchangeably with, the word" and/or, "unless the context clearly dictates otherwise. The word "such as" is used herein to mean, and is used interchangeably with, the phrase "such as but not limited to".
It is also noted that in the devices, apparatuses, and methods of the present disclosure, each component or step can be decomposed and/or recombined. Such decomposition and/or recombination should be considered as equivalents of the present disclosure.
The previous description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present disclosure. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects without departing from the scope of the disclosure. Thus, the present disclosure is not intended to be limited to the aspects shown herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.
The foregoing description has been presented for purposes of illustration and description. Furthermore, this description is not intended to limit embodiments of the disclosure to the form disclosed herein. While a number of example aspects and embodiments have been discussed above, those of skill in the art will recognize certain variations, modifications, alterations, additions and sub-combinations thereof.
Claims (10)
1. A method for pushing target object information comprises the following steps:
determining a target passenger corresponding to the received voice signal;
judging whether the voice signal meets a preset condition or not;
when the voice signal meets a preset condition, determining the voice signal as an inquiry statement, and acquiring the sight line direction of the target passenger at the moment of sending the inquiry;
determining target object information according to the sight line direction of the target passenger;
pushing the target object information to the target passenger;
before the target object information is pushed to the target passenger, the method further comprises the following steps:
and judging that no passenger sends out a reply sentence corresponding to the inquiry sentence within a preset time after the inquiry sentence.
2. The method of claim 1, wherein determining the target passenger to which the received voice signal corresponds comprises:
carrying out sound source positioning on the voice signal to obtain the sound source position of the voice signal;
and determining the target passenger corresponding to the voice signal according to the sound source position.
3. The method of claim 1, wherein the determining whether the voice signal satisfies a preset condition comprises:
carrying out voice recognition on the voice signal to obtain recognized text information;
judging whether preset keywords are contained in the text information; or
Performing semantic recognition on the voice signal to obtain the intention of a user;
and judging whether the intention is a preset intention.
4. The method of claim 1, wherein said obtaining a line of sight direction of the target passenger at a time of issuing the interrogation comprises:
acquiring at least one frame of target image, wherein the target image comprises eyes of a target passenger;
and carrying out sight tracking on the eyes of the target passenger to obtain the sight direction of the target passenger.
5. The method of claim 4, wherein said determining target object information based on the gaze direction of the target passenger comprises:
when the target image is shot, acquiring the position information of the movable equipment where the target passenger is located;
and determining target object information according to the position information and the sight line direction of the target passenger.
6. The method of claim 5, wherein the number of target images is at least two frames, and wherein determining target object information based on the position information and the target passenger's gaze direction comprises:
determining the intersection point of the sight line direction corresponding to each target image according to the position information corresponding to each target image;
and determining the intersection point of the sight line directions as the position of the target object, and determining the information of the target object according to the position of the target object.
7. The method of any of claims 1-6, wherein the object information comprises a plurality of object information, and wherein pushing the object information to the targeted passenger comprises:
and pushing the plurality of target object information to the target passenger according to a preset sequence.
8. A target object information pushing device comprises:
the target passenger determining module is used for determining a target passenger corresponding to the received voice signal;
the voice signal judgment module is used for judging whether the voice signal meets a preset condition or not;
the sight direction determining module is used for determining the voice signal as an inquiry statement when the voice signal meets a preset condition, and acquiring the sight direction of the target passenger at the moment of sending the inquiry;
the target object information determining module is used for determining target object information according to the sight direction of the target passenger;
the pushing module is used for pushing the target object information to the target passenger;
the push module is further configured to: before the target object information is pushed to the target passenger, judging that no passenger sends out a reply sentence corresponding to the inquiry sentence within a preset time after the inquiry sentence.
9. A computer-readable storage medium storing a computer program for executing the method for pushing the object information according to any one of claims 1 to 7.
10. An electronic device, the electronic device comprising:
a processor;
a memory for storing the processor-executable instructions;
the processor is configured to read the executable instructions from the memory, and execute the instructions to implement the method for pushing the object information according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910964968.9A CN110784523B (en) | 2019-10-11 | 2019-10-11 | Target object information pushing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910964968.9A CN110784523B (en) | 2019-10-11 | 2019-10-11 | Target object information pushing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110784523A CN110784523A (en) | 2020-02-11 |
| CN110784523B true CN110784523B (en) | 2022-08-02 |
Family
ID=69385037
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910964968.9A Active CN110784523B (en) | 2019-10-11 | 2019-10-11 | Target object information pushing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110784523B (en) |
Families Citing this family (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110929703B (en) * | 2020-02-04 | 2020-11-13 | 北京未动科技有限公司 | Information determination method and device and electronic equipment |
| CN113947632A (en) * | 2020-07-15 | 2022-01-18 | 北京芯海视界三维科技有限公司 | Method and device for realizing target object positioning and display device |
| CN113147779A (en) * | 2021-04-29 | 2021-07-23 | 前海七剑科技(深圳)有限公司 | Vehicle control method and device |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107705793A (en) * | 2017-09-22 | 2018-02-16 | 百度在线网络技术（北京）有限公司 | Information-pushing method, system and its equipment based on Application on Voiceprint Recognition |
| CN108897867A (en) * | 2018-06-29 | 2018-11-27 | 北京百度网讯科技有限公司 | For the data processing method of knowledge question, device, server and medium |
| WO2019076120A1 (en) * | 2017-10-19 | 2019-04-25 | 格力电器（武汉）有限公司 | Image processing method, device, storage medium and electronic device |
| CN109712250A (en) * | 2019-01-24 | 2019-05-03 | 深圳前海达闼云端智能科技有限公司 | Display method, device and terminal based on vehicle |
| CN109815409A (en) * | 2019-02-02 | 2019-05-28 | 北京七鑫易维信息技术有限公司 | A kind of method for pushing of information, device, wearable device and storage medium |
| CN110310634A (en) * | 2019-06-19 | 2019-10-08 | 广州小鹏汽车科技有限公司 | Vehicle-mounted voice method for pushing, terminal, server and supplying system |
-
2019
- 2019-10-11 CN CN201910964968.9A patent/CN110784523B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107705793A (en) * | 2017-09-22 | 2018-02-16 | 百度在线网络技术（北京）有限公司 | Information-pushing method, system and its equipment based on Application on Voiceprint Recognition |
| WO2019076120A1 (en) * | 2017-10-19 | 2019-04-25 | 格力电器（武汉）有限公司 | Image processing method, device, storage medium and electronic device |
| CN108897867A (en) * | 2018-06-29 | 2018-11-27 | 北京百度网讯科技有限公司 | For the data processing method of knowledge question, device, server and medium |
| CN109712250A (en) * | 2019-01-24 | 2019-05-03 | 深圳前海达闼云端智能科技有限公司 | Display method, device and terminal based on vehicle |
| CN109815409A (en) * | 2019-02-02 | 2019-05-28 | 北京七鑫易维信息技术有限公司 | A kind of method for pushing of information, device, wearable device and storage medium |
| CN110310634A (en) * | 2019-06-19 | 2019-10-08 | 广州小鹏汽车科技有限公司 | Vehicle-mounted voice method for pushing, terminal, server and supplying system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110784523A (en) | 2020-02-11 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110784523B (en) | Target object information pushing method and device | |
| CN109643158B (en) | Command processing using multi-modal signal analysis | |
| US9937922B2 (en) | Collision avoidance using auditory data augmented with map data | |
| US10810881B2 (en) | Method of providing sound tracking information, sound tracking apparatus for vehicles, and vehicle having the same | |
| AU2021201597A1 (en) | Systems and Methods for Supplementing Captured Data | |
| US7792328B2 (en) | Warning a vehicle operator of unsafe operation behavior based on a 3D captured image stream | |
| US11158343B2 (en) | Systems and methods for cross-redaction | |
| US20130030811A1 (en) | Natural query interface for connected car | |
| US9936025B2 (en) | Application for vehicle-to-vehicle communication | |
| JP2019179372A (en) | Learning data creation method, learning method, risk prediction method, learning data creation device, learning device, risk prediction device, and program | |
| JP6448880B1 (en) | Danger information collection device | |
| CN113619588A (en) | Driving behavior reminding method and device, storage medium and electronic equipment | |
| US10926699B2 (en) | Method and system for historical state based advanced driver assistance | |
| CN113593572A (en) | Method and apparatus for performing sound zone localization in spatial region, device and medium | |
| CN110194181B (en) | Driving support method, vehicle, and driving support system | |
| US20230125724A1 (en) | Computer-implemented method and non-transitory computer-readable medium for generating a thumbnail from a video stream or file, and video surveillance system | |
| CN114550720A (en) | Voice interaction method and device, electronic equipment and storage medium | |
| CN116324659A (en) | Emergency alarm flute detection for autonomous vehicles | |
| CN112182256A (en) | Object identification method and device and vehicle | |
| CN114299905A (en) | Path compensation function determination method and device, and active noise reduction method and device | |
| CN209980327U (en) | Intelligent automobile data recorder based on YOLO object image recognition, system and vehicle | |
| WO2021075277A1 (en) | Information processing device, method, and program | |
| CN112951216B (en) | Vehicle-mounted voice processing method and vehicle-mounted information entertainment system | |
| CN117275473A (en) | Multi-screen voice control method, device, equipment and computer readable storage medium | |
| CN117622186A (en) | Autonomous interaction control method for vehicle and vehicle |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |