CN112383826A - Control method and device of vehicle-mounted entertainment terminal, storage medium, terminal and automobile - Google Patents
Control method and device of vehicle-mounted entertainment terminal, storage medium, terminal and automobile Download PDFInfo
- Publication number
- CN112383826A CN112383826A CN202011240794.0A CN202011240794A CN112383826A CN 112383826 A CN112383826 A CN 112383826A CN 202011240794 A CN202011240794 A CN 202011240794A CN 112383826 A CN112383826 A CN 112383826A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- entertainment terminal
- determining
- target user
- target
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/47—End-user applications
- H04N21/472—End-user interface for requesting content, additional data or services; End-user interface for interacting with content, e.g. for content reservation or setting reminders, for requesting event notification, for manipulating displayed content
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/414—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance
- H04N21/41422—Specialised client platforms, e.g. receiver in car or embedded in a mobile appliance located in transportation means, e.g. personal vehicle
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/442—Monitoring of processes or resources, e.g. detecting the failure of a recording device, monitoring the downstream bandwidth, the number of times a movie has been viewed, the storage space available from the internal hard disk
- H04N21/44213—Monitoring of end-user related data
- H04N21/44218—Detecting physical presence or behaviour of the user, e.g. using sensors to detect if the user is leaving the room or changes his face expression during a TV program
Abstract
The invention discloses a control method and device of a vehicle-mounted entertainment terminal, a storage medium, a terminal and an automobile. The method comprises the steps of determining a sight line vector of a target user under a finished automobile coordinate system; determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector; and determining the control intention of the target user according to the attention target, and controlling the vehicle-mounted entertainment terminal according to the control intention. By adopting the control method of the vehicle-mounted entertainment terminal provided by the embodiment of the invention, the vehicle-mounted entertainment terminal can be controlled through the sight vector, the risk of dispersing the attention of a driver due to the control of the vehicle-mounted entertainment terminal is reduced, and the driving safety is improved.
Description
Technical Field
The embodiment of the invention relates to an automobile technology, in particular to a control method and device of a vehicle-mounted entertainment terminal, a storage medium, a terminal and an automobile.
Background
With the improvement of the electrification degree of the automobile, the functions of the automobile tend to be intelligentized and scenarized, so that more consideration needs to be given to user scenes and user experience. Under the condition that various in-vehicle control strategies provide multidimensional operation experience for passengers in the vehicle, more and more screens and larger screens are arranged in the vehicle, and the short plates operated manually are more and more obvious. For example, a driver in a vehicle performs function manipulation by touching a screen, and the attention of the driver may be focused on an in-vehicle entertainment terminal for a long time, thereby affecting normal driving.
Disclosure of Invention
The embodiment of the invention provides a control method and device of a vehicle-mounted entertainment terminal, a storage medium, a terminal and an automobile, which can intelligently control the vehicle-mounted entertainment terminal to respond to the control intention of a user.
In a first aspect, an embodiment of the present invention provides a method for controlling a vehicle-mounted entertainment terminal, including:
determining a sight line vector of a target user under a finished automobile coordinate system;
determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector;
and determining the control intention of the target user according to the attention target, and controlling the vehicle-mounted entertainment terminal according to the control intention.
Optionally, determining a sight line vector of the target user in the entire vehicle coordinate system includes:
acquiring a first sight line vector of a target user in a camera coordinate system;
and determining a second sight vector corresponding to the first sight vector according to a coordinate conversion rule, wherein the second sight vector is a sight vector of the target user in a coordinate system of the whole vehicle, and the coordinate conversion rule is determined based on a mapping relation between a first coordinate of a preset feature point in the vehicle in the coordinate system of the camera and a second coordinate in the coordinate system of the whole vehicle.
Optionally, determining an attention target of the target user on a screen of the in-vehicle entertainment terminal according to the sight vector includes:
determining the position relation between an intersection point of the sight line vector and a screen of the vehicle-mounted entertainment terminal and a preset screen partition, wherein the screen partition comprises a function area and a display area;
and determining the attention target of the target user according to the position relation.
Optionally, determining a control intention of the target user according to the attention target, and controlling the in-vehicle entertainment terminal according to the control intention, includes:
acquiring state information of the concerned target in a preset time period, and determining a control intention of the target user according to the state information, wherein the control intention comprises translation, dragging, telephone dialing or application program opening and closing;
and generating a control instruction according to the control intention, and controlling the vehicle-mounted entertainment terminal based on the control instruction.
Optionally, determining an attention target of the target user on a screen of the in-vehicle entertainment terminal according to the sight vector includes:
and acquiring coordinates of the intersection point of the sight vector and a screen of the vehicle-mounted entertainment terminal, generating an identification area according to the coordinates, and taking the identification area as an attention target of the target user.
Optionally, determining a control intention of the target user according to the attention target, and controlling the in-vehicle entertainment terminal according to the control intention, includes:
acquiring an overlapping area of an identification area corresponding to the attention target and a preset trigger area, and determining a control intention of the target user according to the overlapping area, wherein the control intention comprises whether a function corresponding to the trigger area is triggered or not;
and when the control intention is to trigger the function corresponding to the trigger area, generating a trigger instruction, and controlling the vehicle-mounted entertainment terminal based on the trigger instruction.
In a second aspect, the embodiment of the invention also provides a control device of the vehicle-mounted entertainment terminal, and the device
The method comprises the following steps:
the sight line determining module is used for determining a sight line vector of a target user under a finished automobile coordinate system;
the target determining module is used for determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector;
and the terminal control module is used for determining the control intention of the target user according to the attention target and controlling the vehicle-mounted entertainment terminal according to the control intention.
Optionally, the gaze determining module is specifically configured to:
acquiring a first sight line vector of a target user in a camera coordinate system;
and determining a second sight vector corresponding to the first sight vector according to a coordinate conversion rule, wherein the second sight vector is a sight vector of the target user in a coordinate system of the whole vehicle, and the coordinate conversion rule is determined based on a mapping relation between a first coordinate of a preset feature point in the vehicle in the coordinate system of the camera and a second coordinate in the coordinate system of the whole vehicle.
Optionally, the target determining module is specifically configured to:
determining the position relation between an intersection point of the sight line vector and a screen of the vehicle-mounted entertainment terminal and a preset screen partition, wherein the screen partition comprises a function area and a display area;
and determining the attention target of the target user according to the position relation.
Optionally, the terminal control module is specifically configured to:
acquiring state information of the concerned target in a preset time period, and determining a control intention of the target user according to the state information, wherein the control intention comprises translation, dragging, telephone dialing or application program opening and closing;
and generating a control instruction according to the control intention, and controlling the vehicle-mounted entertainment terminal based on the control instruction.
Optionally, the target determining module is specifically configured to:
and acquiring coordinates of the intersection point of the sight vector and a screen of the vehicle-mounted entertainment terminal, generating an identification area according to the coordinates, and taking the identification area as an attention target of the target user.
Optionally, the terminal control module is specifically configured to:
acquiring an overlapping area of an identification area corresponding to the attention target and a preset trigger area, and determining a control intention of the target user according to the overlapping area, wherein the control intention comprises whether a function corresponding to the trigger area is triggered or not;
and when the control intention is to trigger the function corresponding to the trigger area, generating a trigger instruction, and controlling the vehicle-mounted entertainment terminal based on the trigger instruction.
In a third aspect, the present invention further provides a computer-readable storage medium, on which a computer program is stored, which, when executed by a processor, implements the control method of the in-vehicle entertainment terminal as described in the first aspect above.
In a fourth aspect, an embodiment of the present invention further provides an in-vehicle entertainment terminal, including a display screen, a memory, a processor, and a computer program stored on the memory and executable on the processor, where the processor executes the computer program to implement the control method of the in-vehicle entertainment terminal according to the first aspect.
In a fifth aspect, the embodiment of the present invention further provides an automobile, including the vehicle-mounted entertainment terminal described in the fourth aspect.
The embodiment of the invention provides a control scheme of a vehicle-mounted entertainment terminal, which is characterized in that a sight line vector of a target user under a whole vehicle coordinate system is determined; determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector; and determining the control intention of the target user according to the attention target, and controlling the vehicle-mounted entertainment terminal according to the control intention, so that the vehicle-mounted entertainment terminal is controlled through the sight vector, the risk of dispersing the attention of a driver due to the control of the vehicle-mounted entertainment terminal is reduced, and the driving safety is improved.
Drawings
FIG. 1 is a flowchart illustrating a control method for a vehicle-mounted entertainment terminal according to an embodiment of the present invention;
FIG. 2 is a schematic diagram illustrating a relationship between a camera, eyes and a screen in a control method of a vehicle-mounted entertainment terminal according to an embodiment of the present invention;
FIG. 3 is a flowchart illustrating another control method for a vehicle-mounted entertainment terminal according to a second embodiment of the present invention;
fig. 4 is a schematic diagram of a divided screen according to a second embodiment of the present invention;
FIG. 5 is a flowchart illustrating a control method for a vehicle-mounted entertainment terminal according to a third embodiment of the present invention;
FIG. 6 is a schematic diagram of a map move provided by the present invention;
fig. 7 is a schematic diagram illustrating selection of contacts with the same name according to a third embodiment of the present invention;
FIG. 8 is a schematic structural diagram of a control device of a vehicle-mounted entertainment terminal according to a fourth embodiment of the present invention;
fig. 9 is a schematic diagram of a vehicle-mounted entertainment terminal according to a fifth embodiment of the present invention.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Example one
Fig. 1 is a flowchart of a control method of an in-vehicle entertainment terminal according to an embodiment of the present invention, where the method may be performed by a control apparatus of the in-vehicle entertainment terminal, the apparatus may be implemented by software and/or hardware, and the apparatus is generally integrated on the in-vehicle entertainment terminal. The method comprises the following steps:
and step 110, determining a sight line vector of the target user in the whole vehicle coordinate system.
Wherein the target user may be an in-vehicle person who intends to control the in-vehicle entertainment terminal. For example, the target user may be a driver or an in-vehicle passenger, etc.
The sight line vector may be a line connecting the pupil of the target user to a point on the screen of the in-vehicle entertainment terminal. The gaze vector of the target user may be determined by eye movement detection techniques. For example, a camera is provided in an automobile, and a user image of a target user is captured by the camera. The camera sends the user image to a state detection module, in which an eye movement recognition model has been configured. The state detection module inputs the user image into the eye movement recognition model and obtains the sight line vector output by the eye movement recognition model. The eye movement recognition model may be a machine learning model, and the adopted algorithm may include a Recurrent Neural Network (RNN), a Long Short Term Memory (LSTM) network, a threshold cycle unit, a simple cycle unit, an automatic encoder, a decision tree, a random forest, a feature mean classification, a classification regression tree, a hidden markov, a K-nearest neighbor (KNN) algorithm, a logistic regression model, a bayesian model, a gaussian model, and a KL divergence (Kullback-Leibler divergence), and the like.
Fig. 2 is a schematic diagram illustrating a relationship between a camera, eyes and a screen in a control method of a vehicle-mounted entertainment terminal according to an embodiment of the present invention. As shown in FIG. 2, when a target user looks at the screen of the in-vehicle entertainment terminal, a camera captures a user image of the target user. And determining the sight line vector of the target user according to the user image through a state detection module. As shown in fig. 2, the screens include a main screen 210, a sub-screen 220, an air-conditioning control screen 230, and the like.
It should be noted that the user image acquired by the state detection module is based on the polar coordinates or the triangular coordinates of the camera, so that the sight line vector determined based on the user image is also a vector in the polar coordinate system or the triangular coordinate system. The sight line vector needs to be converted into a vector in the entire vehicle coordinate system. The first coordinate of the preset feature point in the vehicle under the camera coordinate system and the second coordinate of the preset feature point in the vehicle under the vehicle coordinate system can be respectively obtained in advance, and the coordinate conversion rule is determined according to the mapping relation of the first coordinate and the second coordinate. Since the position of the preset feature point in the vehicle is fixed, that is, the second coordinate of the preset feature point in the coordinate system of the whole vehicle is known. The camera shoots the preset feature points, and first coordinates of the preset feature points in a camera coordinate system can be obtained. And determining a coordinate conversion rule according to the mapping relation between the first coordinate and the second coordinate of the preset feature point. The coordinate conversion rule may be that a preset feature point shot by the camera is converted into a point in the rectangular coordinate system according to a coordinate conversion relationship between the polar coordinate and the rectangular coordinate system, an association relationship between an origin of the rectangular coordinate system and an origin of the finished vehicle coordinate system is determined based on the mapping relationship, and the point in the rectangular coordinate system is converted into a point in the finished vehicle coordinate system based on the association relationship. For example, two characteristic areas of 1cm by 1cm in the vehicle are determined as preset characteristic points. Optionally, the color of the preset feature point has a significant color difference with other interior parts, so that the preset feature point can be quickly identified from the image of the preset feature point. It should be noted that the number of the preset feature points is not limited to two, so that the two preset feature points are selected for triangulation, and if the feature area is large enough, the positioning effect can be achieved.
Optionally, under the condition that a first sight line vector of the target user in a camera coordinate system is determined, a second sight line vector corresponding to the first sight line vector is determined according to a coordinate conversion rule, wherein the second sight line vector is a sight line vector of the target user in a vehicle coordinate system. For example, the first sight line vector is represented as (ρ)0，θ0) The rectangular coordinate corresponding to the first sight line vector can be obtained according to the coordinate conversion relation between the polar coordinate and the rectangular coordinate, and is expressed as (x)0,y0,z0) (ii) a And then according to a coordinate conversion rule, converting the rectangular coordinate of the first sight line vector into the coordinate of the whole vehicle, wherein the coordinate is expressed as (x)0+a,y0+b,z0+c)。
And step 120, determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight line vector.
It should be noted that, when the target user looks at the screen of the in-vehicle entertainment terminal, the sight line vector is a straight line having a direction, and intersects with the screen of the in-vehicle entertainment terminal. According to the intersection information, the attention target of the target user can be determined.
And step 130, determining the control intention of the target user according to the attention target, and controlling the vehicle-mounted entertainment terminal according to the control intention.
It should be noted that the control intention indicates that the target user wishes the in-vehicle entertainment terminal to perform some operation. The control intention of the target user can be determined according to the interface content corresponding to the position information of the focus target in the screen. And generating a control instruction according to the control intention, and controlling the vehicle-mounted entertainment terminal based on the control instruction. For example, control intent may include searching, panning, dragging, placing a call, or switching an application, among others.
According to the technical scheme of the embodiment, the sight line vector of the target user under the whole vehicle coordinate system is determined; determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector; and determining the control intention of the target user according to the attention target, and controlling the vehicle-mounted entertainment terminal according to the control intention, so that the vehicle-mounted entertainment terminal is controlled through the sight vector, the risk of dispersing the attention of a driver due to the control of the vehicle-mounted entertainment terminal is reduced, and the driving safety is improved.
Example two
Fig. 3 is a flowchart of another control method for a vehicle-mounted entertainment terminal according to a second embodiment of the present invention. As shown in fig. 3, the method includes:
and 310, determining a sight line vector of the target user in the whole vehicle coordinate system.
Specifically, a first sight line vector of a target user under a camera coordinate system is obtained; and determining a second sight vector corresponding to the first sight vector according to a coordinate conversion rule, wherein the second sight vector is the sight vector of the target user in the whole vehicle coordinate system. The coordinate conversion rule is determined based on the mapping relation between a first coordinate of a preset feature point in the vehicle in a camera coordinate system and a second coordinate of the whole vehicle coordinate system.
And 320, dividing the screen of the vehicle-mounted entertainment terminal into at least one screen partition according to the sight line vector.
It should be noted that the screen of the in-vehicle entertainment terminal may include a plurality of touch screens. For example, the screen of the in-vehicle entertainment terminal may be a triple screen including a main screen, a sub screen, and an air-conditioning control screen. It is understood that the screen of the in-vehicle entertainment terminal may include more or less touch screens based on different functions. For example, there may be a single screen, a dual screen, a quad screen, a quintuplet screen, and so forth.
The in-vehicle entertainment terminal may be divided into at least one screen partition based on the sight line vector, the recognition accuracy, and a size of a screen of the in-vehicle entertainment terminal. The screen partition comprises a function area and a display area. The sight line vector comprises pupil position information and sight line included angle information. The distance between the screen and the eyes of the user is different, and the included angle of the sight line is also different. In addition, the included angle of sight also changes with the pupil position.
Table 1 is a parameter table of distance between eyes and screen and included angle of sight line
And step 330, determining the position relation between the intersection point of the sight line vector and the screen of the vehicle-mounted entertainment terminal and a preset screen partition.
Fig. 4 is a schematic diagram of a divided screen according to a second embodiment of the present invention. As shown in fig. 4, the screen of the in-vehicle entertainment terminal is triple screen, and the main screen is divided into 6 screen partitions and numbered sequentially, and the sub-screen is taken as one screen partition as a whole. Each screen partition corresponds to a certain screen area.
In the case where the control accuracy is the area control, each screen area has its own function, for example, some screen sections are used for displaying function keys, some screen sections are used for displaying data, and the like. And calculating the position of the intersection point of the sight vector and the screen of the vehicle-mounted entertainment terminal, and matching the position of the intersection point with the area contained in the screen partition, thereby determining the position relation between the intersection point and the preset screen partition.
And step 340, determining the attention target of the target user according to the position relation.
Illustratively, the screen partition in which the position is located is judged, and the attention target of the target user is determined according to the screen partition containing the intersection. As shown in FIG. 4, if the intersection of the sight line vector and the screen of the in-vehicle entertainment terminal is in the 3 rd screen partition, the focus of the target user is determined to be in the 3 rd screen partition.
And 350, acquiring state information of the concerned target in a preset time period, and determining the control intention of the target user according to the state information.
Illustratively, the state information of the attention target within a preset time period is recorded. Wherein the state information of the attention target includes a time when the target user gazes at the attention target. Alternatively, the state information of the attention target may further include handover information or the like in which the attention target is handed over from a to B. Taking fig. 4 as an example, if the focus target is switched from the 7 th screen partition to the 3 rd screen partition, the control intention may be to push the content displayed by the 7 th screen partition to the 3 rd screen partition for display. For another example, the 1 st screen partition, the 3 rd screen partition, the 4 th screen partition and the 6 th screen partition respectively represent an upper left translation, an upper right translation, a lower left translation and a lower right translation, and the 2 nd screen partition and the 5 th screen partition are used for displaying a map. Assuming that the focus target is in the 1 st screen area, the control intent may be to pan the map to the upper left. If the focus target is in the 3 rd screen partition, the control intent may be to pan the map to the upper right. As another example, assume that the 1 st and 4 th screens represent moving up and down a record, respectively, the 2 nd and 5 th screen partitions are used to display the names of contacts, the 3 rd screen partition is used to display the contact phone of the contact corresponding to the 2 nd screen partition, and the 6 th screen partition is used to display the contact phone of the contact corresponding to the 5 th screen partition. If the focus target is in the 2 nd screen partition, the control intent is to dial the contact call displayed on the 3 rd screen partition.
It is to be understood that there are many ways of determining the control intention based on the attention target, and not limited to the above-listed ways.
And 360, generating a control instruction according to the control intention, and controlling the vehicle-mounted entertainment terminal based on the control instruction.
According to the technical scheme of the embodiment, the sight line vector of the target user under the whole vehicle coordinate system is determined; dividing a screen of the vehicle-mounted entertainment terminal into a plurality of screen partitions according to the sight vector, and further determining an attention target of the target user on the screen of the vehicle-mounted entertainment terminal according to the relation between the intersection point of the sight vector and the screen partitions; and determining the control intention of the target user according to the attention target, controlling the vehicle-mounted entertainment terminal according to the control intention, and realizing the function represented by the screen partition executed by controlling the vehicle-mounted entertainment terminal based on the sight vector, thereby providing a regional control mode based on the sight vector, reducing the risk of dispersing the attention of the driver due to the control of the vehicle-mounted entertainment terminal, and improving the driving safety.
Fig. 5 is a flowchart of a control method of another in-vehicle entertainment terminal according to a third embodiment of the present invention. As shown in fig. 5, the method includes:
and step 510, determining a sight line vector of the target user in the whole vehicle coordinate system.
And step 520, obtaining coordinates of the intersection point of the sight line vector and a screen of the vehicle-mounted entertainment terminal, generating an identification area according to the coordinates, and taking the identification area as an attention target of the target user.
Illustratively, coordinates of an intersection point of a sight line vector and a screen of the vehicle-mounted entertainment terminal under a whole vehicle coordinate system are obtained, and an identification area with the coordinates as a circle center and a preset radius is generated. The identified region may be targeted for attention by a targeted user. Optionally, the identification area may be displayed on a screen of the in-vehicle entertainment terminal, or may not be displayed on the screen of the in-vehicle entertainment terminal.
The shape of the recognition area is not limited to a circle, and may be a polygon such as a square, a rectangle, a rhombus, or a trapezoid with a predetermined length as a diagonal line centered on the coordinate.
The coordinates are output to the host of the in-vehicle entertainment control terminal through a bus by a driver state monitoring system comprising a state detection module.
Wherein the control intention comprises whether to trigger a function corresponding to the trigger area. The trigger area may be an area having a setting function in the screen, and in a case that it is detected that the target user looks at the trigger area, the setting function corresponding to the trigger area is triggered.
Fig. 6 is a schematic diagram of a map movement according to the present invention. As shown in fig. 6, a trigger area 620 is provided at the edge of the screen display area. The identification area 610 is generated based on coordinates at an intersection of the sight line vector and the screen of the in-vehicle entertainment terminal. The overlapping area of the identification region 610 and the trigger region 620 is calculated. When the overlapping area exceeds the set threshold, it is determined that the control intention of the target user is to trigger the function corresponding to the trigger region 620. For example, if the overlapping area of the recognition area 610 and the right trigger area 620 exceeds a set threshold, it is determined that the control is intended to control the map to pan to the right. Alternatively, any shape of scrolling icon may be defined as the trigger area of the upper diagram, i.e., any shape of icon may be used to define the trigger area of the precise matrix table.
Fig. 7 is a schematic diagram illustrating selection of a contact with the same name according to a third embodiment of the present invention. As shown in fig. 7, a trigger area 710 is provided in the display area corresponding to each contact. The identification area 720 is generated based on coordinates at the intersection of the sight line vector and the screen of the in-vehicle entertainment terminal. Assuming that the overlapping area of the trigger area 710 corresponding to the identification area 720 and the left contact exceeds a set threshold, it is determined that the control intent of the target user is to place a contact call of the left contact.
And 540, when the control intention is to trigger the function corresponding to the trigger area, generating a trigger instruction, and controlling the vehicle-mounted entertainment terminal based on the trigger instruction.
According to the technical scheme of the embodiment, the sight line vector of the target user under the whole vehicle coordinate system is determined; acquiring coordinates of an intersection point of the sight vector and a screen of the vehicle-mounted entertainment terminal, generating an identification area according to the coordinates, and taking the identification area as an attention target of the target user; the method comprises the steps of obtaining the overlapping area of an identification area corresponding to the attention target and a preset trigger area, determining the control intention of the target user according to the overlapping area, generating a trigger instruction when the control intention is to trigger a function corresponding to the trigger area, controlling the vehicle-mounted entertainment terminal based on the trigger instruction, realizing the control of the vehicle-mounted entertainment terminal based on a sight line vector, providing a screen control mode of the vehicle-mounted entertainment terminal based on a coordinate point, reducing the risk of dispersing the attention of a driver due to the control of the vehicle-mounted entertainment terminal, and improving the driving safety.
Example four
Fig. 8 is a schematic structural diagram of a control device of a vehicle-mounted entertainment terminal according to a fourth embodiment of the present invention, where the control device may be implemented by software and/or hardware, and may be generally integrated in an automobile, and may implement control of the vehicle-mounted entertainment terminal through a line of sight by executing a control method of the vehicle-mounted entertainment terminal. As shown in fig. 8, the apparatus includes:
the sight line determining module 810 is configured to determine a sight line vector of the target user in the entire vehicle coordinate system;
a target determination module 820, configured to determine, according to the sight vector, an attention target of the target user on a screen of the in-vehicle entertainment terminal;
and the terminal control module 830 is configured to determine a control intention of the target user according to the attention target, and control the vehicle-mounted entertainment terminal according to the control intention.
The embodiment of the invention provides a control device of a vehicle-mounted entertainment terminal, which is characterized in that a sight line vector of a target user under a whole vehicle coordinate system is determined; determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector; and determining the control intention of the target user according to the attention target, and controlling the vehicle-mounted entertainment terminal according to the control intention, so that the vehicle-mounted entertainment terminal is controlled through the sight vector, the risk of dispersing the attention of a driver due to the control of the vehicle-mounted entertainment terminal is reduced, and the driving safety is improved.
Optionally, the method further includes:
the coordinate conversion module is used for acquiring the camera coordinate and the in-vehicle position coordinate of the preset feature point in the vehicle before determining the sight line vector of the target user in the whole vehicle coordinate system, and determining a coordinate conversion rule according to the mapping relation between the camera coordinate and the in-vehicle position coordinate.
Optionally, the gaze determining module 810 is specifically configured to:
acquiring a first sight line vector of a target user in a camera coordinate system;
and determining a second sight vector corresponding to the first sight vector according to a coordinate conversion rule, wherein the second sight vector is a sight vector of the target user in a finished automobile coordinate system.
Optionally, the target determining module 820 is specifically configured to:
determining the position relation between an intersection point of the sight line vector and a screen of the vehicle-mounted entertainment terminal and a preset screen partition, wherein the screen partition comprises a function area and a display area;
and determining the attention target of the target user according to the position relation.
Optionally, the terminal control module 830 is specifically configured to:
acquiring state information of the concerned target in a preset time period, and determining a control intention of the target user according to the state information, wherein the control intention comprises translation, dragging, telephone dialing or application program opening and closing;
and generating a control instruction according to the control intention, and controlling the vehicle-mounted entertainment terminal based on the control instruction.
Optionally, the target determining module 820 is specifically configured to:
and acquiring coordinates of the intersection point of the sight vector and a screen of the vehicle-mounted entertainment terminal, generating an identification area according to the coordinates, and taking the identification area as an attention target of the target user.
Optionally, the terminal control module 830 is specifically configured to:
acquiring an overlapping area of an identification area corresponding to the attention target and a preset trigger area, and determining a control intention of the target user according to the overlapping area, wherein the control intention comprises whether a function corresponding to the trigger area is triggered or not;
and when the control intention is to trigger the function corresponding to the trigger area, generating a trigger instruction, and controlling the vehicle-mounted entertainment terminal based on the trigger instruction.
The control device of the vehicle-mounted entertainment terminal provided by the embodiment of the invention can execute the control method of the vehicle-mounted entertainment terminal provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE five
Fig. 9 is a schematic view of a vehicle-mounted entertainment terminal according to a fifth embodiment of the present invention. Referring now to FIG. 9, an in-vehicle entertainment terminal suitable for use in implementing embodiments of the present invention is shown. As shown in FIG. 9, the in-vehicle entertainment terminal 900 includes a display 910, a processor 920, a memory 930, and a computer program stored on the memory 930 and operable on the processor 920, and when the processor 920 executes the computer program, the method for controlling the in-vehicle entertainment terminal according to any embodiment of the present invention is implemented.
It should be noted that the in-vehicle entertainment terminal shown in fig. 9 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present disclosure.
As shown in fig. 9, the in-vehicle entertainment terminal 900 may include a processor 920 that may perform various appropriate actions and processes according to a program stored in a read-only memory (ROM)931 or a program loaded from a storage device 933 into a Random Access Memory (RAM) 932. In the RAM932, various programs and data necessary for the operation of the in-vehicle entertainment terminal 900 are also stored. The memory 930 includes a ROM931, a RAM932, a storage device 933, and the like.
The processor 920, the ROM931, and the RAM932 are connected to each other through a bus 950. An input/output (I/O) interface 940 is also connected to bus 950. The display screen 910 is connected to an I/O interface 960. Optionally, the display screen 910 is a touch display screen.
EXAMPLE six
The sixth embodiment of the invention further provides an automobile, which comprises the vehicle-mounted entertainment terminal in the fifth embodiment.
EXAMPLE seven
An embodiment of the present invention further provides a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a method of controlling an in-vehicle entertainment terminal, the method including:
determining a sight line vector of a target user under a finished automobile coordinate system;
determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector;
and determining the control intention of the target user according to the attention target, and controlling the vehicle-mounted entertainment terminal according to the control intention.
Of course, the storage medium containing the computer-executable instructions provided by the embodiments of the present invention is not limited to the method operations described above, and may also perform related operations in the control method of the in-vehicle entertainment terminal provided by any embodiments of the present invention.
From the above description of the embodiments, it is obvious for those skilled in the art that the present invention can be implemented by software and necessary general hardware, and certainly, can also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present invention may be embodied in the form of a software product, which can be stored in a computer-readable storage medium, such as a floppy disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a FLASH Memory (FLASH), a hard disk or an optical disk of a computer, and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device) to execute the methods according to the embodiments of the present invention.
It should be noted that, in the embodiment of the control device of the vehicle-mounted entertainment terminal, the units and modules included in the control device are only divided according to the function logic, but are not limited to the above division as long as the corresponding functions can be realized; in addition, specific names of the functional units are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present invention.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A control method of an in-vehicle entertainment terminal, comprising:
determining a sight line vector of a target user under a finished automobile coordinate system;
determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector;
and determining the control intention of the target user according to the attention target, and controlling the vehicle-mounted entertainment terminal according to the control intention.
2. The method of claim 1, wherein determining a gaze vector of the target user in the vehicle coordinate system comprises:
acquiring a first sight line vector of a target user in a camera coordinate system;
and determining a second sight vector corresponding to the first sight vector according to a coordinate conversion rule, wherein the second sight vector is a sight vector of the target user in a coordinate system of the whole vehicle, and the coordinate conversion rule is determined based on a mapping relation between a first coordinate of a preset feature point in the vehicle in the coordinate system of the camera and a second coordinate in the coordinate system of the whole vehicle.
3. The method of claim 1, wherein determining the target of interest of the target user on a screen of an in-vehicle entertainment terminal according to the gaze vector comprises:
determining the position relation between the intersection point of the sight line vector and the screen of the vehicle-mounted entertainment terminal and a preset screen partition;
and determining the attention target of the target user according to the position relation.
4. The method according to claim 3, wherein determining the control intention of the target user according to the attention target, and controlling the in-vehicle entertainment terminal according to the control intention comprises:
acquiring state information of the concerned target in a preset time period, and determining a control intention of the target user according to the state information, wherein the control intention comprises searching, translating, dragging, making a call or switching on and off an application program;
and generating a control instruction according to the control intention, and controlling the vehicle-mounted entertainment terminal based on the control instruction.
5. The method of claim 1, wherein determining the target of interest of the target user on a screen of an in-vehicle entertainment terminal according to the gaze vector comprises:
and acquiring coordinates of the intersection point of the sight vector and a screen of the vehicle-mounted entertainment terminal, generating an identification area according to the coordinates, and taking the identification area as an attention target of the target user.
6. The method of claim 5, wherein determining the control intention of the target user according to the attention target, and controlling the in-vehicle entertainment terminal according to the control intention comprises:
acquiring an overlapping area of an identification area corresponding to the attention target and a preset trigger area, and determining a control intention of the target user according to the overlapping area, wherein the control intention comprises whether a function corresponding to the trigger area is triggered or not;
and when the control intention is to trigger the function corresponding to the trigger area, generating a trigger instruction, and controlling the vehicle-mounted entertainment terminal based on the trigger instruction.
7. A control apparatus of an in-vehicle entertainment terminal, characterized by comprising:
the sight line determining module is used for determining a sight line vector of a target user under a finished automobile coordinate system;
the target determining module is used for determining an attention target of the target user on a screen of the vehicle-mounted entertainment terminal according to the sight vector;
and the terminal control module is used for determining the control intention of the target user according to the attention target and controlling the vehicle-mounted entertainment terminal according to the control intention.
8. A computer-readable storage medium on which a computer program is stored, characterized in that the program, when executed by a processor, implements the control method of the in-vehicle entertainment terminal according to any of claims 1-6.
9. An in-vehicle entertainment terminal comprising a display screen, a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor implementing the control method of the in-vehicle entertainment terminal according to any one of claims 1 to 6 when executing the computer program.
10. An automobile, characterized by comprising the in-vehicle entertainment terminal of claim 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011240794.0A CN112383826A (en) | 2020-11-09 | 2020-11-09 | Control method and device of vehicle-mounted entertainment terminal, storage medium, terminal and automobile |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011240794.0A CN112383826A (en) | 2020-11-09 | 2020-11-09 | Control method and device of vehicle-mounted entertainment terminal, storage medium, terminal and automobile |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112383826A true CN112383826A (en) | 2021-02-19 |
Family
ID=74579104
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011240794.0A Pending CN112383826A (en) | 2020-11-09 | 2020-11-09 | Control method and device of vehicle-mounted entertainment terminal, storage medium, terminal and automobile |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112383826A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114610432A (en) * | 2022-03-17 | 2022-06-10 | 芜湖汽车前瞻技术研究院有限公司 | Graphic display control method, device, equipment and storage medium for vehicle-mounted display screen |
| CN114647319A (en) * | 2022-03-28 | 2022-06-21 | 重庆长安汽车股份有限公司 | Method, system and storage medium for transferring screen display information in user vehicle |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010259605A (en) * | 2009-05-01 | 2010-11-18 | Nippon Hoso Kyokai <Nhk> | Visual line measuring device and visual line measuring program |
| CN104461005A (en) * | 2014-12-15 | 2015-03-25 | 东风汽车公司 | Vehicle-mounted screen switch control method |
| US20150089440A1 (en) * | 2013-09-24 | 2015-03-26 | Lg Electronics Inc. | Mobile terminal and control method thereof |
| US20170192500A1 (en) * | 2015-12-31 | 2017-07-06 | Le Holdings (Beijing) Co., Ltd. | Method and electronic device for controlling terminal according to eye action |
| US20200150427A1 (en) * | 2018-11-14 | 2020-05-14 | Beijing 7Invensun Technology Co., Ltd. | Method, apparatus and system for determining line of sight, and wearable eye movement device |
-
2020
- 2020-11-09 CN CN202011240794.0A patent/CN112383826A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010259605A (en) * | 2009-05-01 | 2010-11-18 | Nippon Hoso Kyokai <Nhk> | Visual line measuring device and visual line measuring program |
| US20150089440A1 (en) * | 2013-09-24 | 2015-03-26 | Lg Electronics Inc. | Mobile terminal and control method thereof |
| CN104461005A (en) * | 2014-12-15 | 2015-03-25 | 东风汽车公司 | Vehicle-mounted screen switch control method |
| US20170192500A1 (en) * | 2015-12-31 | 2017-07-06 | Le Holdings (Beijing) Co., Ltd. | Method and electronic device for controlling terminal according to eye action |
| US20200150427A1 (en) * | 2018-11-14 | 2020-05-14 | Beijing 7Invensun Technology Co., Ltd. | Method, apparatus and system for determining line of sight, and wearable eye movement device |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114610432A (en) * | 2022-03-17 | 2022-06-10 | 芜湖汽车前瞻技术研究院有限公司 | Graphic display control method, device, equipment and storage medium for vehicle-mounted display screen |
| CN114647319A (en) * | 2022-03-28 | 2022-06-21 | 重庆长安汽车股份有限公司 | Method, system and storage medium for transferring screen display information in user vehicle |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111931579B (en) | Automatic driving assistance system and method using eye tracking and gesture recognition techniques | |
| JP6030430B2 (en) | Control device, vehicle and portable terminal | |
| CN106502570B (en) | Gesture recognition method and device and vehicle-mounted system | |
| US10635184B2 (en) | Information processing device, information processing method, and program | |
| US9703380B2 (en) | Vehicle operation input device | |
| US9235269B2 (en) | System and method for manipulating user interface in vehicle using finger valleys | |
| US9501693B2 (en) | Real-time multiclass driver action recognition using random forests | |
| CN109242765B (en) | Face image processing method and device and storage medium | |
| JP6671288B2 (en) | Gesture device, operation method thereof, and vehicle equipped with the same | |
| CN112383826A (en) | Control method and device of vehicle-mounted entertainment terminal, storage medium, terminal and automobile | |
| CN113253908B (en) | Key function execution method, device, equipment and storage medium | |
| CN108846271B (en) | Device control method, device, storage medium and electronic device | |
| US20150205483A1 (en) | Object operation system, recording medium recorded with object operation control program, and object operation control method | |
| CN113548061B (en) | Man-machine interaction method and device, electronic equipment and storage medium | |
| CN107544740B (en) | Application processing method and device, storage medium and electronic equipment | |
| US10752261B2 (en) | Driver distraction warning control apparatus and method | |
| JP5558899B2 (en) | Information processing apparatus, processing method thereof, and program | |
| CN114510142B (en) | Gesture recognition method based on two-dimensional image, gesture recognition system based on two-dimensional image and electronic equipment | |
| CN110944084B (en) | Single-hand mode control method, terminal and computer storage medium | |
| CN114217716A (en) | Menu bar display method and device and electronic equipment | |
| JP7375611B2 (en) | operating device | |
| JP2021135843A (en) | Operating device | |
| US10466799B2 (en) | Information processing apparatus, information processing method, and program | |
| US20230331162A1 (en) | Display controller | |
| US20230123623A1 (en) | Gesture detecting apparatus and gesture detecting method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20210219 |