CN116394756A - Screen content auxiliary display method and device, vehicle and storage medium - Google Patents
Screen content auxiliary display method and device, vehicle and storage medium Download PDFInfo
- Publication number
- CN116394756A CN116394756A CN202310253316.0A CN202310253316A CN116394756A CN 116394756 A CN116394756 A CN 116394756A CN 202310253316 A CN202310253316 A CN 202310253316A CN 116394756 A CN116394756 A CN 116394756A
- Authority
- CN
- China
- Prior art keywords
- content
- screen
- secondary driving
- auxiliary display
- auxiliary
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 52
- 230000009471 action Effects 0.000 claims description 42
- 238000001514 detection method Methods 0.000 claims description 26
- 210000003128 head Anatomy 0.000 claims description 18
- 210000001747 pupil Anatomy 0.000 claims description 12
- 238000004590 computer program Methods 0.000 claims description 6
- 206010063385 Intellectualisation Diseases 0.000 abstract description 4
- 230000006870 function Effects 0.000 description 4
- 230000008569 process Effects 0.000 description 4
- 238000013135 deep learning Methods 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 238000012546 transfer Methods 0.000 description 3
- 238000004891 communication Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000012015 optical character recognition Methods 0.000 description 2
- 241001125843 Trichiuridae Species 0.000 description 1
- 230000004913 activation Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 230000003287 optical effect Effects 0.000 description 1
- 238000004513 sizing Methods 0.000 description 1
- 230000007723 transport mechanism Effects 0.000 description 1
Images
Classifications
-
- B60K35/60—
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
- B60K35/00—Arrangement of adaptations of instruments
-
- B60K35/22—
-
- B60K35/28—
-
- B60K35/29—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0481—Interaction techniques based on graphical user interfaces [GUI] based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0484—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range
- G06F3/04845—Interaction techniques based on graphical user interfaces [GUI] for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range for image manipulation, e.g. dragging, rotation, expansion or change of colour
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/14—Digital output to display device ; Cooperation and interconnection of the display device with other functional units
- G06F3/1423—Digital output to display device ; Cooperation and interconnection of the display device with other functional units controlling a plurality of local displays, e.g. CRT and flat panel display
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/16—Sound input; Sound output
- G06F3/167—Audio in a user interface, e.g. using voice commands for navigating, audio feedback
-
- B60K2360/164—
-
- B60K2360/166—
-
- B60K2360/182—
-
- B60K2360/184—
-
- B60K2360/21—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- General Physics & Mathematics (AREA)
- Mechanical Engineering (AREA)
- Chemical & Material Sciences (AREA)
- Transportation (AREA)
- Combustion & Propulsion (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- Audiology, Speech & Language Pathology (AREA)
- General Health & Medical Sciences (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The embodiment of the application discloses a screen content auxiliary display method, a device, a vehicle and a storage medium, wherein the method comprises the following steps: detecting whether the content on the secondary driving screen meets a preset auxiliary display condition; the auxiliary display conditions include: the content of the secondary driving screen needs to be displayed in an auxiliary way, and/or the content on the secondary driving screen is continuously concerned by the main driving; when the content on the secondary driving screen meets the auxiliary display condition, the content on the secondary driving screen is displayed in an auxiliary mode; the auxiliary display mode comprises any one or more of the following modes: content highlighting, shifting content display locations, and converting content display to content speakable. By the scheme, auxiliary display of the content of the secondary driving screen is automatically realized, and the intellectualization and humanization of in-vehicle control are improved.
Description
Technical Field
The embodiment of the application relates to a vehicle control technology, in particular to a screen content auxiliary display method and device, a vehicle and a storage medium.
Background
Along with the intellectualization of cabins/car machines, the number/size of screens in the cabins is always increased/increased, such as double-screen, triple-screen and hairtail screen, so that a touch screen which is independently operable is also arranged at the secondary driving position.
When the secondary driver wants to share the information on the current secondary driving screen with the primary driver (for example, the secondary driver searches a plurality of restaurants to be selected in the map application of the secondary driving screen, and wants to confirm from the primary driver, or the secondary driver selects a route, and wants to confirm whether the route is suitable or not by the primary driver), the information on the secondary driving screen cannot be well seen due to the fact that the primary driver is far away from the secondary driving screen and the angle is inclined.
The current solution is that the secondary driver actively moves the window of the application APP to be displayed to the central control screen through specific operation (such as three-finger sliding) so that the primary driver can see clearly.
In some multi-screen display application schemes, a user needs to transfer a window from a secondary driving screen to a central control screen through a line-of-sight operation window, for example, the primary driving needs to look at the secondary driving screen first and then look at the central control screen.
Disclosure of Invention
The embodiment of the application provides a screen content auxiliary display method, a device, a vehicle and a storage medium, which can automatically perform auxiliary display on auxiliary driving screen content, and improve the intellectualization and humanization of in-vehicle control.
The embodiment of the application provides a screen content auxiliary display method, which can comprise the following steps:
detecting whether the content on the secondary driving screen meets a preset auxiliary display condition; the auxiliary display condition includes: the content of the secondary driving screen needs auxiliary display, and/or the content on the secondary driving screen is continuously focused by a main driver;
when the content on the secondary driving screen meets the auxiliary display condition, the content on the secondary driving screen is displayed in an auxiliary mode; the auxiliary display mode comprises any one or more of the following modes: content highlighting, shifting content display locations, and converting content display to content speakable.
In an exemplary embodiment of the present application, the detecting whether the content on the secondary driving screen meets the auxiliary display condition may include:
and when the content on the secondary driving screen meets any one or more of the following, determining that the content on the secondary driving screen meets the content of the secondary driving screen contained in the secondary display condition, wherein the content of the secondary driving screen needs to be displayed in a secondary mode:
the type of the application APP activated on the secondary driving screen belongs to a preset type needing auxiliary display;
the text size on the secondary driving screen is smaller than a preset threshold value;
the content on the secondary driving screen enables the secondary driving to have sharing intention; the method comprises the steps of,
and the content information on the secondary driving screen belongs to preset information needing auxiliary display.
In an exemplary embodiment of the present application, a method for detecting whether content on the secondary driving screen enables the secondary driving to have a sharing intention may include:
the voice content of the secondary driving is recognized through a vehicle-mounted voice recognition device;
and when the voice content contains relevant information about the content on the secondary driving screen to be shared, determining the content on the secondary driving screen to enable the secondary driving to have the sharing intention.
In an exemplary embodiment of the present application, a method for detecting whether content on the secondary driving screen enables the secondary driving to have a sharing intention may include:
acquiring picture information and/or video information of the secondary driving through a camera preset in the vehicle;
detecting whether the secondary driving has the action of sharing the content on the secondary driving screen according to the picture information and/or the video information;
and when the auxiliary driving is detected to have the action of sharing the content on the auxiliary driving screen, determining the content on the auxiliary driving screen so that the auxiliary driving has the sharing intention.
In an exemplary embodiment of the present application, the detecting whether the secondary driving has the action of sharing the content on the secondary driving screen according to the picture information and/or the video information may include:
inputting the picture information and/or the video information into a preset motion detection model;
detecting whether the motion in the picture information and/or the video information accords with a pre-learned sharing motion or not by the motion detection model;
when the action detection model detects that the actions in the picture information and/or the video information accord with the sharing action, determining that the secondary driving has the action of sharing the content on the secondary driving screen;
and when the action detection model detects that the actions in the picture information and/or the video information do not accord with the sharing action, determining that the secondary driving does not have the action of sharing the content on the secondary driving screen.
In an exemplary embodiment of the present application, the detecting whether the content on the secondary driving screen meets the auxiliary display condition may include:
detecting the sight direction of the main driver, and when the sight direction of the main driver points to the auxiliary driver screen, and the duration of the sight direction of the main driver pointing to the auxiliary driver screen is greater than or equal to a preset duration threshold value, and/or the frequency of the sight direction of the main driver pointing to the auxiliary driver screen within the preset duration is greater than or equal to a preset frequency threshold value, determining that the content on the auxiliary driver screen meets the content on the auxiliary driver screen contained in the auxiliary display condition is continuously focused by the main driver.
In an exemplary embodiment of the present application, the detecting the line of sight direction of the primary driver may include:
acquiring the image of the main driver through a camera preset in the vehicle in a preset acquisition period;
detecting a head image from the image of the main driver according to a preset head detection model;
detecting a rotation direction and a rotation angle of the head from the head image;
detecting an pupil image from the head image according to a preset face key point detection model;
detecting the deflection angle of the pupil according to the pupil image;
and calculating the sight line direction of the main driver according to the rotation direction and the rotation angle of the head and the offset angle of the pupil.
In an exemplary embodiment of the present application, when the manner of auxiliary display includes content highlighting, the auxiliary display of the content on the secondary driving screen may include:
amplifying and displaying an APP window and/or part or all of display contents in the APP window according to a preset ratio;
increasing the contrast of the color of the display content in the APP window;
adjusting the layout of the display content in the APP window;
when the auxiliary display mode includes transferring the content display position, the auxiliary display of the content on the secondary driving screen includes: and transferring the content on the secondary driving screen to a central control screen for display.
In an exemplary embodiment of the present application, the method may further include: and selecting a corresponding auxiliary display mode according to the APP type, the current functional interface and/or the current state of the central control screen.
The embodiment of the application also provides a screen content auxiliary display device, which can comprise a processor and a memory, wherein the memory stores instructions, and when the instructions are executed by the processor, the screen content auxiliary display method is realized.
The embodiment of the application also provides a vehicle which can comprise a camera and the screen content auxiliary display device.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, the computer program realizing the screen content auxiliary display method when being executed by a processor.
The screen content auxiliary display method of the embodiment of the application can comprise the following steps: detecting whether the content on the secondary driving screen meets a preset auxiliary display condition; the auxiliary display condition includes: the content of the secondary driving screen needs auxiliary display, and/or the content on the secondary driving screen is continuously focused by a main driver; when the content on the secondary driving screen meets the auxiliary display condition, the content on the secondary driving screen is displayed in an auxiliary mode; the auxiliary display mode comprises any one or more of the following modes: content highlighting, shifting content display locations, and converting content display to content speakable. By the scheme, auxiliary display of the content of the secondary driving screen is automatically realized, and the intellectualization and humanization of in-vehicle control are improved.
Additional features and advantages of the application will be set forth in the description which follows, and in part will be obvious from the description, or may be learned by practice of the application. Other advantages of the present application may be realized and attained by the structure particularly pointed out in the written description and drawings.
Drawings
The accompanying drawings are included to provide an understanding of the technical aspects of the present application, and are incorporated in and constitute a part of this specification, illustrate the technical aspects of the present application and together with the examples of the present application, and not constitute a limitation of the technical aspects of the present application.
FIG. 1 is a flowchart of a method for assisting in displaying screen content according to an embodiment of the present application;
FIG. 2 is a block diagram of an auxiliary display device for screen content according to an embodiment of the present application;
fig. 3 is a block diagram of a vehicle component according to an embodiment of the present application.
Detailed Description
The present application describes a number of embodiments, but the description is illustrative and not limiting and it will be apparent to those of ordinary skill in the art that many more embodiments and implementations are possible within the scope of the embodiments described herein. Although many possible combinations of features are shown in the drawings and discussed in the detailed description, many other combinations of the disclosed features are possible. Any feature or element of any embodiment may be used in combination with or in place of any other feature or element of any other embodiment unless specifically limited.
The present application includes and contemplates combinations of features and elements known to those of ordinary skill in the art. The embodiments, features and elements of the present disclosure may also be combined with any conventional features or elements to form a unique inventive arrangement as defined in the claims. Any feature or element of any embodiment may also be combined with features or elements from other inventive arrangements to form another unique inventive arrangement as defined in the claims. Thus, it should be understood that any of the features shown and/or discussed in this application may be implemented alone or in any suitable combination. Accordingly, the embodiments are not to be restricted except in light of the attached claims and their equivalents. Further, various modifications and changes may be made within the scope of the appended claims.
Furthermore, in describing representative embodiments, the specification may have presented the method and/or process as a particular sequence of steps. However, to the extent that the method or process does not rely on the particular order of steps set forth herein, the method or process should not be limited to the particular sequence of steps described. Other sequences of steps are possible as will be appreciated by those of ordinary skill in the art. Accordingly, the particular order of the steps set forth in the specification should not be construed as limitations on the claims. Furthermore, the claims directed to the method and/or process should not be limited to the performance of their steps in the order written, and one skilled in the art can readily appreciate that the sequences may be varied and still remain within the spirit and scope of the embodiments of the present application.
The embodiment of the application provides a screen content auxiliary display method, as shown in fig. 1, the method may include steps S101-S102:
s101, detecting whether the content on a secondary driving screen meets a preset auxiliary display condition; the auxiliary display condition includes: the content of the secondary driving screen needs auxiliary display, and/or the content on the secondary driving screen is continuously focused by a main driver;
s102, when the content on the secondary driving screen meets the auxiliary display condition, displaying the content on the secondary driving screen in an auxiliary mode; the auxiliary display mode comprises any one or more of the following modes: content highlighting, shifting content display locations, and converting content display to content speakable.
In the current scheme, when the secondary driver wants to share the information on the current secondary driving screen with the primary driver, the primary driver or the secondary driving is required to actively operate to transfer the content of the secondary driving screen to the central control screen, special conditions of a cabin environment (for example, three-finger sliding is not easy to operate under the condition of bumping a vehicle) are not considered in operation, and the content transfer which is realized by the actual operation of personnel is required, so that the control in the vehicle is not intelligent enough.
In the exemplary embodiment of the application, the content information on the secondary driving screen and the intention of the primary driving to view the content information are automatically detected, and the information to be shared on the secondary driving screen is displayed to the primary driving by automatically selecting a proper secondary display mode based on the result, so that the primary driving or the secondary driving is not required to perform specific operation.
In an exemplary embodiment of the present application, the detecting whether the content on the secondary driving screen meets the auxiliary display condition may include:
and when the content on the secondary driving screen meets any one or more of the following, determining that the content on the secondary driving screen meets the content of the secondary driving screen contained in the secondary display condition, wherein the content of the secondary driving screen needs to be displayed in a secondary mode:
the type of the application APP activated on the secondary driving screen belongs to a preset type needing auxiliary display;
the text size on the secondary driving screen is smaller than a preset threshold value;
the content on the secondary driving screen enables the secondary driving to have sharing intention; the method comprises the steps of,
and the content information on the secondary driving screen belongs to preset information needing auxiliary display.
In the exemplary embodiment of the present application, first, it may be determined whether the content on the secondary driving screen meets the secondary display condition according to the APP type of the active window on the current secondary driving screen: before implementing the scheme of the embodiment of the application, some APP or some functions of APP may be preselected as objects to be displayed in an auxiliary manner, for example, navigation APP, map APP (APP may have a navigation or destination function interface), or APP with content sharing requirements, such as picture and/or video play APP.
In the exemplary embodiment of the present application, it may also be determined, according to the text size in the current secondary driving screen activation window, whether the content on the secondary driving screen meets the secondary display condition: in order to facilitate sharing the content on the secondary driving screen with the primary driving, when the text size (the pixel area occupied by a single text) on the secondary driving screen is smaller than a preset threshold, the current content state is considered to be unfavorable for sharing with the primary driving, so that the content needs to be displayed in an auxiliary mode, for example, the text can be amplified; here, the character sizing may be achieved by OCR (Optical Character Recognition ) methods or APP output.
In the exemplary embodiment of the present application, it may also be determined whether the content on the secondary driving screen satisfies the secondary display condition by detecting whether the secondary driving has an intention to share the content on the secondary driving screen: for example, it may be determined whether the secondary drive has a sharing intention for the content on the secondary drive screen through voice detection and motion detection.
In an exemplary embodiment of the present application, a method for detecting whether content on the secondary driving screen enables the secondary driving to have a sharing intention may include:
the voice content of the secondary driving is recognized through a vehicle-mounted voice recognition device;
and when the voice content contains relevant information about the content on the secondary driving screen to be shared, determining the content on the secondary driving screen to enable the secondary driving to have the sharing intention.
In the exemplary embodiment of the application, whether the assistant driver has the sharing intention may be determined through the in-vehicle voice, for example, if the assistant driver is detected to say sentences such as "you see which of these choices", "fast-looking na", the assistant driver is considered to have the sharing intention, and the intention detection model based on deep learning natural language processing may be used to detect the sharing intention in the voice content of the assistant driver.
In an exemplary embodiment of the present application, the method for detecting whether the content on the secondary driving screen enables the secondary driving to have the sharing intention may further include:
acquiring picture information and/or video information of the secondary driving through a camera preset in the vehicle;
detecting whether the secondary driving has the action of sharing the content on the secondary driving screen according to the picture information and/or the video information;
and when the auxiliary driving is detected to have the action of sharing the content on the auxiliary driving screen, determining the content on the auxiliary driving screen so that the auxiliary driving has the sharing intention.
In the exemplary embodiment of the application, whether the current action of the secondary driving is to operate the secondary driving screen to share the content of the secondary driving screen can be judged by acquiring the image/video of the secondary driving through the in-vehicle monitoring camera, and the action detection model based on deep learning can be used for realizing the action detection model.
In an exemplary embodiment of the present application, the detecting whether the secondary driving has the action of sharing the content on the secondary driving screen according to the picture information and/or the video information may include:
inputting the picture information and/or the video information into a preset motion detection model;
detecting whether the motion in the picture information and/or the video information accords with a pre-learned sharing motion or not by the motion detection model;
when the action detection model detects that the actions in the picture information and/or the video information accord with the sharing action, determining that the secondary driving has the action of sharing the content on the secondary driving screen;
and when the action detection model detects that the actions in the picture information and/or the video information do not accord with the sharing action, determining that the secondary driving does not have the action of sharing the content on the secondary driving screen.
In the exemplary embodiment of the present application, the sharing action refers to an action of sharing the content on the secondary driving screen, for example, an action of turning up the screen display content, an action of searching how to share the screen content, an action of making a designation (the designation action indicates that the secondary driving wants to share the screen content), and the like.
In the exemplary embodiment of the present application, it may also be determined whether the content on the secondary driving screen satisfies the secondary display condition by confirming whether the information reflected by the content on the secondary driving screen is predetermined information that needs to be presented to the primary driving: for example, some pop-up messages (such as road condition emergency broadcasting, important notification, etc.) and the like need to be displayed to the main driver.
In the exemplary embodiment of the present application, when the content on the secondary driving screen accords with any one or more of the above conditions, the content of the secondary driving screen may be considered to need auxiliary presentation to the primary driving.
In the exemplary embodiment of the present application, whether the content of the secondary driving screen needs to be presented to the primary driving in an auxiliary manner may also be considered from another angle (the angle of the primary driving), for example, if the primary driving is interested in the content of the secondary driving screen and continuously pays attention, the content of the secondary driving screen may also be considered to need to be presented to the primary driving in an auxiliary manner.
In an exemplary embodiment of the present application, the detecting whether the content on the secondary driving screen meets the auxiliary display condition may further include:
detecting the sight direction of the main driver, and when the sight direction of the main driver points to the auxiliary driver screen, and the duration of the sight direction of the main driver pointing to the auxiliary driver screen is greater than or equal to a preset duration threshold value, and/or the frequency of the sight direction of the main driver pointing to the auxiliary driver screen within the preset duration is greater than or equal to a preset frequency threshold value, determining that the content on the auxiliary driver screen meets the content contained in the auxiliary display condition, wherein the content on the auxiliary driver screen is continuously focused.
In the exemplary embodiment of the application, the object of interest of the main driver can be determined by detecting the sight direction of the main driver, so as to judge whether the main driver is interested in the content of the secondary driving screen.
In an exemplary embodiment of the present application, the detecting the line of sight direction of the primary driver may include:
acquiring the image of the main driver through a camera preset in the vehicle in a preset acquisition period;
detecting a head image from the image of the main driver according to a preset head detection model;
detecting a rotation direction and a rotation angle of the head from the head image;
detecting an pupil image from the head image according to a preset face key point detection model;
detecting the deflection angle of the pupil according to the pupil image;
and calculating the sight line direction of the main driver according to the rotation direction and the rotation angle of the head and the offset angle of the pupil.
In the exemplary embodiment of the application, the voice of the main driver can be detected and identified through the preset voice detection module, the voice content of the main driver is detected to judge whether the main driver pays attention to the content of the secondary driving screen, for example, if the voice content of the main driver including 'let me see at one' and 'what you display there' and the like is detected, the content of the main driver pays attention to the content of the secondary driving screen can be judged, and the content of the secondary driving screen can be displayed in an auxiliary mode.
In an exemplary embodiment of the present application, the manner of auxiliary display may include, but is not limited to, any one or more of the following: content highlighting, shifting content display locations, and converting content display to content speakable.
In an exemplary embodiment of the present application, when the manner of auxiliary display includes content highlighting, the auxiliary display of the content on the secondary driving screen may include:
amplifying and displaying an APP window and/or part or all of display contents in the APP window according to a preset ratio;
increasing the contrast of the color of the display content in the APP window;
and adjusting the layout of the display content in the APP window.
In the exemplary embodiment of the application, the corresponding characters or pictures can be enlarged by enlarging the APP window or enlarging the content, so that the visibility and the recognition rate are increased.
In the exemplary embodiment of the application, the contrast of the added color can be a changed text background, a changed text color or a combination of the two, so that the screen display content is more striking and easy to identify by improving the contrast of the background color and the text color. For example, red text or yellow text on a white background may be modified to black text so that contrast is improved by black-and-white contrast.
In an exemplary embodiment of the present application, adjusting the layout of the display content in the APP window may include, but is not limited to, displaying text content or pictures (e.g., destination, accident site, signage building, etc.) that need to be focused on to a central area of the screen, or to an area biased toward the main driver, so that the main driver can conveniently and clearly see the relevant content.
In an exemplary embodiment of the present application, when the manner of auxiliary display includes transferring a content display position, the auxiliary display of the content on the secondary driving screen includes: and transferring the content on the secondary driving screen to a central control screen for display.
In the exemplary embodiment of the application, the content is transferred to the central control screen, so that the main driver and the auxiliary driver can conveniently watch the content on the screen, and the screen is more humanized.
In the exemplary embodiment of the present application, in order to further facilitate the main driver to understand the screen content, the APP window content may be further read aloud: for example, text or picture descriptions displayed by the current APP window of the secondary driving screen can be read according to a preset sequence.
In the exemplary embodiment of the present application, the pictures included in the current APP window may be pictures that cooperate with text presentation, or may be pause pictures of the current video play, and descriptions of these pictures may be through a deep learning-based picture description model or a corresponding picture text description, and voice reading is performed on the picture text description.
In an exemplary embodiment of the present application, the method may further include: and selecting a corresponding auxiliary display mode according to the APP type, the current functional interface and/or the current state of the central control screen.
In an exemplary embodiment of the present application, for example, if APP is a map navigation APP and is currently on the navigation destination search result display interface, the APP window may be transferred to the central control screen, and the display font of the search result may be enlarged.
In the exemplary embodiment of the present application, for example, if APP is a gallery APP, and when displaying a picture, it may be detected whether the central control screen is idle at first, and when the central control screen is in a state of being occupied and unable to display other content (such as in continuous navigation), the content description of the picture may be displayed through voice reading; when the central control screen is unoccupied and other contents can be displayed, the APP window can be moved to the central control screen for display.
In an exemplary embodiment of the present application, for example, if the APP window is a pop-up message, the screen display portion of the pop-up message is enlarged and the text and background are changed to a high contrast color while the message content is read aloud.
In exemplary embodiments of the present application, at least the following advantages are included:
1. the content of the secondary driving screen is actively displayed in an auxiliary mode through automatic detection of a specific scene without operation of a user, so that the cabin is more intelligent and humanized;
2. the requirements on detection precision are lower (compared with sight tracking), and landing is easier to realize at low cost.
The embodiment of the application further provides a screen content auxiliary display device 1, as shown in fig. 2, may include a processor 11 and a memory 12, where the memory 12 stores instructions, and when the instructions are executed by the processor 11, the screen content auxiliary display method is implemented.
In the exemplary embodiments of the present application, the foregoing embodiments of the method for assisting in displaying screen content are all applicable to the embodiments of the apparatus, and are not described herein in detail.
The embodiment of the application also provides a vehicle 2, as shown in fig. 3, which may include a camera 3 and the screen content auxiliary display device 1.
In the exemplary embodiments of the present application, the foregoing embodiments of the auxiliary display method for screen content are applicable to the vehicle embodiment, and are not described in detail herein.
The embodiment of the application also provides a computer readable storage medium, on which a computer program is stored, which is characterized in that the computer program is executed by a processor to realize the screen content auxiliary display method.
Those of ordinary skill in the art will appreciate that all or some of the steps, systems, functional modules/units in the apparatus, and methods disclosed above may be implemented as software, firmware, hardware, and suitable combinations thereof. In a hardware implementation, the division between the functional modules/units mentioned in the above description does not necessarily correspond to the division of physical components; for example, one physical component may have multiple functions, or one function or step may be performed cooperatively by several physical components. Some or all of the components may be implemented as software executed by a processor, such as a digital signal processor or microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on computer readable media, which may include computer storage media (or non-transitory media) and communication media (or transitory media). The term computer storage media includes both volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data, as known to those skilled in the art. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital Versatile Disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium which can be used to store the desired information and which can be accessed by a computer. Furthermore, as is well known to those of ordinary skill in the art, communication media typically embodies computer readable instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.
Claims (12)
1. A method for auxiliary display of screen content, the method comprising:
detecting whether the content on the secondary driving screen meets a preset auxiliary display condition; the auxiliary display condition includes: the content of the secondary driving screen needs auxiliary display, and/or the content on the secondary driving screen is continuously focused by a main driver;
when the content on the secondary driving screen meets the auxiliary display condition, the content on the secondary driving screen is displayed in an auxiliary mode; the auxiliary display mode comprises any one or more of the following modes: content highlighting, shifting content display locations, and converting content display to content speakable.
2. The screen content auxiliary display method according to claim 1, wherein the detecting whether the content on the secondary driving screen satisfies the auxiliary display condition comprises:
and when the content on the secondary driving screen meets any one or more of the following, determining that the content on the secondary driving screen meets the content of the secondary driving screen contained in the secondary display condition, wherein the content of the secondary driving screen needs to be displayed in a secondary mode:
the type of the application APP activated on the secondary driving screen belongs to a preset type needing auxiliary display;
the text size on the secondary driving screen is smaller than a preset threshold value;
the content on the secondary driving screen enables the secondary driving to have sharing intention; the method comprises the steps of,
and the content information on the secondary driving screen belongs to preset information needing auxiliary display.
3. The screen content auxiliary display method according to claim 2, wherein the method for detecting whether the content on the secondary driving screen enables the secondary driving to have a sharing intention comprises:
the voice content of the secondary driving is recognized through a vehicle-mounted voice recognition device;
and when the voice content contains relevant information about the content on the secondary driving screen to be shared, determining the content on the secondary driving screen to enable the secondary driving to have the sharing intention.
4. The screen content auxiliary display method according to claim 2, wherein the method for detecting whether the content on the secondary driving screen enables the secondary driving to have a sharing intention comprises:
acquiring picture information and/or video information of the secondary driving through a camera preset in the vehicle;
detecting whether the secondary driving has the action of sharing the content on the secondary driving screen according to the picture information and/or the video information;
and when the auxiliary driving is detected to have the action of sharing the content on the auxiliary driving screen, determining the content on the auxiliary driving screen so that the auxiliary driving has the sharing intention.
5. The method according to claim 4, wherein the detecting whether the secondary driving has an action of sharing the content on the secondary driving screen according to the picture information and/or the video information comprises:
inputting the picture information and/or the video information into a preset motion detection model;
detecting whether the motion in the picture information and/or the video information accords with a pre-learned sharing motion or not by the motion detection model;
when the action detection model detects that the actions in the picture information and/or the video information accord with the sharing action, determining that the secondary driving has the action of sharing the content on the secondary driving screen;
and when the action detection model detects that the actions in the picture information and/or the video information do not accord with the sharing action, determining that the secondary driving does not have the action of sharing the content on the secondary driving screen.
6. The screen content auxiliary display method according to claim 1, wherein the detecting whether the content on the secondary driving screen satisfies the auxiliary display condition comprises:
detecting the sight direction of the main driver, and when the sight direction of the main driver points to the auxiliary driver screen, and the duration of the sight direction of the main driver pointing to the auxiliary driver screen is greater than or equal to a preset duration threshold value, and/or the frequency of the sight direction of the main driver pointing to the auxiliary driver screen within the preset duration is greater than or equal to a preset frequency threshold value, determining that the content on the auxiliary driver screen meets the content on the auxiliary driver screen contained in the auxiliary display condition is continuously focused by the main driver.
7. The screen content auxiliary display method according to claim 6, wherein the detecting the line of sight direction of the main drive includes:
acquiring the image of the main driver through a camera preset in the vehicle in a preset acquisition period;
detecting a head image from the image of the main driver according to a preset head detection model;
detecting a rotation direction and a rotation angle of the head from the head image;
detecting an pupil image from the head image according to a preset face key point detection model;
detecting the deflection angle of the pupil according to the pupil image;
and calculating the sight line direction of the main driver according to the rotation direction and the rotation angle of the head and the offset angle of the pupil.
8. The screen content auxiliary display method according to claim 6, wherein when the auxiliary display manner includes content highlighting, the auxiliary display of the content on the secondary driving screen includes:
amplifying and displaying an APP window and/or part or all of display contents in the APP window according to a preset ratio;
increasing the contrast of the color of the display content in the APP window;
adjusting the layout of the display content in the APP window;
when the auxiliary display mode includes transferring the content display position, the auxiliary display of the content on the secondary driving screen includes: and transferring the content on the secondary driving screen to a central control screen for display.
9. The screen content auxiliary display method according to claim 8, wherein the method further comprises: and selecting a corresponding auxiliary display mode according to the APP type, the current functional interface and/or the current state of the central control screen.
10. A screen content auxiliary display device comprising a processor and a memory, the memory having instructions stored therein, wherein the instructions, when executed by the processor, implement the screen content auxiliary display method of any one of claims 1-9.
11. A vehicle comprising a camera and the screen content auxiliary display device according to claim 10.
12. A computer-readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the screen content auxiliary display method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310253316.0A CN116394756A (en) | 2023-03-10 | 2023-03-10 | Screen content auxiliary display method and device, vehicle and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310253316.0A CN116394756A (en) | 2023-03-10 | 2023-03-10 | Screen content auxiliary display method and device, vehicle and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116394756A true CN116394756A (en) | 2023-07-07 |
Family
ID=87013388
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310253316.0A Pending CN116394756A (en) | 2023-03-10 | 2023-03-10 | Screen content auxiliary display method and device, vehicle and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116394756A (en) |
-
2023
- 2023-03-10 CN CN202310253316.0A patent/CN116394756A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US7783422B2 (en) | Navigation device and method of navigating vehicle | |
| US20080281508A1 (en) | Vehicle navigation system and method thereof | |
| US10801858B2 (en) | Map based navigation method, apparatus and storage medium | |
| US7636629B2 (en) | Map information display apparatus and method thereof | |
| JP2008001120A (en) | Display control device for vehicle | |
| EP3922950A2 (en) | Road information processing method and apparatus, electronic device, storage medium and program | |
| US20190001992A1 (en) | Apparatus, vehicle, method and computer program for computing at least one video signal or control signal | |
| US20030062827A1 (en) | Method of setting display, method of setting instrument panel, setting structure of instrument panel and setting structure of display | |
| JP2018088118A (en) | Display control device, control method, program and storage media | |
| CN102729824B (en) | Image processing determining apparatus | |
| JP7232604B2 (en) | NAVIGATION SYSTEM, NAVIGATION DISPLAY METHOD AND NAVIGATION DISPLAY PROGRAM | |
| JP2018151903A (en) | Virtual image display device and computer program | |
| JP2010256554A (en) | Information processor and image display control method | |
| CN116394756A (en) | Screen content auxiliary display method and device, vehicle and storage medium | |
| JP2008213759A (en) | On-vehicle display device | |
| JP2014037172A (en) | Display controller and display control method for head-up display | |
| JP4111444B2 (en) | Navigation device | |
| JP2006078360A (en) | Navigation device | |
| US10971020B1 (en) | Aircraft system and method to provide linear map of traffic | |
| JP6891752B2 (en) | Virtual image display device, virtual image display method and virtual image display program | |
| US20210270626A1 (en) | Image control program, image control device, and image control method | |
| JPH07324940A (en) | Vehicle navigation apparatus | |
| US10564001B2 (en) | Navigation system with route displaying mechanism and method of operation thereof | |
| JP2015148468A (en) | Navigation system, room mirror monitor, navigation method, and program | |
| JP6475413B2 (en) | Navigation system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |