US11302124B2 - Method and apparatus for evaluating vehicle, device and computer readable storage medium - Google Patents
Method and apparatus for evaluating vehicle, device and computer readable storage medium Download PDFInfo
- Publication number
- US11302124B2 US11302124B2 US16/668,380 US201916668380A US11302124B2 US 11302124 B2 US11302124 B2 US 11302124B2 US 201916668380 A US201916668380 A US 201916668380A US 11302124 B2 US11302124 B2 US 11302124B2
- Authority
- US
- United States
- Prior art keywords
- information
- vehicle
- driving mode
- determining
- automatic driving
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active, expires
Links
- 238000000034 method Methods 0.000 title claims abstract description 41
- 230000006399 behavior Effects 0.000 claims description 21
- 230000008921 facial expression Effects 0.000 claims description 17
- 238000004590 computer program Methods 0.000 claims description 6
- 230000000007 visual effect Effects 0.000 claims description 5
- 238000005516 engineering process Methods 0.000 description 9
- 238000010586 diagram Methods 0.000 description 7
- 230000005611 electricity Effects 0.000 description 7
- 239000000446 fuel Substances 0.000 description 7
- 238000012545 processing Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 3
- 238000004891 communication Methods 0.000 description 3
- 238000011156 evaluation Methods 0.000 description 3
- 238000005457 optimization Methods 0.000 description 3
- 230000004044 response Effects 0.000 description 3
- 230000008909 emotion recognition Effects 0.000 description 2
- 230000006870 function Effects 0.000 description 2
- 230000014509 gene expression Effects 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 235000019504 cigarettes Nutrition 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000000391 smoking effect Effects 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M17/00—Testing of vehicles
- G01M17/007—Wheeled or endless-tracked vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
- B60W40/09—Driving style or behaviour
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
- B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/04—Monitoring the functioning of the control system
- B60W50/045—Monitoring control system parameters
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W60/00—Drive control systems specially adapted for autonomous road vehicles
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01D—MEASURING NOT SPECIALLY ADAPTED FOR A SPECIFIC VARIABLE; ARRANGEMENTS FOR MEASURING TWO OR MORE VARIABLES NOT COVERED IN A SINGLE OTHER SUBCLASS; TARIFF METERING APPARATUS; MEASURING OR TESTING NOT OTHERWISE PROVIDED FOR
- G01D21/00—Measuring or testing not otherwise provided for
- G01D21/02—Measuring two or more variables by means not covered by a single other subclass
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01M—TESTING STATIC OR DYNAMIC BALANCE OF MACHINES OR STRUCTURES; TESTING OF STRUCTURES OR APPARATUS, NOT OTHERWISE PROVIDED FOR
- G01M17/00—Testing of vehicles
- G01M17/08—Railway vehicles
-
- G06K9/00845—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/50—Context or environment of the image
- G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
- G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/008—Registering or indicating the working of vehicles communicating information to a remotely located station
-
- G—PHYSICS
- G07—CHECKING-DEVICES
- G07C—TIME OR ATTENDANCE REGISTERS; REGISTERING OR INDICATING THE WORKING OF MACHINES; GENERATING RANDOM NUMBERS; VOTING OR LOTTERY APPARATUS; ARRANGEMENTS, SYSTEMS OR APPARATUS FOR CHECKING NOT PROVIDED FOR ELSEWHERE
- G07C5/00—Registering or indicating the working of vehicles
- G07C5/08—Registering or indicating performance data other than driving, working, idle, or waiting time, with or without registering driving, working, idle or waiting time
- G07C5/0841—Registering performance data
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/04—Monitoring the functioning of the control system
- B60W50/045—Monitoring control system parameters
- B60W2050/046—Monitoring control system parameters involving external transmission of data to or from the vehicle, e.g. via telemetry, satellite, Global Positioning System [GPS]
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
- B60W50/08—Interaction between the driver and the control system
- B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
- B60W2050/146—Display means
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2540/00—Input parameters relating to occupants
- B60W2540/221—Physiology, e.g. weight, heartbeat, health or special needs
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2556/00—Input parameters relating to data
- B60W2556/10—Historical data
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
- B60W2556/00—Input parameters relating to data
- B60W2556/45—External transmission of data to or from the vehicle
- B60W2556/50—External transmission of data to or from the vehicle of positioning data, e.g. GPS [Global Positioning System] data
Definitions
- Embodiments of the present disclosure relate to a vehicle technology field, and more particularly, to a method and an apparatus for evaluating a vehicle, a device and a computer readable storage medium.
- the automatic driving mode of the vehicle may be enabled in response to a user's command for starting the automatic driving mode and disabled in response to a user's command for ending the automatic driving mode.
- the contents recognized and executed by the vehicle's automatic driving system can be visually displayed, such as the increasing proximity of other vehicles, objects and other entities to the vehicle.
- a scheme for evaluating a vehicle is provided.
- a method for evaluating a vehicle includes: determining driving information of the vehicle in an automatic driving mode, the driving information comprising at least one of trip information, power consumption information and driver state of the vehicle in the automatic driving mode; determining a usage situation of the vehicle in the automatic driving mode based on the driving information; and providing the usage situation of the vehicle.
- an apparatus for evaluating a vehicle includes a processor and a memory.
- the memory is configured to store instructions executable by the processor.
- the processor is configured to run a program corresponding to the instructions by reading the instructions stored in the memory, so as to perform the method according to the first aspect of the present disclosure.
- a device in a third aspect of the present disclosure, includes one or more processors and a storage device configured to store one or more programs. When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method according to the first aspect of the present disclosure.
- a computer readable storage medium is provided.
- the computer readable storage medium is stored thereon with a computer program.
- the program is executed by a processor, the method according to the first aspect of the present disclosure is implemented.
- FIG. 1 illustrates an example environment in which embodiments of the present disclosure may be implemented.
- FIG. 2 illustrates a flowchart of a method for evaluating a vehicle according to some embodiments of the present disclosure.
- FIG. 3 illustrates a flowchart of a method for evaluating time saving in an automatic driving mode according to some embodiments of the present disclosure.
- FIG. 4 illustrates a flowchart of a method for evaluating power consumption saving in an automatic driving mode according to some embodiments of the present disclosure.
- FIG. 5 illustrates a flowchart of a method for evaluating a relax degree of a driver in an automatic driving mode according to some embodiments of the present disclosure.
- FIG. 6 illustrates a schematic diagram of an example of providing a usage situation of an evaluated vehicle according to some embodiments of the present disclosure.
- FIG. 7 illustrates a schematic block diagram of an apparatus for evaluating a vehicle according to some embodiments of the present disclosure.
- FIG. 8 illustrates a block diagram of a computing device that can implement embodiments of the present disclosure.
- the term “including” and similar terms shall be understood as open inclusion, i.e. “including but not limited to”.
- the term “based on” should be understood to mean “at least partially based on”.
- the term “an embodiment” or “such embodiment” shall be understood as “at least one embodiment”.
- the terms “first”, “second”, and so on can refer to different or identical objects. Other explicit and implicit definitions may be included below.
- a scheme for evaluating a vehicle in an automatic driving mode is proposed.
- the usage situation of the vehicle in the automatic driving mode is evaluated based on the driving information of the vehicle in the automatic driving mode, the driving information including at least one of trip information, power consumption information and driver state.
- the driving information including at least one of trip information, power consumption information and driver state.
- FIG. 1 illustrates an example environment in which embodiments of the present disclosure may be implemented.
- a vehicle 110 is schematically illustrated.
- the vehicle 110 may be any type of vehicle that can carry people/or things and move through power systems such as engines, including but not limited to cars, trucks, buses, electric cars, motorcycles, RVS, trains, etc.
- the vehicle 110 may have an automatic driving mode and a normal driving mode. In the automatic driving mode, the vehicle 110 may operate with the help of the intelligent driving system equipped with the vehicle itself, without the participation of the driver 111 . Based on the automatic perception of the vehicle on surrounding environment, the intelligent driving system determines the executable driving behavior of the vehicle in the current environment, and then controls the vehicle to realize the corresponding driving behavior.
- the scheme of embodiments of the present disclosure is suitable for any intelligent driving system, which is not repeated here to avoid confusion.
- the driver 111 is required to control the entire operation of the vehicle 110 .
- the vehicle 100 may have only the automatic driving mode, and such a vehicle is also known as a driverless vehicle.
- the vehicle 110 may be driven by the driver 111 in the normal driving mode or may be driven in the automatic driving mode.
- the vehicle 110 may include an in-vehicle camera 112 , a driving computer 113 and an onboard GPS 114 .
- the vehicle 110 may also be equipped with any other suitable facilities, which not shown here for the sake of illustrating.
- the vehicle 110 may include more in-vehicle cameras, which is not limited in any way in embodiments of the present disclosure.
- the in-vehicle camera 112 may at least record operating information of the driver, such as attention area information, facial expression information, non-driving behavior information and so on.
- the in-vehicle camera 112 may also record information associated with the passenger or other things in the vehicle. It should be understood that the in-vehicle camera 112 may be placed inside the vehicle 110 or outside the vehicle 110 , as long as the in-vehicle camera 112 may record the information inside the vehicle 110 . Embodiments of the present disclosure is not limited thereto.
- the driving computer 113 may record information about fuel consumption and speed of the vehicle, for example, immediate fuel consumption (or electricity consumption), average fuel consumption (or electricity consumption), remaining amount of oil (or electricity), cruising range, engine speed, water temperature, and so on.
- the onboard GPS (global positioning system) 114 may be used to locate the current position of the vehicle 110 .
- the storage device 120 and the computing device 130 are also shown in the environment 100 . According to embodiments of the present disclosure, information obtained from the vehicle may be stored to the storage device 120 , for performing evaluation of the vehicle at the computing device 130 , thereby outputting the evaluation result 150 .
- the storage device 120 may be any suitable storage device, which is not elaborated here.
- the computing device 130 may be any server or terminal device with computing capabilities. It should be understood that although the storage device 120 and the computing device 130 are shown here as independent of the vehicle 110 , the storage device 120 and the computing device 130 may also be integrated with the vehicle 110 in alternative embodiments.
- FIG. 2 illustrates a flowchart of a method 200 for evaluating a vehicle according to some embodiments of the present disclosure.
- the method 200 may be implemented at the computing device 130 in FIG. 1 , and thus it is described below in combination with the environment of FIG. 1 .
- driving information of the vehicle 110 in the automatic driving mode may be determined.
- the driving information may include at least one of trip information, power consumption information and driver state of the vehicle 110 in the automatic driving mode.
- related information of the vehicle 110 in the automatic driving mode may be obtained from data recorded by the in-vehicle 112 , the driving computer 113 and the onboard GPS 114 , and further, the driving information of the vehicle 110 in the automatic driving mode is determined, including the trip information, the power consumption information, and the driver state. It should be understood that there may be no driver state when the vehicle 110 is the driverless vehicle.
- usage situation of the vehicle in the automatic driving mode is determined based on the driving information.
- the usage situation of the vehicle may include at least one of time saving information, power consumption saving information and relax degree of driver of the vehicle in the automatic driving mode. It should be understood that, embodiments of the present disclosure are not limited to this, but are intended to cover any information related to the usage situation of the vehicle in the automatic driving mode.
- time saving information, power consumption saving information and relax degree of driver are only taken as examples, and specific embodiments are described in combination with FIGS. 3-5 .
- FIG. 3 illustrates a flowchart of a method 300 for evaluating time saving in an automatic driving mode according to some embodiments of the present disclosure.
- the method 300 may be implemented at the computing device 130 of FIG. 1 , and thus it is described below in combination with the environment in FIG. 1 .
- map data, onboard GPS data and time information of a section of trip of the vehicle 110 in the automatic driving mode are obtained.
- information related to the whole trip in the automatic driving mode may be obtained.
- only information related to a part of the trip in the automatic driving mode may be obtained.
- the map data may be obtained from the vehicle 110 , for example, if the vehicle 110 has an onboard navigator. In an alternative embodiment, the map data may be obtained from outside of the vehicle 110 . For example, in some embodiments, map data stored locally in the computing device 130 may be obtained. In an alternative embodiment, the computing device 130 may obtain the map data from a remote server that maintains the map data. According to some embodiments of the present disclosure, the computing device 130 may obtain the onboard GPS data and time information from an onboard GPS 140 of the vehicle 110 .
- the trip information is determined based on the obtained map data, onboard GPS data and time information.
- the position and time of the start and end of the automatic driving mode may be determined, and then the trip length and trip time under the automatic driving mode may be determined.
- the condition of the road through which the vehicle 110 passes in the automatic driving mode such as the lengths of the congested road section and slow-moving road section and the corresponding passing time.
- historical trip information of the vehicle in a normal driving mode is obtained.
- the map data, onboard GPS data and time information of the vehicle 110 passing the same positions of start and end in the normal driving mode in a previous period of time may be obtained, and based on the obtained information, the historical trip information may be determined.
- the trip length and trip time in normal driving mode may be determined.
- the condition of the road through which the vehicle 110 passes in the normal driving mode may be determined, such as the lengths of the congested road section and slow-moving road section and the corresponding passing time.
- This historical trip information may be determined by operations similar to those described at blocks 310 and 320 . Other details on this point are described at blocks 310 and 320 , and will not be repeated here.
- time saving information in the automatic driving mode compared to the normal driving mode may be determined based on the trip information determined at block 320 and the historical trip information determined at block 330 .
- the total time saved in the whole trip in the automatic driving mode compared to the normal driving mode may be determined based on the whole trip time in the automatic driving mode and the whole trip time in the normal driving mode.
- the time saved in the congested road section in the automatic driving mode may be determined compared with the normal driving mode. It should be understood that the embodiments of the present disclosure are not limited to this and, for the sake of brevity, will not be enumerated here.
- FIG. 4 illustrates a flowchart of a method 400 for evaluating power consumption saving in an automatic driving mode according to some embodiments of the present disclosure.
- the method 400 may be implemented at the computing device 130 of FIG. 1 , and thus it is described below in combination with the environment in FIG. 1 .
- driving computer data of the vehicle 110 in the automatic driving mode is obtained, for example, from the driving computer 113 .
- information related to the whole trip in the automatic driving mode may be obtained.
- only information related to a part of the trip in the automatic driving mode may be obtained.
- power consumption information in the automatic driving mode may be determined based on the driving computer data.
- an average power consumption of the whole trip in the automatic driving mode may be determined from the driving computer data.
- the fuel (or electricity) consumption for the entire trip in the automatic driving mode may be determined from the driving computer data.
- historical power consumption information of the vehicle 110 in the normal driving mode is obtained.
- the driving computer data of the vehicle 110 passing the same positions of start and end in the normal driving mode in a previous period of time may be obtained, and based on the obtained data, the historical power consumption information may be determined.
- the average power consumption on the same trip in the normal driving mode may be determined.
- the fuel (electricity) consumption on the same trip in the normal driving mode may be determined.
- This historical power consumption information may be determined by operations similar to those described at blocks 410 and 420 . Other details on this point are described at blocks 410 and 420 , and will not be repeated here.
- power consumption saving information in the automatic driving mode compared to the normal driving mode may be determined based on the power consumption information determined at block 420 and the historical power consumption information determined at block 430 .
- the power consumption saved in the whole trip in the automatic driving mode compared to the normal driving mode may be determined based on the corresponding average power consumption or total fuel (electricity) consumption in the whole trip in the automatic driving mode and in the normal driving mode.
- the power consumption saved in the congested road section in the automatic driving mode may be determined compared with the normal driving mode. It should be understood that the embodiments of the present disclosure are not limited to this and, for the sake of brevity, will not be enumerated here.
- FIG. 5 illustrates a flowchart of a method 500 for evaluating a relax degree of a driver in an automatic driving mode according to some embodiments of the present disclosure.
- the method 500 may be implemented at the computing device 130 of FIG. 1 , and thus it is described below in combination with the environment in FIG. 1 .
- camera data of the vehicle 110 in the automatic driving mode is obtained, for example, from the in-vehicle camera 112 .
- image data captured in this time period may be obtained from the camera data.
- operating information of the driver is determined based on the camera data.
- attention area information, facial expression information, non-driving behavior information and the like of the driver 111 may be determined by analyzing the obtained image data.
- the image data may be analyzed by machine vision technology to determine the driver 111 's attention to lane area and other areas.
- the lane area includes the lane in which the driver is driving and the lanes on both sides of the driving. Other areas include the dashboard, rearview mirror, and pedestrians on the side of the road.
- the positions of facial feature points of the driver may be extracted by machine vision, the head deflection angle of the driver may be calculated, and the discriminant index of the degree of attention may be established, so that the driver's current degree of attention to the predetermined area may be distinguished.
- the driver's attention to the lane area may be determined by determining whether the time in which the driver pays attention to the lane area or other areas exceeds the predetermined threshold. It should be understood that this can be achieved using any suitable machine vision technology known or developed in the future in the field.
- the image data may be analyzed through emotion recognition technology to determine the facial expression information of the driver Ill, such as calmness, tension, fear, happiness, anger, surprise and so on.
- the more frequent facial expressions of the driver 111 in the automatic driving mode may be determined as the facial expression information of the driver 111 . It should be understood that this can be achieved using any appropriate emotion recognition technology known or developed in the future in the field.
- the image data may be analyzed by machine vision technology to determine the non-driving behavior information of the driver 111 , such as talking on the phone, looking down to play with the phone, talking with others on the side, smoking cigarettes and so on.
- the duration of the driver's non-driving behavior in the automatic driving mode may be determined as the non-driving behavior information of the driver 111 .
- the number of types of the driver's non-driving behavior in the automatic driving mode may be determined as the non-driving behavior information of the driver 111 . It should be understood that this can be achieved using any suitable machine vision technology known or developed in the future in the field.
- a relax level of the driver 11 I is determined based on the operating information obtained at block 520 .
- the relax level indicates a relax degree of the driver 111 in the automatic driving mode.
- the relax level may be defined based on one or more of the attention area information, facial expression information, and non-driving behavior information.
- the relax level may include relaxed, highly relaxed, and completely relaxed. For example, a situation in which the hands are freed but the driver is required to pay attention to the road is identified as relaxed. The situation of freeing your hands and partially freeing your brain without paying attention to the road is defined as highly relaxed. The situation in which the hands are freed and reading, games and other activities can be carried on is determined as completely relaxed. It should be understood that the definition and determination of relax levels is not limited to this example, and any other appropriate division can be used.
- the relax level when the driver 111 's attention area during the automatic driving is lane area, the relax level may be determined as relaxed, while when the attention area is other areas, the relax level may be determined as highly relaxed. In some embodiments, when the attention area is determined to be other area, the relax level may be determined as completely relaxed if it is further determined that the driver's facial expression information during the automatic driving is happy and the duration exceeds the predetermined threshold. In some embodiments, when the attention area is determined to be other area, the relax level may be determined as completely relaxed if it is further determined that the duration of the non-driving behavior of the driver during the automatic driving exceeds the threshold or the number of types of the driver's non-driving behavior exceeds the predetermined threshold.
- weights may be set for each item in this information to correct the total score and improve accuracy.
- a low score may be given for high attention to the lane area in the attention area information, while a high score may be given for low attention to the lane area in the attention area information.
- a low score could be given for calm expressions in the facial expression information, while a high score could be given for happy expressions in the facial expression information.
- a high score could be given for looking down and playing with the phone in non-driving behavior information. The higher the total score is, the higher the relax level is.
- the method for determining the relax degree is not limited to the above example, and any other appropriate method can be used. By quantifying the relax degree of the driver, it is possible to evaluate the vehicle in the automatic driving mode more accurately, which is beneficial to the optimization, management and use of the vehicle.
- the evaluated usage situation of the vehicle may be provided.
- the usage situation of the vehicle may be provided at the end of the automatic driving mode.
- FIG. 6 illustrates a schematic diagram of an example 600 of providing a usage situation of an evaluated vehicle according to some embodiments of the present disclosure.
- the example 600 shows the vehicle usage 601 , which includes time saving information 610 , power consumption saving information 620 , and driver state information 630 .
- the vehicle usage 601 may include only one or two of the time saving information 610 , power consumption saving information 620 and driver state information 630 .
- the vehicle usage 601 may only include the time saving information 610 and power consumption saving information 620 .
- the vehicle usage is not limited to being provided in the manner shown in 610 - 630 and may be provided in any other manner as appropriate.
- the vehicle usage 601 may be presented in a visual form via a display device of the vehicle 110 . In some embodiments, it may be presented through the dashboard of the vehicle. In some embodiments, it may be presented through the central control display of the vehicle.
- the vehicle usage 601 may be presented in an audio form through a speaker system of the vehicle 110 .
- the vehicle usage 601 may also be presented on terminals of users.
- the users may include at least one of the driver, passengers, and managers of the vehicle.
- the managers may be for example, vendors, manufacturers, and so on.
- the terminals here may include mobile phones, desktop computers, portable computers, personal digital assistants, and so on.
- the presenting may be performed in response to a request from the user. In the alternative embodiment, the presenting may be voluntarily provided to the user by the computing device 130 .
- the method for evaluating the vehicle in the automatic driving mode is described.
- the vehicle in the automatic driving mode may be evaluated.
- the evaluation results can be beneficial to the subsequent optimization of vehicle management and use.
- FIG. 7 illustrates a schematic block diagram of an apparatus 700 for evaluating a vehicle according to some embodiments of the present disclosure.
- the apparatus 700 may be implemented at the computing device 130 in FIG. 1 .
- the apparatus 700 may include an information determining unit 710 , a situation determining unit 720 and a situation providing unit 730 .
- the information determining unit 710 may be configured to determine driving information of the vehicle in an automatic driving mode, the driving information including at least one of trip information, power consumption information and driver state of the vehicle in the automatic driving mode.
- the situation determining unit 720 may be configured to determine a usage situation of the vehicle in the automatic driving mode based on the driving information.
- the situation providing unit 730 may be configure to providing the usage situation of the vehicle.
- the information determining unit may include: an information obtaining unit, configured to obtain map data, onboard GPS data and time information of a section of trip of the vehicle in the automatic driving mode; and a tip determining unit, configured to determine the trip information based on the map data, the onboard GPS data, and the time information.
- the situation determining unit 720 may include: a historical trip obtaining unit, configured to obtain historical trip information of the vehicle in the automatic driving mode; and a time saving determining unit, configured to determine time saving information in the automatic driving mode compared to a normal driving mode based on the trip information and the historical trip information.
- the information determining unit 710 may include: a driving data obtaining unit, configured to obtain driving computer data of the vehicle in the automatic driving mode; and a power consumption determining unit, configured to determine the power consumption information in the automatic driving mode based on the driving computer data.
- the situation determining unit 720 may include: a historical power consumption obtaining unit, configured to obtaining historical power consumption information of the vehicle in a normal driving mode; and a power consumption saving determining unit, configured to determine power consumption saving information in the automatic driving mode compared to the normal driving mode based on the power consumption information and the historical power consumption information.
- the information determining unit 710 may include: a camera data obtaining unit, configured to obtain camera data of the vehicle in the automatic driving mode; and a state determining unit, configured to determine the driver state based on the camera data.
- the situation determining unit 720 may include: an operating information determining unit, configured to determine operating information of a driver based on the camera data, the operating information including at least one of attention area information, facial expression information and non-driving behavior information; and a relax level determining unit, configured to determine a relax level of the driver based on the operating information, the relax level indicating a relax degree of the driver in the automatic driving mode.
- the situation providing unit 730 may include at least one of: a visual presenting unit, configured to present the usage situation of the vehicle in a visual form via a display device of the vehicle; an audio presenting unit, configured to present the usage situation of the vehicle in an audio form via a speaker system of the vehicle; and a terminal presenting unit, configured to present the usage situation of the vehicle on a terminal of a user, the user comprising at least one of a driver, a passenger, and a manager of the vehicle.
- a visual presenting unit configured to present the usage situation of the vehicle in a visual form via a display device of the vehicle
- an audio presenting unit configured to present the usage situation of the vehicle in an audio form via a speaker system of the vehicle
- a terminal presenting unit configured to present the usage situation of the vehicle on a terminal of a user, the user comprising at least one of a driver, a passenger, and a manager of the vehicle.
- FIG. 8 illustrates a block diagram of an exemplary device 800 suitable for implementing embodiments of the present disclosure.
- the device 800 may be used to implement the computing device 130 in FIG. 1 .
- the device 800 includes a computing unit (CPU) 801 that can perform various appropriate actions and processes based on computer program instructions stored in the read only memory (ROM) 802 or computer program instructions loaded from the memory unit 808 into the random access memory (RAM) 803 .
- ROM 802 read only memory
- RAM 803 random access memory
- various programs and data needed for the operation of the device 800 may also be stored.
- the CPU 801 , ROM 802 , and RAM 803 are connected to each other via the bus 804 .
- the input/output (I/O) interface 805 is also connected to the bus 804 .
- the 110 interface 805 Multiple components of the device 800 are connected to the 110 interface 805 , including: the input unit 806 , such as keyboard, mouse, etc.; the output unit 807 , such as various types of monitors, speakers, etc.; storage unit 808 , such as disk, CD, etc; and the communication unit 809 , such as network cards, modems, wireless transceiver, etc.
- the communication unit 809 allows the device 800 to exchange information/data with other devices via the computer network such as Internet and/or various telecommunication networks.
- the computing unit 801 may be any kind of generic and/or dedicated processing component with processing and computing capability. Examples of the computing unit 801 include but are not limited to the central processing unit (CPU), graphics processing unit (GPU), all kinds of dedicated artificial intelligence (AI) computing chip, all kinds of computing unit running the machine learning model algorithms, digital signal processor (DSP), and any appropriate processor, controller and micro controller, etc.
- the computing unit 801 performs the various methods and processes described above, such as any of the methods 200 , 300 , 400 and 500 .
- any of the methods 200 , 300 , 400 and 500 may be implemented as a computer software program that is physically contained in machine-readable media, such as storage unit 808 .
- part or all of a computer program may be loaded and/or installed onto the device 800 via the ROM 802 and/or the communication unit 809 .
- the computer program When the computer program is loaded into the RAM 803 and executed by the CPU 801 , one or more steps of any of the methods 200 , 300 , 400 and 500 described above can be executed.
- the computing unit 801 may be configured to execute any of the methods 200 , 300 , 400 and 500 by any other appropriate means (for example, with the help of firmware).
- FPGA field programmable gate arrays
- ASIC application specific integrated circuits
- ASSP application specific standard products
- SOC systems on a chip
- CPLD complex programmable logic devices
- Program codes used to implement the method of the present disclosure may be written in any combination of one or more programming languages.
- the program codes may be provided to a processor or a controller of a general computer, a dedicated computer or other programmable data processing device, such that the program codes, when executed by the processor or the controller, cause the functions/operations defined in the flowchart and/or block diagram to be implemented.
- the program codes may be executed completely on a machine, partly on the machine, executed partly on the machine as a separate package and partly on a remote machine, or completely executed on a remote machine or a server.
- machine readable media may be tangible media that may contain or store programs for use by or in conjunction with an instruction execution system, apparatus, or device.
- the machine readable media may be machine readable signal media or machine readable storage media.
- the machine readable media may include, but not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatuses or devices, or any suitable combination of the above.
- machine readable storage medium examples include the electrical connection based on one or more lines, portable computer disk, hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM) or flash memory, optical fiber, convenient type compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage device, or any suitable combination of the above content.
- RAM random access memory
- ROM read-only memory
- EPROM erasable programmable read-only memory
- flash memory optical fiber
- CD-ROM compact disc read-only memory
- CD-ROM compact disc read-only memory
- magnetic storage device or any suitable combination of the above content.
Landscapes
- Engineering & Computer Science (AREA)
- Automation & Control Theory (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Transportation (AREA)
- Mechanical Engineering (AREA)
- Human Computer Interaction (AREA)
- Mathematical Physics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- Traffic Control Systems (AREA)
- Control Of Driving Devices And Active Controlling Of Vehicle (AREA)
Abstract
Description
Claims (13)
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201811280541.9 | 2018-10-30 | ||
CN201811280541.9A CN109406167B (en) | 2018-10-30 | 2018-10-30 | Method, apparatus, device and computer-readable storage medium for evaluating a vehicle |
Publications (2)
Publication Number | Publication Date |
---|---|
US20200134946A1 US20200134946A1 (en) | 2020-04-30 |
US11302124B2 true US11302124B2 (en) | 2022-04-12 |
Family
ID=65470542
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/668,380 Active 2040-06-09 US11302124B2 (en) | 2018-10-30 | 2019-10-30 | Method and apparatus for evaluating vehicle, device and computer readable storage medium |
Country Status (3)
Country | Link |
---|---|
US (1) | US11302124B2 (en) |
EP (1) | EP3647147B1 (en) |
CN (1) | CN109406167B (en) |
Families Citing this family (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN112009494B (en) * | 2019-05-30 | 2022-12-27 | 华为技术有限公司 | Vehicle abnormity processing method and device, control equipment and storage medium |
CN110751823A (en) * | 2019-10-25 | 2020-02-04 | 上海商汤临港智能科技有限公司 | Monitoring method and device for automatic driving fleet |
CN111619576B (en) * | 2020-06-03 | 2021-06-25 | 中国第一汽车股份有限公司 | Control method, device, equipment and storage medium |
CN111814904B (en) * | 2020-07-22 | 2021-05-28 | 同济大学 | Method and system for distinguishing test driving mode of automatic driving road |
CN113353065A (en) * | 2021-05-19 | 2021-09-07 | 广州小鹏汽车科技有限公司 | Interaction method and device based on automatic driving |
CN113320535B (en) * | 2021-07-20 | 2022-08-19 | 文思海辉智科科技有限公司 | Driving behavior monitoring method and device, electronic equipment and storage medium |
CN113954854B (en) * | 2021-11-30 | 2023-04-07 | 广汽埃安新能源汽车有限公司 | Intelligent driving function optimization method and device, electronic equipment and storage medium |
Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN202243010U (en) | 2010-12-27 | 2012-05-30 | 上海博泰悦臻电子设备制造有限公司 | Anti-fatigue driving device and vehicle-mounted device |
CN103234763A (en) | 2013-04-09 | 2013-08-07 | 北京理工大学 | System and method for quantitatively evaluating unmanned vehicles |
DE102014209854A1 (en) | 2014-05-23 | 2015-11-26 | Continental Automotive Gmbh | Vehicle control device |
CN105225478A (en) | 2015-08-13 | 2016-01-06 | 中国科学院自动化研究所 | Intelligent vehicle behavior record and evaluation method |
US20160362118A1 (en) * | 2011-08-31 | 2016-12-15 | Pulsar Informatics, Inc. | Driver performance metric |
CN107063713A (en) | 2017-04-27 | 2017-08-18 | 百度在线网络技术(北京)有限公司 | Method of testing and device applied to pilotless automobile |
CN107543726A (en) | 2016-07-28 | 2018-01-05 | 北京百度网讯科技有限公司 | A kind of method of testing and device of unmanned vehicle Ride Control System manipulation precision |
US20180118219A1 (en) * | 2016-10-27 | 2018-05-03 | Toyota Motor Engineering & Manufacturing North America, Inc. | Driver and vehicle monitoring feedback system for an autonomous vehicle |
CN108609019A (en) | 2018-05-09 | 2018-10-02 | 浙江金之路信息科技有限公司 | A kind of electric vehicle automatic Pilot method based on artificial intelligence platform |
US10185999B1 (en) * | 2014-05-20 | 2019-01-22 | State Farm Mutual Automobile Insurance Company | Autonomous feature use monitoring and telematics |
US20190179308A1 (en) * | 2016-06-07 | 2019-06-13 | Denso Corporation | Vehicle apparatus and vehicle system |
US10324463B1 (en) * | 2016-01-22 | 2019-06-18 | State Farm Mutual Automobile Insurance Company | Autonomous vehicle operation adjustment based upon route |
US10599155B1 (en) * | 2014-05-20 | 2020-03-24 | State Farm Mutual Automobile Insurance Company | Autonomous vehicle operation feature monitoring and evaluation of effectiveness |
-
2018
- 2018-10-30 CN CN201811280541.9A patent/CN109406167B/en active Active
-
2019
- 2019-10-30 EP EP19206215.6A patent/EP3647147B1/en active Active
- 2019-10-30 US US16/668,380 patent/US11302124B2/en active Active
Patent Citations (13)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN202243010U (en) | 2010-12-27 | 2012-05-30 | 上海博泰悦臻电子设备制造有限公司 | Anti-fatigue driving device and vehicle-mounted device |
US20160362118A1 (en) * | 2011-08-31 | 2016-12-15 | Pulsar Informatics, Inc. | Driver performance metric |
CN103234763A (en) | 2013-04-09 | 2013-08-07 | 北京理工大学 | System and method for quantitatively evaluating unmanned vehicles |
US10185999B1 (en) * | 2014-05-20 | 2019-01-22 | State Farm Mutual Automobile Insurance Company | Autonomous feature use monitoring and telematics |
US10599155B1 (en) * | 2014-05-20 | 2020-03-24 | State Farm Mutual Automobile Insurance Company | Autonomous vehicle operation feature monitoring and evaluation of effectiveness |
DE102014209854A1 (en) | 2014-05-23 | 2015-11-26 | Continental Automotive Gmbh | Vehicle control device |
CN105225478A (en) | 2015-08-13 | 2016-01-06 | 中国科学院自动化研究所 | Intelligent vehicle behavior record and evaluation method |
US10324463B1 (en) * | 2016-01-22 | 2019-06-18 | State Farm Mutual Automobile Insurance Company | Autonomous vehicle operation adjustment based upon route |
US20190179308A1 (en) * | 2016-06-07 | 2019-06-13 | Denso Corporation | Vehicle apparatus and vehicle system |
CN107543726A (en) | 2016-07-28 | 2018-01-05 | 北京百度网讯科技有限公司 | A kind of method of testing and device of unmanned vehicle Ride Control System manipulation precision |
US20180118219A1 (en) * | 2016-10-27 | 2018-05-03 | Toyota Motor Engineering & Manufacturing North America, Inc. | Driver and vehicle monitoring feedback system for an autonomous vehicle |
CN107063713A (en) | 2017-04-27 | 2017-08-18 | 百度在线网络技术(北京)有限公司 | Method of testing and device applied to pilotless automobile |
CN108609019A (en) | 2018-05-09 | 2018-10-02 | 浙江金之路信息科技有限公司 | A kind of electric vehicle automatic Pilot method based on artificial intelligence platform |
Non-Patent Citations (3)
Title |
---|
Office Action for Chinese Application No. 201811280541.9, dated Apr. 20, 2020. |
Office Action for European Application No. 19206215.6, dated Mar. 23, 2020. |
Study on the Test and Evaluation Method System of Autonomous Vehicles, 3 pages; Abstract. |
Also Published As
Publication number | Publication date |
---|---|
CN109406167B (en) | 2022-05-20 |
EP3647147B1 (en) | 2023-12-27 |
EP3647147A1 (en) | 2020-05-06 |
CN109406167A (en) | 2019-03-01 |
US20200134946A1 (en) | 2020-04-30 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11302124B2 (en) | Method and apparatus for evaluating vehicle, device and computer readable storage medium | |
US12032730B2 (en) | Methods and systems for using artificial intelligence to evaluate, correct, and monitor user attentiveness | |
CN106803423B (en) | Man-machine interaction voice control method and device based on user emotion state and vehicle | |
CN113460070B (en) | Vehicle control method and device | |
US20150345981A1 (en) | Adaptive navigation and location-based services based on user behavior patterns | |
Lin et al. | Adasa: A conversational in-vehicle digital assistant for advanced driver assistance features | |
CN109920410B (en) | Apparatus and method for determining reliability of recommendation based on environment of vehicle | |
JP7159368B2 (en) | Vehicle operation control method, vehicle operation control device, computer equipment, computer-readable storage medium and computer program | |
CN110203154B (en) | Recommendation method and device for vehicle functions, electronic equipment and computer storage medium | |
JP6697998B2 (en) | Mobile insurance premium calculation method, mobile insurance calculation device, mobile insurance calculation program, and mobile | |
WO2021067380A1 (en) | Methods and systems for using artificial intelligence to evaluate, correct, and monitor user attentiveness | |
CN114684152A (en) | Method, device, vehicle and medium for processing driving experience data | |
CN108429817A (en) | One bulb vehicle vehicle-mounted terminal system | |
Lashkov et al. | Ontology-based approach and implementation of ADAS system for mobile device use while driving | |
WO2023193652A1 (en) | Car computer avatar interaction method and system, and vehicle and storage medium | |
CN114084144A (en) | Method and system for determining driver's mood in conjunction with driving environment | |
JP6819529B2 (en) | Information processing equipment, information processing system, and information processing method | |
Fu et al. | Evaluation of the visual-manual resources required to perform calling and navigation tasks in conventional mode with a portable phone and in full-touch mode with an embedded system | |
US11565726B2 (en) | Vehicle and safe driving assistance method therefor | |
KR20190088710A (en) | Method for evaluating usability on vehicle infortainment syste, method for evaluating cognitive load of driver using vehicle infortainment syste | |
CN113299286A (en) | Interaction method, device and system based on vehicle-mounted robot and readable medium | |
CN112052325A (en) | Voice interaction method and device based on dynamic perception | |
CN111993997A (en) | Pedestrian avoidance prompting method, device, equipment and storage medium based on voice | |
CN114296680B (en) | Virtual test driving device, method and storage medium based on facial image recognition | |
Du et al. | Towards Proactive Interactions for In-Vehicle Conversational Assistants Utilizing Large Language Models |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ZHANG, QIUYU;REEL/FRAME:050862/0768 Effective date: 20190724 |
|
FEPP | Fee payment procedure |
Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
AS | Assignment |
Owner name: APOLLO INTELLIGENT CONNECTIVITY (BEIJING) TECHNOLOGY CO., LTD., CHINA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:BAIDU ONLINE NETWORK TECHNOLOGY (BEIJING) CO., LTD.;REEL/FRAME:057779/0075 Effective date: 20210923 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: PUBLICATIONS -- ISSUE FEE PAYMENT VERIFIED |
|
STCF | Information on status: patent grant |
Free format text: PATENTED CASE |