WO2025255801A1 - 图像显示方法、装置和车辆 - Google Patents

图像显示方法、装置和车辆

Info

Publication number
WO2025255801A1
WO2025255801A1 PCT/CN2024/099207 CN2024099207W WO2025255801A1 WO 2025255801 A1 WO2025255801 A1 WO 2025255801A1 CN 2024099207 W CN2024099207 W CN 2024099207W WO 2025255801 A1 WO2025255801 A1 WO 2025255801A1
Authority
WO
WIPO (PCT)
Prior art keywords
sub
region
user
vehicle
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/099207
Other languages
English (en)
French (fr)
Inventor
沈涛
温裕祥
闫智海
李云龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Yinwang Intelligent Technology Co Ltd
Original Assignee
Shenzhen Yinwang Intelligent Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Yinwang Intelligent Technology Co Ltd filed Critical Shenzhen Yinwang Intelligent Technology Co Ltd
Priority to PCT/CN2024/099207 priority Critical patent/WO2025255801A1/zh
Priority to CN202480008984.XA priority patent/CN121605052A/zh
Publication of WO2025255801A1 publication Critical patent/WO2025255801A1/zh
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60K—ARRANGEMENT OR MOUNTING OF PROPULSION UNITS OR OF TRANSMISSIONS IN VEHICLES; ARRANGEMENT OR MOUNTING OF PLURAL DIVERSE PRIME-MOVERS IN VEHICLES; AUXILIARY DRIVES FOR VEHICLES; INSTRUMENTATION OR DASHBOARDS FOR VEHICLES; ARRANGEMENTS IN CONNECTION WITH COOLING, AIR INTAKE, GAS EXHAUST OR FUEL SUPPLY OF PROPULSION UNITS IN VEHICLES
    • B60K35/00—Instruments specially adapted for vehicles; Arrangement of instruments in or on vehicles
    • B60K35/20—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor
    • B60K35/28—Output arrangements, i.e. from vehicle to user, associated with vehicle functions or specially adapted therefor characterised by the type of the output information, e.g. video entertainment or vehicle dynamics information; characterised by the purpose of the output information, e.g. for attracting the attention of the driver
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three-dimensional [3D] modelling for computer graphics
    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B29/00—Maps; Plans; Charts; Diagrams, e.g. route diagram
    • G09B29/10—Map spot or coordinate position indicators; Map reading aids

Definitions

  • This application relates to the field of intelligent driving, and more specifically, to an image display method, apparatus, and vehicle.
  • Points of interest generally refer to products or destinations that users are interested in.
  • POIs can only be presented in the form of text or graphic information.
  • This application provides an image display method, apparatus, and vehicle that enables users to have a comprehensive understanding of POIs and facilitates destination confirmation.
  • this application provides an image display method, which includes: acquiring three-dimensional (3D) information of a first point of interest (POI) sent by a cloud server, wherein the three-dimensional information is determined by images captured by cameras when one or more vehicles are around the first POI and the pose information corresponding to the images; and controlling a display device to display the three-dimensional information.
  • 3D three-dimensional
  • POI point of interest
  • displaying the 3D information of a Point of Interest (POI) on a display device allows users to gain a comprehensive understanding of the POI through 3D information, thereby facilitating destination confirmation and enhancing the user experience. For example, taking a park as the first POI, displaying the park's 3D information on a display device allows users to easily confirm their destination (e.g., a sculpture within the park) from the park's 3D information, thus facilitating their journey to that destination.
  • POI Point of Interest
  • obtaining the three-dimensional information of the first POI sent by the cloud server includes: obtaining the three-dimensional information of the first POI sent by the cloud server and the surrounding environment of the first POI.
  • a display device can show three-dimensional information about the elevator lobby and its surrounding environment. This allows users to easily see the information of the nearest parking space to the elevator lobby entrance, thus facilitating their driving to that parking space. This avoids users relying on experience to find the elevator lobby entrance in the underground parking garage, improving the efficiency of driving from their current location to their destination.
  • the vehicle can automatically drive itself from its current location to the parking space. This avoids the need for the user to take over the vehicle and manually drive it to the elevator lobby based on their experience after the vehicle has automatically driven to the entrance of the underground parking garage (the in-vehicle map application can include the location of the entrance but not the location of the elevator lobby). This helps to expand the capabilities of autonomous driving and thus improves the user's intelligent driving experience.
  • the three-dimensional information of the first POI is determined by a cloud server based on images uploaded by one or more vehicles when they are around the first POI and the pose information of those images.
  • the cloud server can perform one or more of the following operations to obtain the 3D information of the first POI: segmenting the image uploaded when one or more vehicles are around the first POI to obtain common objects in different regions of the image and their categories; extracting text from the image using text extraction technology and matching the text content with the common objects; reconstructing and locating the environment based on the image and the pose information corresponding to the image; and performing super-resolution reconstruction of the image based on the image, the image's depth information, and the image's pose information to achieve enhancement from low-quality image to high-quality image.
  • the method further includes: upon detecting a user's first touch operation on the 3D model, controlling the display device to show the 3D model rotating or translating.
  • the first touch operation is a swipe operation.
  • the method further includes switching from displaying an image in the frontal view orientation of a first position in the 3D information to displaying an image in the frontal view orientation of a second position in the 3D information, based on the sliding distance of the user's finger on the display screen.
  • the distance between the first position and the second position can be determined by the sliding distance.
  • the method further includes: upon detecting a second touch operation by the user on the 3D model, controlling the display device to display an image of a frontal view at a certain location in the 3D model.
  • the second touch operation can be a click operation.
  • the method further includes: controlling the display device to display first prompt information, the first prompt information being used to request the user to confirm whether to navigate to the first POI; and, upon receiving a first instruction from the user, controlling the display device to display navigation information from the current location of the terminal device to the first POI, the first instruction indicating navigation to the first POI.
  • the vehicle can detect the accuracy of the 3D information of the first POI constructed by the cloud server through user feedback. If the user reports navigation to the first POI, the accuracy of the first POI information can be confirmed, and navigation information from the current location to the first POI can be displayed.
  • the terminal device is a vehicle
  • the method further includes: controlling the vehicle to travel from the current location to the first POI based on the navigation information.
  • the vehicle can autonomously drive from its current location to the first POI. This avoids the need for the user to take over the vehicle and manually drive it to the first POI based on their experience after the vehicle has autonomously driven to a location near the first POI (which is a POI in the map application). This helps to expand the boundaries of the vehicle's autonomous driving capabilities, thereby improving the user's intelligent driving experience.
  • the first POI is a POI that is not included in the map application.
  • controlling the vehicle to travel from its current location to the first POI based on the navigation information includes: controlling the vehicle to travel from its current location to the nearest drop-off point to the first POI based on the navigation information.
  • the three-dimensional information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the method further includes: when an operation by a user to select a first sub-region from the plurality of sub-regions is detected, controlling the display device to display navigation information from the current position of the terminal device to the first sub-region.
  • the user after viewing the 3D information, the user can select a sub-region (navigation destination) from multiple sub-regions, and the terminal device can then display navigation information from the current location to that sub-region.
  • the first POI is a park
  • the first sub-region is a building within the park.
  • the first POI is an underground parking lot
  • the first sub-area is a parking space.
  • the first POI is a charging area
  • the first sub-area is a charging station.
  • the three-dimensional information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the method further includes: controlling the display device to recommend a first sub-region to the user and controlling the display device to display second prompt information, the second prompt information being used to request the user to confirm whether to navigate to the first sub-region, the plurality of sub-regions including the first sub-region; and, upon receiving a second instruction from the user, controlling the display device to display navigation information from the current location of the terminal device to the first sub-region, the second instruction being used to instruct navigation to the first sub-region.
  • the vehicle can recommend a sub-region as a navigation destination to the user using 3D information. After the user confirms the navigation destination, the vehicle can then display navigation information from the current location to that sub-region.
  • the first sub-region is a parking space.
  • obtaining the three-dimensional information of a first point of interest (POI) sent by a cloud server includes: obtaining the three-dimensional information of the first POI and its surrounding environment sent by the cloud server; the method further includes: controlling a display device to recommend the nearest parking space to the first POI to the user based on the three-dimensional information.
  • POI point of interest
  • the terminal device is a vehicle
  • the method further includes: controlling the vehicle to travel from the current location to the first sub-area based on the navigation information.
  • the three-dimensional information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the method further includes: controlling the display device to recommend a second sub-region to the user and controlling the display device to display a third prompt message, the third prompt message being used to request the user to confirm whether to navigate to the second sub-region, the plurality of sub-regions including the second sub-region; upon obtaining the user's third instruction, sending a fourth instruction to the cloud server, the third instruction being used to indicate not to navigate to the second sub-region, the fourth instruction being used to instruct the cloud server to re-identify the three-dimensional information or the second sub-region.
  • the method further includes: controlling the display device to display a fourth prompt message.
  • the fourth prompt is used to request the user to select from the plurality of sub-regions; upon receiving the user's fifth instruction, the display device is controlled to display navigation information from the current location of the terminal device to the third sub-region, the fifth instruction instructing the user to select the third sub-region, the plurality of sub-regions including the third sub-region.
  • the terminal device is a vehicle
  • the method further includes: controlling the vehicle to travel from the current location to the third sub-region based on the navigation information.
  • the first POI is a parking lot or a charging area.
  • an image display device comprising: an acquisition unit for acquiring three-dimensional information of a first point of interest (POI) sent by a cloud server, the three-dimensional information being determined by images captured by cameras when one or more vehicles are around the first POI and the pose information corresponding to the images; and a control unit for controlling the display device to display the three-dimensional information.
  • POI point of interest
  • control unit is further configured to: control the display device to display a first prompt message, the first prompt message being used to request the user to confirm whether to navigate to the first POI; and when the acquisition unit acquires a first instruction from the user, control the display device to display navigation information from the current location of the terminal device to the first POI, the first instruction indicating navigation to the first POI.
  • the terminal device is a vehicle
  • the control unit is further configured to: control the vehicle to travel from the current location to the first POI based on the navigation information.
  • the three-dimensional information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the control unit is further configured to: control the display device to recommend a first sub-region to the user and control the display device to display a second prompt message, the second prompt message being used to request the user to confirm whether to navigate to the first sub-region, the plurality of sub-regions including the first sub-region; and when the acquisition unit acquires a second instruction from the user, control the display device to display navigation information from the current location of the terminal device to the first sub-region, the second instruction being used to instruct navigation to the first sub-region.
  • the terminal device is a vehicle
  • the control unit is further configured to: control the vehicle to travel from the current location to the first sub-area based on the navigation information.
  • the three-dimensional information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the control unit is further configured to control the display device to recommend a second sub-region to the user and to control the display device to display a third prompt message, the third prompt message being used to request the user to confirm whether to navigate to the second sub-region.
  • the plurality of sub-regions include the second sub-region.
  • the device further includes: a sending unit, configured to send a fourth instruction to the cloud server when the acquisition unit obtains the user's third instruction.
  • the third instruction is used to indicate not to navigate to the second sub-region, and the fourth instruction is used to instruct the cloud server to re-identify the three-dimensional information or the second sub-region.
  • control unit is further configured to control the display device to display a fourth prompt message, the fourth prompt message being used to request the user to select from the plurality of sub-regions; when the acquisition unit acquires the user's fifth instruction, it controls the display device to display navigation information from the current location of the terminal device to the third sub-region, the fifth instruction instructing the user to select the third sub-region, the plurality of sub-regions including the third sub-region.
  • the first POI is a parking lot or a charging area.
  • an image display device including a memory and a processor, the memory for storing a computer program and the processor for executing the computer program in the memory, such that the device can implement the method in any of the possible implementations of the first aspect described above.
  • a vehicle that includes the device possible in either the second or third aspect described above, or includes the system described in the fourth aspect described above.
  • vehicle in this application is used in a broad sense and can refer to means of transportation (such as commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.), industrial vehicles (such as forklifts, trailers, tractors, etc.), engineering vehicles (such as excavators, bulldozers, cranes, etc.), agricultural equipment (such as lawnmowers, harvesters, etc.), amusement equipment, toy vehicles, etc.
  • transportation such as commercial vehicles, passenger cars, motorcycles, flying cars, trains, etc.
  • industrial vehicles such as forklifts, trailers, tractors, etc.
  • engineering vehicles such as excavators, bulldozers, cranes, etc.
  • agricultural equipment such as lawnmowers, harvesters, etc.
  • amusement equipment toy vehicles, etc.
  • toy vehicles etc.
  • the embodiments of this application do not specifically limit the type of vehicle.
  • a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the method in any possible implementation of the first aspect.
  • FIG. 3 shows a set of user graphical interfaces (GUIs) provided in an embodiment of this application.
  • GUIs user graphical interfaces
  • Figure 6 is another GUI provided in an embodiment of this application.
  • Figure 9 is a schematic flowchart of the image display method provided in an embodiment of this application.
  • Figure 10 is a schematic block diagram of an image display device provided in an embodiment of this application.
  • A/B can mean A or B;
  • and/or in this document is merely a description of the association relationship between related objects, indicating that three relationships can exist.
  • a and/or B can represent: A existing alone, A and B existing simultaneously, and B existing alone.
  • At least one refers to one or more.
  • at least one of A and B similar to "A and/or B," describes the association relationship between related objects, indicating that three relationships can exist.
  • at least one of A and B can represent: A existing alone, A and B existing simultaneously, and B existing alone.
  • prefixes such as “first” and “second” used in this application embodiment are merely for distinguishing different descriptive objects and do not limit the position, order, priority, quantity, or content of the described objects.
  • the use of ordinal numbers and other prefixes used to distinguish descriptive objects in this application embodiment does not constitute a limitation on the described objects.
  • the description of the described objects is given in the claims or the context of the embodiments, and should not constitute unnecessary restrictions due to the use of such prefixes.
  • “multiple" means two or more.
  • FIG. 1 is a functional block diagram of a vehicle 100 provided in an embodiment of this application.
  • the vehicle 100 may include a sensing system 110, a computing platform 120, and a display device 130.
  • the sensing system 110 may include one or more sensors for sensing information about the environment surrounding the vehicle 100.
  • the sensing system 110 may include a positioning system, which may be a Global Positioning System (GPS), a BeiDou system, or another positioning system.
  • the sensing system 110 may include an inertial measurement unit (IMU), an accelerometer, a lidar, a millimeter-wave radar, an ultrasonic radar, and a camera device.
  • IMU inertial measurement unit
  • Computing platform 120 may include one or more processors, such as processors 121 to 12n (n is a positive integer).
  • a processor is a circuit with signal processing capabilities.
  • the processor may be a circuit with instruction read and execute capabilities, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which can be understood as a type of microprocessor), or a digital signal processor (DSP).
  • the processor can implement certain functions through the logical relationship of hardware circuits. The logical relationship of the hardware circuits is fixed or reconfigurable.
  • the processor is a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as a field-programmable gate array (FPGA).
  • ASIC application-specific integrated circuit
  • PLD programmable logic device
  • FPGA field-programmable gate array
  • the process of a processor loading a configuration document and configuring the hardware circuit can be understood as the process of a processor loading instructions to implement the functions of some or all of the aforementioned units.
  • the processor can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as a neural network processing unit (NPU), tensor processing unit (TPU), or deep learning processing unit (DPU).
  • the computing platform 120 may also include a memory for storing instructions. Some or all of the processors 121 to 12n can call the instructions in the memory to implement the corresponding functions.
  • the in-cabin display devices 130 are mainly divided into two categories: the first is the in-vehicle display screen; the second is the projection display screen, such as the head-up display (HUD).
  • An in-vehicle display screen is a physical display screen and an important component of the in-vehicle infotainment system. Multiple displays can be installed in the cabin, such as the digital instrument cluster display, the central control screen, the display screen in front of the front passenger (also known as the front-seat passenger), the display screen in front of the left rear passenger, the display screen in front of the right rear passenger, and even the car windows can be used as displays.
  • a head-up display also known as a head-up display system, is mainly used to display driving information such as speed and navigation on a display device in front of the driver (such as the windshield). This reduces the driver's eye-shift time, avoids pupil changes caused by eye-shifting, and improves driving safety and comfort.
  • HUDs include, for example, combiner-HUD (C-HUD) systems, windshield-HUD (W-HUD) systems, and augmented reality HUD (AR-HUD) systems. It should be understood that other types of HUD systems may emerge as technology evolves, and this application does not limit them.
  • the display device 130 uses an in-vehicle display screen and a projection display screen as examples, but the embodiments of this application are not limited thereto.
  • the display device 130 can also be a light display screen or a projection screen.
  • the structure of the vehicle 100 described above is merely illustrative. In actual applications, various components of the vehicle 100 may be added or removed as needed.
  • Vehicle 100 may include an intelligent driving system, which may include an advanced driving assistance system (ADAS) and an autonomous driving system (ADS).
  • the intelligent driving system uses various sensors on the vehicle (including but not limited to: lidar, millimeter-wave radar, camera devices, ultrasonic sensors, global positioning system, inertial measurement unit) to acquire information from the vehicle's surroundings, and analyzes and processes the acquired information to achieve functions such as obstacle perception, target recognition, vehicle positioning, path planning, and driver monitoring/alert, thereby improving the safety, automation, and comfort of driving the vehicle.
  • sensors on the vehicle including but not limited to: lidar, millimeter-wave radar, camera devices, ultrasonic sensors, global positioning system, inertial measurement unit
  • FIG. 2 shows a schematic block diagram of an intelligent driving system provided in an embodiment of this application.
  • the intelligent driving system may include three functional modules: a perception module 210, a planning module 220, and a control module 230.
  • the perception module 210 perceives the environment surrounding the vehicle through sensors and outputs corresponding perception data to the planning module 220.
  • the planning module 220 obtains information such as road topology and target objects based on the information acquired by the perception module 210. Based on the road topology and target object information, the planning module 220 can determine a planned trajectory over a period of time.
  • the planning module 220 can send this planned trajectory to the control module 230.
  • the control module 230 can output control signals to control the actuators to take corresponding actions, such as steering, acceleration, and deceleration.
  • the above-mentioned sensing module 210 can be the above-mentioned sensing system 110, and the planning module 220 and the control module 230 can be located in the above-mentioned computing platform 120.
  • Vehicle-based driving automation systems are classified into five levels (or L0-L5) based on the degree to which they can perform dynamic driving tasks, according to the role allocation in performing these tasks and the presence or absence of an operational design domain (ODD), such as the external conditions (road, traffic, weather, lighting, etc.) defined during the system's design.
  • Levels 0-2 represent driver assistance, where the system assists humans in performing dynamic driving tasks, but the driver remains the primary driver.
  • Levels 3-5 represent autonomous driving, where the system performs dynamic driving tasks in place of the human under the designed operating conditions; when activated, the system becomes the primary driver.
  • ODD operational design domain
  • Level 0 driving automation also known as emergency assistance
  • Level 1 Level 1 driving automation continuously performs lateral or longitudinal motion control of the vehicle during dynamic driving tasks under its design operating conditions, and possesses the ability to detect and respond to some targets and events adapted to the lateral or longitudinal motion control performed.
  • Level 2 driving automation also known as combined driver assistance continuously performs lateral and longitudinal motion control of the vehicle during dynamic driving tasks under its design operating conditions, and possesses the ability to detect and respond to some targets and events adapted to the lateral and longitudinal motion control performed.
  • Level 3 driving automation continuously performs all dynamic driving tasks under its design operating conditions.
  • Level 4 driving automation also known as highly automated driving continuously performs all dynamic driving tasks under its design operating conditions and automatically executes a minimum risk strategy.
  • Level 5 driving automation also known as fully automated driving continuously performs all dynamic driving tasks under any drivable conditions and automatically executes a minimum risk strategy.
  • intelligent driving systems fall into the L2-L5 category, such as ADAS (Advanced Driver Assistance Systems) which is L2 and ADS (Advanced Driver Assistance Systems) which is L3-L5.
  • POIs can only be presented in the form of text or planar information.
  • This application provides an image display method, apparatus, and vehicle that displays 3D information of a Point of Interest (POI) through a terminal device, enabling users to have a comprehensive understanding of the POI and facilitating destination confirmation.
  • POI Point of Interest
  • FIG. 3 illustrates a set of graphical user interfaces (GUIs) provided in embodiments of this application.
  • GUIs graphical user interfaces
  • search results box 302 when vehicle 200 detects that "Elevator Hall 1, basement Level of Shopping Mall A" is entered into the search box 301 of the in-vehicle map application, three search results can be displayed in the search results box 302, such as the North Gate of Shopping Mall A, the South Gate of Shopping Mall A, and the car wash on the basement level of Shopping Mall A. At this time, the search results box 302 does not include the search result "Elevator Hall 1, Basement Level of Shopping Mall A". The user needs to manually drive the vehicle to find the location of Elevator Hall 1 after entering the basement level of Shopping Mall A.
  • vehicle 200 when vehicle 200 detects that a user has driven the vehicle into the underground parking lot of shopping mall A and parked it near elevator hall number 1, vehicle 200 can collect environmental information about the surrounding environment through sensors outside the cabin. For example, vehicle 200 can collect images through a camera outside the cabin. Vehicle 200 can then send the image and its corresponding pose information to a cloud server.
  • the pose information corresponding to the above images can be the pose information of the vehicle 200 (or the camera of the vehicle 200) when the image is captured, or it can be the pose information of the image itself.
  • vehicle 200 can collect point cloud data using lidar or millimeter-wave radar outside the cockpit. Vehicle 200 can then send this point cloud data, along with its pose information during the data collection, to a cloud server.
  • the vehicle 200 when it detects a user's parking operation, it can send the data collected by the external sensors in the cabin and the corresponding pose information to the cloud server; or, the vehicle 200 can send the data collected by the external sensors in the cabin and the corresponding pose information to the cloud server within a preset time period before detecting the user's parking operation.
  • the vehicle 200 when the vehicle 200 detects that a user is searching for information about the first POI through an in-vehicle map application and the in-vehicle map application does not provide search results for the first POI, it can send the data collected by the external sensors and the corresponding pose information to the cloud server.
  • vehicle 200 detects that a user searches for "Elevator Hall 1, Shopping Mall A” through an in-vehicle map application, but the search results provided by the in-vehicle map application do not include "Elevator Hall 1, Shopping Mall A”.
  • vehicle 200 detects that the user has parked, it can send an image of the area near the parking spot, along with the vehicle 200's pose information at the time the image was captured, to a cloud server.
  • the vehicle 200 when the vehicle 200 detects that the user has issued a voice command to find the second POI, it can send the image captured by the external camera and the vehicle 200's pose when the image was captured to the cloud server.
  • the information collection process involved in the embodiments of this application (such as image data collection, point cloud data collection, voice collection, and user search data collection in vehicle map applications) is performed with the user's knowledge and permission, that is, the information collection process complies with the requirements of laws and regulations and does not constitute an act that harms the public interest.
  • the cloud server can process the information sent by vehicle 200.
  • the cloud server can segment the image using an image segmentation algorithm and perform category matching on the segmentation results. As shown in Figure 3(c), after segmenting the image and matching the categories, the cloud server can establish the association between different regions in the image and their corresponding categories (e.g., parking space, wall, column, elevator hall automatic door, ground).
  • categories e.g., parking space, wall, column, elevator hall automatic door, ground.
  • the cloud server can also recognize text in images using text recognition algorithms. For example, it can recognize the text "Elevator Hall No. 1" from an image uploaded by vehicle 200. The cloud server can then determine the association between the recognized text content and the aforementioned regions and categories. Verification and fusion are performed. When the cloud server determines that the text content "Elevator Hall No. 1" is located near the automatic door of the elevator hall, it can confirm that the automatic door in the image is the automatic door of Elevator Hall No. 1.
  • the cloud server can also enrich and update existing POIs in the base map of the in-vehicle map application.
  • the cloud server can match newly identified POIs (e.g., "Elevator Lobby 1") with existing POIs near the user's parking location, updating and supplementing them based on the existing POIs.
  • the in-vehicle map application includes the location information of a POI but excludes the 3D information of the POI and its surrounding environment.
  • the cloud server can process images collected by one or more vehicles around the POI and the corresponding pose information to obtain the 3D information of the POI.
  • the cloud server can then associate the POI's location information with its 3D information.
  • the cloud server can send the associated POI's location information and 3D information to vehicle 100; or, the cloud server can send the POI's 3D information to vehicle 100, allowing the vehicle to associate the POI's location information with its 3D information.
  • vehicle 200 can send multiple frames of images (or videos) of the area surrounding elevator hall 1, along with the pose information corresponding to each frame, to a cloud server.
  • the cloud server can then perform environment reconstruction based on the multiple frames of images sent by vehicle 200 and the pose information corresponding to each frame.
  • three-dimensional information of elevator hall 1 and its surrounding environment is obtained.
  • users can continuously rotate or pan to view the three-dimensional information of elevator hall 1 and its surrounding environment.
  • the cloud server can perform super-resolution reconstruction of the image based on multiple frames of images, the depth information of each frame in the multiple frames of images, and the pose information corresponding to each frame of images, thereby enhancing the image from low quality to high quality.
  • the above embodiments illustrate the sending of multiple frames of images and pose information corresponding to each frame by vehicle 200 to a cloud server.
  • the cloud server can receive images and pose information of elevator hall 1 and its surroundings from multiple vehicles (which can be understood as crowdsourced vehicles), and construct three-dimensional information of elevator hall 1 and its surrounding environment based on the images and pose information sent by the multiple vehicles.
  • a low-cost crowdsourced POI recommendation layer generation method provides users with a user-friendly and interactive way to view 3D POI information, enhancing the user experience when using navigation functions. Simultaneously, for vehicles with intelligent driving capabilities, it can expand the boundaries of intelligent driving capabilities.
  • vehicle 100 when vehicle 100 detects that "A Shopping Mall" is entered through the search box 303 of the in-vehicle map application, it can display four search results in the search results box 304, such as North Gate of A Shopping Mall, South Gate of A Shopping Mall, Car Wash on Basement Level of A Shopping Mall, and Elevator Hall No. 1 on Basement Level of A Shopping Mall.
  • search results box 304 such as North Gate of A Shopping Mall, South Gate of A Shopping Mall, Car Wash on Basement Level of A Shopping Mall, and Elevator Hall No. 1 on Basement Level of A Shopping Mall.
  • vehicle 100 can display the GUI shown in Figure 3(e) on the display screen.
  • vehicle 100 in response to detecting a user's selection of "Elevator Hall 1, Basement Level, Shopping Mall A", can display an image from a certain perspective of the 3D information of Elevator Hall 1 and its surrounding environment. This image includes an arrow 305 pointing to the entrance of Elevator Hall 1. Simultaneously, vehicle 100 can also display a prompt box 306 on its screen, containing the message "Please confirm whether you want to navigate to Elevator Hall 1?". When a user clicks control 307, vehicle 100 can display the GUI shown in Figure 3(f) on its screen.
  • vehicle 100 in response to detecting a user click on control 307, can plan a driving route from its current location to elevator hall 1 on the basement level of shopping mall A. Simultaneously, vehicle 100 can display a prompt box 308 on its screen, which includes the message "A driving route from your current location to elevator hall 1 on the basement level of shopping mall A has been planned for you.”
  • the vehicle 100 when the vehicle 100 detects that the user has activated the intelligent driving function, it can control the vehicle 100 to drive automatically from its current location to the No. 1 elevator hall according to the driving path.
  • vehicle 100 can also prompt the user to select a drop-off location. For example, when detecting a user's click on a parking space, the vehicle can plan a driving trajectory from its current location to that parking space.
  • the vehicle when the vehicle detects a user clicking on a pillar, it can plan a driving trajectory from its current location to that pillar.
  • the vehicle detects that the user has activated the intelligent driving function, it can control itself to automatically drive from its current location to the vicinity of the pillar based on this driving path. After detecting that the user has alighted, the vehicle can automatically park in any suitable parking space.
  • Figure 4 illustrates another set of GUIs provided in an embodiment of this application.
  • vehicle 100 detects the user's input of "B Shopping Mall" through the search box 401 of the in-vehicle map application.
  • search results box 402 When the user selects the parking lot on the second basement level of B shopping mall, four search results can be displayed through the search results box 402, including the north gate of B shopping mall, the south gate of B shopping mall, the underground parking lot on the first basement level of B shopping mall, and the underground parking lot on the second basement level of B shopping mall.
  • the GUI shown in Figure 4(b) can be displayed on the screen.
  • the underground parking lots on the first and second basement levels of Shopping Mall B can be Points of Interest (POIs) obtained through the aforementioned low-cost crowdsourcing method.
  • the cloud server can construct the 3D information of the underground parking lot on the second basement level of Shopping Mall B based on images uploaded by multiple vehicles and the corresponding pose information.
  • vehicle 100 in response to detecting that the user has selected the underground parking level 2 of Shopping Mall B, vehicle 100 can display three-dimensional information of the underground parking level 2 of Shopping Mall B on the display screen. The user can click, drag, zoom, and rotate the three-dimensional information.
  • a user can switch to displaying the image from the front view of that location.
  • a user can view it from different perspectives.
  • zooming in or out a user can view the 3D information at closer or farther distances.
  • the vehicle 100 in response to detecting a user's click operation on a location in the 3D information, the vehicle 100 can display an image of the front view of that location on the display screen.
  • a GUI as shown in Figure 4(d) can be displayed.
  • Both (b) and (c) in Figure 4 above can be referred to as the three-dimensional information of the underground parking lot on the first basement level of Shopping Mall B.
  • the GUI in Figure 4(b) displays the three-dimensional model of the underground parking lot on the second basement level of Shopping Mall B
  • the GUI in Figure 4(c) displays the front view image of a certain location in the three-dimensional model.
  • the vehicle 100 in response to detecting a user's swipe operation on the display screen, the vehicle 100 can display an image in a frontal view direction at another location on the display screen.
  • the vehicle 100 can highlight the parking space on the display screen and display the GUI shown in Figure 4(e).
  • vehicle 100 in response to detecting a user clicking on a parking space in the image, vehicle 100 can display a prompt box 403 on the screen, which includes the prompt message "Would you like to navigate to this parking space?".
  • the GUI shown in Figure 4(f) can be displayed.
  • vehicle 100 in response to detecting a user's click on control 404, can plan a driving route from its current location to the parking space. Simultaneously, vehicle 100 can display a prompt box 405 on its screen, which includes the message "A driving route from your current location to your selected parking space has been planned for you.”
  • the POI Point of Interest
  • the POI is an underground parking lot, which may include multiple parking spaces. Users can view the 3D information of the underground parking lot to gain a comprehensive understanding and select their preferred parking space (e.g., the parking space closest to the elevator entrance).
  • the 3D information of the underground parking lot constructed by the cloud server allows the vehicle 100 to automatically drive from its current location to a specific parking space after activating its intelligent driving function. This avoids the process of the user taking over the vehicle after reaching the entrance and then manually driving to a parking space based on experience, thus helping to improve the boundaries of the vehicle's autonomous driving capabilities and enhancing the user's intelligent driving experience.
  • Figure 5 illustrates another set of GUIs provided in an embodiment of this application.
  • the vehicle 100 when the vehicle 100 detects that the user has entered "C Park" through the search box 501 of the in-vehicle map application, it can display four search results through the search results box 502, including C Park North Gate, C Park South Gate, C Park Charging Area, and C Park Parking Lot.
  • the vehicle detects that the user has selected C Park Charging Area, it can display the GUI shown in Figure 5(b) on the display screen.
  • the C Park charging area described above can be a Point of Interest (POI) obtained through the aforementioned low-cost crowdsourcing method.
  • a cloud server can construct 3D information of the C Park charging area based on images of the C Park charging area uploaded by multiple vehicles and the corresponding pose information.
  • vehicle 100 in response to the user's selection of the C park charging area, vehicle 100 can highlight a charging station on the display screen and display a prompt box 503, which includes the prompt message "Navigate to this charging station?". From this 3D information, it can be seen that the pillar behind the highlighted parking space is mistakenly identified as a charging station by the cloud server.
  • the GUI shown in Figure 5(c) can be displayed.
  • the vehicle 100 may also prompt the user to confirm that the charging station is incorrectly identified, or that the charging station is correctly identified and the user wishes to change to a different charging station.
  • the GUI shown in Figure 5(c) can be displayed on the screen.
  • the GUI shown in Figure 5(d) can be displayed on the screen.
  • vehicle 100 may send an instruction to the cloud server to instruct the reconstruction of the three-dimensional information of the C Park charging area, or to instruct the re-identification of the charging pile shown in Figure 5(b).
  • vehicle 100 in response to detecting a user click on control 504, can display a prompt box 505, which includes the message "Is there information about a charging station?".
  • a user click on control 506 the GUI shown in Figure 5(d) can be displayed.
  • vehicle 100 in response to detecting a user's click on control 506, can switch the display screen to show an image of a front view of a location in the C Park charging area and display a prompt box 507, which includes the message "Please select a charging station".
  • a prompt box 507 which includes the message "Please select a charging station”.
  • vehicle 100 can display navigation information from the vehicle's current location to the parking space on the screen.
  • Figure 7 shows a schematic diagram of the system architecture provided in an embodiment of this application.
  • Vehicle 1 can be the vehicle used for data collection
  • Vehicle 2 can be the vehicle used for data collection.
  • the vehicle used for data collection can also be called a crowdsourced vehicle, and there can be multiple crowdsourced vehicles.
  • a cloud server can use detection large models to identify general objects, such as the segment anything model (SAM). This involves segmenting and detecting all content in an image and performing category matching to achieve content detection.
  • SAM segment anything model
  • a cloud server can enrich and update existing Points of Interest (POIs) on a base map.
  • POIs Points of Interest
  • an in-vehicle map application might include the location information of a POI but not the 3D information of that POI and its surrounding environment.
  • the cloud server can process images captured by one or more vehicles near that POI to obtain its 3D information.
  • the cloud server can then associate the POI's location information with this 3D information.
  • the cloud server can perform Gaussian spatting environment reconstruction on the images sent by vehicle 1 and the pose of the images. Based on the images from sparse perspectives and the corresponding pose input, it can realize dense and continuous rotation and translation viewing in three dimensions.
  • the cloud server can perform super-resolution reconstruction of the image based on the image sent by vehicle 1, the depth information of the image, and the pose information corresponding to the image, thereby enhancing the image from low quality to high quality.
  • Figure 8 shows a schematic diagram of a human-machine interaction process provided in an embodiment of this application.
  • This human-machine interaction process can be executed by either vehicle 100 or vehicle 2.
  • the human-machine interaction process includes:
  • determining whether the recommended sub-region in the POI is accurate includes: determining whether the recommended sub-region is accurate based on user feedback. If the recommended sub-region is inaccurate, proceed to step S802; if the recommended sub-region is accurate, then the recommended sub-region can be used as the navigation destination and step S805 can be executed.
  • vehicle 2 can control the prompting device to display the recommended sub-area to the user and prompt the user to determine whether the recommended sub-area is accurate, or control the prompting device to prompt the user to determine whether to navigate to the recommended sub-area.
  • the vehicle 100 may also prompt the user to confirm that the charging station is incorrectly identified, or that the charging station is correctly identified and the user wishes to replace it with a different charging station.
  • steps S802 and S804 can be executed.
  • S803 prompts the user to select a sub-region from multiple sub-regions in the POI.
  • control display device displays navigation information from the current location to the navigation destination.
  • Figure 9 shows a schematic flowchart of an image display method 900 provided in an embodiment of this application.
  • the method 900 can be executed by the aforementioned vehicle 100 or the vehicle using the data; or, the method 900 can be executed by the aforementioned computing platform 120; or, the method 900 can be executed by a processor, chip, or circuit in the aforementioned computing platform 120.
  • the method 900 includes:
  • the three-dimensional information is determined by images captured by cameras when one or more vehicles are around the first POI and the pose information corresponding to the images.
  • the vehicle can obtain the three-dimensional information of the first POI from the cloud server through over-the-air (OTA) technology.
  • OTA over-the-air
  • control display device displays this three-dimensional information.
  • Points of Interest can only be presented as text or 2D information.
  • the search results may only include the park's entrance. If the user wants to get to a building within the park, they have to drive the vehicle to the park entrance and rely on their experience to manually locate the building. This increases the time it takes for the driver to navigate to the building, resulting in a poor driving experience.
  • the cloud server which may include images of various buildings in the park
  • users can have a comprehensive understanding of the park through this three-dimensional information, thereby facilitating users to confirm their destination (e.g., a building in the park) and helping to improve the user experience.
  • the method further includes: detecting a user's instruction to search for a first location.
  • obtaining the three-dimensional information of the first POI sent by the cloud server includes: obtaining the three-dimensional information of the first POI sent by the cloud server and the surrounding environment of the first POI.
  • the system when the system detects that the user enters “A Shopping Mall” in the search box 303 and clicks on the No. 1 elevator hall on the basement floor of A Shopping Mall in the search results, it can control the display screen to show the three-dimensional information of the No. 1 elevator hall and the surrounding environment of the No. 1 elevator hall.
  • the display device shows three-dimensional information of the elevator lobby and its surrounding environment. This allows users to easily view information about the nearest parking space to the elevator lobby entrance, thus facilitating their driving to that parking space. This avoids users relying on experience to find the elevator lobby entrance in the underground parking garage, improving the efficiency of driving from the current location to the destination.
  • the vehicle can automatically drive from its current location to the parking space. This avoids the process of the user taking over the vehicle after it has automatically driven to the entrance of the underground parking garage and then manually driving it to the parking space around the elevator lobby based on the user's experience. This helps to expand the boundaries of the vehicle's autonomous driving capabilities, thereby improving the user's intelligent driving experience.
  • the three-dimensional information of the first POI is determined by a cloud server based on images uploaded by one or more vehicles when they are around the first POI and the pose information of those images.
  • the cloud server can perform the following operations to obtain the 3D information of the first POI: segment the image uploaded when one or more vehicles are around the first POI to obtain the common objects in different regions of the image and their categories; extract the text in the image using text extraction technology and match the text content with the common objects; reconstruct and locate the environment based on the image and the pose information corresponding to the image; and perform super-resolution reconstruction of the image based on the image, the depth information of the image, and the pose information corresponding to the image to achieve the enhancement from low-quality image to high-quality image.
  • the method 900 further includes: when a user's first touch operation on the three-dimensional model is detected, controlling the display device to display the three-dimensional model as rotated or translated.
  • the method 900 further includes: when a second touch operation by the user on the three-dimensional model is detected, controlling the display device to display an image of the front view at a first position in the three-dimensional model.
  • the method 900 further includes: controlling the display device to display first prompt information, the first prompt information being used to request the user to confirm whether to navigate to the first POI; and, upon receiving a first instruction from the user, controlling the display device to display navigation information from the current location of the terminal device to the first POI, the first instruction indicating navigation to the first POI.
  • the vehicle can display a prompt message on the screen: "Please confirm whether you want to navigate to Elevator Hall 1?".
  • the vehicle can display navigation information from the current location to Elevator Hall 1 on the screen.
  • the vehicle can detect the accuracy of the 3D information of the first POI constructed by the cloud server. If the user reports navigation to the first POI, the accuracy of the first POI's information can be confirmed, and navigation information from the current location to the first POI can be displayed.
  • the terminal device is a vehicle
  • the method 900 further includes: controlling the vehicle to travel from the current location to the first POI based on the navigation information.
  • Elevator Hall 1 Taking the first Point of Interest (POI) as Elevator Hall 1 mentioned above, this Elevator Hall 1 could be located on the basement level of Shopping Mall A.
  • the in-vehicle map application might only include the entrance to the basement level of Shopping Mall A.
  • the vehicle could autonomously drive from its current location to this entrance. After reaching the entrance, the vehicle would prompt the user to take over, and the user would then manually drive the vehicle to the vicinity of Elevator Hall 1 based on their experience. This necessitates manual driving from the entrance to Elevator Hall 1, increasing the travel time and impacting the user's driving experience.
  • the cloud server by receiving the location and 3D information of Elevator Hall 1 sent by the cloud server, users can easily view the 3D information of Elevator Hall 1 and plan their driving trajectory from their current location to Elevator Hall 1.
  • the vehicle can autonomously drive from its current location to Elevator Hall 1 or the nearest parking space. This avoids the need for the user to take over the vehicle and manually drive it to Elevator Hall 1 based on their experience after the vehicle has autonomously driven to the entrance of the basement level of Shopping Mall A. This helps to expand the boundaries of the vehicle's autonomous driving capabilities, thereby improving the user's intelligent driving experience.
  • controlling the vehicle to travel from the current location to the first POI based on the navigation information includes: controlling the vehicle to travel from the current location to the nearest drop-off point to the first POI based on the navigation information.
  • the vehicle can automatically drive to the nearest alighting point to the elevator hall No. 1 (the pillar shown in Figure 3(e)). After detecting that the user has alighted at the nearest alighting point to the elevator hall No. 1, the vehicle can automatically park in any reasonable parking space.
  • the three-dimensional information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the method further includes: when detecting an operation by a user selecting a first sub-region from the plurality of sub-regions, controlling the display device to display from the terminal device. Navigation information from the current location to the first sub-region.
  • the user after viewing the three-dimensional information, the user can select a sub-area (navigation destination) from multiple sub-areas in the parking lot or charging area, so that the terminal device can display navigation information from the current location to the sub-area.
  • a sub-area novigation destination
  • the first POI is a park
  • the first sub-region is a building (e.g., a sculpture) in the park.
  • the first POI is an underground parking lot
  • the first sub-area is a parking space in the underground parking lot.
  • the first POI is a charging area
  • the first sub-area is a charging pile in the charging area.
  • the three-dimensional information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the method further includes: controlling the display device to recommend a first sub-region to the user and controlling the display device to display a second prompt message, the second prompt message being used to request the user to confirm whether to navigate to the first sub-region, the plurality of sub-regions including the first sub-region; and when a second instruction is obtained from the user, controlling the display device to display navigation information from the current location of the terminal device to the first sub-region, the second instruction being used to instruct navigation to the first sub-region.
  • the vehicle can display three-dimensional information about Elevator Hall 1 and its surrounding environment on a screen.
  • This first sub-area represents the parking space closest to Elevator Hall 1.
  • the three-dimensional information includes arrow 305, parking space information, pillar information, entrance information for Elevator Hall 1, and wall information.
  • Arrow 305 indicates the entrance to Elevator Hall 1.
  • the shaded area in the three-dimensional information indicates the parking space closest to Elevator Hall 1.
  • the terminal device is a vehicle
  • the method further includes: controlling the vehicle to travel from the current location to the first sub-area based on the navigation information.
  • the 3D information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the method further includes: controlling the display device to recommend a second sub-region to the user and controlling the display device to display a third prompt message, the third prompt message being used to request the user to confirm whether to navigate to the second sub-region, the plurality of sub-regions including the second sub-region; upon receiving the user's third instruction, sending a fourth instruction to the cloud server, the third instruction being used to indicate not to navigate to the second sub-region, the fourth instruction being used to instruct the cloud server to re-identify the 3D information or the second sub-region.
  • the first POI can be the charging area of Park C
  • the second sub-area can be the shaded area shown in Figure 5(b).
  • the vehicle detects a user searching for the charging area of Park C, it can display a prompt box 503 on the screen and recommend a charging station in the charging area to the user.
  • the prompt box 503 includes the prompt message "Navigate to this charging station?".
  • the vehicle detects a user clicking the control 504, it can send the fourth instruction to the cloud server.
  • the method 900 further includes: controlling the display device to display a fourth prompt message, the fourth prompt message being used to request the user to select from the plurality of sub-regions; and, upon receiving a fifth instruction from the user, controlling the display device to display navigation information from the current location of the terminal device to a third sub-region, the fifth instruction instructing the user to select the third sub-region, the plurality of sub-regions including the third sub-region.
  • the terminal device is a vehicle
  • the method further includes: controlling the vehicle to travel from the current location to the third sub-area based on the navigation information.
  • FIG. 10 shows a schematic block diagram of an image display device 1000 provided in an embodiment of this application.
  • the device 1000 includes: an acquisition unit 1010, configured to acquire three-dimensional information of a first point of interest (POI) sent by a cloud server, the three-dimensional information being determined by images captured by cameras when one or more vehicles are around the first POI and the pose information corresponding to the images; and a control unit 1020, configured to control a display device to display the three-dimensional information.
  • POI point of interest
  • control unit 1020 is further configured to: control the display device to display a first prompt message, the first prompt message being used to request the user to confirm whether to navigate to the first POI; and when the acquisition unit acquires the user's first instruction, control the display device to display navigation information from the current location of the terminal device to the first POI, the first instruction indicating navigation to the first POI.
  • the terminal device is a vehicle
  • the control unit 1020 is further configured to: control the vehicle to travel from the current location to the first POI based on the navigation information.
  • the three-dimensional information includes an image of each sub-region in the plurality of sub-regions and pose information of the image of each sub-region.
  • the device 1000 further includes: a detection unit for detecting an operation by which a user selects a first sub-region from the plurality of sub-regions; and a control unit 1020 for controlling the display device to display navigation information from the current location of the terminal device to the first sub-region in response to detecting the operation of selecting the first sub-region.
  • the three-dimensional information includes an image of each sub-region in a plurality of sub-regions and pose information of the image of each sub-region.
  • the control unit 1020 is further configured to: control the display device to recommend a first sub-region to the user and control the display device to display a second prompt message, the second prompt message being used to request the user to confirm whether to navigate to the first sub-region, the plurality of sub-regions including the first sub-region; when the acquisition unit 1010 acquires a second instruction from the user, control the display device to display navigation information from the current location of the terminal device to the first sub-region, the second instruction being used to instruct navigation to the first sub-region.
  • the first sub-area can be a parking space.
  • the terminal device is a vehicle
  • the control unit 1020 is further configured to: control the vehicle to travel from the current location to the first sub-area based on the navigation information.
  • the three-dimensional information includes an image of each sub-region in multiple sub-regions and pose information of the image of each sub-region.
  • the control unit 1020 is further configured to control the display device to recommend a second sub-region to the user and to control the display device to display a third prompt message.
  • the third prompt message is used to request the user to confirm whether to navigate to the second sub-region.
  • the multiple sub-regions include the second sub-region.
  • the device 1000 further includes a sending unit, configured to send a fourth instruction to the cloud server when the acquisition unit obtains the user's third instruction.
  • the third instruction is used to indicate not to navigate to the second sub-region, and the fourth instruction is used to instruct the cloud server to re-identify the three-dimensional information or the second sub-region.
  • control unit 1020 is further configured to control the display device to display a fourth prompt message, which requests the user to select from the plurality of sub-regions; when the acquisition unit 1010 acquires the user's fifth instruction, it controls the display device to display navigation information from the current location of the terminal device to the third sub-region, which instructs the user to select the third sub-region, and the plurality of sub-regions includes the third sub-region.
  • the terminal device is a vehicle
  • the control unit 1020 is further configured to control the vehicle to travel from the current location to the third sub-area based on the navigation information.
  • the first POI can be a parking lot or a charging area.
  • the functions implemented by the acquisition unit 1010 can be implemented by the processor, circuit, or chip in the computing platform 120. Taking the function implemented by the acquisition unit 1010 as an example, the processor 121 can acquire the three-dimensional information of the POI.
  • control unit 1020 can be implemented by the processor, circuit, or chip in the computing platform 120.
  • the processor 122 can control the display device to display the three-dimensional information of the POI.
  • the functions implemented by the acquisition unit 1010 and the control unit 1020 can be implemented by the same processor or by different processors. This application embodiment does not specifically limit this.
  • the division of units in the above device is only a logical functional division. In actual implementation, they can be fully or partially integrated into a single physical entity, or they can be physically separated.
  • the units in the device can be implemented by a processor calling software; for example, the device includes a processor connected to memory, which stores instructions. The processor calls the instructions stored in memory to implement any of the above methods or to implement the functions of each unit in the device.
  • the processor can be, for example, a general-purpose processor, such as a CPU or microprocessor, and the memory can be internal or external to the device.
  • the units in the device can be implemented as hardware circuits. The functions of some or all units can be implemented through the design of the hardware circuits, which can be understood as one or more processors.
  • the hardware circuit is an ASIC, and the functions of some or all units are implemented through the design of the logical relationships between the components within the circuit.
  • the hardware circuit can be implemented using a PLD, such as an FPGA, which can include a large number of logic gates. The connection relationships between the logic gates are configured through configuration files, thereby implementing the functions of some or all units. All units of the above devices can be implemented entirely through processor calling software, or entirely through hardware circuits, or partially through processor calling software with the remaining parts implemented through hardware circuits.
  • a processor is a circuit with signal processing capabilities.
  • the processor can be a circuit with instruction reading and execution capabilities, such as a CPU, microprocessor, GPU, or DSP.
  • the processor can implement certain functions through the logical relationships of hardware circuits. These logical relationships are fixed or reconfigurable.
  • the processor may be a hardware circuit implemented as an ASIC or PLD, such as an FPGA.
  • the process of the processor loading a configuration document and configuring the hardware circuit can be understood as the processor loading instructions to implement the functions of some or all of the above units.
  • it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, or DPU.
  • each unit in the above device can be one or more processors (or processing circuits) configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.
  • processors or processing circuits configured to implement the above methods, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.
  • the units in the above devices can be integrated in whole or in part, or they can be implemented independently. In one implementation, these units are integrated together as a System-on-a-Chip (SoC).
  • SoC may include at least one processor for implementing any of the above methods or implementing the functions of the units in the device.
  • the at least one processor may be of different types, such as CPU and FPGA, CPU and AI processor, CPU and GPU, etc.
  • This application also provides an image display device, which includes a processing unit and a storage unit.
  • the storage unit is used to store instructions, and the processing unit executes the instructions stored in the storage unit to cause the device to perform the methods or steps described in the above embodiments.
  • the processing unit may be one or more of the processors 121-12n shown in FIG1.
  • This application also provides an image display system, which includes a display device and a computing platform, the computing platform including the image display device 1000 described above.
  • This application also provides a vehicle that may include the image display device 1000 or the image display system described above.
  • This application also provides a computer program product, which includes computer program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.
  • This application also provides a computer-readable medium storing program code that, when run on a computer, causes the computer to perform the methods described in the above embodiments.
  • This application also provides a chip, which includes a circuit for performing the methods described in the above embodiments.
  • each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software.
  • the method disclosed in the embodiments of this application can be directly implemented by a hardware processor, or by a combination of hardware and software modules within the processor.
  • the software modules can reside in random access memory, flash memory, read-only memory, programmable read-only memory, power-on erasable programmable memory, registers, or other mature storage media in the art. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps of the above method. To avoid repetition, detailed descriptions are omitted here.
  • the memory may include read-only memory and random access memory, and provides instructions and data to the processor.
  • the disclosed systems, apparatuses, and methods can be implemented in other ways.
  • the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods.
  • multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.
  • the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
  • the units described as separate components may or may not be physically separate.
  • the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.
  • the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.
  • the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application.
  • the aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Chemical & Material Sciences (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Business, Economics & Management (AREA)
  • Mathematical Physics (AREA)
  • Combustion & Propulsion (AREA)
  • Transportation (AREA)
  • Mechanical Engineering (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Navigation (AREA)

Abstract

一种图像显示方法、装置和车辆，该方法包括：获取云端服务器发送的第一兴趣点POI的三维信息，该三维信息由一个或者多个车辆处于该第一POI周围时通过摄像头采集的图像以及该图像对应的位姿信息确定；控制显示装置显示该三维信息。该方法可以应用于智能汽车或者电动汽车中，可以使得用户对POI有全方位了解，方便用户对于目的地进行确认。

Description

图像显示方法、装置和车辆 技术领域
本申请涉及智能驾驶领域，并且更具体地，涉及一种图像显示方法、装置和车辆。
背景技术
兴趣点(point of interest，POI)泛指用户关注的商品或者目的地。当前POI只能以文字或者平面信息的方式呈现，对于复杂区域的POI，仅靠文字和平面信息难以在范围内进行目的地确认。
发明内容
本申请提供一种图像显示方法、装置和车辆，可以使得用户对POI有全方位了解，方便用户对于目的地进行确认。
第一方面，本申请提供了一种图像显示方法，该方法包括：获取云端服务器发送的第一兴趣点POI的三维(three-dimensional，3D)信息，该三维信息由一个或者多个车辆处于该第一POI周围时通过摄像头采集的图像以及该图像对应的位姿信息确定；控制显示装置显示该三维信息。
基于上述技术方案，通过显示装置显示POI的三维信息，可以使得用户通过三维信息对于POI有全方位了解，从而方便用户对于目的地进行确认，有助于提升用户的体验。示例性的，以该第一POI为某个公园为例，通过显示装置显示该公园的三维信息，可以方便用户从该公园的三维信息中确认目的地(例如，公园中的某个雕塑)的信息，从而方便用户前往该目的地。
在一些可能的实现方式中，获取云端服务器发送的第一POI的三维信息，包括：获取云端服务器发送的第一POI以及第一POI周围环境的三维信息。
示例性的，以该第一POI为电梯厅入口为例，通过显示装置显示该电梯厅以及该电梯厅周围环境的三维信息，可以方便用户查看到距离该电梯厅入口最近的停车位的信息，从而可以方便用户驾驶车辆行驶至该停车位。这样，可以避免用户靠经验在地下车库内寻找该电梯厅入口，有助于提升用户从当前位置行驶至目的地的效率。
或者，在检测到用户启动智能驾驶功能时车辆可以从当前位置自动驾驶至该停车位。这样，避免车辆自动驾驶至地下车库的入口(车载地图应用可以包括该入口的位置且不包括电梯厅的位置)后用户接管车辆并依据用户的经验人工驾驶车辆行驶至该电梯厅的过程，有助于拓展自动驾驶的能力边界，从而有助于提升用户的智能驾驶体验。
在一些可能的实现方式中，该第一POI的三维信息由云端服务器基于一个或者多个车辆处于第一POI周围时上传的图像以及该图像的位姿信息确定。
示例性的，云端服务器可以执行如下一项或者多项操作，得到该第一POI的三维信息：对一个或者多个车辆处于第一POI周围时上传的图像进行分割，得到图像中不同区域的通用物体以及其类别；使用文字提取技术对图像中的文字进行提取并将文本内容与通用物体进行匹配；基于图像以及图像对应的位姿信息进行环境重建和定位；基于图像、图像的深度信息以及图像对应的位姿信息，对图像超分辨重建，实现从低质量图像到高质量图像的增强。
在一些可能的实现方式中，以三维信息是三维模型为例，该方法还包括：在检测到用户针对该三维模型的第一触控操作时，控制显示装置显示该三维模型旋转或者平移。示例性的，该第一触控操作为滑动操作。、
在一些可能的实现方式中，该方法还包括：根据用户手指在显示屏上的滑动距离，从显示该三维信息中第一位置处的正视图方向的图像切换至显示该三维信息中第二位置处的正视图方向的图像。该第一位置和该第二位置之间的距离可以由该滑动距离确定。
在一些可能的实现方式中，以三维信息是三维模型为例，该方法还包括：在检测到用户针对该三维模型的第二触控操作时，控制该显示装置显示该三维模型中某个位置处正视图方向的图像。示例性的，该第二触控操作可以为点击操作。
结合第一方面，在第一方面的某些实现方式中，该方法还包括：控制该显示装置显示第一提示信息，该第一提示信息用于请求用户确认是否导航至该第一POI；在获取到用户的第一指令时，控制该显示装置显示从终端设备的当前位置到该第一POI的导航信息，该第一指令指示导航至该第一POI。
基于上述技术方案，通过用户的反馈，车辆可以检测云端服务器构建的第一POI的三维信息的准确性。若用户反馈导航至该第一POI，可以确认该第一POI的信息准确，从而可以显示从当前位置到该第一POI的导航信息。
结合第一方面，在第一方面的某些实现方式中，该终端设备为车辆，该方法还包括：根据该导航信息，控制该车辆从该当前位置行驶至该第一POI。
基于上述技术方案，车辆可以从当前位置自动驾驶至该第一POI。这样，避免车辆自动驾驶至第一POI附近的某个地点(该地点为地图应用中的POI)后，用户接管车辆并依据用户的经验人工驾驶车辆行驶至该第一POI的过程，有助于拓展车辆自动驾驶的能力边界，从而有助于提升用户的智能驾驶体验。
在一些可能的实现方式中，该第一POI为地图应用中不包括的POI。
在一些可能的实现方式中，根据该导航信息，控制该车辆从该当前位置行驶至该第一POI，包括：根据该导航信息，控制该车辆从该当前位置行驶至距离该第一POI最近的下车地点。这样，在检测到用户开启智能驾驶功能后，车辆可以自动驾驶至距离该第一POI最近的下车地点。在检测到用户在距离该第一POI最近的下车地点下车后，车辆可以自动泊入任意合理的停车位中。
结合第一方面，在第一方面的某些实现方式中，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该方法还包括：在检测到用户从该多个子区域中选择第一子区域的操作时，控制该显示装置显示从终端设备的当前位置到该第一子区域的导航信息。
基于上述技术方案，用户在查看到该三维信息后可以从多个子区域中选择某个子区域(导航目的地)，从而终端设备可以显示从当前位置到该子区域的导航信息。
示例性的，该第一POI为公园，该第一子区域为公园中的某个建筑物。
示例性的，该第一POI为地下停车场，该第一子区域为某个停车位。
示例性的，该第一POI为充电区，该第一子区域为某个充电桩。
结合第一方面，在第一方面的某些实现方式中，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该方法还包括：控制该显示装置向用户推荐第一子区域且控制该显示装置显示第二提示信息，该第二提示信息用于请求用户确认是否导航至该第一子区域，该多个子区域包括该第一子区域；在获取到用户的第二指令时，控制该显示装置显示从终端设备的当前位置到该第一子区域的导航信息，该第二指令用于指示导航至该第一子区域。
基于上述技术方案，车辆可以通过三维信息向用户推荐某个子区域作为导航目的地。这样，用户在确认该导航目的地后，车辆可以显示从当前位置到该子区域的导航信息。
结合第一方面，在第一方面的某些实现方式中，该第一子区域为停车位。
在一些可能的实现方式中，获取云端服务器发送的第一兴趣点POI的三维信息，包括：获取云端服务器发送的第一POI以及第一POI周围环境的三维信息；该方法还包括：控制显示装置在该三维信息中，向用户推荐距离该第一POI最近的停车位。
结合第一方面，在第一方面的某些实现方式中，该终端设备为车辆，该方法还包括：根据该导航信息，控制该车辆从该当前位置行驶至该第一子区域。
结合第一方面，在第一方面的某些实现方式中，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该方法还包括：控制该显示装置向用户推荐第二子区域且控制该显示装置显示第三提示信息，该第三提示信息用于请求用户确认是否导航至该第二子区域，该多个子区域包括该第二子区域；在获取到用户的第三指令时，向该云端服务器发送第四指令，该第三指令用于指示不导航至该第二子区域，该第四指令用于指示该云端服务器重新对该三维信息或者该第二子区域进行识别。
基于上述技术方案，在用户反馈不导航至该第二子区域后，可以确认云端服务器构建的三维信息或者该第二子区域可能不准确，从而可以指示云端服务器重新对该三维信息或者第二子区域进行识别。这样，通过用户的反馈可以实现对云端服务器构建的三维信息的准确性进行验证，在不准确的情况下对该三维信息进行重新构建，有助于提升终端设备显示的三维信息的准确性。
结合第一方面，在第一方面的某些实现方式中，该方法还包括：控制该显示装置显示第四提示信息， 该第四提示信息用于请求用户从该多个子区域中选择；在获取到用户的第五指令时，控制该显示装置显示从终端设备的当前位置到第三子区域的导航信息，该第五指令指示用户选择该第三子区域，该多个子区域包括该第三子区域。
基于上述技术方案，通过提示用户从多个子区域进行选择且检测到用户选择了第三子区域后，可以确认该第一POI中存在准确的子区域。这样，在通过用户的反馈确定三维信息中某些子区域不准确且某些区域准确的情况下，可以通过显示装置显示从当前位置到该准确的子区域的导航信息。这样，一方面可以让云端服务器对不准确的子区域进行重新识别，另一方面不会影响用户的导航体验。
结合第一方面，在第一方面的某些实现方式中，该终端设备为车辆，该方法还包括：根据该导航信息，控制该车辆从该当前位置行驶至该第三子区域。
结合第一方面，在第一方面的某些实现方式中，该第一POI为停车场或者充电区。
基于上述技术方案，对于停车场或者充电区，可能会存在多个子区域的情况。通过三维信息可以向用户显示该多个子区域，用户可以对该多个子区域中任意一个子区域进行选择。这样，可以实现驾驶员基于导航信息，人工驾驶车辆行驶至停车场或者充电区的任意一个子区域；或者，可以实现在车辆检测到用户开启智能驾驶功能后，控制车辆从当前位置行驶该子区域，这一期间无需用户接管车辆并依据用户的经验人工驾驶车辆行驶至该子区域的过程，有助于拓展车辆自动驾驶的能力边界，从而有助于提升用户的智能驾驶体验。
第二方面，提供了一种图像显示装置，该装置包括：获取单元，用于获取云端服务器发送的第一兴趣点POI的三维信息，该三维信息由一个或者多个车辆处于该第一POI周围时通过摄像头采集的图像以及该图像对应的位姿信息确定；控制单元，用于控制显示装置显示该三维信息。
结合第二方面，在第二方面的某些实现方式中，该控制单元，还用于：控制该显示装置显示第一提示信息，该第一提示信息用于请求用户确认是否导航至该第一POI；在该获取单元获取到用户的第一指令时，控制该显示装置显示从终端设备的当前位置到该第一POI的导航信息，该第一指令指示导航至该第一POI。
结合第二方面，在第二方面的某些实现方式中，该终端设备为车辆，该控制单元，还用于：根据该导航信息，控制该车辆从该当前位置行驶至该第一POI。
结合第二方面，在第二方面的某些实现方式中，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该装置还包括：检测单元，用于检测到用户从该多个子区域中选择第一子区域的操作；该控制单元，还用于响应于检测到选择该第一子区域的操作，控制该显示装置显示从终端设备的当前位置到该第一子区域的导航信息。
结合第二方面，在第二方面的某些实现方式中，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该控制单元，还用于：控制该显示装置向用户推荐第一子区域且控制该显示装置显示第二提示信息，该第二提示信息用于请求用户确认是否导航至该第一子区域，该多个子区域包括该第一子区域；在该获取单元获取到用户的第二指令时，控制该显示装置显示从终端设备的当前位置到该第一子区域的导航信息，该第二指令用于指示导航至该第一子区域。
结合第二方面，在第二方面的某些实现方式中，该第一子区域为停车位。
结合第二方面，在第二方面的某些实现方式中，该终端设备为车辆，该控制单元，还用于：根据该导航信息，控制该车辆从该当前位置行驶至该第一子区域。
结合第二方面，在第二方面的某些实现方式中，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该控制单元，还用于控制该显示装置向用户推荐第二子区域且控制该显示装置显示第三提示信息，该第三提示信息用于请求用户确认是否导航至该第二子区域，该多个子区域包括该第二子区域；该装置还包括：发送单元，用于在该获取单元获取到用户的第三指令时，向该云端服务器发送第四指令，该第三指令用于指示不导航至该第二子区域，该第四指令用于指示该云端服务器重新对该三维信息或者该第二子区域进行识别。
结合第二方面，在第二方面的某些实现方式中，该控制单元，还用于控制该显示装置显示第四提示信息，该第四提示信息用于请求用户从该多个子区域中选择；在该获取单元获取到用户的第五指令时，控制该显示装置显示从终端设备的当前位置到第三子区域的导航信息，该第五指令指示用户选择该第三子区域，该多个子区域包括该第三子区域。
结合第二方面，在第二方面的某些实现方式中，该终端设备为车辆，该控制单元，还用于根据该导 航信息，控制该车辆从该当前位置行驶至该第三子区域。
结合第二方面，在第二方面的某些实现方式中，该第一POI为停车场或者充电区。
第三方面，提供了一种图像显示装置，该装置包括存储器和处理器，该存储器用于存储计算机程序，该处理器用于执行该存储器中的计算机程序，使得该装置可以实现上述第一方面中任一种可能的实现方式中的方法。
第四方面，提供了一种图像显示系统，该系统包括显示装置和上述第二方面或者第三方面中任一项所述的装置。
第五方面，提供了一种车辆，包括上述第二方面或者第三方面中任一方面可能的装置，或者，包括上述第四方面所述的系统。
本申请中的车辆为广义概念上的车辆，可以是交通工具(如商用车、乘用车、摩托车、飞行车、火车等)，工业车辆(如：叉车、挂车、牵引车等)，工程车辆(如挖掘机、推土车、吊车等)，农用设备(如割草机、收割机等)，游乐设备，玩具车辆等，本申请实施例对车辆的类型不作具体限定。
第六方面，提供了一种计算机程序产品，上述计算机程序产品包括：计算机程序代码，当上述计算机程序代码在计算机上运行时，使得计算机执行上述第一方面中任一种可能实现方式中的方法。
第七方面，提供了一种计算机可读存储介质，上述计算机可读介质存储有计算机程序，当上述计算机程序在计算机上运行时，使得计算机执行上述第一方面中任一种可能实现方式中的方法。
第八方面，提供了一种芯片，该芯片包括电路，用于执行上述第一方面中任一种可能实现方式中的方法。
附图说明
图1是本申请实施例提供的车辆的功能框图示意。
图2是本申请实施例提供的智能驾驶系统的示意性框图。
图3是本申请实施例提供的一组用户图形界面GUI。
图4是本申请实施例提供的另一组GUI。
图5是本申请实施例提供的另一组GUI。
图6是本申请实施例提供的另一GUI。
图7是本申请实施例提供的系统架构的示意图。
图8是本申请实施例提供的人机交互过程的示意图。
图9是本申请实施例提供的图像显示方法的示意性流程图。
图10是本申请实施例提供的图像显示装置的示意性框图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行描述。其中，在本申请实施例的描述中，除非另有说明，“/”表示或的意思，例如，A/B可以表示A或B；本文中的“和/或”仅仅是一种描述关联对象的关联关系，表示可以存在三种关系，例如，A和/或B，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。“至少一项”是指一项或一项以上。例如，“A和B中的至少一项”，类似于“A和/或B”，描述关联对象的关联关系，表示可以存在三种关系，例如，A和B中的至少一项，可以表示：单独存在A，同时存在A和B，单独存在B这三种情况。
本申请实施例中采用诸如“第一”、“第二”的前缀词，仅仅为了区分不同的描述对象，对被描述对象的位置、顺序、优先级、数量或内容等没有限定作用。本申请实施例中对序数词等用于区分描述对象的前缀词的使用不对所描述对象构成限制，对所描述对象的陈述参见权利要求或实施例中上下文的描述，不应因为使用这种前缀词而构成多余的限制。此外，在本实施例的描述中，除非另有说明，“多个”的含义是两个或两个以上。
图1是本申请实施例提供的车辆100的一个功能框图示意。车辆100可以包括感知系统110、计算平台120和显示装置130，其中，感知系统110可以包括感测关于车辆100周边的环境的信息的一种或多种传感器。例如，感知系统110可以包括定位系统，定位系统可以是全球定位系统(global positioning system，GPS)，也可以是北斗系统或者其他定位系统。又例如，感知系统110可以包括惯性测量单元(inertial measurement unit，IMU)、加速度传感器、激光雷达、毫米波雷达、超声雷达以及摄像装置中 的一种或者多种。
车辆100的部分或所有功能可以由计算平台120控制。计算平台120可以包括一个或多个处理器，例如处理器121至12n(n为正整数)，处理器是一种具有信号的处理能力的电路，在一种实现中，处理器可以是具有指令读取与运行能力的电路，例如中央处理单元(central processing unit，CPU)、微处理器、图形处理器(graphics processing unit，GPU)(可以理解为一种微处理器)、或数字信号处理器(digital signal processor，DSP)等；在另一种实现中，处理器可以通过硬件电路的逻辑关系实现一定功能，该硬件电路的逻辑关系是固定的或可以重构的，例如处理器为专用集成电路(application-specific integrated circuit，ASIC)或可编程逻辑器件(programmable logic device，PLD)实现的硬件电路，例如现场可编程门阵列(field programmable gate array，FPGA)。在可重构的硬件电路中，处理器加载配置文档，实现硬件电路配置的过程，可以理解为处理器加载指令，以实现以上部分或全部单元的功能的过程。此外，处理器还可以是针对人工智能设计的硬件电路，其可以理解为一种ASIC，例如神经网络处理单元(neural network processing unit，NPU)、张量处理单元(tensor processing unit，TPU)、深度学习处理单元(deep learning processing unit，DPU)等。此外，计算平台120还可以包括存储器，存储器用于存储指令，处理器121至12n中的部分或全部处理器可以调用存储器中的指令，以实现相应的功能。
座舱内的显示装置130主要分为两类，第一类是车载显示屏；第二类是投影显示屏，例如抬头显示装置(head up display，HUD)。车载显示屏是一种物理显示屏，是车载信息娱乐系统的重要组成部分，座舱内可以设置有多块显示屏，如数字仪表显示屏，中控屏，副驾驶位上的乘客(也称为前排乘客)面前的显示屏，左侧后排乘客面前的显示屏以及右侧后排乘客面前的显示屏，甚至是车窗也可以作为显示屏进行显示。抬头显示，也称平视显示系统。主要用于在驾驶员前方的显示设备(例如挡风玻璃)上显示例如时速、导航等驾驶信息。以降低驾驶员视线转移时间，避免因驾驶员视线转移而导致的瞳孔变化，提升行驶安全性和舒适性。HUD例如包括组合型抬头显示(combiner-HUD，C-HUD)系统、风挡型抬头显示(windshield-HUD，W-HUD)系统、增强现实型抬头显示系统(augmented reality HUD，AR-HUD)。应理解，HUD也可以随着技术演进出现其他类型的系统，本申请对此不作限定。
以上显示装置130是以车载显示屏和投影显示屏为例进行说明的，本申请实施例并不限于此。例如，该显示装置130还可以为光显示屏或者投影幕布。
可选地，以上车辆100的结构仅仅是示意性的，实际应用中，上述车辆100中的各个部件可以根据实际需要增添或者删除。
车辆100可以包括智能驾驶系统，智能驾驶系统可以包括高级驾驶辅助系统(advanced driving assistant system，ADAS)与自动驾驶系统(autonomous driving system，ADS)，智能驾驶系统利用在车辆上的多种传感器(包括但不限于：激光雷达、毫米波雷达、摄像装置、超声波传感器、全球定位系统、惯性测量单元)从车辆周围获取信息，并对获取的信息进行分析和处理，实现例如障碍物感知、目标识别、车辆定位、路径规划、驾驶员监控/提醒等功能，从而提升车辆驾驶的安全性、自动化程度和舒适度。
示例性的，图2示出了本申请实施例提供的智能驾驶系统的示意性框图。智能驾驶系统可以包括三个功能模块：感知模块210、规划模块220和控制模块230，其中，感知模块210通过传感器感知车身周围环境，输出相应感知数据至规控模块220。规划模块220根据感知模块210获取的信息，获得道路拓扑与目标物等信息。规划模块220可以基于道路拓扑与目标物信息，确定一段时间内的规划轨迹。规划模块220可以向控制模块230发送该规划轨迹。控制模块230从规划模块220接收到该规划轨迹后可以输出控制信号，可控制执行器采取相应行动，如转向、加速、减速等。
以上感知模块210可以为上述感知系统110，规划模块220与控制模块230可以位于上述计算平台120中。
基于车辆的驾驶自动化系统能够执行动态驾驶任务的程度，根据在执行动态驾驶任务中的角色分配以及有无设计运行范围(operational design domain，ODD)，如驾驶自动化系统设计时确定的适用于其功能运行的外部条件，如道路、交通、天气、光照等)限制，将驾驶自动化分成0级至5级(或称为L0-L5)。在驾驶自动化的6个等级之中，0-2级为驾驶辅助，系统辅助人类执行动态驾驶任务，驾驶主体仍为驾驶员；3-5级为自动驾驶，系统在设计运行条件下代替人类执行动态驾驶任务，当功能激活时，驾驶主体是系统。各级名称及定义如下：
0级驾驶自动化(也可以称为应急辅助(emergency assistance))系统不能持续执行动态驾驶任务中的车辆横向或纵向运动控制，但具备持续执行动态驾驶任务中的部分目标和事件探测与响应的能力。1级 驾驶自动化(也可以称为部分驾驶辅助(partial driver assistance))系统在其设计运行条件下持续地执行动态驾驶任务中的车辆横向或纵向运动控制，且具备与所执行的车辆横向或纵向运动控制相适应的部分目标和事件探测与响应的能力。2级驾驶自动化(也可以称为组合驾驶辅助(combined driver assistance))系统在其设计运行条件下持续地执行动态驾驶任务中的车辆横向和纵向运动控制，且具备与所执行的车辆横向和纵向运动控制相适应的部分目标和事件探测与响应的能力。3级驾驶自动化(也可以称为有条件自动驾驶(conditionally automated driving))系统在其设计运行条件下持续地执行全部动态驾驶任务。4级驾驶自动化(也可以称为高度自动驾驶(highly automated driving))系统在其设计运行条件下持续地执行全部动态驾驶任务并自动执行最小风险策略。5级驾驶自动化(也可以称为完全自动驾驶(fully automated driving))系统在任何可行驶条件下持续地执行全部动态驾驶任务并自动执行最小风险策略。通常，智能驾驶系统一般属于L2-L5，如ADAS属于L2，ADS属于L3-L5。
如前所述，当前POI只能以文字或者平面信息的方式呈现，对于复杂区域的POI，仅靠文字和平面信息难以在范围内进行目的地确认。
本申请实施例提供了一种图像显示方法、装置和车辆，通过终端设备显示POI的三维信息，可以使得用户对POI有全方位了解，方便用户对于目的地进行确认。
图3示出了本申请实施例提供的一组用户图形界面(graphical user interface，GUI)。
如图3中的(a)所示，车辆200检测到通过车载地图应用的搜索框301输入“A商场地下一层1号电梯厅”时，可以在搜索结果框302中显示3个搜索结果，例如，A商场北门、A商场南门和A商场地下一层洗车店。此时，搜索结果框302中并不包括A商场地下一层1号电梯厅这一搜索结果，需要用户在开车进入A商场地下一层后人工驾驶车辆寻找1号电梯厅的位置。
如图3中的(b)所示，车辆200在检测到用户驾驶车辆进入A商场地下一层停车场且将车辆200停在1号电梯厅附近时，车辆200可以通过座舱外的传感器对车辆周围的环境信息进行采集。例如，车辆200可以通过座舱外的摄像头采集图像。车辆200可以将该图像以及该图像对应的位姿信息发送给云端服务器。
以上图像对应的位姿信息可以为车辆200(或者车辆200的摄像头)采集该图像时的位姿信息，或者，也可以是该图像本身的位姿信息。
又例如，车辆200可以通过座舱外的激光雷达或者毫米波雷达采集点云数据。车辆200可以将该点云数据以及采集该点云数据时车辆200的位姿信息发送给云端服务器。
可选地，车辆200可以在检测到用户驻车的操作时，将座舱外传感器采集的数据以及该数据对应的位姿信息发送给云端服务器；或者，车辆200可以将检测到用户驻车操作前预设时长内，座舱外传感器采集的数据以及该数据对应的位姿信息发送给云端服务器。
可选地，车辆200在检测到用户通过车载地图应用搜索第一POI的信息且车载地图应用未提供该第一POI的搜索结果时，可以将座舱外传感器采集的数据以及该数据对应的位姿信息发送给云端服务器。
示例性的，车辆200检测到用户通过车载地图应用搜索“A商场1号电梯厅”且该车载地图应用提供的搜索结果中不包括“A商场1号电梯厅”的搜索结果。车辆200在检测到用户驻车操作时，可以将驻车点附近的图像以及采集该图像时车辆200的位姿信息发送给云端服务器。
可选地，车辆200在检测到用户发出寻找第二POI的语音指令时，可以将座舱外摄像头采集的图像以及采集该图像时车辆200的位姿发送给云端服务器。
需要说明的是，本申请实施例中所涉及的信息采集过程(如图像数据采集、点云数据采集、语音采集、用户在车载地图应用中的搜索数据采集等)，是在用户知情且经得用户允许的情况下执行的，即信息采集过程符合法律法规要求，不属于妨害公共利益行为。
如图3中的(c)所示，云端服务器在接收到车辆200发送的信息后，可以对车辆200发送的信息进行处理。
例如，以车辆200向云端服务器发送的信息为图像和位姿信息为例，云端服务器可以通过图像分割算法对图像进行分割并对分割结果进行类别匹配。如图3中的(c)所示，云端服务器可以在对图像进行分割且对类别进行匹配后，可以建立图像中不同区域与其对应类别(例如，停车位、墙面、立柱、电梯厅自动门、地面)的关联关系。
可选地，云端服务器还可以通过文字识别算法对图像中的文字进行识别。如通过车辆200上传的图像识别到文本内容“1号电梯厅”。云端服务器可以基于识别到的文本内容对上述区域与类别的关联关系 进行校验融合。云端服务器在确定该文本内容“1号电梯厅”位于电梯厅自动门附近时，可以确定图像中电梯厅自动门为1号电梯厅的自动门。
可选地，云端服务器还可以对车载地图应用的底图中已有的POI进行丰富和更新。云端服务器可以将新识别到的POI(例如，“1号电梯厅”)与用户停车位置附近已有的POI进行匹配，在已有的POI基础上进行更新和补充。
示例性的，车载地图应用中的多个POI中包括A商场地下一层洗车店以及洗车店的位置信息。云端服务器可以基于洗车店的位置信息，以及车辆200采集图像时的位姿信息，在车载地图应用中增加1号电梯厅这一POI以及在已有的洗车店附近增加1号电梯厅的图像。
可选地，车载地图应用中包括某个POI的位置信息且不包括该POI和该POI周围环境的三维信息。云端服务器可以对一个或者多个车辆在该POI周围采集的图像以及图像对应的位姿信息处理，从而得到该POI的三维信息。云端服务器可以将该POI的位置信息和该三维信息关联起来。可选地，云端服务器可以将关联后的POI的位置信息和该三维信息发送给车辆100；或者，云端服务器可以向车辆100发送该POI的三维信息，由车辆对该POI的位置信息和该三维信息进行关联。
可选地，车辆200可以将1号电梯厅周围的多帧图像(或者视频)以及每帧图像对应的位姿信息发送给云端服务器。云端服务器可以基于车辆200发送的多帧图像以及每帧图像对应的位姿信息进行环境重建。基于稀疏视角的图像和对应位姿信息的输入，得到该1号电梯厅以及1号电梯厅周围环境的三维信息。这样，通过将该三维信息发送给车辆，可以使得用户可以连续地旋转，或者，平移观看1号电梯厅以及1号电梯厅周围环境的三维信息。
可选地，云端服务器可以基于多帧图像、多帧图像中每帧图像的深度信息以及每帧图像对应的位姿信息，对图像进行超分辨率重建，实现从低质量图像到高质量图像的增强。
以上实施例中是以车辆200向云端服务器发送多帧图像以及每帧图像对应的位姿信息为例进行说明的，本申请实施例对此不作具体限定。例如，云端服务器可以接收多个车辆(可以理解为众包车辆)发送的1号电梯厅以及1号电梯厅周围的图像以及位姿信息，并基于多个车辆发送的图像以及位姿信息构建在1号电梯厅以及1号电梯厅周围环境的三维信息。
本申请实施例中，基于一种低成本的众包POI推荐图层生成方法，可以给用户一种友好交互的3DPOI信息查看方式，有助于提升用户在使用导航功能时的体验。同时，对于具有智能驾驶功能的车辆来说，可以拓展智能驾驶能力的边界。
如图3中的(d)所示，车辆100检测到通过车载地图应用的搜索框303输入“A商场”时，可以在搜索结果框304中显示4个搜索结果，例如，A商场北门、A商场南门、A商场地下一层洗车店以及A商场地下一层1号电梯厅。在检测到用户在搜索框304中选择“A商场地下一层1号电梯厅”的操作时，车辆100可以通过显示屏显示如图3中的(e)所示的GUI。
如图3中的(e)所示，响应于检测到用户选择“A商场地下一层1号电梯厅”的操作，车辆100可以显示1号电梯厅以及1号电梯厅周围环境的三维信息中某个视角下的图像，该图像中包括箭头305，该箭头305指向1号电梯厅的入口。同时，车辆100还可以通过显示屏显示提示框306，该提示框306中包括提示信息“请您确认是否导航至1号电梯厅？”。在检测到用户点击控件307的操作时，车辆100可以通过显示屏显示如图3中的(f)所示的GUI。
如图3中的(f)所示，响应于检测到用户点击控件307的操作，车辆100可以规划从当前位置到A商场地下一层1号电梯厅的行驶路径。同时，车辆100可以通过显示屏显示提示框308，该提示框308中包括提示信息“已为您规划好从当前位置到A商场地下一层1号电梯厅的行驶路径”。
可选地，车辆100在检测到用户开启智能驾驶功能时，可以根据该行驶路径，控制车辆100从当前位置自动驾驶至该1号电梯厅。
可选地，在检测到用户点击控件307的操作后，车辆100还可以提示用户对下车地点进行选择。例如，在检测到用户点击某个停车位的操作时，车辆可以规划从当前位置到该停车位的行驶轨迹。
又例如，在检测到用户点击立柱的操作时，车辆可以规划从当前位置到该立柱的行驶轨迹。车辆100在检测到用户开启智能驾驶功能时，可以根据该行驶路径，控制车辆100从当前位置自动驾驶至该立柱附近。车辆100在检测到用户下车后，可以自动泊入任意合理的停车位中。
示例性的，图4示出了本申请实施例提供的另一组GUI。
如图4中的(a)所示，车辆100检测到用户通过车载地图应用的搜索框401输入“B商场”的操作 时，可以通过搜索结果框402显示4个搜索结果，包括B商场北门、B商场南门、B商场地下一层停车场和B商场地下二层停车场。在检测到用户选择B商场地下二层停车场的操作时，可以通过显示屏显示如图4中的(b)所示的GUI。
以上B商场地下一层停车场和B商场地下二层停车场可以为通过上述低成本的众包方式得到的POI。示例性的，云端服务器可以基于多个车辆上传的B商场地下二层停车场的图像以及图像对应的位姿信息，构建B商场地下二层停车场的三维信息。
如图4中的(b)所示，响应于检测到用户选择B商场地下二层停车场的操作，车辆100可以通过显示屏显示B商场地下二层停车场的三维信息。用户可以对该三维信息进行点击、拖动、缩放以及旋转操作。
例如，用户通过对三维信息的某个位置进行点击，可以切换至显示该位置处正视图方向的图像。又例如，用户通过对三维信息像进行旋转操作，可以观看该三维信息的不同视角。又例如，用户通过对三维信息进行缩放，可以近距离或者远距离观看该三维信息。在检测到用户对三维信息某个位置的点击操作时，可以显示如图4中的(c)所示的GUI。
如图4中的(c)所示，响应于检测到用户对三维信息某个位置的点击操作，车辆100可以通过显示屏显示该位置的正视图方向的图像。在检测到用户在显示屏上的滑动操作，可以显示如图4中的(d)所示的GUI。
以上图4中的(b)和(c)均可以称之为B商场地下一层停车场的三维信息。图4中的(b)中的GUI显示该B商场地下二层停车场的三维模型，图4中的(c)中的GUI显示三维模型中某个位置处的正视图方向下的图像。
如图4中的(d)所示，响应于检测到用户在显示屏上的滑动操作，车辆100可以通过显示屏显示另一位置处正视图方向的图像。在检测到用户在该图像中点击某个停车位的操作时，车辆100可以通过显示屏对该停车位高亮显示，并显示如图4中的(e)所示的GUI。
如图4中的(e)所示，响应于检测到用户在该图像中点击停车位的操作，车辆100可以通过显示屏显示提示框403，该提示框403中包括提示信息“是否想到导航至该车位？”。在检测到用户点击控件404的操作时，可以显示如图4中的(f)所示的GUI。
如图4中的(f)所示，响应于检测到用户点击控件404的操作，车辆100可以规划从当前位置到该停车位的行驶路径。同时，车辆100可以通过显示屏显示提示框405，该提示框405中包括提示信息“已为您规划好从当前位置到您选择的车位的行驶路径”。
本申请实施例中，针对地下停车场这一POI，其中可能包括多个停车位。用户可以通过查看该地下停车场的三维信息，对该地下停车场有全方位了解并可以选择自己偏好的停车位(例如，距离电梯口最近的停车位)。这样，在车载地图应用中仅包括地下停车场的入口这一POI时，通过云端服务器构建的地下停车场的三维信息，可以使得车辆100在开启智能驾驶功能后，从当前位置自动驾驶至某个停车位。避免了车辆在行驶至地下停车场的入口后由用户接管车辆，再凭借经验人工驾驶至某个停车位的过程，有助于提升车辆自动驾驶的能力边界，也有助于提升用户的智能驾驶体验。
示例性的，图5示出了本申请实施例提供的另一组GUI。
如图5中的(a)所示，车辆100在检测到用户通过车载地图应用的搜索框501输入“C公园”的操作时，可以通过搜索结果框502显示4个搜索结果，包括C公园北门、C公园南门、C公园充电区和C公园停车场。在检测到用户选择C公园充电区的操作时，可以通过显示屏显示如图5中的(b)所示的GUI。
以上C公园充电区可以为通过上述低成本的众包方式得到的POI。示例性的，云端服务器可以基于多个车辆上传的C公园充电区的图像以及图像对应的位姿信息，构建C公园充电区的三维信息。
如图5中的(b)所示，响应于检测到用户选择择C公园充电区的操作，车辆100可以通过显示屏高亮显示某个充电桩且显示提示框503，其中，提示框503中包括提示信息“是否导航至该充电桩？”。从该三维信息中可以看出，高亮显示的停车位后面的立柱被云端服务器误识别为充电桩。在检测到用户点击控件504的操作时，可以显示如图5中的(c)所示的GUI。
可选地，在检测到用户点击控件504的操作时，车辆100还可以提示用户确认该充电桩识别有误，或者，该充电桩识别准确且用户希望更换一个充电桩。在检测到用户确认该充电桩识别有误的操作时，可以通过显示屏显示如图5中的(c)所示的GUI。或者，在检测到用户确认该充电桩识别准确且用户希望更换一个充电桩时，可以通过显示屏显示如图5中的(d)所示的GUI。
可选地，在检测到用户点击控件504的操作，或者，在获取到用户确认该充电桩识别有误的操作时，车辆100可以向云端服务器发送指令，该指令用于指示对C公园充电区的三维信息进行重新构建，或者，该指令用于指示对如图5中的(b)所示的充电桩进行重新识别。
如图5中的(c)所示，响应于检测到用户点击控件504的操作，车辆100可以显示提示框505，其中，提示框505中包括提示信息“是否存在充电桩的信息？”。在检测到用户点击控件506的操作时，可以显示如图5中的(d)所示的GUI。
如图5中的(d)所示，响应于检测到用户点击控件506的操作，车辆100可以通过显示屏切换至显示C公园充电区某个位置处正视图方向的图像且显示提示框507，其中，提示框507包括提示信息“请您选择一个充电桩”。在检测到用户选择了某个充电桩的操作时，可以显示如图5中的(e)所示的GUI。
如图5中的(e)所示，响应于检测到用户选择某个充电桩的操作，车辆100可以规划从当前位置到该充电桩的行驶路径。同时，车辆100可以通过显示屏显示提示框508，该提示框508中包括提示信息“已为您规划好从当前位置到您选择的充电桩的行驶路径”。
本申请实施例中，可以基于用户的反馈判断云端服务器构建的充电区的三维信息是否准确。若不准确，车辆可以指示云端服务器对充电区的三维信息进行重新构建，同时，车辆还可以通过提示信息询问用户是否存在准确的充电桩。若用户确认存在准确的充电桩且选择了某个充电桩，车辆也可以规划从当前位置到该充电桩的行驶路径。这样，在车载地图应用中仅包括C公园北门或者C公园南门这一POI时，通过云端服务器构建的C公园充电区的三维信息，可以使得车辆100在开启智能驾驶功能后，从当前位置自动驾驶至该充电桩。避免了车辆在行驶至C公园北门或者C公园南门后由用户接管车辆，再凭借经验人工驾驶至充电区中的某个充电桩的过程，有助于提升车辆自动驾驶的能力边界，也有助于提升用户的智能驾驶体验。
示例性的，图6示出了本申请实施例提供的另一GUI。
如图6所示，响应于检测到用户选择“A商场地下一层1号电梯厅”的操作，车辆100可以显示1号电梯厅以及1号电梯厅周围环境的三维信息中某个位置正视图方向的图像，该图像中包括箭头305，该箭头305指向1号电梯厅的入口。同时，车辆100还可以通过显示屏显示推荐的停车位(例如，阴影所示的停车位)以及提示框601，该提示框601中包括提示信息“是否导航至距离1号电梯厅最近的停车位”。
示例性的，在检测到用户点击控件602的操作时，车辆100可以通过显示屏显示从车辆的当前位置到该停车位的导航信息。
示例性的，图7示出了本申请实施例提供的系统架构的示意图。
如图7所示，该系统架构中包括车辆1、云端服务器和车辆2，其中，车辆1可以将POI(或者，该POI以及该POI周围环境)的图像以及图像对应的位姿信息发送给云端服务器；云端服务器可以对车辆1采集的数据进行处理，得到POI的三维信息并将POI的三维信息发送给车辆2；车辆2可以通过显示装置显示该POI的三维信息。
以上车辆1可以为进行数据采集的车辆，车辆2可以为数据使用的车辆。进行数据采集的车辆也可以称之为众包车辆且众包车辆的数量可以为多个。
示例性的，云端服务器可以使用检测大模型识别通用物体，如分割一切大模型(segment anything model，SAM)。将图像中的所有内容进行分割检测，并进行类别匹配，实现内容检测。
示例性的，云端服务器可以通过光学字符识别(optical character recognition，OCR)图文大模型识别文字信息，如CLIP4STR技术。将图像中的内容用文字描述，并将识别到的图像中已有的文字内容，与上述检测大模型得到的检测结果进行校验融合，确定图像中POI的存在性。例如，通过检测大模型可以得到电梯厅自动门的图像以及位置且通过OCR图文大模型得到的文字内容“1号电梯厅”位于该电梯厅自动门的图像的附近，那么可以确定检测大模型的检测结果准确，同时，云端服务器可以将该文字内容“1号电梯厅”与电梯厅自动门的图像进行关联。
示例性的，云端服务器可以对底图已有的POI进行丰富和更新。例如，车载地图应用中包括某个POI的位置信息且不包括该POI和该POI周围环境的三维信息。云端服务器可以对一个或者多个车辆在该POI附近采集的图像处理，从而得到该POI的三维信息。云端服务器可以将该POI的位置信息和该三维信息关联起来。
示例性的，云端服务器可以对车辆1发送的图像以及图像的位姿进行高斯溅射(Gaussian spatting)环境重建，基于稀疏视角的图像和对应位姿的输入，实现在三维中稠密连续地旋转、平移观看。
示例性的，云端服务器可以根据车辆1发送的图像、图像的深度信息以及图像对应的位姿信息，对图像进行超分辨率重建，实现从低质量图像到高质量图像的增强。
以上通过云端服务器的处理过程可以得到该POI的三维信息。可选地，若该POI为地图应用中已有的POI，云端服务器可以将该POI的三维信息发送给车辆2；或者，若该POI为地图应用中不包括的POI，云端服务器可以将该POI的位置信息和三维信息发送给车辆2。
示例性的，图8示出了本申请实施例提供的人机交互过程的示意图。该人机交互过程可以由上述车辆100或者车辆2执行。该人机交互过程包括：
S801，判断该POI中的推荐子区域是否准确。
可选地，判断该POI中的推荐子区域是否准确，包括：基于用户的反馈，判断该推荐子区域是否准确。若该推荐子区域不准确，则执行S802；若该推荐子区域准确，则可以将该推荐子区域作为导航目的地且执行S805。
示例性的，在检测到用户搜索某个POI时，车辆2可以控制提示装置向用户显示该推荐子区域且提示用户确定该推荐子区域是否准确，或者，控制提示装置提示用户确定是否导航至该推荐子区域。
示例性的，如图5中的(a)和(b)所示，车辆在检测到用户搜索C公园充电区的操作时，可以通过显示屏显示提示框503，该提示框503中包括提示信息“是否导航至该充电桩？”。
可选地，在检测到用户点击控件504的操作时，车辆100还可以提示用户确认该充电桩识别有误，或者，该充电桩识别准确且用户希望更换一个充电桩。在检测到用户确认该充电桩识别有误的操作时，可以执行S802和S804。
S802，若该推荐子区域不准确，判断该POI中是否存在准确的子区域。
示例性的，如图5中的(c)所示，车辆可以提示用户确定三维信息中是否存在充电桩的信息。
若该POI中存在准确的子区域，则执行S803；否则，执行S804。
S803，提示用户从该POI中的多个子区域中选择一个子区域。
示例性的，如图5中的(d)所示，车辆可以提示用户从该充电区的多个充电桩中选择一个充电桩。在检测到用户选择某个充电桩的操作时，可以将该充电桩作为导航目的地。
S804，向云端服务器发送指令，该指令用于指示重新构建该POI的三维信息，或者，该指令用于指示对该不准确的子区域进行重新识别。
S805，控制显示装置显示从当前位置到导航目的地的导航信息。
图9示出了本申请实施例提供的图像显示方法900的示意性流程图。该方法900可以由上述车辆100或者数据使用的车辆执行；或者，该方法900可以由上述计算平台120执行；或者，该方法900可以由上述计算平台120中的处理器、芯片或者电路执行。该方法900包括：
S910，获取云端服务器发送的第一兴趣点POI的三维信息，该三维信息由一个或者多个车辆处于该第一POI周围时通过摄像头采集的图像以及该图像对应的位姿信息确定。
可选地，以该方法900的执行主体是车辆为例，车辆可以通过空中下载(over the air，OTA)技术从云端服务器获取到该第一POI的三维信息。
S920，控制显示装置显示该三维信息。
当前POI只能以文字或者平面信息的方式呈现，对于复杂区域的POI，仅靠文字和平面图难以在范围内进行位置确认。例如，用户通过地图应用搜索某个公园时，搜索结果中可能只包括公园的入口位置。如果用户想要前往公园中某个建筑物附近，只能在车辆行驶至公园的入口位置后凭借用户的经验人工驾驶车辆寻找该建筑物，这样会增加驾驶员导航前往该建筑物的时间，导致用户的驾驶体验较差。
基于上述技术方案，通过接收云端服务器发送的公园的三维信息，该三维信息中可以包括公园中各个建筑物的图像等，可以使得用户通过该三维信息对于公园有全方位了解，从而方便用户对于目的地(例如，公园中的某个建筑物)进行确认，有助于提升用户的体验。
可选地，控制显示装置显示该三维信息之前，该方法还包括：检测到用户的指令，该指令用于指示搜索第一地点。
示例性的，如图4中的(b)所示，在检测到用户在搜索框401中输入“B商场”并点击搜索结果中的B商场地下一层停车场时，可以控制显示屏显示B商场地下一层停车场的三维信息。
可选地，获取云端服务器发送的第一POI的三维信息，包括：获取云端服务器发送的第一POI以及第一POI周围环境的三维信息。
示例性的，如图3中的(d)所示，在检测到用户在搜索框303中输入“A商场”并点击搜索结果中的A商场地下一层1号电梯厅时，可以控制显示屏显示1号电梯厅以及1号电梯厅周围环境的三维信息。
示例性的，如图3中的(e)所示，以该第一POI为电梯厅入口为例，通过显示装置显示该电梯厅以及该电梯厅周围环境的三维信息，可以方便用户查看到距离该电梯厅入口最近的停车位的信息，从而可以方便用户驾驶车辆行驶至该停车位。这样，可以避免用户靠经验在地下车库内寻找该电梯厅入口，有助于提升用户从当前位置行驶至目的地的效率。
或者，在检测到用户开启智能驾驶功能后，车辆可以从当前位置自动驾驶至该停车位。这样，避免车辆自动驾驶至地下车库的入口后由用户接管车辆并依据用户的经验人工驾驶车辆行驶至该电梯厅周围的停车位的过程，有助于拓展车辆自动驾驶的能力边界，从而有助于提升用户的智能驾驶体验。
可选地，该第一POI的三维信息由云端服务器基于一个或者多个车辆处于第一POI周围时上传的图像以及该图像的位姿信息确定。
示例性的，云端服务器可以执行如下操作，得到该第一POI的三维信息：对一个或者多个车辆处于第一POI周围时上传的图像进行分割，得到图像中不同区域的通用物体以及其类别；使用文字提取技术对图像中的文字进行提取并将文本内容与通用物体进行匹配；基于图像以及图像对应的位姿信息进行环境重建和定位；基于图像、图像的深度信息以及图像对应的位姿信息，对图像超分辨重建，实现从低质量图像到高质量图像的增强。
可选地，以该三维信息是三维模型为例，该方法900还包括：在检测到用户针对该三维模型的第一触控操作时，控制显示装置显示该三维模型旋转或者平移。
可选地，以该三维信息是三维模型为例，该方法900还包括：在检测到用户针对该三维模型的第二触控操作时，控制显示装置显示该三维模型中第一位置处正视图的图像。
可选地，该方法900还包括：控制显示装置显示第一提示信息，该第一提示信息用于请求用户确认是否导航至该第一POI；在获取到用户的第一指令时，控制该显示装置显示从终端设备的当前位置到该第一POI的导航信息，该第一指令指示导航至该第一POI。
示例性的，如图3中的(e)所示，车辆可以通过显示屏显示提示信息“请您确认是否导航至1号电梯厅？”。在检测到用户点击控件307的操作时，车辆可以通过显示屏显示从当前位置到1号电梯厅的导航信息。
本申请实施例中，通过用户的反馈，车辆可以检测云端服务器构建的第一POI的三维信息的准确性。若用户反馈导航至该第一POI，可以确认该第一POI的信息准确，从而可以显示从当前位置到该第一POI的导航信息。
可选地，该终端设备为车辆，该方法900还包括：根据该导航信息，控制该车辆从该当前位置行驶至该第一POI。
以第一POI为上述1号电梯厅为例，该1号电梯厅可以位于A商场地下一层。车载地图应用中可能仅包括A商场地下一层的入口位置，当前车辆可以实现从当前位置自动驾驶至该A商场地下一层的入口位置。在车辆到达该入口位置后，车辆会提示用户接管车辆，随后由用户凭借经验人工驾驶车辆行驶至该1号电梯厅附近。这样，需要用户人工驾驶车辆从入口到1号电梯厅，会增加从当前位置到1号电梯厅的时长，影响用户的驾驶体验。
本申请实施例中，通过接收云端服务器发送的1号电梯厅的位置信息和三维信息，可以方便用户对1号电梯厅的三维信息进行查看并规划从当前位置到1号电梯厅的行驶轨迹。车辆可以从当前位置自动驾驶至该1号电梯厅或者距离1号电梯厅最近的停车位。这样，避免车辆自动驾驶至A商场地下一层的入口后，用户接管车辆并依据用户的经验人工驾驶车辆行驶至该该1号电梯厅的过程，有助于拓展车辆自动驾驶的能力边界，从而有助于提升用户的智能驾驶体验。
可选地，根据该导航信息，控制该车辆从该当前位置行驶至该第一POI，包括：根据该导航信息，控制该车辆从该当前位置行驶至距离该第一POI最近的下车地点。
示例性的，在检测到用户开启智能驾驶功能后，车辆可以自动驾驶至距离该1号电梯厅最近的下车地点(如图3中的(e)所示的立柱处)。在检测到用户在距离该1号电梯厅最近的下车地点下车后，车辆可以自动泊入任意合理的停车位中。
可选地，该三维信息包括多个子区域中每个子区域的图像以及每个子区域的图像的位姿信息，该方法还包括：在检测到用户从该多个子区域中选择第一子区域的操作时，控制该显示装置显示从终端设备 的当前位置到该第一子区域的导航信息。
本申请实施例中，用户在查看到该三维信息后可以从停车场或者充电区中的多个子区域中选择某个子区域(导航目的地)，从而终端设备可以显示从当前位置到该子区域的导航信息。
示例性的，该第一POI为公园，该第一子区域为公园中的某个建筑物(例如，雕塑)。
示例性的，该第一POI为地下停车场，该第一子区域为地下停车场的某个停车位。
示例性的，该第一POI为充电区，该第一子区域为充电区的某个充电桩。
可选地，该三维信息包括多个子区域中每个子区域的图像以及每个子区域的图像的位姿信息，该方法还包括：控制显示装置向用户推荐第一子区域且控制显示装置显示第二提示信息，该第二提示信息用于请求用户确认是否导航至该第一子区域，该多个子区域包括该第一子区域；在获取到用户的第二指令时，控制该显示装置显示从终端设备的当前位置到该第一子区域的导航信息，该第二指令用于指示导航至该第一子区域。
示例性的，如图6所示，车辆可以通过显示屏显示1号电梯厅以及1号电梯厅周围环境的三维信息，该第一子区域为距离1号电梯厅最近的停车位。该三维信息中包括箭头305、停车位的信息、立柱的信息、1号电梯厅入口的信息以及墙面的信息，箭头305用于指示1号电梯厅的入口。同时，该三维信息中的阴影部分指示距离1号电梯厅最近的停车位。在检测到用户点击控件602的操作时，车辆可以通过显示屏显示从当前位置到该停车位的导航信息。
可选地，该终端设备为车辆，该方法还包括：根据该导航信息，控制该车辆从该当前位置行驶至该第一子区域。
可选地，该三维信息包括多个子区域中每个子区域的图像以及每个子区域的图像的位姿信息，该方法还包括：控制该显示装置向用户推荐第二子区域且控制该显示装置显示第三提示信息，该第三提示信息用于请求用户确认是否导航至该第二子区域，该多个子区域包括该第二子区域；在获取到用户的第三指令时，向该云端服务器发送第四指令，该第三指令用于指示不导航至该第二子区域，该第四指令用于指示该云端服务器重新对该三维信息或者该第二子区域进行识别。
示例性的，该第一POI可以为C公园的充电区，该第二子区域可以为如图5中的(b)所示的阴影部分区域。车辆在检测到用户搜索C公园充电区的操作时，可以通过显示屏显示提示框503且向用户推荐充电区中的某个充电桩，该提示框503中包括提示信息“是否导航至该充电桩？”。在检测到用户点击控件504的操作时，车辆可以向云端服务器发送该第四指令。
本申请实施例中，在用户反馈不导航至该第二子区域后，可以确认云端服务器构建的三维信息或者该第二子区域可能不准确，从而可以指示云端服务器重新对该三维信息进行构建或者对第二子区域进行重新识别。这样，通过用户的反馈可以实现对云端服务器构建的三维信息的准确性进行验证，在不准确的情况下对该三维信息进行重新构建，有助于提升终端设备显示的三维信息的准确性。
可选地，该方法900还包括：控制显示装置显示第四提示信息，该第四提示信息用于请求用户从该多个子区域中选择；在获取到用户的第五指令时，控制该显示装置显示从终端设备的当前位置到第三子区域的导航信息，该第五指令指示用户选择该第三子区域，该多个子区域包括该第三子区域。
示例性的，如图5中的(c)所示，在检测到用户点击控件504的操作时，车辆可以通过显示屏显示提示信息“是否存在充电桩的信息？”。在检测到用户点击控件506的操作时，车辆可以显示提示信息“请您选择一个充电桩”。在检测到用户选择了某个充电桩的信息时，车辆可以通过显示屏显示从当前位置到该充电桩的导航信息。
本申请实施例中，通过提示用户从多个子区域进行选择且检测到用户选择了第三子区域后，可以确认该第一POI中存在准确和不准确的充电桩。这样，通过用户的反馈确定三维信息中某些子区域不准确且某些区域准确的情况下，可以通过显示装置显示从当前位置到用户选择的准确的子区域的导航信息，有助于提升用户的导航体验。
可选地，该终端设备为车辆，该方法还包括：根据该导航信息，控制该车辆从该当前位置行驶至该第三子区域。
图10示出了本申请实施例提供的图像显示装置1000的示意性框图。该装置1000包括：获取单元1010，用于获取云端服务器发送的第一兴趣点POI的三维信息，该三维信息由一个或者多个车辆处于该第一POI周围时通过摄像头采集的图像以及该图像对应的位姿信息确定；控制单元1020，用于控制显示装置显示该三维信息。
可选地，该控制单元1020，还用于：控制该显示装置显示第一提示信息，该第一提示信息用于请求用户确认是否导航至该第一POI；在该获取单元获取到用户的第一指令时，控制该显示装置显示从终端设备的当前位置到该第一POI的导航信息，该第一指令指示导航至该第一POI。
可选地，该终端设备为车辆，该控制单元1020，还用于：根据该导航信息，控制该车辆从该当前位置行驶至该第一POI。
可选地，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该装置1000还包括：检测单元，用于检测到用户从该多个子区域中选择第一子区域的操作；该控制单元1020，还用于响应于检测到选择该第一子区域的操作，控制该显示装置显示从终端设备的当前位置到该第一子区域的导航信息。
可选地，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该控制单元1020，还用于：控制该显示装置向用户推荐第一子区域且控制该显示装置显示第二提示信息，该第二提示信息用于请求用户确认是否导航至该第一子区域，该多个子区域包括该第一子区域；在该获取单元1010获取到用户的第二指令时，控制该显示装置显示从终端设备的当前位置到该第一子区域的导航信息，该第二指令用于指示导航至该第一子区域。
可选地，该第一子区域为停车位。
可选地，该终端设备为车辆，该控制单元1020，还用于：根据该导航信息，控制该车辆从该当前位置行驶至该第一子区域。
可选地，该三维信息包括多个子区域中每个子区域的图像以及该每个子区域的图像的位姿信息，该控制单元1020，还用于控制该显示装置向用户推荐第二子区域且控制该显示装置显示第三提示信息，该第三提示信息用于请求用户确认是否导航至该第二子区域，该多个子区域包括该第二子区域；该装置1000还包括：发送单元，用于在该获取单元获取到用户的第三指令时，向该云端服务器发送第四指令，该第三指令用于指示不导航至该第二子区域，该第四指令用于指示该云端服务器重新对该三维信息或者该第二子区域进行识别。
可选地，该控制单元1020，还用于控制该显示装置显示第四提示信息，该第四提示信息用于请求用户从该多个子区域中选择；在该获取单元1010获取到用户的第五指令时，控制该显示装置显示从终端设备的当前位置到第三子区域的导航信息，该第五指令指示用户选择该第三子区域，该多个子区域包括该第三子区域。
可选地，该终端设备为车辆，该控制单元1020，还用于根据该导航信息，控制该车辆从该当前位置行驶至该第三子区域。
可选地，该第一POI为停车场或者充电区。
示例性的，该获取单元1010实现的功能可以由上述计算平台120中的处理器、电路或者芯片实现。以该获取单元1010实现的功能由处理器121实现为例，处理器121可以获取该POI的三维信息。
示例性的，该控制单元1020实现的功能可以由上述计算平台120中的处理器、电路或者芯片实现。以该控制元1020实现的功能由处理器122实现为例，处理器122可以控制显示装置显示该POI的三维信息。
以上获取单元1010和控制单元1020实现的功能可以由相同的处理器实现，也可以由不同的处理器实现，本申请实施例对此不作具体限定。
应理解以上装置中各单元的划分仅是一种逻辑功能的划分，实际实现时可以全部或部分集成到一个物理实体上，也可以物理上分开。此外，装置中的单元可以以处理器调用软件的形式实现；例如装置包括处理器，处理器与存储器连接，存储器中存储有指令，处理器调用存储器中存储的指令，以实现以上任一种方法或实现该装置各单元的功能，其中处理器例如为通用处理器，例如CPU或微处理器，存储器为装置内的存储器或装置外的存储器。或者，装置中的单元可以以硬件电路的形式实现，可以通过对硬件电路的设计实现部分或全部单元的功能，该硬件电路可以理解为一个或多个处理器；例如，在一种实现中，该硬件电路为ASIC，通过对电路内元件逻辑关系的设计，实现以上部分或全部单元的功能；再如，在另一种实现中，该硬件电路为可以通过PLD实现，以FPGA为例，其可以包括大量逻辑门电路，通过配置文件来配置逻辑门电路之间的连接关系，从而实现以上部分或全部单元的功能。以上装置的所有单元可以全部通过处理器调用软件的形式实现，或全部通过硬件电路的形式实现，或部分通过处理器调用软件的形式实现，剩余部分通过硬件电路的形式实现。
在本申请实施例中，处理器是一种具有信号的处理能力的电路，在一种实现中，处理器可以是具有指令读取与运行能力的电路，例如CPU、微处理器、GPU、或DSP等；在另一种实现中，处理器可以通过硬件电路的逻辑关系实现一定功能，该硬件电路的逻辑关系是固定的或可以重构的，例如处理器为ASIC或PLD实现的硬件电路，例如FPGA。在可重构的硬件电路中，处理器加载配置文档，实现硬件电路配置的过程，可以理解为处理器加载指令，以实现以上部分或全部单元的功能的过程。此外，还可以是针对人工智能设计的硬件电路，其可以理解为一种ASIC，例如NPU、TPU、DPU等。
可见，以上装置中的各单元可以是被配置成实施以上方法的一个或多个处理器(或处理电路)，例如：CPU、GPU、NPU、TPU、DPU、微处理器、DSP、ASIC、FPGA，或这些处理器形式中至少两种的组合。
此外，以上装置中的各单元可以全部或部分可以集成在一起，或者可以独立实现。在一种实现中，这些单元集成在一起，以SoC的形式实现。该SoC中可以包括至少一个处理器，用于实现以上任一种方法或实现该装置各单元的功能，该至少一个处理器的种类可以不同，例如包括CPU和FPGA，CPU和人工智能处理器，CPU和GPU等。
本申请实施例还提供了一种图像显示装置，该装置包括处理单元和存储单元，其中存储单元用于存储指令，处理单元执行存储单元所存储的指令，以使该装置执行上述实施例执行的方法或者步骤。
可选地，若该图像显示装置位于车辆中，上述处理单元可以是图1所示的处理器121-12n中的一个或者多个。
本申请实施例还提供了一种图像显示系统，该系统包括显示装置和计算平台，该计算平台包括上述图像显示装置1000。
本申请实施例还提供了一种车辆，该车辆可以包括上述图像显示装置1000或者上述图像显示系统。
本申请实施例还提供了一种计算机程序产品，所述计算机程序产品包括：计算机程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述实施例中的方法。
本申请实施例还提供了一种计算机可读介质，所述计算机可读介质存储有程序代码，当所述计算机程序代码在计算机上运行时，使得计算机执行上述实施例中的方法。
本申请实施例还提供了一种芯片，所述芯片包括电路，所述电路用于执行上述实施例中的方法。
在实现过程中，上述方法的各步骤可以通过处理器中的硬件的集成逻辑电路或者软件形式的指令完成。结合本申请实施例所公开的方法可以直接体现为硬件处理器执行完成，或者用处理器中的硬件及软件模块组合执行完成。软件模块可以位于随机存储器，闪存、只读存储器，可编程只读存储器或者上电可擦写可编程存储器、寄存器等本领域成熟的存储介质中。该存储介质位于存储器，处理器读取存储器中的信息，结合其硬件完成上述方法的步骤。为避免重复，这里不再详细描述。
应理解，本申请实施例中，该存储器可以包括只读存储器和随机存取存储器，并向处理器提供指令和数据。
还应理解，在本申请的各种实施例中，上述各过程的序号的大小并不意味着执行顺序的先后，各过程的执行顺序应以其功能和内在逻辑确定，而不应对本申请实施例的实施过程构成任何限定。
本领域普通技术人员可以意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、或者计算机软件和电子硬件的结合来实现。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
所属领域的技术人员可以清楚地了解到，为描述的方便和简洁，上述描述的系统、装置和单元的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
在本申请所提供的几个实施例中，应该理解到，所揭露的系统、装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，所述单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
所述作为分离部件说明的单元可以是或者也可以不是物理上分开的，作为单元显示的部件可以是或者也可以不是物理单元，即可以位于一个地方，或者也可以分布到多个网络单元上。可以根据实际的需要选择其中的部分或者全部单元来实现本实施例方案的目的。
另外，在本申请各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。
所述功能如果以软件功能单元的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是个人计算机，服务器，或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(read-only memory，ROM)、随机存取存储器(random access memory，RAM)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖。在本申请的保护范围之内。因此，本申请的保护范围应以所述权利要求的保护范围为准。

Claims (28)

  1. 一种图像显示方法，其特征在于，包括：
    获取云端服务器发送的第一兴趣点POI的三维信息，所述三维信息由一个或者多个车辆处于所述第一POI周围时通过摄像头采集的图像以及所述图像对应的位姿信息确定；
    控制显示装置显示所述三维信息。
  2. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    控制所述显示装置显示第一提示信息，所述第一提示信息用于请求用户确认是否导航至所述第一POI；
    在获取到用户的第一指令时，控制所述显示装置显示从终端设备的当前位置到所述第一POI的导航信息，所述第一指令指示导航至所述第一POI。
  3. 根据权利要求2所述的方法，其特征在于，所述终端设备为车辆，所述方法还包括：
    根据所述导航信息，控制所述车辆从所述当前位置行驶至所述第一POI。
  4. 根据权利要求1所述的方法，其特征在于，所述三维信息包括多个子区域中每个子区域的图像以及所述每个子区域的图像的位姿信息，所述方法还包括：
    在检测到用户从所述多个子区域中选择第一子区域的操作时，控制所述显示装置显示从终端设备的当前位置到所述第一子区域的导航信息。
  5. 根据权利要求1所述的方法，其特征在于，所述三维信息包括多个子区域中每个子区域的图像以及所述每个子区域的图像的位姿信息，所述方法还包括：
    控制所述显示装置向用户推荐第一子区域且控制所述显示装置显示第二提示信息，所述第二提示信息用于请求用户确认是否导航至所述第一子区域，所述多个子区域包括所述第一子区域；
    在获取到用户的第二指令时，控制所述显示装置显示从终端设备的当前位置到所述第一子区域的导航信息，所述第二指令用于指示导航至所述第一子区域。
  6. 根据权利要求5所述的方法，其特征在于，所述第一子区域为停车位。
  7. 根据权利要求4至6中任一项所述的方法，其特征在于，所述终端设备为车辆，所述方法还包括：
    根据所述导航信息，控制所述车辆从所述当前位置行驶至所述第一子区域。
  8. 根据权利要求1所述的方法，其特征在于，所述三维信息包括多个子区域中每个子区域的图像以及所述每个子区域的图像的位姿信息，所述方法还包括：
    控制所述显示装置向用户推荐第二子区域且控制所述显示装置显示第三提示信息，所述第三提示信息用于请求用户确认是否导航至所述第二子区域，所述多个子区域包括所述第二子区域；
    在获取到用户的第三指令时，向所述云端服务器发送第四指令，所述第三指令用于指示不导航至所述第二子区域，所述第四指令用于指示所述云端服务器重新对所述三维信息或者所述第二子区域进行识别。
  9. 根据权利要求8所述的方法，其特征在于，所述方法还包括：
    控制所述显示装置显示第四提示信息，所述第四提示信息用于请求用户从所述多个子区域中选择；
    在获取到用户的第五指令时，控制所述显示装置显示从终端设备的当前位置到第三子区域的导航信息，所述第五指令指示用户选择所述第三子区域，所述多个子区域包括所述第三子区域。
  10. 根据权利要求8或9所述的方法，其特征在于，所述终端设备为车辆，所述方法还包括：
    根据所述导航信息，控制所述车辆从所述当前位置行驶至所述第三子区域。
  11. 根据权利要求4至10中任一项所述的方法，其特征在于，所述第一POI为停车场或者充电区。
  12. 一种图像显示装置，其特征在于，包括：
    获取单元，用于获取云端服务器发送的第一兴趣点POI的三维信息，所述三维信息由一个或者多个车辆处于所述第一POI周围时通过摄像头采集的图像以及所述图像对应的位姿信息确定；
    控制单元，用于控制显示装置显示所述三维信息。
  13. 根据权利要求12所述的装置，其特征在于，
    所述控制单元，还用于：控制所述显示装置显示第一提示信息，所述第一提示信息用于请求用户确认是否导航至所述第一POI；
    在所述获取单元获取到用户的第一指令时，控制所述显示装置显示从终端设备的当前位置到所述第一POI的导航信息，所述第一指令指示导航至所述第一POI。
  14. 根据权利要求13所述的装置，其特征在于，所述终端设备为车辆，
    所述控制单元，还用于：根据所述导航信息，控制所述车辆从所述当前位置行驶至所述第一POI。
  15. 根据权利要求12所述的装置，其特征在于，所述三维信息包括多个子区域中每个子区域的图像以及所述每个子区域的图像的位姿信息，所述装置还包括：
    检测单元，用于检测到用户从所述多个子区域中选择第一子区域的操作；
    所述控制单元，还用于响应于检测到选择所述第一子区域的操作，控制所述显示装置显示从终端设备的当前位置到所述第一子区域的导航信息。
  16. 根据权利要求12所述的装置，其特征在于，所述三维信息包括多个子区域中每个子区域的图像以及所述每个子区域的图像的位姿信息，
    所述控制单元，还用于：控制所述显示装置向用户推荐第一子区域且控制所述显示装置显示第二提示信息，所述第二提示信息用于请求用户确认是否导航至所述第一子区域，所述多个子区域包括所述第一子区域；
    在所述获取单元获取到用户的第二指令时，控制所述显示装置显示从终端设备的当前位置到所述第一POI的导航信息，所述第二指令用于指示导航至所述第一子区域。
  17. 根据权利要求16所述的装置，其特征在于，所述第一子区域为停车位。
  18. 根据权利要求15至17中任一项所述的装置，其特征在于，所述终端设备为车辆，所述控制单元，还用于：
    根据所述导航信息，控制所述车辆从所述当前位置行驶至所述第一子区域。
  19. 根据权利要求12所述的装置，其特征在于，所述三维信息包括多个子区域中每个子区域的图像以及所述每个子区域的图像的位姿信息，
    所述控制单元，还用于控制所述显示装置向用户推荐第二子区域且控制所述显示装置显示第三提示信息，所述第三提示信息用于请求用户确认是否导航至所述第二子区域，所述多个子区域包括所述第二子区域；
    所述装置还包括：
    发送单元，用于在所述获取单元获取到用户的第三指令时，向所述云端服务器发送第四指令，所述第三指令用于指示不导航至所述第二子区域，所述第四指令用于指示所述云端服务器重新对所述三维信息或者所述第二子区域进行识别。
  20. 根据权利要求19所述的装置，其特征在于，
    所述控制单元，还用于控制所述显示装置显示第四提示信息，所述第四提示信息用于请求用户从所述多个子区域中选择；
    在所述获取单元获取到用户的第五指令时，控制所述显示装置显示从终端设备的当前位置到第三子区域的导航信息，所述第五指令指示用户选择所述第三子区域，所述多个子区域包括所述第三子区域。
  21. 根据权利要求19或20所述的装置，其特征在于，所述终端设备为车辆，
    所述控制单元，还用于根据所述导航信息，控制所述车辆从所述当前位置行驶至所述第三子区域。
  22. 根据权利要求15至21中任一项所述的装置，其特征在于，所述第一POI为停车场或者充电区。
  23. 一种图像显示装置，其特征在于，包括：
    存储器，用于存储计算机程序；
    处理器，用于执行所述存储器中存储的计算机程序，以使得所述装置执行如权利要求1至11中任一项所述的方法。
  24. 一种图像显示系统，其特征在于，所述系统包括显示装置和计算平台，所述计算平台包括如权利要求12-23中任一项所述的装置。
  25. 一种车辆，其特征在于，包括如权利要求12至23中任一项所述的装置，或者，包括如权利要求24所述的系统。
  26. 一种计算机可读存储介质，其特征在于，其上存储有指令，所述指令被处理器执行时，以使得处理器实现如权利要求1至11中任一项所述的方法。
  27. 一种计算机程序产品，其特征在于，所述计算机程序产品包括计算机程序代码，当所述计算机程序代码在计算机上运行时，以使得计算机实现如权利要求1至11中任一项所述的方法。
  28. 一种芯片，其特征在于，所述芯片包括电路，所述电路用于执行如权利要求1至11中任一项所 述的方法。
PCT/CN2024/099207 2024-06-14 2024-06-14 图像显示方法、装置和车辆 Pending WO2025255801A1 (zh)

Priority Applications (2)

Application Number Priority Date Filing Date Title
PCT/CN2024/099207 WO2025255801A1 (zh) 2024-06-14 2024-06-14 图像显示方法、装置和车辆
CN202480008984.XA CN121605052A (zh) 2024-06-14 2024-06-14 图像显示方法、装置和车辆

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2024/099207 WO2025255801A1 (zh) 2024-06-14 2024-06-14 图像显示方法、装置和车辆

Publications (1)

Publication Number Publication Date
WO2025255801A1 true WO2025255801A1 (zh) 2025-12-18

Family

ID=98049994

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/099207 Pending WO2025255801A1 (zh) 2024-06-14 2024-06-14 图像显示方法、装置和车辆

Country Status (2)

Country Link
CN (1) CN121605052A (zh)
WO (1) WO2025255801A1 (zh)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103258472A (zh) * 2012-02-16 2013-08-21 北京四维图新科技股份有限公司 电子地图的处理方法和装置、服务器以及系统
US20140320674A1 (en) * 2013-04-28 2014-10-30 Tencent Technology (Shenzhen) Company Limited Providing navigation information to a point of interest on real-time street views using a mobile device
KR101459005B1 (ko) * 2014-05-29 2014-11-07 (주)이지스 3차원 지도의 관심지점 표시 제어 방법
US20170330034A1 (en) * 2016-05-11 2017-11-16 Baidu Usa Llc System and method for providing augmented virtual reality content in autonomous vehicles
CN113112859A (zh) * 2021-04-07 2021-07-13 万翼科技有限公司 基于建筑信息模型的停车位确定方法与装置、相关设备
CN113610702A (zh) * 2021-08-09 2021-11-05 北京百度网讯科技有限公司 一种建图方法、装置、电子设备及存储介质
CN117128959A (zh) * 2023-04-18 2023-11-28 荣耀终端有限公司 寻车导航方法、电子设备、服务器及系统
CN117870716A (zh) * 2023-11-27 2024-04-12 广州汽车集团股份有限公司 地图兴趣点的显示方法、装置、电子设备及存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103258472A (zh) * 2012-02-16 2013-08-21 北京四维图新科技股份有限公司 电子地图的处理方法和装置、服务器以及系统
US20140320674A1 (en) * 2013-04-28 2014-10-30 Tencent Technology (Shenzhen) Company Limited Providing navigation information to a point of interest on real-time street views using a mobile device
KR101459005B1 (ko) * 2014-05-29 2014-11-07 (주)이지스 3차원 지도의 관심지점 표시 제어 방법
US20170330034A1 (en) * 2016-05-11 2017-11-16 Baidu Usa Llc System and method for providing augmented virtual reality content in autonomous vehicles
CN113112859A (zh) * 2021-04-07 2021-07-13 万翼科技有限公司 基于建筑信息模型的停车位确定方法与装置、相关设备
CN113610702A (zh) * 2021-08-09 2021-11-05 北京百度网讯科技有限公司 一种建图方法、装置、电子设备及存储介质
CN117128959A (zh) * 2023-04-18 2023-11-28 荣耀终端有限公司 寻车导航方法、电子设备、服务器及系统
CN117870716A (zh) * 2023-11-27 2024-04-12 广州汽车集团股份有限公司 地图兴趣点的显示方法、装置、电子设备及存储介质

Also Published As

Publication number Publication date
CN121605052A (zh) 2026-03-03

Similar Documents

Publication Publication Date Title
CN108139756B (zh) 为自动驾驶车辆构建周围环境以制定驾驶决策的方法和系统
CN111144211B (zh) 点云显示方法和装置
US10860868B2 (en) Lane post-processing in an autonomous driving vehicle
CN111874006A (zh) 路线规划处理方法和装置
CN107851394A (zh) 驾驶辅助装置、驾驶辅助系统、驾驶辅助方法以及自动驾驶车辆
EP3588007B1 (en) Information processing method and information processing device
US12573212B2 (en) Information processing device, information processing method, and program
US20210291653A1 (en) Display control apparatus, display control method, and computer-readable storage medium storing program
CN114771510B (zh) 基于路线图的泊车方法、泊车系统及电子设备
US12450275B2 (en) Context-based searching systems and methods for vehicles
CN120792866A (zh) 应用于智能驾驶的可行驶区域判定的方法、智能驾驶系统和智能汽车
US20210382560A1 (en) Methods and System for Determining a Command of an Occupant of a Vehicle
CN114694115A (zh) 一种道路障碍物检测方法、装置、设备及存储介质
JP2019020128A (ja) 運転支援車両の走行ルート生成方法及び走行ルート生成装置
CN115050203A (zh) 地图生成装置以及车辆位置识别装置
WO2023184468A1 (zh) 交通信号状态检测方法、装置及存储介质
WO2023028764A1 (zh) 一种自动泊车的方法、装置以及车辆
WO2024032148A1 (zh) 一种窄道通行的方法、装置和车辆
CN119590408B (zh) 基于双目摄像头的代客泊车方法、装置、设备及存储介质
CN120348313A (zh) 车辆的控制方法、装置、车辆及存储介质
CN114919598A (zh) 虚拟车道方法和系统
JP2023028609A (ja) 自動運転システム、自動運転方法およびプログラム
WO2025255801A1 (zh) 图像显示方法、装置和车辆
CN115107749B (zh) 基于自选车位的自动泊车方法
JP7614261B2 (ja) 画像認識装置、画像認識方法、およびプログラム

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24942970

Country of ref document: EP

Kind code of ref document: A1