WO2022088766A1 - Procédé de traitement d'images, support, dispositif et système de traitement d'images - Google Patents

Procédé de traitement d'images, support, dispositif et système de traitement d'images Download PDF

Info

Publication number
WO2022088766A1
WO2022088766A1 PCT/CN2021/106506 CN2021106506W WO2022088766A1 WO 2022088766 A1 WO2022088766 A1 WO 2022088766A1 CN 2021106506 W CN2021106506 W CN 2021106506W WO 2022088766 A1 WO2022088766 A1 WO 2022088766A1
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
image
angle
panoramic image
information
Prior art date
Application number
PCT/CN2021/106506
Other languages
English (en)
Chinese (zh)
Inventor
侯金龙
Original Assignee
上海博泰悦臻网络技术服务有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海博泰悦臻网络技术服务有限公司 filed Critical 上海博泰悦臻网络技术服务有限公司
Publication of WO2022088766A1 publication Critical patent/WO2022088766A1/fr

Links

Images

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
    • B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
    • B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00—Details of television systems
    • H04N5/222—Studio circuitry; Studio devices; Studio equipment
    • H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
    • B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
    • B60R2300/8073—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for vehicle security, e.g. parked vehicle surveillance, burglar detection

Definitions

  • the invention belongs to the technical field of image processing, and relates to an image processing method, medium, equipment and image processing system.
  • the panoramic images generated in the prior art are mostly realized by splicing and synthesizing 4 to 8 cameras, and the When the panoramic image is displayed, there will be problems such as lack of three-dimensional image, smear, superimposition, deformation, and display dead angle. Further, the car owner has a poor experience when viewing the displayed panoramic image or obtaining relevant information from the panoramic image.
  • one object of the present invention is to provide an image processing method, medium, electronic device and image processing system, the advantages of which are that the panoramic image around the vehicle can be highly processed in combination with the current driving information of the vehicle. Partial image display of quality picture.
  • Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, the advantages of which are that the display angle of the collected panoramic image can be automatically adjusted according to the traveling direction of the vehicle.
  • Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, the advantages of which are that the display angle of the collected panoramic image can be automatically adjusted according to the position of the target object outside the vehicle.
  • Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, the advantages of which are that a complete and clear image at any angle in a panoramic image can be manually adjusted and viewed according to user requirements.
  • Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, the advantages of which are that when a panoramic image is partially displayed, the lack of three-dimensionality, smear, ghosting, and deformation of the image is avoided. , There are problems such as display dead corners.
  • Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, the advantages of which are that the panoramic images collected by the vehicle can be shared on different terminals in real time for viewing by different target groups.
  • Another object of the present invention is to provide an image processing method, a medium, an electronic device and an image processing system, which have the advantage of being able to identify the congestion situation and prompt the owner to move the car in a parking situation.
  • Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, the advantages of which are that the identity and intention of persons approaching the vehicle can be identified and the vehicle owner is prompted.
  • Another object of the present invention is to provide an image processing system, the advantage of which is that, through the communication between the panoramic image device and the electronic device, the panoramic image generated by the panoramic image device is sent to the electronic device for image processing, so that in the subsequent The electronic device can automatically adjust the display angle of the panoramic image according to the traveling direction of the vehicle/position of the target object in the parking space, wherein the installation position of the panoramic image device of the present invention on the vehicle is optimized, so that the probability of the panoramic image device being collided is the lowest in the whole vehicle , and the integrated panoramic lens greatly reduces the difficulty of installation and post-maintenance.
  • one aspect of the present invention provides an image processing method, the method includes the following steps: acquiring a panoramic image around the vehicle and current driving information of the vehicle; angle information; obtain image display requirements that match the current driving information of the vehicle; the image display requirements include a display angle; based on the angle information consistent with the display angle, extract the panoramic image portion corresponding to the angle information , so that the panoramic image portion is presented in the in-vehicle display device in combination with the change of the image display requirements.
  • the current driving information of the vehicle includes at least current steering wheel angle information, travel speed information and gear information of the vehicle. Therefore, the corresponding partial panoramic image can be displayed in combination with the current steering wheel angle information, driving speed information and gear information of the vehicle.
  • the image display requirement further includes at least a refresh direction and a refresh frequency;
  • the vehicle determines the current steering wheel angle information and gear information of the vehicle through preset sensors and/or navigation positioning data. and driving speed information;
  • the acquiring the image display requirement matching the current driving information of the vehicle includes the following steps: calculating the display angle of the panoramic image based on the sensor and/or the navigation and positioning data;
  • the bit information detects the advancing and retreating state of the vehicle, and then determines the refresh direction of the panoramic image based on the advancing and retreating state; and determines the refresh frequency of the panoramic image based on the traveling speed information. Therefore, the image display requirement can be determined through the information provided by the vehicle data collected by the sensor and/or the navigation and positioning data, so that the displayed partial panoramic image changes with the state of the vehicle.
  • the extracting the panoramic image portion corresponding to the angle information based on the angle information consistent with the display angle includes the following steps: establishing an angle coordinate system based on the angle information to The angular coordinate system is superimposed on the panoramic image; the display angle is positioned in the angular coordinate system; and the picture to be displayed at the position where the display angle is positioned is extracted. Therefore, the deflection angle of the vehicle can be associated with the angle information of the panoramic image through the angle coordinate system, so as to locate the content to be displayed in the panoramic image through the determined display angle.
  • the method further includes the step of: presenting the panoramic image via an electronic device.
  • the electronic device for image processing can be associated with the server to provide multi-channel display of different electronic devices for the generated panoramic image, and to share the image with others; in addition, the generated panoramic image can also be backed up storage and system management.
  • the method further includes: when the vehicle is in a parked state, performing image recognition on the panoramic image to recognize the passing situation outside the vehicle; and performing voice recognition on the panoramic image , to recognize the voice content of the person outside the vehicle; based on the meeting situation and the voice content, determine whether the car needs to be moved; and in response to determining that the car needs to be moved, send a car-moving warning to the owner of the vehicle.
  • the “sentry” mode can be turned on through the recognition of the panoramic image to intelligently identify the traffic jam, so as to issue a car-moving warning to the car owner, so that the car owner can move the vehicle at the first time, and the road can be viewed more quickly. Clear the situation.
  • the method further includes the following steps: when the vehicle is in a parked state, performing image recognition on the panoramic image through machine learning to identify the identity features of persons outside the vehicle; identity feature, infer the intention of the person outside the vehicle to approach the vehicle; and in response to determining the intention of the person outside the vehicle to approach the vehicle, send prompt information to the owner of the vehicle for indicating the person outside the vehicle The intent of the person to approach the vehicle and/or the recommended response.
  • the vehicle when the vehicle is in a parked state, people around the vehicle can be intelligently identified through the recognition of panoramic images, the intention of people outside the vehicle to approach the vehicle can be predicted, and a prompt message can be sent to the vehicle owner, so that the vehicle owner can take timely measures for various abnormal situations. Countermeasures can also make the owner try to avoid abnormal parking.
  • another aspect of the present invention provides a non-transitory computer-readable storage medium having stored thereon machine-executable instructions that, when executed, cause a machine to execute the steps of the method.
  • an electronic device comprising: a memory configured to store one or more computer programs; and a processor coupled to the memory and configured to execute the The one or more computer programs cause the electronic device to perform the method.
  • a final aspect of the present invention provides an image processing system
  • the image processing system includes: a panoramic imaging device and the electronic device; the panoramic imaging device is connected to the electronic device; The panoramic image device is used to generate a panoramic image and send it to the electronic device; the electronic device is used to obtain the panoramic image around the vehicle and the current driving information of the vehicle; angle information; obtain image display requirements that match the current driving information of the vehicle; the image display requirements include a display angle; based on the angle information consistent with the display angle, extract the panoramic image portion corresponding to the angle information , so that the panoramic image portion is presented in the in-vehicle display device in combination with the change of the image display requirements.
  • FIG. 1 is a schematic flowchart of an image processing method of the present invention in an embodiment
  • FIG. 2 is a schematic diagram of driving information in an embodiment of the image processing method of the present invention.
  • FIG. 3 is a flowchart showing an image display requirement acquisition process in an embodiment of the image processing method of the present invention
  • FIG. 4A is a schematic diagram showing the calculation of a display angle in an embodiment of the image processing method of the present invention.
  • FIG. 4B is a schematic diagram showing the calculation of the display angle in another embodiment of the image processing method of the present invention.
  • FIG. 5 is a schematic diagram illustrating panoramic image extraction in an embodiment of the image processing method of the present invention.
  • FIG. 6A shows a flow chart of establishing an angular coordinate system in an embodiment of the image processing method of the present invention
  • 6B is a schematic diagram showing the establishment of an angular coordinate system in an embodiment of the image processing method of the present invention.
  • FIG. 7 is a flowchart showing a panoramic image display in an embodiment of the image processing method of the present invention.
  • FIG. 8 is a schematic diagram showing the monitoring of persons outside the vehicle in an embodiment of the image processing method of the present invention.
  • FIG. 9 is a schematic diagram showing a traffic jam detection in an embodiment of the image processing method of the present invention.
  • FIG. 10 is a schematic diagram of abnormal parking in an embodiment of the image processing method of the present invention.
  • FIG. 11 is a schematic diagram showing monitoring of children and carried items in an embodiment of the image processing method of the present invention.
  • FIG. 12 is a schematic diagram showing the structural connection of the electronic device of the present invention in an embodiment
  • FIG. 13 is a schematic structural diagram of an image processing system of the present invention in an embodiment
  • FIG. 14 is a schematic diagram illustrating an application of the image processing system of the present invention in an embodiment.
  • the image processing method of the present invention can display images from various angles of the vehicle by providing panoramic images, and process the collected panoramic images, thereby bringing more safety assurance and more convenient information to the life of the vehicle owner.
  • FIG. 1 is a schematic flow chart of the image processing method of the present invention in one embodiment. As shown in Figure 1, the image processing method specifically includes the following steps:
  • S11 Acquire a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image includes angle information corresponding to its presentation content.
  • the panoramic image is acquired through a 720° panoramic lens.
  • the 720° panoramic lens can be produced by integrating two front and rear wide-angle fisheye lenses.
  • the obtained image is a full-angle image of 720° up, down, left, and right.
  • the fixation scheme of the panoramic lens is preferably installed on the top of the vehicle, and the picture quality can be improved.
  • the panoramic lens is installed at the top of the vehicle, the collision probability is the lowest in the whole vehicle; in addition, the difficulty of installation and post-maintenance is greatly reduced by having only one panoramic lens.
  • FIG. 2 is a schematic diagram of driving information in an embodiment of the image processing method of the present invention.
  • the current driving information of the vehicle includes at least current steering wheel angle information, travel speed information and gear information of the vehicle.
  • S12 Acquire an image display requirement matching the current driving information of the vehicle; the image display requirement includes a display angle.
  • the image display requirement further includes at least a refresh direction and a refresh frequency; the vehicle determines the current steering wheel angle information, gear information and driving speed of the vehicle through preset sensors and/or navigation positioning data. information.
  • the steering wheel angle information is detected by a steering wheel angle sensor of the vehicle
  • the gear information is detected by a transmission gear sensor of the vehicle
  • the odometer sensor or wheel speed sensor of the vehicle is used to detect all the information.
  • the display angle, refresh direction, and refresh frequency of the panoramic image are determined correspondingly through the detected information.
  • the small rotational displacement of the vehicle is determined by the high-precision navigation positioning data, so that the rotational direction of the vehicle is estimated from the side, and then the display angle is determined according to the rotational direction, Combined with the gear information and driving speed information measured by the vehicle sensors, the image display requirements of the panoramic image are comprehensively determined. For example, if the user turns the vehicle to the left at an intersection, at this time, the steering wheel is turned to the left for many times, and the tangent direction of each point on the entire trajectory of the vehicle's U-turn is taken as the turning direction of the vehicle. Further, the centerline of the road can be selected.
  • the angle of the tangential direction of each point is determined from this, and the steering angle of the vehicle is then known.
  • the three parameters of the steering wheel angle information, gear information and driving speed information are inferred from the navigation and positioning data, and then the image display requirements are determined according to the three parameters, wherein the estimated value of the navigation and positioning data is determined.
  • the method can be that the vehicle communicates with the navigation and positioning device, and transmits the steering wheel angle information, gear information and driving speed information to the navigation and positioning device to obtain, specifically, the collected steering wheel angle information, gear information and driving speed through the vehicle sensor.
  • the information is transmitted to the navigation and positioning device through Bluetooth, wifi and other communication methods (for example, when the execution device of the image processing method is a mobile phone, the mobile phone receives the steering wheel angle information, gear information and driving speed information collected by the vehicle sensor to determine the display angle of the panoramic image) ; It can also be the navigation positioning data to determine the vehicle's deflection direction, forward and backward, or travel speed and other information according to the change of the vehicle's position on the map. For example, the vehicle is compared to a point in the navigation map. The position on the map is constantly changing. When the point is shifted to the left relative to the current direction, it is determined that the vehicle is turning left. When the point moves backward relative to the current position, it is determined that the vehicle is moving backward. To a certain extent, it can reflect the speed of the vehicle.
  • FIG. 3 is a flow chart of acquiring an image display requirement in an embodiment of the image processing method of the present invention.
  • S12 includes:
  • S121 Calculate the display angle of the panoramic image based on the sensor and/or the navigation and positioning data. For example, when the vehicle is driving forward, the front screen is displayed, and when the vehicle turns right, the screen can be automatically rotated with the rotation of the steering wheel.
  • FIG. 4A is a schematic diagram showing the calculation of the display angle in an embodiment of the image processing method of the present invention.
  • the gear information includes D gear and R gear
  • the steering wheel angle information is detected by a steering wheel angle sensor of the vehicle
  • the steering wheel angle information includes a left turn angle and a right turn angle.
  • the calculation of the display angle is illustrated by the following four cases.
  • the steering wheel angle information is the D gear and the left turn angle
  • the right turn angle is used as the display angle. For example, if the gear is D-forward and the steering wheel angle is 20 degrees to the right, the corresponding coordinate image is the direction of the display angle of 20 degrees.
  • the second preset angle is added to the left turn angle as the display angle.
  • the second preset angle is 180°.
  • the gear is R-reverse
  • the steering wheel angle is 40 degrees to the left
  • FIG. 4B is a schematic diagram of calculating a display angle in another embodiment of the image processing method of the present invention.
  • the display angle of the panoramic image is calculated through the navigation and positioning data, so as to realize the tracking and shooting of the driving image.
  • the panoramic image can be used to record the surrounding images, especially the target objects or objects that the owner is interested in. Predict objects that may be of interest to car owners.
  • the POI (Point of Information, information point or point of interest) location can be identified through the panoramic image, and the driver can be prompted by voice according to the surrounding POI location data, "Your right front is the Dongfeng Pearl TV Tower, whether you need to track and shoot", or The driver can use voice control or click the screen position to circle the surrounding POI content for tracking shooting.
  • the POI is identified in the obtained panoramic image based on the GPS coordinates, outline, and obvious marks of the POI location, so as to track and shoot the POI location image.
  • the POI point is the Oriental Pearl TV Tower
  • the angle between the POI point and the vehicle is calculated according to the latitude and longitude: the positioning information of the vehicle is determined through the navigation and positioning data to obtain the vehicle coordinates.
  • , so ⁇ a arctan(
  • the POI image is displayed in real time by calculating the angle every second, and the POI point image is displayed on the vehicle.
  • the clip between the Oriental Pearl TV Tower and the vehicle is determined by the navigation and positioning data.
  • the angle gradually changes from 30 degrees to 75 degrees, and accordingly, the display angle of the panoramic image also changes from 30 degrees to 75 degrees.
  • the POI can be intelligently identified according to the owner's preferences and the instructions issued, and the display angle of the POI in the panoramic image can be determined, and then the corresponding panoramic image part can be continuously tracked and displayed; and the POI coordinates in the map can be intelligently predicted through contour recognition or map.
  • the vehicle passes through landmarks (targets or events) in the area, such as landmark buildings that the owner may be interested in, so as to realize automatic tracking shooting and display of panoramic images and provide audio guide functions.
  • S122 Detect the forward/backward state of the vehicle based on the gear information, and then determine the refresh direction of the panoramic image based on the forward/backward state.
  • FIG. 5 is a schematic diagram of panoramic image extraction in an embodiment of the image processing method of the present invention.
  • S13 includes:
  • FIG. 6A is a flow chart of establishing an angular coordinate system in an embodiment of the image processing method of the present invention.
  • S131 includes:
  • the panoramic image is a 2:1 ratio image
  • the image is folded into a spherical image through a deformed graphics processing algorithm.
  • the u(x, y) motion vector of each pixel point then, the rectangular image is meshed, and the new position of each mesh vertex is recorded according to -u(x, y); finally , the new position of the mesh vertex is used as the control ver
  • the horizontal line is used as the center line, and the center horizontal image is determined.
  • FIG. 6B is a schematic diagram of establishing an angular coordinate system in an embodiment of the image processing method of the present invention.
  • Fig. 6B presents a real view of the center horizontal image, and the vehicle is located at the center of the circular angular coordinate system, so that all scenes around the circumference can be viewed.
  • the gear is D-forward and the steering wheel angle is 90 degrees to the left.
  • the orientation of the vehicle is positioned to the 270° direction in FIG. 6B , so the 270° direction is used as the display angle.
  • a tall building can be photographed directly in front of the vehicle as the part of the presented panoramic image corresponding to the display angle, that is, the picture to be displayed at the current moment.
  • FIG. 7 is a flow chart of displaying a panoramic image in an embodiment of the image processing method of the present invention.
  • the image processing method further includes S14.
  • S14 includes:
  • S142 Display a plurality of the pictures to be displayed in chronological order, so as to perform corresponding display of different parts of the panoramic image according to the real-time changes of the image display requirements.
  • the image processing method further includes:
  • the panoramic image is sent to an electronic device for displaying the panoramic image, so as to present the panoramic image.
  • the panoramic image is sent to a server, so as to send the panoramic image to an electronic device for displaying the panoramic image via the server, so as to present the panoramic image.
  • the panoramic images uploaded on the cloud can be used by family members and friends to view the panoramic videos around driving.
  • family members can see all the scenery on the road at home.
  • the viewing method of the panoramic image may be through a car, a mobile phone, an IOT (The Internet of Things) device, a HUD (Head Up Display, head-up display) device, a holographic image device, or other devices with display. function of the device is displayed.
  • the panoramic image can be viewed anytime, anywhere, not only when driving under safe conditions, such as in traffic jams or when driving normally, to view the images in the blind spot of the driving, but also through the in-vehicle display device when parking, or when relatives and friends are in the car. It can be viewed at home or other venues through mobile phones or IoT devices, or when a traffic accident occurs, it can be retrieved and viewed through the work equipment of the traffic department.
  • the panoramic image described in the present invention performs monitoring in the "sentry" mode during the parking period of the vehicle owner, and judges the possible abnormality or sudden situation of the vehicle by recognizing the shooting content of the panoramic image.
  • the panoramic image records abnormal vehicle movement, collision, scratches and other information throughout the entire process, providing evidence for later traceability.
  • FIG. 8 is a schematic diagram of monitoring of persons outside the vehicle in an embodiment of the image processing method of the present invention.
  • a 720-degree panoramic lens captures a person outside the vehicle approaching the vehicle, and then performs image recognition on the panoramic image to analyze the approaching intention of the person outside the vehicle.
  • FIG. 9 is a schematic diagram of traffic jam detection in an embodiment of the image processing method of the present invention.
  • the panoramic image records the surrounding road conditions of the vehicle.
  • the image information in the panoramic image shows that the distance between car B and car C and the owner's vehicle is relatively small.
  • the lateral distance between the rear of the owner's vehicle and the fronts of cars B and C is small, and there is a possibility of collision when continuing to drive, and it is determined that the meeting is poor.
  • the rear of the vehicle in the panoramic image and the front of car B Perform contour extraction with the front of car C, calculate the lateral distance D1 between the rear of the vehicle and the front of car B, calculate the lateral distance D2 between the rear of the vehicle and the front of car C, and calculate D1 and D2 according to the shooting ratio of the panoramic image.
  • the conversation information makes a judgment on whether the car owner needs to move the car.
  • the keywords "distance” and "moving car” in the speech are extracted, and then the natural language processing method is used to analyze the
  • the extracted keywords are semantically recognized to determine that the car owner needs to move the car. Specifically include the following steps:
  • (A2) Perform voice recognition on the panoramic image to recognize the voice content of people outside the vehicle.
  • the person outside the vehicle is the person outside the vehicle who advertises, the traffic police who post the ticket, or the security guard of a nearby building.
  • FIG. 10 is a schematic diagram of abnormal parking in an embodiment of the image processing method of the present invention.
  • the panoramic image obtained by the 720-degree panoramic lens it is recognized that there is a person outside the vehicle in work clothes approaching the vehicle. Further identify the work clothes as security clothes, volunteer clothes, traffic police clothes or company shop clothes, and identify the carry-on items of people outside the car.
  • the person outside the vehicle is a building security guard, holding a flashlight, passing by at night to check for abnormally parked vehicles in the parking space of the building. Specifically include the following steps:
  • (B1-2) Input the static picture into a pre-trained person identification model to perform image recognition, and the factors of the image recognition include the clothing of the person and the identification items carried by the person.
  • FIG. 11 is a schematic diagram of monitoring children and carried items in an embodiment of the image processing method of the present invention. As shown in Figure 11, it is recognized through the panoramic image that the car owner carries a document when entering the car and is accompanied by children, and only the car owner leaves when leaving the car.
  • the protection scope of the image processing method of the present invention is not limited to the execution order of the steps listed in this embodiment, and all the solutions implemented by adding and subtracting steps and replacing steps in the prior art based on the principles of the present invention are included in the present invention. within the scope of protection.
  • This embodiment provides a non-transitory computer-readable storage medium on which machine-executable instructions are stored, the machine-executable instructions, when executed, cause a machine to execute the image processing method.
  • the image processing method includes: acquiring a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image includes angle information corresponding to its presentation content; acquiring an image display requirement matching the current driving information of the vehicle; the The image display requirement includes a display angle; based on the angle information consistent with the display angle, extract the panoramic image part corresponding to the angle information, so as to present the image display device in the in-vehicle display device in combination with the change of the image display requirement Panoramic image section.
  • the aforementioned computer program may be stored in a computer-readable storage medium.
  • the program executes the steps including the above method embodiments; and the aforementioned computer readable storage medium includes: ROM, RAM, magnetic disk or optical disk and other computer storage media that can store program codes.
  • FIG. 12 is a schematic diagram of the structure and connection of the electronic device of the present invention in an embodiment.
  • the electronic device 1 of the present invention includes: a processor 11 and a memory 12; the memory 12 is configured to store one or more computer programs; the processor 11 is coupled to the memory 12 and It is configured to execute the one or more computer programs to cause the electronic device 1 to perform the image processing methods shown in FIGS. 1-5 , 6A and 7 .
  • the electronic device includes, but is not limited to, a tablet computer, a smart phone, a Personal Digital Assistant (PDA for short), a car terminal or a smart glasses, a smart watch or other wearable devices that can obtain panoramic images through a network connection.
  • PDA Personal Digital Assistant
  • the above-mentioned processor 11 can be a general-purpose processor, including a central processing unit (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (Digital Signal Processing, referred to as DSP) ), Application Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, and discrete hardware components.
  • the processor 11 is used to execute the computer program of the image processing method shown in FIG. 1-FIG. 5, FIG. 6A, and FIG.
  • the image processing method includes: acquiring panoramic images around the vehicle and current driving information of the vehicle;
  • the panoramic image includes angle information corresponding to its presentation content; obtains image display requirements that match the current driving information of the vehicle; the image display requirements include a display angle; based on the angle information consistent with the display angle, extracting
  • the panoramic image portion corresponding to the angle information is used to present the panoramic image portion in the in-vehicle display device in combination with the change of the image display requirements.
  • the above-mentioned memory 12 may include random access memory (Random Access Memory, RAM for short), and may also include non-volatile memory (non-volatile memory), such as at least one disk memory.
  • the memory 12 is used to store the computer program of the image processing method shown in FIG. 1-FIG. 5, FIG. 6A, and FIG. 7.
  • the image processing method includes: acquiring panoramic images around the vehicle and current driving information of the vehicle;
  • the panoramic image includes angle information corresponding to its presentation content; obtains an image display requirement that matches the current driving information of the vehicle; the image display requirement includes a display angle; based on the angle information consistent with the display angle, extracting and
  • the panoramic image portion corresponding to the angle information is used to present the panoramic image portion in the in-vehicle display device in combination with changes in the image display requirements.
  • the electronic device is a car machine, which consists of a microprocessor (MCU), a memory (ROM, RAM), an input/output interface (I/O), an analog-to-digital converter (A/D), and a plasticizer. , driver and other large-scale integrated circuits.
  • MCU microprocessor
  • ROM read-only memory
  • RAM random access memory
  • I/O input/output interface
  • A/D analog-to-digital converter
  • plasticizer e.g., driver and other large-scale integrated circuits.
  • the microprocessor (MCU) is used for executing the computer program of the image processing method
  • the memory (ROM, RAM) is used for storing the computer program of the image processing method.
  • the image processing method includes: acquiring a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image includes angle information corresponding to its presentation content; acquiring an image display requirement matching the current driving information of the vehicle; the The image display requirement includes a display angle; based on the angle information consistent with the display angle, extract the panoramic image part corresponding to the angle information, so as to present the image display device in the in-vehicle display device in combination with the change of the image display requirement Panoramic image section.
  • FIG. 13 is a schematic structural diagram of an image processing system according to an embodiment of the present invention.
  • the image processing system of the present invention includes: an electronic device 1 and a panoramic imaging device 2 ; the panoramic imaging device 2 is connected to the electronic device 1 .
  • the panoramic image device 2 is used to generate a panoramic image and send it to the electronic device 1 .
  • the electronic device 1 is used to obtain a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image includes angle information corresponding to its presentation content; obtains image display requirements that match the current driving information of the vehicle; the The image display requirement includes a display angle; based on the angle information consistent with the display angle, extract the panoramic image part corresponding to the angle information, so as to present the image display device in the in-vehicle display device in combination with the change of the image display requirement Panoramic image section.
  • FIG. 14 is a schematic diagram of an application of the image processing system of the present invention in an embodiment.
  • the panoramic imaging device is a panoramic camera
  • the electronic device is a vehicle end.
  • the panoramic camera can be installed on the top of the vehicle before the vehicle leaves the factory, or the panoramic camera can be added in the later modification process.
  • the panoramic camera is installed in the middle of the roof of the vehicle, and is connected to the Tbox controller of the vehicle through a USB data cable, so as to transmit the obtained panoramic image to the vehicle for image processing.
  • the panoramic image is uniformly calculated, processed, and output to the central control screen through the on-board controller (ie, the Tbox controller) at the vehicle end.
  • the on-board controller ie, the Tbox controller
  • the obtained panoramic images can be stored in a storage device (for example: U disk) associated with the data transmission of the Tbox controller, for the owner or the relatives and friends of the owner who has the authority and the personnel who need to view it in the follow-up. View the driving panorama video recording video.
  • the Tbox controller can also upload the imaged panoramic images to the cloud server for storage in the cloud through the 4G-SIM card, so that the uploaded panoramic images can be stored and backed up, systematically managed, and distributed to the electronic devices to be displayed according to the viewing requirements. corresponding panoramic image.
  • the image processing system of the present invention can implement the image processing method of the present invention, but the implementation devices of the image processing method of the present invention include but are not limited to the structures of the image processing system listed in this embodiment.
  • the structural deformation and replacement of the prior art made by the principle of the invention are all included in the protection scope of the present invention.
  • the image processing method, medium, device and image processing system of the present invention obtain a panoramic image through a panoramic lens, which can be dynamically imaged at 720° on a display device, and can display images from various angles of driving.
  • the image that can be automatically converted to the image while driving with the vehicle can be realized.
  • a cloud account can be used to log in to the application interface associated with the panoramic lens, so that users with permissions can view the surrounding of the vehicle from anywhere.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Mechanical Engineering (AREA)
  • Traffic Control Systems (AREA)

Abstract

L'invention concerne un procédé de traitement d'images, un support, un dispositif et un système de traitement d'images. Le procédé de traitement d'images consiste : à acquérir une image panoramique autour d'un véhicule et d'informations de déplacement actuel du véhicule, l'image panoramique comportant des informations d'angle correspondant à un contenu de présentation de l'image panoramique ; à acquérir une exigence d'affichage d'image qui correspond aux informations de déplacement actuel du véhicule, l'exigence d'affichage d'image comprenant un angle d'affichage ; et sur la base des informations d'angle qui sont cohérentes avec l'angle d'affichage, à extraire une partie d'image panoramique correspondant aux informations d'angle, de manière à présenter, dans un dispositif d'affichage embarqué et en combinaison avec un changement de l'exigence d'affichage d'image, la partie d'image panoramique. Au moyen de la fourniture d'une image panoramique, une image de chaque angle d'un véhicule peut être affichée, et l'image panoramique collectée est traitée, de sorte qu'une meilleure garantie de sécurité est amenée à environ la durée de vie d'un propriétaire de véhicule, et des informations plus pratiques et rapides sont fournies.
PCT/CN2021/106506 2020-10-29 2021-07-15 Procédé de traitement d'images, support, dispositif et système de traitement d'images WO2022088766A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011186836.7 2020-10-29
CN202011186836.7A CN114425991B (zh) 2020-10-29 2020-10-29 图像处理方法、介质、设备及图像处理系统

Publications (1)

Publication Number Publication Date
WO2022088766A1 true WO2022088766A1 (fr) 2022-05-05

Family

ID=81309778

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/106506 WO2022088766A1 (fr) 2020-10-29 2021-07-15 Procédé de traitement d'images, support, dispositif et système de traitement d'images

Country Status (2)

Country Link
CN (1) CN114425991B (fr)
WO (1) WO2022088766A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117278697A (zh) * 2023-11-21 2023-12-22 中汽传媒(天津)有限公司 一种用于汽车碰撞全景视频采集的系统和方法

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150353010A1 (en) * 2014-06-10 2015-12-10 Lg Electronics Inc. Around view provision apparatus and vehicle including the same
CN107347091A (zh) * 2017-06-30 2017-11-14 深圳市众谷智能科技有限公司 一种挪车通知处理方法及挪车指示装置
CN108382305A (zh) * 2018-02-11 2018-08-10 北京车和家信息技术有限公司 一种影像显示方法、装置及车辆
CN108583435A (zh) * 2018-04-25 2018-09-28 北京新能源汽车股份有限公司 基于车辆的显示控制方法、装置和车辆
CN110300215A (zh) * 2018-03-23 2019-10-01 上海擎感智能科技有限公司 挪车方法、挪车系统、存储介质、及电子设备
CN110341597A (zh) * 2018-04-02 2019-10-18 杭州海康威视数字技术股份有限公司 一种车载全景视频显示系统、方法及车载控制器
CN110798658A (zh) * 2018-08-03 2020-02-14 上海博泰悦臻网络技术服务有限公司 远程挪车控制方法、移动终端及车辆
CN111688698A (zh) * 2020-06-19 2020-09-22 山东交通学院 一种智能汽车的挪车方法及系统

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101376210B1 (ko) * 2012-08-06 2014-03-21 현대모비스 주식회사 어라운드 뷰 모니터 시스템 및 모니터링 방법
KR101704201B1 (ko) * 2015-05-20 2017-02-15 주식회사 와이즈오토모티브 파노라마 뷰 가변 시스템 및 이의 제어방법
CN105809701B (zh) * 2016-03-25 2019-03-08 成都易瞳科技有限公司 全景视频姿态标定方法
CN110992687B (zh) * 2019-11-25 2021-06-15 珠海格力电器股份有限公司 一种应用于停车场的车辆挪车提醒方法及存储介质
CN111757057B (zh) * 2020-06-17 2022-06-17 广州市泰睿科技有限公司 一种全景环视显示方法、装置、设备及存储介质

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150353010A1 (en) * 2014-06-10 2015-12-10 Lg Electronics Inc. Around view provision apparatus and vehicle including the same
CN107347091A (zh) * 2017-06-30 2017-11-14 深圳市众谷智能科技有限公司 一种挪车通知处理方法及挪车指示装置
CN108382305A (zh) * 2018-02-11 2018-08-10 北京车和家信息技术有限公司 一种影像显示方法、装置及车辆
CN110300215A (zh) * 2018-03-23 2019-10-01 上海擎感智能科技有限公司 挪车方法、挪车系统、存储介质、及电子设备
CN110341597A (zh) * 2018-04-02 2019-10-18 杭州海康威视数字技术股份有限公司 一种车载全景视频显示系统、方法及车载控制器
CN108583435A (zh) * 2018-04-25 2018-09-28 北京新能源汽车股份有限公司 基于车辆的显示控制方法、装置和车辆
CN110798658A (zh) * 2018-08-03 2020-02-14 上海博泰悦臻网络技术服务有限公司 远程挪车控制方法、移动终端及车辆
CN111688698A (zh) * 2020-06-19 2020-09-22 山东交通学院 一种智能汽车的挪车方法及系统

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117278697A (zh) * 2023-11-21 2023-12-22 中汽传媒(天津)有限公司 一种用于汽车碰撞全景视频采集的系统和方法
CN117278697B (zh) * 2023-11-21 2024-03-15 中汽传媒(天津)有限公司 一种用于汽车碰撞全景视频采集的系统和方法

Also Published As

Publication number Publication date
CN114425991A (zh) 2022-05-03
CN114425991B (zh) 2023-12-29

Similar Documents

Publication Publication Date Title
US11967109B2 (en) Vehicle localization using cameras
TWI633524B (zh) 交通監控系統
US20210211576A1 (en) Camera peek into turn
US10553113B2 (en) Method and system for vehicle location
García et al. Driver monitoring based on low-cost 3-D sensors
Nambi et al. Hams: Driver and driving monitoring using a smartphone
CN109664889B (zh) 一种车辆控制方法、装置和系统以及存储介质
JP2022169621A (ja) 再生装置および再生方法ならびにそのプログラムならびに記録装置および記録装置の制御方法等
WO2016138942A1 (fr) Système d'assistance pour un véhicule
US20240037935A1 (en) Artificial intelligence based real time vehicle parking verification
WO2022088766A1 (fr) Procédé de traitement d'images, support, dispositif et système de traitement d'images
JP2017129973A (ja) 運転支援装置および運転支援方法
CN114298908A (zh) 一种障碍物展示方法、装置、电子设备及存储介质
CN113570751A (zh) 一种全景环带相机的行车记录仪系统
KR20210147410A (ko) 차량 카메라를 이용한 정보 제공 서비스 방법 및 장치
JP5557036B2 (ja) 退出判定装置、退出判定プログラム及び退出判定方法
JP2009159241A (ja) 駐車時停止支援システム
JP7021899B2 (ja) 画像生成装置および画像生成方法
Li et al. Rear-end collision prevention using mobile devices
JP2014154125A (ja) 走行支援システム、走行支援方法及びコンピュータプログラム
TWI719330B (zh) 電子裝置及行車安全提醒方法
CN117755206A (zh) 车辆视野调节方法、装置、计算机设备和存储介质
JP2023104122A (ja) 情報処理装置、情報処理方法、および、情報処理装置用プログラム
CN117835052A (zh) 影像数据采集方法、终端设备以及存储介质
Peláez Coronado et al. Driver monitoring based on low-cost 3-D sensors

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21884502

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21884502

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205 DATED 04/10/2023)