CN114425991A - Image processing method, medium, device and image processing system - Google Patents
Image processing method, medium, device and image processing system Download PDFInfo
- Publication number
- CN114425991A CN114425991A CN202011186836.7A CN202011186836A CN114425991A CN 114425991 A CN114425991 A CN 114425991A CN 202011186836 A CN202011186836 A CN 202011186836A CN 114425991 A CN114425991 A CN 114425991A
- Authority
- CN
- China
- Prior art keywords
- vehicle
- angle
- image
- panoramic image
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 54
- 238000012545 processing Methods 0.000 title claims abstract description 34
- 238000000034 method Methods 0.000 claims abstract description 50
- 230000008859 change Effects 0.000 claims abstract description 19
- 230000004044 response Effects 0.000 claims description 14
- 238000004590 computer program Methods 0.000 claims description 12
- 238000013459 approach Methods 0.000 claims description 10
- 238000010801 machine learning Methods 0.000 claims description 4
- 230000003068 static effect Effects 0.000 claims description 3
- 238000003384 imaging method Methods 0.000 claims description 2
- 230000000007 visual effect Effects 0.000 claims description 2
- 230000008569 process Effects 0.000 abstract description 9
- 238000010586 diagram Methods 0.000 description 15
- 230000002159 abnormal effect Effects 0.000 description 6
- 238000004364 calculation method Methods 0.000 description 6
- 238000012544 monitoring process Methods 0.000 description 5
- 238000012986 modification Methods 0.000 description 4
- 230000004048 modification Effects 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 238000000605 extraction Methods 0.000 description 3
- 230000006870 function Effects 0.000 description 3
- 238000009434 installation Methods 0.000 description 3
- 230000005540 biological transmission Effects 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000007547 defect Effects 0.000 description 2
- 238000001514 detection method Methods 0.000 description 2
- 230000033001 locomotion Effects 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 208000012661 Dyskinesia Diseases 0.000 description 1
- 206010039203 Road traffic accident Diseases 0.000 description 1
- 230000005856 abnormality Effects 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 238000006243 chemical reaction Methods 0.000 description 1
- 238000011161 development Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 229940079593 drug Drugs 0.000 description 1
- 239000003814 drug Substances 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000003058 natural language processing Methods 0.000 description 1
- 238000007493 shaping process Methods 0.000 description 1
- 239000004984 smart glass Substances 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000002194 synthesizing effect Effects 0.000 description 1
- 230000009466 transformation Effects 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/20—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
- B60R1/22—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle
- B60R1/23—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view
- B60R1/27—Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles for viewing an area outside the vehicle, e.g. the exterior of the vehicle with a predetermined field of view providing all-round vision, e.g. using omnidirectional cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N5/00—Details of television systems
- H04N5/222—Studio circuitry; Studio devices; Studio equipment
- H04N5/262—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
- H04N5/2624—Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects for obtaining an image which is composed of whole input images, e.g. splitscreen
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/30—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the type of image processing
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B60—VEHICLES IN GENERAL
- B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
- B60R2300/00—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle
- B60R2300/80—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement
- B60R2300/8073—Details of viewing arrangements using cameras and displays, specially adapted for use in a vehicle characterised by the intended use of the viewing arrangement for vehicle security, e.g. parked vehicle surveillance, burglar detection
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Signal Processing (AREA)
- Mechanical Engineering (AREA)
- Traffic Control Systems (AREA)
Abstract
The invention provides an image processing method, medium, device and image processing system, wherein the image processing method comprises the following steps: acquiring a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirements comprise display angles; based on the angle information that coincides with the display angle, a panoramic image portion corresponding to the angle information is extracted so as to be presented in the in-vehicle display device in conjunction with a change in image display demand. The invention can display images of various angles of the vehicle by providing the panoramic image, and process the collected panoramic image, thereby bringing more safety guarantee and providing more convenient information for the life of the vehicle owner.
Description
Technical Field
The invention belongs to the technical field of image processing, and relates to an image processing method, medium, equipment and an image processing system.
Background
Along with the development of image technology, acquire the panoramic image around the vehicle and bring more convenience for the user, especially drive the in-process through 360 degrees looks around and bring better panorama visual experience for the user.
However, when the panoramic image generated in the prior art is used for image processing and analysis, the following defects exist: the panoramic image generated by the existing vehicle-mounted 360-degree panoramic system is mostly realized by splicing and synthesizing 4-8 cameras, the problems of lack of stereoscopic impression, smear, ghost, deformation, display dead angle and the like of images can exist during panoramic image display, and furthermore, the vehicle owner experiences poor experience when viewing the displayed panoramic image or acquiring related information according to the panoramic image.
Disclosure of Invention
In view of the above-described drawbacks of the prior art, it is an object of the present invention to provide an image processing method, medium, electronic device, and image processing system, which are advantageous in that a partial image display of a high-quality screen is performed on a panoramic image around a vehicle in conjunction with current driving information of the vehicle.
Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, which are advantageous in that the display angle of the captured panoramic image can be automatically adjusted according to the traveling direction of the vehicle.
Another object of the present invention is to provide an image processing method, medium, electronic device, and image processing system, which are advantageous in that the display angle of the captured panoramic image can be automatically adjusted according to the position of the target object outside the vehicle.
Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, which are advantageous in that any angle of a panoramic image can be manually adjusted and viewed according to user requirements.
Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, which are advantageous in that when a panoramic image is displayed locally, problems of lack of stereoscopic impression, smear, ghost, deformation, dead angle of display, and the like of the image are avoided.
Another object of the present invention is to provide an image processing method, medium, electronic device and image processing system, which are advantageous in that panoramic images captured by a vehicle can be shared on different terminals in real time for different target groups to view.
Another object of the present invention is to provide an image processing method, medium, electronic device, and image processing system, which are advantageous in that they can recognize a congestion situation in a parking situation and prompt a vehicle owner to move.
Another object of the present invention is to provide an image processing method, medium, electronic device, and image processing system, which are advantageous in that the identity and intention of a person approaching a vehicle can be recognized to prompt the owner of the vehicle.
Another object of the present invention is to provide an image processing system, which is advantageous in that a panoramic image generated by a panoramic image device is transmitted to an electronic device for image processing through communication between the panoramic image device and the electronic device, so that a display angle of the panoramic image can be automatically adjusted by the subsequent electronic device according to a vehicle traveling direction/a parking space target object position, wherein an installation position of the panoramic image device on a vehicle is optimized, so that a collision probability of the panoramic image device on the vehicle is the lowest, and an installation difficulty and a later maintenance difficulty are greatly reduced through an integrated panoramic lens.
To achieve the above and other related objects, an aspect of the present invention provides an image processing method, including: acquiring a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirement comprises a display angle; and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
In an embodiment of the invention, the current driving information of the vehicle at least includes current steering wheel angle information, driving speed information and gear information of the vehicle. Therefore, the corresponding local panoramic image can be displayed by combining the current steering wheel angle information, the current running speed information and the current gear information of the vehicle.
In an embodiment of the present invention, the image display requirements further include at least a refresh direction and a refresh frequency; the vehicle determines the current steering wheel angle information, gear information and running speed information of the vehicle through preset sensors and/or navigation positioning data; the method for acquiring the image display requirement matched with the current driving information of the vehicle comprises the following steps: calculating a display angle of the panoramic imagery based on the sensor and/or the navigational positioning data; detecting the advancing and retreating state of the vehicle based on the gear information, and further determining the refreshing direction of the panoramic image based on the advancing and retreating state; and determining a refresh frequency of the panoramic image based on the driving speed information. Therefore, the image display requirement can be determined through information provided by vehicle data and/or navigation positioning data collected by the sensor, so that the displayed local panoramic image changes along with the state change of the vehicle.
In an embodiment of the present invention, the extracting the panoramic image portion corresponding to the angle information based on the angle information consistent with the display angle includes the following steps: establishing an angle coordinate system based on the angle information so as to superimpose the angle coordinate system in the panoramic image; positioning the display angle in the angle coordinate system; and extracting the picture to be displayed with the display angle at the positioning position. Therefore, the deflection angle of the vehicle can be associated with the angle information of the panoramic image through the angle coordinate system, so that the content to be displayed in the panoramic image can be positioned through the determined display angle.
In an embodiment of the present invention, the method further includes the following steps: presenting the panoramic imagery via an electronic device. Therefore, the electronic equipment for image processing can be associated with the server so as to provide multi-channel display of different electronic equipment for the generated panoramic image and share the image with other people; in addition, backup storage and system management can be performed on the generated panoramic image.
In an embodiment of the present invention, the method further includes: when the vehicle is in a parking state, carrying out image recognition on the panoramic image so as to recognize the vehicle meeting condition outside the vehicle; carrying out voice recognition on the panoramic image so as to recognize the voice content of the person outside the vehicle; determining whether the vehicle needs to be moved or not based on the vehicle meeting condition and the voice content; and in response to determining that vehicle removal is required, sending a vehicle removal alert to an owner of the vehicle. From this, can be when the vehicle is in parking state, open "sentry" mode through panoramic image's discernment, the traffic congestion condition is discerned intelligently to send the warning of moving the car to the car owner, make the car owner move the vehicle at the very first time, more fast dredge the road situation.
In an embodiment of the present invention, the method further includes the following steps: when the vehicle is in a parking state, performing image recognition on the panoramic image through machine learning to recognize the identity characteristics of people outside the vehicle; inferring an intent of the person outside the vehicle to be proximate to the vehicle based on the identity feature; and in response to determining the intention of the person outside the vehicle to approach the vehicle, sending a prompt message to an owner of the vehicle for indicating the intention of the person outside the vehicle to approach the vehicle and/or recommending a countermeasure. Therefore, when the vehicle is in a parking state, people around the vehicle can be intelligently identified through identification of the panoramic image, the intention of people outside the vehicle to approach the vehicle is predicted, and a prompt message is sent to the vehicle owner, so that the vehicle owner can timely take countermeasures against various abnormal conditions, and the vehicle owner can avoid abnormal parking as much as possible.
To achieve the above and other related objects, another aspect of the present invention provides a non-transitory computer-readable storage medium having stored thereon machine-executable instructions that, when executed, cause a machine to perform the steps of the method.
To achieve the above and other related objects, a further aspect of the present invention provides an electronic device, comprising: a memory configured to store one or more computer programs; and a processor coupled to the memory and configured to execute the one or more computer programs to cause the electronic device to perform the method.
To achieve the above and other related objects, a last aspect of the present invention provides an image processing system comprising: panoramic image equipment and the electronic equipment; the panoramic image equipment is connected with the electronic equipment; the panoramic image equipment is used for generating a panoramic image and sending the panoramic image to the electronic equipment; the electronic equipment is used for acquiring a panoramic image around the vehicle and the current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirement comprises a display angle; and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
Drawings
FIG. 1 is a schematic flow chart diagram illustrating an image processing method according to an embodiment of the invention.
Fig. 2 is a schematic view of driving information of an embodiment of an image processing method according to the present invention.
Fig. 3 is a flowchart illustrating an image display requirement obtaining method according to an embodiment of the present invention.
Fig. 4A is a schematic view illustrating a display angle calculation of an image processing method according to an embodiment of the invention.
FIG. 4B is a schematic view illustrating a display angle calculation of the image processing method according to another embodiment of the invention.
Fig. 5 is a schematic view illustrating panoramic image extraction in an embodiment of the image processing method of the present invention.
Fig. 6A is a flowchart illustrating establishment of an angular coordinate system according to an embodiment of an image processing method of the present invention.
Fig. 6B is a schematic view illustrating establishment of an angular coordinate system in an embodiment of an image processing method according to the present invention.
Fig. 7 is a flowchart illustrating a panoramic image display method according to an embodiment of the present invention.
Fig. 8 is a schematic view of monitoring personnel outside a vehicle according to an embodiment of the image processing method of the present invention.
Fig. 9 is a schematic diagram illustrating a traffic jam detection method according to an embodiment of the present invention.
Fig. 10 is a schematic diagram illustrating an abnormal parking in an embodiment of an image processing method according to the present invention.
FIG. 11 is a schematic diagram illustrating a child and carried object monitoring method according to an embodiment of the present invention.
Fig. 12 is a schematic structural connection diagram of an electronic device according to an embodiment of the invention.
FIG. 13 is a schematic diagram of an embodiment of an image processing system according to the present invention.
FIG. 14 is a schematic diagram illustrating an application of the image processing system according to an embodiment of the present invention.
Description of the element reference numerals
1 electronic device
11 processor
12 memory
2 panoramic image equipment
S11-S14
S121 to S123
S131 to S133
S131A-S131C
S141 to S142
Detailed Description
The embodiments of the present invention are described below with reference to specific embodiments, and other advantages and effects of the present invention will be easily understood by those skilled in the art from the disclosure of the present specification. The invention is capable of other and different embodiments and of being practiced or of being carried out in various ways, and its several details are capable of modification in various respects, all without departing from the spirit and scope of the present invention. It is to be noted that the features in the following embodiments and examples may be combined with each other without conflict.
It should be noted that the drawings provided in the following embodiments are only for illustrating the basic idea of the present invention, and the drawings only show the components related to the present invention rather than the number, shape and size of the components in actual implementation, and the type, quantity and proportion of the components in actual implementation may be changed freely, and the layout of the components may be more complicated.
The image processing method can display images of all angles of the vehicle by providing the panoramic image, and process the acquired panoramic image, thereby bringing more safety guarantee and providing more convenient information for the life of the vehicle owner.
The principles and embodiments of an image processing method, medium, apparatus and image processing system according to the present embodiment will be described in detail below with reference to fig. 1 to 14, so that those skilled in the art can understand the image processing method, medium, apparatus and image processing system according to the present embodiment without creative work.
Referring to fig. 1, a schematic flow chart of an embodiment of an image processing method according to the invention is shown. As shown in fig. 1, the image processing method specifically includes the following steps:
s11, acquiring the panoramic image around the vehicle and the current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image.
Specifically, the panoramic image is acquired through a 720 ° panoramic lens. The 720-degree panoramic lens can be formed by integrally producing a front large wide-angle fisheye lens and a rear large wide-angle fisheye lens, the obtained images are images with an overall angle of 720 degrees from top to bottom and from left to right, the fixing scheme of the panoramic lens is preferably installed at the top end of a vehicle, the picture quality can reach the level of 4K, and therefore the problem of lack of stereoscopic impression and ghost shadow is avoided. The panoramic lens is arranged at the top end of the vehicle, so that the collision probability is the lowest for the whole vehicle; in addition, the installation difficulty and the later maintenance difficulty are greatly reduced by only one panoramic lens.
Please refer to fig. 2, which is a schematic view of driving information of an embodiment of an image processing method according to the present invention. As shown in fig. 2, the current driving information of the vehicle at least includes current steering wheel angle information, running speed information, and gear information of the vehicle.
S12, acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirements include a display angle.
In one embodiment, the image display requirements further include at least a refresh direction and a refresh frequency; the vehicle determines the current steering wheel angle information, gear information and running speed information of the vehicle through preset sensors and/or navigation positioning data. In an embodiment of this embodiment, the steering wheel angle information is detected by a steering wheel angle sensor of the vehicle, the gear information is detected by a transmission gear sensor of the vehicle, the driving speed information is detected by an odometer sensor or a wheel speed sensor of the vehicle, and the display angle, the refresh direction and the refresh frequency of the panoramic image are determined according to the detected information. In another embodiment of this embodiment, when the steering wheel rotates for a plurality of turns, the micro rotational displacement of the vehicle is determined by the high-precision navigation positioning data, so as to laterally estimate the rotational direction of the vehicle, determine the display angle according to the rotational direction, and comprehensively determine the image display requirement of the panoramic image by combining the gear information and the driving speed information measured by the vehicle sensor. For example, a user performs a vehicle turning left operation at a road junction, at this time, the steering wheel turns left for multiple turns, the tangential direction of each point on the whole trajectory of the vehicle turning left is taken as the turning direction of the vehicle, further, the road center line can be selected as a reference line, the angle of the tangential direction of each point is determined, and the steering angle of the vehicle is known. In another embodiment of this embodiment, the navigation positioning data is used to estimate three parameters, namely, the steering wheel angle information, the gear information and the driving speed information, and further determine an image display requirement according to the three parameters, where the navigation positioning data is estimated by communicating the vehicle with the navigation positioning device and transmitting the steering wheel angle information, the gear information and the driving speed information to the navigation positioning device for obtaining, and specifically, transmitting the collected steering wheel angle information, the gear information and the driving speed information to the navigation positioning device through a vehicle sensor by using a communication method, such as bluetooth and wifi (for example, when the execution device of the image processing method is a mobile phone, the mobile phone receives the steering wheel angle information, the gear information and the driving speed information and determines a display angle of the panoramic image from the vehicle sensor); the navigation positioning data may determine information such as a yaw direction, forward and backward directions, or a traveling speed of the vehicle according to a change in a position of the vehicle on a map, for example, compare the vehicle with a point in the navigation map, where the position of the point on the map changes continuously during traveling of the vehicle, determine that the vehicle turns left when the point deviates to the left with respect to the current direction, determine that the vehicle backs when the point moves backward with respect to the current position, and determine that the speed of the point moves may reflect the traveling speed of the vehicle to some extent.
Please refer to fig. 3, which illustrates a flowchart of an image display requirement obtaining method according to an embodiment of the present invention. As shown in fig. 3, S12 includes:
and S121, calculating the display angle of the panoramic image based on the sensor and/or the navigation positioning data. For example, when the vehicle is driving forward, the image right ahead is displayed, and at the time when the vehicle turns right, the image can be automatically rotated along with the rotation of the steering wheel.
Please refer to fig. 4A, which is a schematic view illustrating a display angle calculation of an image processing method according to an embodiment of the present invention. As shown in fig. 4A, the shift position information includes a D position and an R position, and the steering wheel angle information, which includes a left turn angle and a right turn angle, is detected by a steering wheel angle sensor of the vehicle. The calculation of the display angle is illustrated by the following four cases.
And in response to determining that the steering wheel angle information is the D gear and the left-turning angle, subtracting the left-turning angle from a first preset angle as the display angle. Wherein the first preset angle is 360 °. For example, if the shift position is D-forward and the steering wheel angle is 30 degrees left, the corresponding coordinate image is a direction in which the display angle is 360 ° -30 ° -330 °.
In response to determining that the steering wheel angle information is the D gear and the right turn angle, taking the right turn angle as the display angle. For example, if the shift position is D-forward and the steering wheel angle is 20 degrees, the corresponding coordinate image is the direction of the display angle of 20 °.
In response to determining that the steering wheel angle information is the R gear and the left turn angle, adding a second preset angle to the left turn angle as the display angle. Wherein the second preset angle is 180 °. For example, if the shift position is the R-reverse position and the steering wheel angle is 40 degrees left turn, the corresponding coordinate image is a direction in which the display angle of 180 ° +40 ° + 220 ° is located.
And in response to determining that the steering wheel angle information is the R gear and the right turning angle, subtracting the right turning angle from a second preset angle as the display angle. For example, if the shift position is the R-reverse position and the steering wheel angle is 10 degrees, the corresponding coordinate image is 180 ° -10 ° -170 ° in the direction of the display angle.
Please refer to fig. 4B, which is a schematic view illustrating a display angle calculation in another embodiment of the image processing method according to the present invention. As shown in fig. 4B, the display angle of the panoramic image calculated by the navigation positioning data is presented to realize the tracking shooting of the driving image, and the surrounding image, especially the target object interested by the vehicle owner or the target object possibly interested by the vehicle owner can be recorded through the panoramic image in the whole driving process.
Specifically, the POI (Point of Information or Point of interest) location is identified through the panoramic image, the driver can be prompted through voice according to the surrounding POI location data to determine whether tracking shooting is needed or not by the fact that the right front of the driver is the Dongfeng Mingzhu television tower, or the driver performs tracking shooting by performing voice control or clicking the screen position by himself to select the surrounding POI content. After the shooting instruction is output, the POI is identified in the obtained panoramic image based on the GPS coordinates, the outline and the obvious identification of the POI, so that the image of the POI is tracked and shot.
In practical application, as shown in (a) of fig. 4B, the POI point is an eastern bright pearl television tower, and the POI point and the vehicle calculate an included angle according to longitude and latitude: the positioning information of the vehicle is determined through the navigation positioning data so as to obtain the coordinates of the vehicle, the coordinates of the vehicle and the coordinates of the POI (point of interest) are determined in the navigation map, and the included angle degree can be calculated through angle calculation. For example, assuming that the vehicle coordinates are (0, 0) and the POI point location is (x, y), tan ═ a ═ x | ÷ | y |, and thus ═ a ═ arctan (| x | ÷ | y |). And displaying the panoramic image of the corresponding angle according to the included angle.
Further, as shown in (B) of fig. 4B, the angle is calculated once per second to display the POI image in real time, so as to display the POI spot image on the vehicle, for example, it is determined by the navigation positioning data that the included angle between the eastern pearl television tower and the vehicle gradually changes from 30 degrees to 75 degrees, and accordingly, the display angle of the panoramic image also changes from 30 degrees to 75 degrees.
Furthermore, when tracking shooting and panoramic image display are carried out on the oriental pearl television tower, voice explanation of the related information of the oriental pearl television tower is provided for the user, and the function of intelligent tour guide is played.
Therefore, POI can be intelligently identified according to the preference of the vehicle owner and the sent instruction, the display angle of the POI in the panoramic image is determined, and the corresponding panoramic image part is continuously tracked and displayed; and landmark objects (objects or events) in the passing area of the vehicle, such as landmark buildings in which the vehicle owner is likely to be interested, are intelligently predicted through contour recognition or POI coordinates in a map, so that automatic tracking shooting and display of a panoramic image are realized, and a voice guide function is provided.
And S122, detecting the advancing and retreating states of the vehicle based on the gear information, and further determining the refreshing direction of the panoramic image based on the advancing and retreating states.
And S123, determining the refreshing frequency of the panoramic image based on the running speed information.
S13, based on the angle information consistent with the display angle, extracting a panoramic image portion corresponding to the angle information so as to present the panoramic image portion in the in-vehicle display device in combination with the change of the image display requirement.
Please refer to fig. 5, which is a schematic view illustrating a panoramic image extraction method according to an embodiment of the present invention. As shown in fig. 5, S13 includes:
s131, establishing an angle coordinate system based on the angle information so as to superimpose the angle coordinate system on the panoramic image.
Please refer to fig. 6A, which is a flowchart illustrating an angular coordinate system establishment procedure of an embodiment of an image processing method according to the present invention. As shown in fig. 6A, S131 includes:
S131A, converting the panoramic image into a spherical image. Specifically, the panoramic image is obtained by dividing 2: and (3) folding the image with the proportion of 1 into a spherical image by a malformed image processing algorithm. Specifically, the implementation steps of the malformed graphics processing algorithm include: first, a panoramic image is enlarged into a rectangular image to perform local shape change, for example, after one source pixel point l (x, y) of an input panoramic image is enlarged by a Seam viewing algorithm, the position of the source pixel point l (x, y) is moved to l '(x, y), so that a motion field (optical flow field) u (x, y) ═ l' (x, y) -l (x, y) can be calculated, and a u (x, y) motion vector of each pixel point after the panoramic image is changed into the rectangular image is calculated; then, carrying out mesh division on the rectangular image, and recording the new position of each mesh vertex according to-u (x, y); and finally, taking the new position of the grid vertex as a control vertex, and carrying out global deformation on the panoramic image by using a deformation method of similarity transformation to obtain a spherical image.
And S131B, determining a central horizontal image by taking the horizontal line as a central marked line in the spherical image.
S131C, establishing the angular coordinate system for the central horizontal image.
S132, positioning the display angle in the angle coordinate system.
And S133, extracting the picture to be displayed with the display angle at the positioning position.
Please refer to fig. 6B, which is a schematic diagram illustrating an angle coordinate system establishment according to an embodiment of the image processing method of the present invention. As shown in fig. 6B, fig. 6B presents a live view of a central horizontal image, with the vehicle centered on a circular angular coordinate system, whereby all views of a circle can be viewed around. For example, when the vehicle is moving in a D-forward range and the steering wheel angle is 90 degrees left turn, the vehicle is oriented to be oriented to 270 ° in fig. 6B, and thus the 270 ° direction is taken as the display angle, and a high-rise building can be shot right in front of the vehicle as a displayed panoramic image portion corresponding to the display angle, that is, a picture to be displayed at the current moment.
Please refer to fig. 7, which is a flowchart illustrating a panoramic image display method according to an embodiment of the present invention. As shown in fig. 7, the image processing method further includes S14, and in one embodiment, S14 includes:
and S141, combining the change of the image display requirement, and continuously determining the pictures to be displayed at different moments.
And S142, displaying the plurality of pictures to be displayed according to the time sequence, and correspondingly displaying different parts of the panoramic image according to the real-time change of the image display requirement.
In an embodiment, the image processing method further includes:
and S15, presenting the panoramic image through the electronic equipment.
In one embodiment, the panoramic image is sent to an electronic device for displaying the panoramic image, so as to present the panoramic image.
In another embodiment, the panoramic image is sent to a server, so that the panoramic image is sent to an electronic device for displaying the panoramic image through the server, so as to present the panoramic image.
Specifically, the panoramic image uploaded at the cloud end can be used for family and friends to check panoramic videos around the travelling vehicle, the panoramic image serves as content support of on-line travelling, and all the scenery on the travelling road can be seen by family at home. In practical applications, The panoramic image may be viewed through a car machine, a mobile phone, an IOT (The Internet of Things) device, a HUD (Head Up Display) device, a holographic image device, or other devices with Display functions. The viewing of the panoramic image can be carried out at any time and any place, the image of a driving blind area can be viewed in the process of driving under the safe condition, such as traffic jam or normal driving, the image can also be viewed through display equipment in a vehicle when the vehicle is parked, the image can also be viewed by relatives and friends at home or other places through mobile phones or Internet of things equipment, and the image can also be viewed after being called by working equipment of a traffic department when a traffic accident occurs.
The panoramic image executes a sentry mode to monitor during the parking of the vehicle owner, and judges the possible abnormality or emergency of the vehicle by identifying the shooting content of the panoramic image.
In an actual application scene, the panoramic image records information of abnormal movement, collision, scratch and the like of the vehicle in the whole process, and evidences are provided for later-stage tracing.
In a practical application scene, when a vehicle owner stops temporarily (to buy medicines or to buy urgent articles), the outside personnel close to the vehicle are identified through recording and identification of the panoramic image. In practical situations, the person outside the vehicle may be a vehicle owner temporarily parking the vehicle to block the road, the vehicle owner is urgently required to be contacted to move the vehicle, the person outside the vehicle can be an advertisement-issuing person, a traffic police attaching a ticket or a security person of a nearby building, and the person outside the vehicle can also be a person about to enter the vehicle and a person about to go out of the vehicle when the vehicle is parked. Please refer to fig. 8, which is a schematic diagram of monitoring personnel outside a vehicle according to an embodiment of the image processing method of the present invention. As shown in fig. 8, it is photographed by a 720-degree panoramic lens that a person outside the vehicle approaches the vehicle, and the panoramic image is subjected to image recognition to analyze the intention of the person outside the vehicle to approach the vehicle.
On the one hand, the person outside the vehicle stops temporarily for the vehicle owner to block the road, and the person who moves the vehicle owner is urgently needed to be contacted. Please refer to fig. 9, which is a schematic diagram illustrating a traffic jam detection method according to an embodiment of the present invention. As shown in fig. 9, during the time when the vehicle owner temporarily parks and leaves the vehicle, the panoramic image records the road conditions around the vehicle in the whole process, the distance between the vehicle B and the vehicle C and the vehicle owner is smaller through the image information in the panoramic image, the transverse distance between the vehicle tail of the vehicle owner and the vehicle heads of the vehicle B and the vehicle C is smaller through monitoring in the panoramic image, and the possibility of collision is continuously generated, the vehicle-meeting situation is determined to be poor, specifically, the vehicle-meeting situation is determined by extracting the outlines of the vehicle tail, the vehicle head of the vehicle B and the vehicle head of the vehicle C in the panoramic image, calculating the transverse distance D1 between the vehicle tail of the vehicle and the vehicle head of the vehicle B, calculating the transverse distance D2 between the vehicle tail of the vehicle and the vehicle head of the vehicle C, converting the distances D1 and D2 into actual transverse distances D1 and D2 according to the shooting proportion of the panoramic image, and determining that the vehicle-meeting situation is poor when the distance D1 or D2 is smaller than 10 cm; whether the car owner needs to move the car is judged according to conversation information of the person (the driver walking out of the car C) outside the car provided by the audio information in the panoramic image. For example, when a person outside the vehicle speaks "the current distance cannot pass and needs to be in contact with the vehicle owner to move", the keywords such as "distance" and "move" in the voice are extracted through voice content recognition, and then the extracted keywords are subjected to semantic recognition by using a natural language processing method to judge that the vehicle owner needs to move. The method specifically comprises the following steps:
(A1) when the vehicle is in a temporary parking state, the panoramic image is subjected to image recognition to recognize the vehicle meeting condition outside the vehicle, and the method specifically comprises the following steps:
(A1-1) identifying vehicles of other people in the panoramic image, and measuring and calculating the distance between the vehicles of other people according to the presenting proportion of the panoramic image and the real scene.
(a1-2) comparing the inter-vehicle distance with a preset inter-vehicle distance threshold to evaluate the possibility of the other vehicle passing normally.
(a1-3) in response to determining that the inter-vehicle distance is less than the preset inter-vehicle distance threshold, determining that the inter-vehicle crossing condition is congested.
(A2) And carrying out voice recognition on the panoramic image so as to recognize the voice content of the person outside the vehicle.
(A3) And determining whether the vehicle needs to be moved according to the vehicle meeting condition and the voice content.
(A4) And responding to the determination that the vehicle moving is required, and sending a vehicle moving warning to an owner of the vehicle.
On the other hand, the person outside the vehicle is the person outside the vehicle who sends the advertisement, the traffic police who posts the ticket, or the security personnel in the nearby building, please refer to fig. 10, which shows the abnormal parking schematic diagram of the image processing method according to an embodiment of the present invention. As shown in fig. 10, it is recognized that a person outside the vehicle, who is wearing a work clothes, is approaching the vehicle in the panoramic image acquired through the 720-degree panoramic lens. Further identifying the work clothes as security clothes, volunteer clothes, traffic police clothes or shop clothes of companies, and identifying the articles carried by personnel outside the vehicle. For example, in fig. 10, the person outside the vehicle is a security guard in a building, holds a flashlight, and passes through the night to check for abnormal parking in a parking space of the building. The method specifically comprises the following steps:
(B1) when the vehicle is in a temporary parking state, the panoramic image is subjected to image recognition through machine learning so as to recognize the identity characteristics of personnel outside the vehicle. The method specifically comprises the following steps:
(B1-1) performing still picture extraction on the panoramic image.
(B1-2) inputting the static picture into a pre-trained personnel identification model for image recognition, wherein the image recognition factors comprise personnel dress and identification articles carried by personnel.
(B2) And presuming the intention of the person outside the vehicle to approach the vehicle according to the identity characteristic.
(B3) In response to determining the intention of the person outside the vehicle to approach the vehicle, sending a prompt message to an owner of the vehicle for indicating the intention of the person outside the vehicle to approach the vehicle and/or recommending a countermeasure.
On the other hand, the outside-vehicle persons are persons who are about to enter the vehicle and persons who are about to exit the vehicle when the vehicle is parked. Please refer to fig. 11, which is a schematic diagram illustrating a monitoring of a child and a carried object according to an embodiment of the image processing method of the present invention. As shown in fig. 11, it is recognized through the panoramic image that the owner carries a file and accompanies a child when entering the vehicle, and only the owner leaves the vehicle when leaving the vehicle. According to the method, the time of vehicle ignition data and flameout data is combined, an image of a vehicle owner entering the vehicle and an image of the vehicle owner leaving the vehicle in the corresponding time are extracted, image comparison is carried out, therefore, the fact that the vehicle owner forgets to carry a file or forget a child into the vehicle when leaving the vehicle is judged, a prompt is sent to the vehicle owner, the vehicle owner can flexibly process the prompt information, whether the owner really forgets the object or the child or leaves the vehicle in a short time is judged, and the object or the child can return to the vehicle immediately.
The protection scope of the image processing method according to the present invention is not limited to the execution sequence of the steps illustrated in the embodiment, and all the solutions of the prior art including step addition, step subtraction, and step replacement according to the principles of the present invention are included in the protection scope of the present invention.
The present embodiments provide a non-transitory computer readable storage medium having stored thereon machine executable instructions that, when executed, cause a machine to perform the image processing method. The image processing method comprises the following steps: acquiring a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirement comprises a display angle; and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
Those of ordinary skill in the art will understand that: all or part of the steps for implementing the above method embodiments may be performed by hardware associated with a computer program. The aforementioned computer program may be stored in a computer readable storage medium. When executed, the program performs steps comprising the method embodiments described above; and the aforementioned computer-readable storage media comprise: various computer storage media that can store program codes, such as ROM, RAM, magnetic or optical disks.
Please refer to fig. 12, which is a schematic structural connection diagram of an electronic device according to an embodiment of the present invention. As shown in fig. 12, the electronic device 1 according to the present invention includes: a processor 11 and a memory 12; the memory 12 is configured to store one or more computer programs; the processor 11 is coupled to the memory 12 and configured to execute the one or more computer programs to cause the electronic device 1 to perform the image processing method as shown in fig. 1-5, 6A, 7. The electronic device includes, but is not limited to, a tablet computer, a smart phone, a Personal Digital Assistant (PDA), a vehicle-mounted terminal or smart glasses, a smart watch, or other wearable devices that can obtain a panoramic image through network connection.
The Processor 11 may be a general-purpose Processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and the like; the Integrated Circuit may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other Programmable logic device, discrete gate or transistor logic device, discrete hardware component. The processor 11 is configured to execute a computer program of an image processing method as shown in fig. 1-5, 6A, and 7, the image processing method comprising: acquiring a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirement comprises a display angle; and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
The Memory 12 may include a Random Access Memory (RAM), and may further include a non-volatile Memory (non-volatile Memory), such as at least one disk Memory. The memory 12 is used for storing a computer program of the image processing method shown in fig. 1-5, 6A and 7, and the image processing method comprises the following steps: acquiring a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirement comprises a display angle; and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
In one embodiment, the electronic device is a vehicle, and is composed of a Microprocessor (MCU), a memory (ROM, RAM), an input/output interface (I/O), an analog-to-digital converter (a/D), and a large-scale integrated circuit such as a shaping circuit and a driving circuit. Wherein the Microprocessor (MCU) is adapted to execute the computer program of the image processing method and the memory (ROM, RAM) is adapted to store the computer program of the image processing method. The image processing method comprises the following steps: acquiring a panoramic image around the vehicle and current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirement comprises a display angle; and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
Please refer to fig. 13, which is a schematic diagram illustrating an embodiment of an image processing system according to the present invention. As shown in fig. 13, the image processing system according to the present invention includes: an electronic device 1 and a panoramic image device 2; the panoramic image device 2 is connected with the electronic device 1.
The panoramic image device 2 is configured to generate a panoramic image and send the panoramic image to the electronic device 1.
The electronic equipment 1 is used for acquiring a panoramic image around the vehicle and the current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirement comprises a display angle; and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
Please refer to fig. 14, which is a schematic diagram illustrating an application of the image processing system according to an embodiment of the present invention. As shown in fig. 14, in practical applications, the panoramic image device is a panoramic camera, and the electronic device is a car end. The panoramic camera can be installed at the top of the vehicle before the vehicle leaves a factory, and the panoramic camera can be added in the later modification process. Preferably, the panoramic camera is installed in the middle of the top of the vehicle and is connected to a Tbox controller of the vehicle through a USB data line, so as to transmit the acquired panoramic image to the vehicle for image processing. Specifically, the panoramic image is uniformly calculated and processed through a vehicle-mounted controller (namely, a Tbox controller) at a vehicle end, and the image is output to a central control screen. Further, the obtained panoramic image can be stored in a storage device (for example, a USB flash disk) which is in data transmission connection with the Tbox controller, so that the owner or relatives and friends of the owner with authority and people with viewing requirements can view the driving panoramic image recorded video in the following process. The Tbox controller can also upload the imaged panoramic image to a cloud server through a 4G-SIM card for cloud storage, so that the uploaded panoramic image can be stored, backed up, systematically managed and transmitted to the electronic equipment to be displayed according to the viewing requirement.
The image processing system according to the present invention can implement the image processing method according to the present invention, but the implementation apparatus of the image processing method according to the present invention includes, but is not limited to, the structure of the image processing system illustrated in the present embodiment, and all the structural modifications and substitutions in the prior art made according to the principle of the present invention are included in the scope of the present invention.
In summary, the image processing method, medium, device and image processing system according to the present invention can dynamically image 720 ° of the panoramic image obtained by the panoramic lens onto the display device, so as to display images of various angles of the vehicle. By combining the steering wheel angle data of the vehicle, the automatic conversion imaging picture can be realized during the vehicle-mounted running. By providing 720-degree panoramic image record of the whole vehicle when driving or parking, prompt information can be fed back to a vehicle owner according to abnormal conditions in the monitored panoramic image, and picture evidence can be provided when accidents, regulations and vehicle scratch and rub occur. The application program interface associated with the panoramic lens can be logged in through the cloud account, so that a user with authority can view the surrounding situation of the vehicle anywhere. The invention effectively overcomes various defects in the prior art and has high industrial utilization value.
The foregoing embodiments are merely illustrative of the principles and utilities of the present invention and are not intended to limit the invention. Any person skilled in the art can modify or change the above-mentioned embodiments without departing from the spirit and scope of the present invention. Accordingly, it is intended that all equivalent modifications or changes which can be made by those skilled in the art without departing from the spirit and technical spirit of the present invention be covered by the claims of the present invention.
Claims (15)
1. An image processing method, characterized in that it comprises the steps of:
acquiring a panoramic image around a vehicle and current driving information of the vehicle, wherein the panoramic image comprises angle information corresponding to the presented content of the panoramic image;
acquiring an image display requirement matched with the current driving information of the vehicle, wherein the image display requirement comprises a display angle; and
and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
2. The method of claim 1, wherein the current driving information of the vehicle includes at least current steering wheel angle information, travel speed information, and gear information of the vehicle.
3. The method of claim 2, wherein the visual display requirements further include at least a refresh direction and a refresh frequency; the vehicle determines the current steering wheel angle information, gear information and running speed information of the vehicle through preset sensors and/or navigation positioning data; the method for acquiring the image display requirement matched with the current driving information of the vehicle comprises the following steps:
calculating a display angle of the panoramic imagery based on the sensor and/or the navigational positioning data;
detecting the advancing and retreating state of the vehicle based on the gear information, and further determining the refreshing direction of the panoramic image based on the advancing and retreating state; and
and determining the refreshing frequency of the panoramic image based on the running speed information.
4. The method of claim 3, wherein the gear information includes a D gear and an R gear, and the steering wheel angle information includes a left turn angle and a right turn angle; calculating the display angle of the panoramic image based on the steering wheel angle information at least comprises one of the following steps:
in response to determining that the steering wheel angle information is the D gear and the left turn angle, subtracting the left turn angle from a first preset angle as the display angle;
in response to determining that the steering wheel angle information is the D gear and the right turn angle, taking the right turn angle as the display angle;
in response to determining that the steering wheel angle information is the R gear and the left turn angle, adding a second preset angle to the left turn angle as the display angle; and
and in response to determining that the steering wheel angle information is the R gear and the right turning angle, subtracting the right turning angle from a second preset angle as the display angle.
5. The method according to claim 1, wherein the extracting the panoramic image portion corresponding to the angle information based on the angle information in accordance with the display angle comprises:
establishing an angle coordinate system based on the angle information so as to superimpose the angle coordinate system in the panoramic image;
positioning the display angle in the angle coordinate system; and
and extracting the picture to be displayed with the display angle at the positioning position.
6. The method of claim 5, wherein said establishing an angular coordinate system based on said angular information comprises the steps of:
converting the panoramic image into a spherical image;
determining a central horizontal image by taking a horizontal line as a central marking line in the spherical image; and
and establishing the angle coordinate system for the central horizontal image.
7. The method of claim 1, wherein the method further comprises the steps of:
continuously determining the pictures to be displayed at different moments based on the change of the image display requirement; and
and displaying the plurality of pictures to be displayed according to the time sequence, so that different parts of the panoramic image are correspondingly displayed based on the real-time change of the image display requirement.
8. The method of claim 1, wherein the method further comprises the steps of: presenting the panoramic imagery via an electronic device.
9. The method of any one of claims 1-8, wherein the method further comprises:
when the vehicle is in a parking state, carrying out image recognition on the panoramic image so as to recognize the vehicle meeting condition outside the vehicle;
carrying out voice recognition on the panoramic image so as to recognize the voice content of the person outside the vehicle;
determining whether the vehicle needs to be moved or not based on the vehicle meeting condition and the voice content; and
and responding to the determination that the vehicle moving is required, and sending a vehicle moving warning to an owner of the vehicle.
10. The method of claim 9, wherein the image recognition of the panoramic imagery to identify an out-of-vehicle meeting condition comprises:
recognizing vehicles of other people in the panoramic image, and measuring and calculating the distance between the vehicles of other people based on the presentation ratio of the panoramic image and the real scene;
comparing the vehicle distance with a preset vehicle distance threshold value to evaluate the possibility that the other people normally pass through the vehicle; and
and in response to determining that the vehicle distance is smaller than the preset vehicle distance threshold value, determining that the vehicle crossing condition outside the vehicle is congestion.
11. The method according to any one of claims 1-8, wherein the method further comprises the steps of:
when the vehicle is in a parking state, performing image recognition on the panoramic image through machine learning to recognize the identity characteristics of people outside the vehicle;
inferring an intent of the person outside the vehicle to be proximate to the vehicle based on the identity feature; and
in response to determining the intention of the person outside the vehicle to approach the vehicle, sending a prompt message to an owner of the vehicle for indicating the intention of the person outside the vehicle to approach the vehicle and/or recommending a countermeasure.
12. The method of claim 11, wherein the image recognition of the panoramic imagery by machine learning to identify identity of the person outside the vehicle comprises the steps of:
extracting a static picture of the panoramic image; and
and inputting the static picture into a pre-trained personnel identity recognition model for image recognition, wherein the image recognition factors comprise personnel dresses and identification articles carried by personnel.
13. A non-transitory computer readable storage medium having stored thereon machine executable instructions which, when executed, cause a machine to perform the steps of the method of any of claims 1-12.
14. An electronic device, comprising: a memory configured to store one or more computer programs; and a processor coupled to the memory and configured to execute the one or more computer programs to cause the electronic device to perform the method of any of claims 1-12.
15. An image processing system, characterized in that the image processing system comprises: a panoramic imaging apparatus and an electronic apparatus as claimed in claim 14; the panoramic image equipment is connected with the electronic equipment;
the panoramic image equipment is used for generating a panoramic image and sending the panoramic image to the electronic equipment;
the electronic equipment is used for acquiring a panoramic image around the vehicle and the current driving information of the vehicle; the panoramic image comprises angle information corresponding to the presentation content of the panoramic image; acquiring an image display requirement matched with the current driving information of the vehicle; the image display requirement comprises a display angle; and extracting a panoramic image part corresponding to the angle information based on the angle information consistent with the display angle so as to present the panoramic image part in the in-vehicle display equipment in combination with the change of the image display requirement.
Priority Applications (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011186836.7A CN114425991B (en) | 2020-10-29 | 2020-10-29 | Image processing method, medium, device and image processing system |
| PCT/CN2021/106506 WO2022088766A1 (en) | 2020-10-29 | 2021-07-15 | Image processing method, medium, device, and image processing system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011186836.7A CN114425991B (en) | 2020-10-29 | 2020-10-29 | Image processing method, medium, device and image processing system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN114425991A true CN114425991A (en) | 2022-05-03 |
| CN114425991B CN114425991B (en) | 2023-12-29 |
Family
ID=81309778
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011186836.7A Active CN114425991B (en) | 2020-10-29 | 2020-10-29 | Image processing method, medium, device and image processing system |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN114425991B (en) |
| WO (1) | WO2022088766A1 (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117278697B (en) * | 2023-11-21 | 2024-03-15 | 中汽传媒(天津)有限公司 | System and method for collecting panoramic video of automobile collision |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103568953A (en) * | 2012-08-06 | 2014-02-12 | 现代摩比斯株式会社 | Around view monitor system and monitoring method |
| CN105809701A (en) * | 2016-03-25 | 2016-07-27 | 成都易瞳科技有限公司 | Panorama video posture calibrating method |
| KR20160136636A (en) * | 2015-05-20 | 2016-11-30 | 주식회사 와이즈오토모티브 | Panorama view variable system and method for controlling the same |
| CN110798658A (en) * | 2018-08-03 | 2020-02-14 | 上海博泰悦臻网络技术服务有限公司 | Remote vehicle moving control method, mobile terminal and vehicle |
| CN110992687A (en) * | 2019-11-25 | 2020-04-10 | 珠海格力电器股份有限公司 | Vehicle moving reminding method applied to parking lot and storage medium |
| CN111757057A (en) * | 2020-06-17 | 2020-10-09 | 广州市泰睿科技有限公司 | Panoramic all-around display method, device, equipment and storage medium |
Family Cites Families (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| KR101587147B1 (en) * | 2014-06-10 | 2016-01-20 | 엘지전자 주식회사 | apparatus for providing around view and Vehicle including the same |
| CN107347091A (en) * | 2017-06-30 | 2017-11-14 | 深圳市众谷智能科技有限公司 | One kind moves car notifier processes method and moves car instruction device |
| CN108382305B (en) * | 2018-02-11 | 2020-04-21 | 北京车和家信息技术有限公司 | Image display method and device and vehicle |
| CN110300215A (en) * | 2018-03-23 | 2019-10-01 | 上海擎感智能科技有限公司 | It moves vehicle method, move vehicle system, storage medium and electronic equipment |
| CN110341597B (en) * | 2018-04-02 | 2020-11-27 | 杭州海康威视数字技术股份有限公司 | Vehicle-mounted panoramic video display system and method and vehicle-mounted controller |
| CN108583435A (en) * | 2018-04-25 | 2018-09-28 | 北京新能源汽车股份有限公司 | Display control method, device based on vehicle and vehicle |
| CN111688698B (en) * | 2020-06-19 | 2021-05-25 | 山东交通学院 | Vehicle moving method and system of intelligent vehicle |
-
2020
- 2020-10-29 CN CN202011186836.7A patent/CN114425991B/en active Active
-
2021
- 2021-07-15 WO PCT/CN2021/106506 patent/WO2022088766A1/en active Application Filing
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103568953A (en) * | 2012-08-06 | 2014-02-12 | 现代摩比斯株式会社 | Around view monitor system and monitoring method |
| KR20160136636A (en) * | 2015-05-20 | 2016-11-30 | 주식회사 와이즈오토모티브 | Panorama view variable system and method for controlling the same |
| CN105809701A (en) * | 2016-03-25 | 2016-07-27 | 成都易瞳科技有限公司 | Panorama video posture calibrating method |
| CN110798658A (en) * | 2018-08-03 | 2020-02-14 | 上海博泰悦臻网络技术服务有限公司 | Remote vehicle moving control method, mobile terminal and vehicle |
| CN110992687A (en) * | 2019-11-25 | 2020-04-10 | 珠海格力电器股份有限公司 | Vehicle moving reminding method applied to parking lot and storage medium |
| CN111757057A (en) * | 2020-06-17 | 2020-10-09 | 广州市泰睿科技有限公司 | Panoramic all-around display method, device, equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| WO2022088766A1 (en) | 2022-05-05 |
| CN114425991B (en) | 2023-12-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11967109B2 (en) | Vehicle localization using cameras | |
| JP2023153944A (en) | Navigating vehicle based on detected barrier | |
| CN110349405A (en) | It is monitored using the real-time traffic of networking automobile | |
| US9888364B2 (en) | Localizing a smartphone in a moving vehicle | |
| García et al. | Driver monitoring based on low-cost 3-D sensors | |
| US20200126418A1 (en) | Method and system for vehicle location | |
| JP2008139320A (en) | Notification apparatus and notification program of road environmental information | |
| JP4093026B2 (en) | Road environment information notification device, in-vehicle notification device, information center device, and road environment information notification program | |
| US11294387B2 (en) | Systems and methods for training a vehicle to autonomously drive a route | |
| CN109664889B (en) | Vehicle control method, device and system and storage medium | |
| US20200120273A1 (en) | Overlooking image generation system for vehicle and method thereof | |
| CN113676702A (en) | Target tracking monitoring method, system and device based on video stream and storage medium | |
| Jiang et al. | SafeCam: Analyzing intersection-related driver behaviors using multi-sensor smartphones | |
| US20240037935A1 (en) | Artificial intelligence based real time vehicle parking verification | |
| CN112289036A (en) | Scene type violation attribute identification system and method based on traffic semantics | |
| CN110741631A (en) | Image providing system for vehicle, server system, and image providing method for vehicle | |
| WO2020210960A1 (en) | Method and system for reconstructing digital panorama of traffic route | |
| CN114425991B (en) | Image processing method, medium, device and image processing system | |
| CN113570751A (en) | Vehicle event data recorder system of panorama clitellum camera | |
| JP7021899B2 (en) | Image generator and image generation method | |
| KR102340902B1 (en) | Apparatus and method for monitoring school zone | |
| CN115134491B (en) | Image processing method and device | |
| WO2023021755A1 (en) | Information processing device, information processing system, model, and model generation method | |
| JP7235438B2 (en) | Driving evaluation device, driving evaluation method, and computer program | |
| WO2020073270A1 (en) | Snapshot image of traffic scenario |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |