WO2017197971A1 - Automobile or mobile device 3d image acquisition and naked-eye 3d head-up display system and 3d image processing method - Google Patents

Abstract

Disclosed are an automobile or mobile device 3D image acquisition and naked-eye 3D head-up display system and a 3D image processing method. The automobile or mobile device 3D image acquisition and naked-eye 3D head-up display system comprises a head-up display (9), a set of naked-eye 3D screens, a set of biomimetic 3D viewpoints (1, 2, 3, 4, 5, 6, 7, 8), a 3D intelligent center (47), automobile left side and right side biomimetic 3D viewpoint mirrors and an in-automobile naked-eye 3D rearview mirror. When a driver drives an automobile or uses a mobile device, the driver does not need to take their eyes off the road ahead to acquire information, and the driver can determine a relative distance between the automobile of the user and another automobile in an adjacent lane or a lane behind, or a pedestrian, according to 3D images from the set of biomimetic 3D viewpoints and processed by a biomimetic 3D image reconstruction technology, and displayed on the set of naked-eye 3D screens in the automobile. Moreover, an automobile naked-eye 3D display platform is provided for a biomimetic 3D navigation map and other third-party 3D navigation maps.

Description

3D image acquisition and naked eye 3D head-up display system and 3D image processing method for automobile or mobile device Technical field

The invention relates to a 3D image acquisition suitable for a car or a mobile device, a naked eye 3D head-up display system, a bionic 3D image reconstruction technology, a bionic 3D navigation map making method and a 3D navigation mode.

Background technique

The head-up display (HUD) has made people realize that if they can get the driver to get the car driving status, traffic conditions, navigation maps, communication and social information without having to bow their heads or take their eyes away from the road ahead, it will greatly improve. The driver's active safe driving ability and the possibility of reducing traffic accidents. But the information provided by the heads-up display is limited, and the driver still needs to bow down to get more information and services from the car center console. Streaming media has entered the automotive industry. It has become a trend. Digital cameras can be seen everywhere in the car, but drivers can't judge the cars they drive and the adjacent lanes through the 2D images provided by traditional rearview mirrors and car cameras. The relative distance between other cars or objects or pedestrians that appear in the rear lane. There are already mobile 3D navigation services specifically for overpasses in the market, but there is still a big gap between applying 3D navigation to the entire city street and major transportation hubs, and the realization of 3D navigation requires a real car 3D playback platform. In the era of increasingly three-dimensional urban transportation network, 3D navigation is the direction of future car navigation development.

Most of the rear-mounted car head-up displays on the market today consist of a processor and a transparent screen that is attached to the dashboard or center console of the car. The original car head-up display hides the processor behind the dashboard. The projector in the processor projects navigation maps and driving status information onto a transparent screen or car front windshield. However, the information provided by the head-up display is limited. The driver needs to look down at the navigation map in the center console from time to time during the driving process, search for the music channel and adjust the air conditioner, turn to look at the left and right rear view mirrors of the car, and look up the car. Inside mirrors, etc. In order to solve the problem that the driver takes the binocular line of sight away from the road ahead to obtain information, the present invention proposes a naked eye 3D head-up display system for a car.

The driver can easily determine the relative distance between the car he is driving and other cars or objects or pedestrians in his hand through his own eyes, but he cannot make the same judgment based on the 2D images provided by the car's traditional rearview mirrors and car cameras. . In order to enable the driver to determine the relative distance between the car he is driving and other cars or objects or pedestrians appearing in the adjacent lane and the rear lane, the present invention proposes a vehicle bionic 3D viewpoint and a bionic 3D image reconstruction technique.

The Internet of Vehicles technology displays the application, navigation, communication, social, entertainment and service information of the mobile phone directly on the car screen, and controls and manages it through the touch screen. However, the hardware parameters of the car screen, including the resolution and response speed, are far less than the smooth and easy to use of the smartphone, and can not meet the requirements for display and operation of various 3D protocols and streaming media information. In order to solve the problem of multi-channel 3D image processing, display, control and management of automobiles, the present invention proposes an automobile 3D intelligent center.

The biggest challenge facing automotive 3D navigation is not only the way of collecting and acquiring 3D navigation map data, lower cost and less time, but also the problem of resolving the stereoscopic depth relationship between objects in the original scene and obtaining A car naked eye 3D playback platform. The invention not only provides a driver with a naked eye 3D playing platform, but also proposes a method for making a bionic 3D navigation map and three different 3D navigation modes.

The driver observes the traffic conditions in the adjacent lanes and rear lanes on the left and right sides of the car through the left and right side rearview mirrors and the interior rearview mirrors of the car. However, traditional rearview mirrors have large blind spots, wind resistance and noise figure, and the driver cannot judge the relative distance between the car he drives and the other cars or objects or pedestrians in the adjacent lane and the rear lane. In order to solve the problem of the conventional rearview mirror, the present invention proposes a left-side and right-side bionic 3D viewing mirror and an in-vehicle naked-eye 3D rearview mirror.

Summary of the invention

The invention provides a 3D image capturing and naked-eye 3D head-up display system and a 3D image processing method for a car or a mobile device, which not only solves the problem that the driver does not need to take his binocular line of sight away from the traffic ahead of the road when driving the car or the mobile device. Car driving status, traffic, navigation, communication and social, and information about nearby lanes and traffic directions in the rear direction, and allows the driver to follow the car's bionic 3D view from a set of naked-eye 3D screens in the car and pass through bionic 3D The 3D image processed by the image reconstruction technology determines the relative distance between the car that it drives and other cars or objects or pedestrians appearing in the adjacent lane and the rear lane, and the bionic 3D navigation map and other third-party production proposed by the present invention. The 3D navigation map provides a car naked eye 3D playback platform.

3D image capture for car or mobile devices and a naked-eye 3D head-up display system, including a heads-up display, a set of naked-eye 3D screens, a set of bionic 3D viewpoints, a 3D smart center, left and right bionic 3D viewing glasses and a car Inside the naked eye 3D rearview mirror.

A heads-up display consists of a transparent plexiglass in the shape of a flat or curved surface and a flexible or non-flexible transparent screen with a flat or curved shape attached to its inner surface. The transparent screen can be a traditional 2D screen or a naked eye 3D. screen. Transparent plexiglass materials are selected to have good transparency, sturdy, lightweight and temperature resistant materials. A layer of polarizing film can be added between the screen and the transparent plexiglass or on the outer surface of the transparent plexiglass to reduce the intensity of sunlight and the effect of various reflections on the image in the screen. The heads-up display is connected to a base, and the angle between the head-up display and the base is adjustable. The base is fixed on the dashboard of the car or on the center console. In order to design the base as a simple and not too large space, the motherboard of the screen can be placed in the 3D smart center. When the driver adjusts the angle between the heads-up display and the base, the image and the font are deformed in the vertical direction because the tilt angle of the screen relative to the driver's eyes changes, and the driver can pass through the 3D smart center. The image processing software and the hand touch screen correct the images and fonts on the heads up display screen. The corrected images, fonts and symbols will be stored in the memory unit of the 3D Smart Center. The content and screen displayed on the heads-up display are designed with simple, clear and required information, including but not limited to car speed, speed limit, safe distance for drivers, speeding or too close to other vehicles, instant messaging and social information. , short version of 3D navigation guidelines, recommended routes, etc.

If the head-up display adopts a naked-eye 3D screen, according to the principle of changing the scene convergence point in the bionic 3D image reconstruction technology, the content convergence point in the head-up display is set in the car through the image processing software built in the 3D smart center and the hand touch screen. At a position in front of the front windshield, different displays can be placed at different positions from the front of the front windshield of the car if needed. When driving in a city, because the distance between cars may be very close, the principle of setting the content convergence point is not to bring the content to the front car. Experience has shown that setting the content convergence point at the forefront of the car is a location that is more reasonable and acceptable to most drivers, considering all possible situations.

The driver gets important and necessary information through the heads-up display, and gets more and detailed content through a set of naked-eye 3D screens, including but not limited to 3D navigation and 3D navigation maps, 3D panoramic images, 3D entertainment, communication, SMS, WeChat, Other content and services. A set of naked-eye 3D screens employ one of two different design patterns described below. The first design mode is the naked-eye 3D workbench mode. The naked-eye 3D table is a flexible or non-flexible naked eye 3D consisting of a flat or curved shaped back plate or a bent flat plate-shaped back plate and one or more flat or curved shapes fixed on the inner surface of the back plate. The screen is composed. The back plate can be directly fixed to the base or connected to the base by a bracket that is fixed on the center console of the front of the car. The driver can adjust the position of the back and the screen up and down, the left and right rotation and the up and down elevation through the bracket to obtain a position for better and convenient viewing of the screen content. If the back panel is fixed to the base, a transparent plexiglass back panel and a transparent naked-eye 3D screen can be used. When the driver does not use the screen, the front and rear passengers can move and rotate the screen to a position that is appropriate for them to view. The screen stand and base are designed with an ultimate blocking mechanism to ensure that the screen does not move with the front passengers when it is moved and rotated to the extreme position. The full airbags collided during the outbreak. In order to reduce the weight and space occupied by the naked-eye 3D workbench, the motherboards of all screens can be placed in the 3D smart center. The second design mode is the naked-eye 3D console mode. This mode is suitable for an in-vehicle 3D display device designed by an automobile manufacturer for a car. The naked-eye 3D console is composed of a plurality of flat or curved shaped flexible or non-flexible naked-eye 3D screens; multiple screens can be stitched together into one continuous set of screens or differently placed in the car according to different display contents or purposes. Location. Each screen can play different content, or you can splicing together to play an image with an extra wide picture. Each screen can vary in size and shape. The number, arrangement, location, and arrangement of screens used in the naked-eye 3D console mode is different for each different make and model of car or mobile device.

Studies have shown that the range of human eyes can be up to 160 degrees from horizontal and 80 degrees from vertical. In general, the central region of the human eye can clearly distinguish images when it is within about 15 degrees of view. When the horizontal viewing angle is between 15 degrees and 35 degrees, the human eye can see the existence and motion of the object, and it is possible to discern the degree of clarity without turning the head, but the resolution has been reduced. When the horizontal angle of view is between 35–60 degrees, the eyes of the person can distinguish the color, but the object and the motion details cannot be determined. If you need to distinguish the object clearly, you need to turn the eyeball or the head to let the object or motion details fall into the eye. The horizontal viewing angle is within 35 degrees. When the horizontal viewing angle is between 60 and 120 degrees, the human eye can only distinguish the color and the blurred object and the direction of motion. At this time, the head must be turned to distinguish the object and the motion details. In fact, the human eye can clearly see that the object is in an elliptical cone with a horizontal viewing angle of about 35 degrees and a vertical viewing angle of about 20 degrees. The human eyes will have a very good visual presence in this elliptical cone area and will not cause fatigue due to frequent eyeball rotation. The driver can look at the information displayed in the head-up display by looking down or down from the road ahead for about 5–10 degrees, and the car dashboard data can be seen at about 20–25 degrees. The position of the equipment in the center console of the car is lower than the position of the car dashboard and is located in the lower right direction of the instrument panel. When the driver views the car navigation screen in the center console, searching for the music channel and adjusting the air conditioning knob requires both eyes and head to be offset 20-20 degrees to the right and 25 to 45 degrees below the head. At this time, the driver has completely left the eyes of the eyes away from the road ahead. If the car speed is faster at this time, the seconds left by the eyes leaving the road ahead will cause the car to travel tens of meters to hundreds of meters, posing a great risk to the life of the driver and the passengers in the car. The screen of the car naked eye 3D table is designed to be close to or higher than the height of the car dashboard in the horizontal direction. When the driver is facing the front of the car, a vertical line at the midpoint of the driver's eyes is the intersection of two vertical planes. The horizontal tangent on one vertical plane faces the front of the car, and the other vertical plane passes. In the above [0012], the center point of the backplane of the naked eye 3D workbench, the angle between the two planes does not exceed 75 degrees. When the driver turned his head to view the content played on the naked-eye 3D workbench screen, part of the residual light of both eyes still included the road traffic condition in front of the car, and did not completely lose the observation of the traffic state ahead.

The most frequently used 3D shooting methods are convergence and parallel. The convergence method is a 3D shooting method that imitates the way people focus on the scene. The left and right images with different viewing angles obtained when shooting using the convergence method will have a trapezoidal distortion phenomenon, and the closer to the attention of the subject, the larger the distortion. The trapezoidal distortion causes the edges of the two images to be unable to perform perfect 3D fusion in the vertical direction, which is one of the main causes of eye fatigue and physical discomfort when the viewer feels the 3D image. The parallel method is a method of 3D shooting of two cameras placed in parallel against the subject. The parallel method does not conform to the way and habits of people's eyes to pay attention to the scene. The 3D image merged in the 3D playback space appears in front of the screen, and is not an ideal 3D playback effect. However, the two images with different viewing angles obtained by the parallel method have no keystone distortion, and the parameters set at the time of shooting can be corrected by manual intervention after the shooting is completed, so that the reconstructed two have different viewing angles. The image is converged on the only corresponding point in the 3D playback space. This method not only conforms to the human eye viewing habits but also minimizes image distortion. Experience has shown that two images with different viewing angles obtained using the parallel method have a natural, comfortable and healthy 3D feel and effect after being processed by 3D image reconstruction technology.

A bionic 3D viewpoint is a device that has a camera or two cameras that can capture 3D images. In a bionic 3D viewpoint with two cameras, the centerlines of the two cameras are on the same horizontal plane and parallel to each other, with a parallel spacing of 5–80 mm between the two centerlines. Manufacturer of two cameras, The brand, model and parameters should be identical, ensuring that the two images with different viewing angles are close in color, white balance, depth of field, expressiveness and quality. The cover plate facing the camera lens in the bionic 3D viewpoint is a transparent glass coated with a transparent coating on the inner and outer surfaces, and the transparent glass surface coated with a transparent coating has a self-cleaning function of repelling water, mist and foreign matter. The human eye is an optical structure that is suitable for optical theory. The closer the camera lens parameters are to the optical system of the human eye, the closer the stereoscopic relationship between the captured image and the surrounding scene seen by the eyes in the original scene. The results of the study show that the human eye has a focal length of approximately 16.65 mm and a corresponding viewing angle of 120 degrees. A wide-angle lens can cause barrel distortion in an image. Two images with barrel distortion cannot achieve perfect 3D image fusion in the vertical direction, so each camera lens in the bionic 3D viewpoint has a viewing angle of less than 120 degrees.

The car 3D image acquisition system collects 3D images around the car and in a specific direction by setting a bionic 3D viewpoint at a plurality of different positions of the car. Each bionic 3D viewpoint is labeled with a different number or name depending on the location of the car. For the car, the bionic 3D viewpoint 1 is placed below or above the center of the inner surface of the front windshield of the car or at the center or near the center of the front of the car, and the camera lens is directed toward the front of the car. The bionic 3D viewpoint 2 is placed below or above the center of the inner surface of the rear window of the automobile or at the center of the rear cover or near the center, and the camera lens is directed toward the front of the vehicle. Bionic 3D viewpoints 3 and 4 and bionic 3D viewpoints 5 and 6 are respectively placed in the left and right side of the car in a conventional mirror rearview mirror or under the mirror mirror housing or on the left and right side of the bionic 3D viewing mirror bracket On the left and right side of the car. The camera lenses in the bionic 3D viewpoints 3 and 4 are respectively oriented toward the left front direction and the left rear direction of the car. The camera lenses in the bionic 3D viewpoints 5 and 6 are respectively oriented toward the right front and rear directions and the right rear direction of the car. The bionic 3D viewpoints 7 and 8 are respectively placed on the left and right side of the car or on the roof of the car, and the camera lenses are respectively directed to the left or right direction of the car.

The number and location of bionic 3D viewpoints installed in the automotive 3D image acquisition system determine the range of viewing angles of the images obtained around the car and the possible blind spots. For example, for a car equipped with bionic 3D viewpoints 1, 2, 3, 4, 5 and 6, there are two symmetrical blind zones on the left and right sides of the car. The blind area on the left side of the car is the area between the left edge of the view of the bionic 3D view 3 on the left side of the car and the right edge of the view of the bionic 3D view 4. The blind area on the right side of the car is the area between the right edge of the view of the bionic 3D view 5 on the right side of the car and the left edge of the view of the bionic 3D view 6. For a car equipped with all of the above-described bionic 3D viewpoints, the car is surrounded by a 360-degree 3D image without a blind spot.

The panoramic image of the car is a 3D image of a continuous and complete ultra-wide image by editing and splicing the images from multiple bionic 3D viewpoints through the built-in image processing software of the 3D intelligent center. The panoramic image of the front of the car comes from the bionic 3D viewpoints 1, 3 and 5. The rear panoramic image of the car comes from the bionic 3D viewpoints 2, 4 and 6. The panoramic image of the left rear of the car comes from the bionic 3D viewpoints 2, 4 and 7. The panoramic image of the rear right of the car comes from the bionic 3D viewpoints 2, 6 and 8. In the above four different global images, at least one of the bionic 3D viewpoint images in each of the whole images includes one or a part of a car or a mobile device as a reference object, for example, in the front panoramic image. The image obtained by the camera in the bionic 3D viewpoint 1 includes a logo on the front of the car or a front edge portion of the front cover of the automobile engine. The driver can determine the relative distance between the car he is driving and other cars or objects or pedestrians appearing in the adjacent lane or in the rear lane based on the reference in the 3D image played in a set of naked-eye 3D screens.

Cars with bionic 3D viewpoints 1, 2, 3, 4, 5 and 6 During normal driving, a set of naked-eye 3D screens plays a panoramic image of the front of the car. When the driver toggles the turn lever or switches the car gear to the reverse gear position to switch lanes, turn, reverse or prepare to stop, the 3D smart center automatically switches a set of naked-eye 3D screen playback content from the front panoramic image of the car to the car. Rear panoramic image. After the driver completes the lane change, turns, reverses or stops, and after the turning lever rebounds or switches the car gear to the forward gear position, the 3D smart center automatically plays the contents of a set of naked-eye 3D screens from the rear panoramic image of the car. Switch back to the panoramic image in front of the car. For a car equipped with all of the above-described bionic 3D viewpoints during normal driving, a set of naked-eye 3D screens plays a panoramic image of the front of the car. When the driver switches the turn lever or switches the car gear to the reverse gear position to switch lanes, turn, reverse or prepare to stop, the 3D smart center automatically switches the content played in a set of naked-eye 3D screens from the front panoramic image of the car. A panoramic image of the left rear or right rear of the car. When the driver completes the conversion lane, turns, reverses or After parking and after the turning lever rebounds or switches the car gear to the forward gear position, the 3D smart center automatically switches the content played in a set of naked-eye 3D screens from the left rear or right rear panoramic image of the car back to the front panoramic image of the car. . The driver can switch the content played in a set of naked-eye 3D screens by voice control at any time during driving.

In order to measure the car in the forward direction, the front direction and the left front direction in the adjacent lane, the right front direction, the left rear direction and the right rear direction, the real-time distance measurement is performed, and the range finder is installed at a corresponding position on the vehicle. . The range finder can be used in any way of sensor ranging technology and equipment in laser, ultrasonic, radar or microwave radar. The distance measurement information obtained by the range finder will directly enter the 3D intelligent center, and the distance measurement data obtained by the 3D intelligent center will be analyzed in real time through software, and the completed data will be corrected and calculated together with the pre-designed and added information. Superimposed on the heads-up display and a set of images in a set of naked-eye 3D screens, an augmented reality display method is used, such as an arrow pointing to a car or pedestrian near an ongoing distance measurement, and the information box of the connection arrow is displayed. A number indicating the distance and a brief description.

The 3D Intelligent Center reconstructs each of the two images with different viewing angles obtained from the bionic 3D viewpoint through bionic 3D image reconstruction technology, so that the reconstructed two images are less distorted after being merged by the human brain. Small chromatic aberration, natural stereo depth, health and comfort. The bionic 3D image reconstruction technology mainly includes; firstly, each of the two images with different viewing angles acquired at any point in the original scene space is translated in the horizontal and opposite directions by h=T/A. The distance is such that the two images after translation have a unique corresponding point in the 3D play space and the two images achieve convergence at the corresponding point. The second is to set the convergence point of the two images with different perspectives in the 3D playback space on the 3D playback screen by appropriately selecting the value of h and adjusting the position of the scene in front of the car. The parallax on the 3D playback screen is zero. At this time, the original scene space and the 3D play space become two ideal spaces corresponding to each other. The ideal imaging is any point in the original scene space. A straight line and a surface reproduce at corresponding points in the corresponding ideal space to a point with a similar stereoscopic depth relationship, without distortion and distortion, a straight line and a surface. The maximum amount of foreground in the original scene space that is closest to the camera lens in the 3D playback space is Z _cnear = 2TxZ _D xz _near / (txfxA). The maximum amount of screen is a parameter that needs to be controlled, because the fused 3D image is located between the viewer and the 3D screen. It is not an ideal expression for 3D images. Long-term viewing will cause fatigue in the eyes of the audience. Physiologically feeling unwell. The third is to use the theoretical maximum viewing angle difference d(a _max )=(txfxA/Z _D )x1/z _near using bionic 3D image reconstruction technology to replace the traditional empirical formula, which can make the stereo depth between different objects in the original scene. The relationship is reproduced in a 3D play space in a similar stereoscopic depth relationship with minimal changes. The current 3D playback technology is called planar 3D display technology, in which two images with different viewing angles are simultaneously projected onto a flat screen, and the horizontal distance of the two images on the screen is parallax. The position of the merged 3D image convergence point is usually controlled by adjusting the parallax of the two images. The left and right eyes of the audience can only see and focus on one of the images that are independent of each other. The images obtained by the left and right eyes are merged by the brain to form a 3D image. But flat 3D display technology is different from the way people use to watch the world. Under normal circumstances, the eyes of the person are focused on the object of interest rather than being separately focused on two planar images that are separated from each other and are independent of each other. This is a phenomenon known as focus-convergence conflict. This kind of conflict is one of the most fundamental and main causes of feelings of fatigue and physical discomfort when people's eyes are watching 3D images. The purpose of setting the maximum viewing angle difference is to reduce the impact of such conflicts on the eyes and physiology of the person. The most common method is to use empirical formulas or safety factors to limit some physical parameters during 3D image acquisition and playback. After years of actual testing, the empirical formula is valid. However, with the continuous development of 3D imaging technology and equipment, it is conservative to look at these empirical formulas based on current technical level, experience and viewpoints. For example, the maximum viewing angle difference often used in empirical formulas is 0.02, which is equivalent to 1.17 degrees or 0.07 radians. The bionic 3D image reconstruction technology theoretically derives the maximum parallax angle and believes that as the viewer's viewing angle difference in viewing 3D images is closer to the theoretical maximum parallax angle of the bionic 3D image reconstruction technology, the closer the 3D image and effect seen by the viewer is Stereo depth and feel in the original scene space. However, increasing the maximum viewing angle difference also increases the impact of the focus-convergence conflict. So how to find the best balance point is the key. For different applications and requirements, the choice and focus of the balance point are also different. The fourth is to include at least one or part of the components on the car or mobile device as a reference in the image captured by the partial bionic 3D viewpoint. The driver can determine the car that he is driving and the other cars or objects or pedestrians appearing in the adjacent lane and the rear lane according to the reference object in the 3D image reconstructed by the bionic 3D image played in a set of naked-eye 3D screens in the car. The relative distance between them.

One of the biggest challenges facing car 3D navigation maps is the way in which 3D map data is collected and produced. If you use the same method of making 2D maps to obtain 3D navigation map data, you need to invest in extremely high production costs and spend more time. 3D navigation maps and data can be obtained by the following four production methods that are more efficient, lower cost, and require less time. The first method is the urban model method. Many cities have a scaled urban model with detailed urban roads, three-dimensional transportation hubs, overpasses, architectural models with the same architectural style and details as physical buildings, parking lots, tree planting on both sides of the road, and more. detail. The model is constantly being updated as the city develops and changes. During production, the camera is 3D shot from a number of different directions and angles to the city model, using a miniature camera or two miniature cameras that can capture 3D images along each street in the urban model by simulating the driving of real people. , squares, intersections, parking lots, tunnels, overpasses and transportation hubs, etc. and 3D shooting in different directions. The two images with different viewing angles were reconstructed and processed by bionic 3D image reconstruction technology. The second method is user sharing. This approach encourages every driver who owns the patented product of the invention to participate in the shared data plan, making each driver a renderer of the 3D navigation map and also making them a user of the 3D navigation map. The driver photographs the real street scene in the car's driving route through the bionic 3D viewpoints 1, 3 and 5 while driving the car and records it in the car smart 3D center. If GPS and related 3D navigation map drawing software are set in the 3D smart center, each car will record the relevant data of the current road conditions, road signs, road signs, signal lights and traffic signs while driving. When the driver arrives at the destination and stops and turns off the fire, the car 3D smart center will automatically search for the wireless hotspot, and automatically collect and process all the collected data and images through the software and upload it to the cloud. The third method is the 2D to 3D method. When cars are driving on city roads, off-city roads and highways, buildings and streetscapes on both sides of the road are not important signs. At this time, the most important thing for the driver is the right road and direction, the entrance and exit of the highway and transportation hub, the location of the highway relative to the nearest city, the distance, the connecting road and direction into the city. The 2D to 3D technology is to divide and separate some urban roads in the current 2D navigation map, different roads and highways according to different road numbers, different road colors or different road directions with different three-dimensional depths. When a driver passes a route with different stereo depths, he or she knows very clearly where the road or lane he is driving. For transportation hubs or overpasses, different directions and intersecting routes are layered and separated by roads or lanes with different depths according to the true proportion between the overpasses, making it easier for the driver to distinguish the correct road. And direction so that it doesn't go the wrong way and direction. This is an ideal solution for fast, simple, and low cost. In the traditional 3D industry, the biggest complaint for viewers watching 3D content through the 2D to 3D method is the tablet effect. The 3D image realized by this technology is often referred to as pseudo 3D, and is one of the main reasons why 2D to 3D technology cannot be popularized. The flat panel effect means that the entire original scene space seems to be artificially divided into a plurality of subspaces, and the scenes and contents in each word space are forcibly compressed onto a drawing board, and each scene on the drawing board has the same three-dimensional shape. depth. A large number of panels are placed in a certain order before and after and along the longitudinal distance, which looks very unnatural and has a significant depth jump. For some urban roads, out-of-city roads and highways, 3D navigation maps with flat panel effects can not only be accepted, but also become one of the most practical tools for three-dimensional navigation maps. The city model navigation map produced by the above method will appear in a set of naked-eye 3D screens when the car is driving on urban roads, the roads outside the city and the highway passing the closest city edge. At this time, the driver knows very clearly the relative position and direction of the car he is driving from the nearest city. The fourth method is to make a short version of the 3D navigation guide. The short 3D navigation guide is a new navigation guide with perspective and 3D visuals. The short 3D navigation guide has an infinity convergence point with a perspective effect that is lower than the driver's eyes. The road guidance with arrows in the simplified 3D navigation guide can extend directly along the road and point forward. Road guidance with arrows and height between roads You can adjust. The driver uses the 3D smart center hand touch screen to adjust the position of the infinity convergence point of the simplified 3D navigation guide and the height between the road guide with the arrow and the real road. The infinity convergence point of the short version of the 3D navigation guide becomes a road guide with arrows that approach or intersect at the end of the infinity direction. The short version of the 3D navigation guide is designed with simple, clear and straightforward expressions in mind so that the driver can quickly identify, understand and react accordingly without any hesitation. When the 3D navigation map data obtained by the above three production methods are subjected to image post-production, the data obtained by different methods are shared with each other, and after being labeled and summarized, the final 3D navigation map and the simplified 3D navigation guide are obtained. If the final 3D navigation map obtained by the above method is reconstructed by the biomimetic 3D image reconstruction technology, the 3D navigation map is called a bionic 3D navigation map. The bionic 3D navigation map intersects the 3D navigation map as a 3D navigation map that restores the original scene more accurately, less deformed, and represents a more precise similar stereo depth relationship.

The 3D Smart Center provides drivers with three different navigation modes; the first mode is the simplified 3D navigation guidance mode, which is used directly in the heads-up display. The second mode is an augmented reality mode of real 3D street view navigation combined with a simplified 3D navigation guide. This is a navigation mode of an augmented reality representation that overlays a simplified 3D navigation guide on a front panoramic image played in a set of naked-eye 3D screens. The third mode is the 3D navigation map mode. The 3D navigation map may be derived from the 3D navigation map obtained by the first three production methods or other production methods in [0022] above. In addition to the first navigation mode being played in the heads-up display, the other two navigation modes are played in a set of naked-eye 3D screens. The driver can switch between the second and third 3D navigation modes by voice control at any time. The above second and third navigation modes not only provide the driver with a clearer and more accurate navigation service, but also provide the driver with a clear and accurate point of interest (POI) positioning value-added service, becoming a brand new, Effective and commercially viable business service model. Each of the different points of interest can be directly marked in the 3D navigation map at the exact location and location, whether the point of interest is in the same building, on a different floor or next to the ground floor. The driver can easily find the location and direction of the point of interest based on the real 3D street view navigation or the 3D navigation map in accordance with the real target shape or architectural features of the point of interest. It will not happen because the same place has appeared at the same time. Many points of interest were piled up and missed points of interest. At the same time, using an augmented reality representation, an arrow points directly to the point of interest, and a box containing simple explanatory text and numbers is attached to the back of the arrow. When the driver directly speaks the number in the box, the 3D Smart Center will take you directly to the point of interest and get a more detailed introduction and description of the 3D image with the naked eye 3D screen.

Traditional mirror mirrors on the left and right sides of the car have been in the market for decades. The traditional mirror rearview mirror technology not only has a small viewing angle, but also has a large wind resistance and wind noise coefficient, and the driver cannot judge from the rearview mirror the other car or object or pedestrian appearing in the car in the adjacent lane or in the rear lane. The relative distance between them. The left and right bionic 3D viewfinder combined with the rangefinder and augmented reality display technology and method not only solve all the problems of the above-mentioned traditional mirror rearview mirror, but also let the driver do not need to turn the head and leave the eyes away from the front. In the case of roads, it can be judged whether it is possible to safely change lanes, reverse, park and obtain information on the traffic conditions around the car, becoming an active safety aid for the driver. As long as the problem of video transmission stability is solved, the use of digital streaming to replace traditional mirror mirrors is not only a fashion and fashion, but also a trend and direction.

The left and right bionic 3D viewing mirrors of the car use one of four different designs. The first solution is to place the bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 in the left and right side of the conventional mirror mirrors of the car, respectively. This design is suitable for the front and rear bionic 3D viewing mirrors of the front or rear mounted car. This design not only has mutual support and backup of the two rear view technologies, but also better protects the 3D camera. The bionic 3D viewpoints 3 and 5 are respectively fixed on the left side and the right side of the left side and the right side of the conventional mirror mirror rear view housing and a support inside the rear view mirror, and the intersection of the two camera center lines and the outer surface of the camera objective lens The midpoint recombination with the outer surface of the objective lens is located on the same tangent plane. The bionic 3D viewpoints 4 and 6 are respectively fixed to a support inside the conventional mirror mirrors on the left and right sides of the car. There is a large enough space between the two camera objectives and the inner surface of the mirror lens. When the driver adjusts the working angle of the mirror lens in the left and right side of the traditional mirror mirror, the camera objective There is no contact with the inner surface of the mirror lens. In the traditional mirror rear view mirror The lens portion of the lens lens facing the bionic 3D viewpoints 4 and 6 has a horizontally rectangular portion that is completely transparent, and the mirror lens has no mirror coating on the back side of the horizontal rectangular region. Floor. However, the inner and outer surfaces of the transparent glass in the horizontally-shaped rectangular region are plated with a clear coating, and the glass surface coated with the transparent coating has a self-cleaning function of repelling water, mist and foreign matter. The camera lens in the bionic 3D viewpoints 4 and 6 captures the outside image through this horizontally-shaped rectangular transparent glass. The second option is to fix the bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 respectively under the left side and right side of the conventional mirror mirror housing. This design is suitable for front-loading or rear-loading automotive bionic 3D viewing mirrors, which are simple to install and easy to maintain. The bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 are designed to have a streamlined shape and are integrated with the conventional conventional mirror mirrors on the left and right sides of the car, respectively. The third option is to design a retractable left and right mirror bracket for the car. One end of the mirror bracket is attached to the car body or a base fixed to the car body, and the other end is suspended. When the car is started, the left and right mirror brackets of the car will automatically move or rotate to the normal working position. When the car is stopped, it will be automatically retracted into the car body or the mirror bracket base to hide. The bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 are respectively disposed on the left side and right side mirror brackets of the automobile. For a bionic 3D viewpoint with two cameras, the two camera centers in each of the 3D viewpoints in the mirror bracket are in the same horizontal plane when the two left and right mirror brackets of the car are in the normal working position. . The mirror bracket can be placed at different positions in the car, such as the position of a conventional mirror mirror, on the front door or on the body in front of the front door. How the positional setting and shape design of the mirror bracket can be perfectly integrated with the shape and design style of the car body will become a challenge for the car designer. The fourth option is to mount 3D viewpoints 3 and 4, 5, 6, 7, and 8 at different positions on the left and right side of the car or on the roof of the car. In any of the above four different bionic 3D viewfinder designs, when the bionic 3D viewfinder is working properly, the centerlines of the two cameras in the bionic 3D viewfinder are at the same level. on flat surface.

The driver observes the traffic conditions on the rear road through the rear view mirror of the car. However, the angle of view of the traditional interior mirrors is limited by the C-pillars and D-pillars behind the car and the rear passengers and cargo. The rear road traffic condition information provided by the 2D camera image does not allow the driver to determine the relative distance between the car he is driving and other cars or objects or pedestrians appearing in the back road. The driver sees the rear panoramic image from the bionic 3D viewpoints 2, 4 and 6 from the naked eye 3D rearview mirror in the car, with a larger viewing angle and no interference and dead ends. A naked-eye 3D rearview mirror in a vehicle is a flexible or non-flexible naked-eye 3D screen consisting of a flat or curved shape backing plate and one or more flat or curved shapes fixed on the inner surface of the backing plate and a set outside the screen. It is composed of coated anti-glare glass. The coating only allows light to penetrate from one direction to the outside. In order to reduce the weight and space occupied by the rearview mirror, the screen motherboard of the naked eye 3D rearview mirror in the vehicle can be placed in the car 3D intelligent center. There is a switch on the naked eye 3D rearview mirror. When the driver turns off the switch, the naked eye 3D rearview mirror becomes a traditional interior mirror rearview mirror.

The 3D Smart Center is a computer with a head-up display connected to the motherboard of the computer, a set of naked-eye 3D screens and a motherboard with a naked-eye 3D rearview mirror screen, multiple ISP image processors, a central control unit, a codec and a decoder. A wireless communication module with a communication chip Sim card slot and a hand touch screen. The 3D Smart Center has a separate operating system, built-in 3D image reconstruction technology, image editing and stitching, 3D format conversion, image encoding and decoding, and various signal processing software packages. The 3D Smart Center uses an operating system and built-in image processing software, through an array connected to the motherboard with an image processing chip, a screen and display motherboard and a central control unit, encoding and decoder for each of the two from each bionic 3D viewpoint Image processing from different perspectives, including: bionic 3D image reconstruction, editing, stitching, panning, 3D format conversion, algorithm, optimization, color grading, white balance, encoding, decoding, output and archiving; with a communication chip The APP software pre-installed in the wireless communication module or in the mobile phone connects and processes various functions and contents from the mobile phone application, navigation map, telephone, WeChat, short message, voice control, etc.; through the built-in signal analysis and processing software pair The information of the car OBD and the range finder are processed and processed; and the processed images and information are respectively output to the head-up display according to a preset program by the central control unit, a set of naked-eye 3D screens and a naked-eye 3D rearview mirror in the vehicle. Play in. After the car starts, all the bionic 3D The camera in the viewpoint is automatically turned on. After a period of time, the 3D smart center simultaneously receives two images with different viewing angles acquired by each bionic 3D viewpoint at the same time. Two images with different viewing angles are reconstructed by bionic 3D image editing, editing, stitching, panning and left and right 3D playback formats to become a complete image with super wide image. Each image of the ultra-wide image is processed by the image processor, and mainly includes: algorithm, optimization, brightness, hue and white balance, etc., and then directly enters into the encoder for compression and encoding. For cars that only use bionic 3D viewpoints 1, 2, 3, 4, 5, and 6, there are two images of such an ultra-wide picture, which are front-wide images and rear panoramic images. For cars that use all eight bionic viewpoints, there are four images of such ultra-wide images, which are front-facing full-field images, rear-wide images, left-back full-field images, and right-back full-field images. The central control unit controls and manages the above two or four different panoramic images, and outputs them to a set of naked-eye 3D screens and a screen main board of the naked-eye 3D rearview mirror in time according to the setting requirements of the program. After being decoded by the decoder, it is played in a set of naked-eye 3D screens and in-car naked-eye 3D rearview mirrors. The 3D intelligent center is provided with a wireless communication module with a SIM card slot. The module is equipped with the main mobile phone operating system and the required APP software in the market, and can directly download and receive 3D navigation maps in different operating system mobile phones. Short-term 3D navigation guides, music and video applications and services, not only will not interrupt the service due to various interferences caused by mobile communication or the outside world during use, and save the flow and cost of the mobile phone. If the mobile phone downloads or connects to the bionic 3D navigation map, the 3D intelligent center will directly output the bionic 3D navigation map to a set of naked-eye 3D screens and the naked-eye 3D rearview mirror through the central control unit. If the 3D navigation map downloaded or connected by the mobile phone comes from a 3D navigation map of a third party, and the 3D image reconstruction technology is not reconstructed and the 3D playback format is not satisfied, the driver can touch the screen through the hand. Whether to perform bionic 3D image reconstruction on such a 3D navigation map or directly output to a set of naked-eye 3D screens and in-car naked-eye 3D rearview mirrors. The 3D Smart Center overlays pre-designed and produced possible or relevant point-of-interest descriptions and profile content information on 3D navigation map images, such as pointing an arrow directly to a point of interest, followed by an arrow containing a simple explanatory text and number. frame. When the driver directly speaks the number in the box by voice control, the 3D Smart Center will take you directly to the point of interest and get a more detailed description and introduction of 3D image effects through a set of naked-eye 3D screens. The 3D Smart Center supports voice-activated interactive control, with multiple general-purpose external interfaces reserved on the backplane and open to third-party applications and services. The 3D Smart Center will be designed and built into a rugged device that will not be easily damaged in traffic accidents and retain data and images from the time of the accident and before.

The 3D Smart Center will perform bionic 3D image reconstruction, image editing and splicing processing on all images with different viewing angles from the bionic 3D viewpoint, and the processed two images will be in the left-right format (Side-by-Side). The 3D playback format is output to the heads-up display, a set of naked-eye 3D screens and a naked-eye 3D rearview mirror in the car.

The technology and device proposed by the invention have the characteristics of easy installation, simple operation, low cost, easy promotion and popularization.

BRIEF DESCRIPTION OF THE DRAWINGS:

Figure 1 Schematic diagram of a car plane shape head-up display

Figure 2 Schematic diagram of the car surface shape back plate and the naked eye 3D screen and the fixed base workbench

Figure 3 Schematic diagram of the car surface shape back plate and the naked eye 3D screen and the connection bracket type workbench

Fig. 4 Schematic diagram of a flat-shaped back plate and a naked-eye 3D screen and a connection bracket type work table in which the automobile is bent twice

Figure 5 car naked eye 3D console schematic

Figure 6 Schematic diagram of automobile bionic 3D viewpoint

Figure 7 Schematic diagram of car bionic 3D viewpoint and bionic 3D viewpoint viewing angle distribution

Figure 8 bionic 3D image reconstruction technology 1 - schematic

Figure 9 bionic 3D image reconstruction technology 2 schematic

Figure 10 Schematic diagram of the design of the left and right bionic 3D viewing point mirrors of the car

Figure 11 Schematic diagram of the design of the left and right bionic 3D viewing point mirrors of the car

Figure 12 Schematic diagram of the design of the left and right bionic 3D viewing mirror brackets of the car

Figure 13 Schematic diagram of the naked eye 3D rear view mirror in the car

Figure 14 Schematic diagram of the car 3D intelligent center and car networking system

Figure 15 Schematic diagram of the car 3D intelligent center

detailed description:

The specific embodiments of the present invention show an example of the embodiment of the present invention, and have a corresponding relationship with the specific matters in the claims. The present invention is not limited to the embodiments, and various embodiments can be realized without departing from the spirit and scope of the invention. The illustrative examples in all of the schematics are examples of the described implementable solutions.

Figure 1. A head-up display 9 has two circular connecting ears on its bottom side and a semi-cylindrical projection on the back of the base 10. The connecting ears of the head-up display 9 and the semicircular projections of the base 10 are connected by two bolts 11. The angle between the head-up display 9 and the base 10 is adjusted and fixed by the two bolts 11. Behind the semi-cylindrical shape of the base 10 is a wiring socket 12 dedicated to the head-up display 9 and a power supply port 13. In the design shown in Figure 1, the base 10 is attached to the dashboard or center console of the car via an plexiglass of an attachment 15. The upper surface of the attachment 15 is a flat surface with a curved surface and adhesive. The attachment 15 is secured to the dashboard or center console of the vehicle by glue on the underside, and the base 10 is secured to the planar surface of the attachment 15 by a suction cup 14. The shape of the bottom surface of the accessory 15 varies depending on the brand and model of the car.

Figure 2 shows a perverted 3D table 16 consisting of a curved shape back plate and a curved shape flexible or non-flexible open-hole 3D screen pasted on its inner surface. The naked-eye 3D table 16 is fixed to a base 18 and is fixed to the upper center of the car together with the base 18.

Figure 3 shows a naked-eye 3D table 16 consisting of a curved-shaped back panel and a curved-shaped flexible or non-flexible open-hole 3D screen pasted on its inner surface. The naked-eye 3D table 16 is connected by a bracket 17 to a base 18 that is fixed to the upper console of the car. The driver can adjust the position and orientation of the naked-eye 3D table 16 through the bracket 17.

Figure 4 is a plan view of a flat-plate back sheet 19 that has been bent twice and a naked-eye 3D stitching screen 20 having three planar shapes fixed on its inner surface. The back plate 19 is connected by a bracket 17 to a base 18 that is fixed to the upper console of the car. The driver can adjust the position and orientation of the back panel 19 and the three stitching screens 20 through the bracket 17.

Figure 5, the car naked eye 3D console is composed of a plurality of flat or curved shape flexible or non-flexible naked eye 3D screen 21; a plurality of screens 21 can be stitched together to form a continuous screen group or according to different display content or purpose Set at different locations. The naked-eye 3D screen 21 used at each different location may be different, such as the shape, size, or transparency of the screen. The number, arrangement, location and arrangement of screens 21 used by different brands of car consoles are different.

Figure 6 shows a car bionic 3D viewpoint with two cameras 22 arranged in parallel. Each bionic 3D viewpoint is composed of a casing 23, two cameras 22 and a transparent glass cover 25 in the forward direction of the camera. The shape of the bionic 3D viewpoint housing 23 is different depending on the setting position on the car. In the normal working position, the centerlines of the two cameras 22 in the bionic 3D viewpoint are on the same horizontal plane and parallel to each other, and the parallel between the giants is 5-80 mm. The wires 24 of the two cameras 22 are respectively passed through the bionic 3D view housing 23 and connected to the car 3D smart center 34. The front cover of the bionic 3D view housing 23 is a transparent glass 25 coated on both the inner and outer surfaces, and the two cameras 22 in the bionic 3D view capture the outer image through the transparent glass 25 in the forward direction.

Figure 7 shows a set of bionic 3D viewpoints set on a car and the viewing angle distribution of these viewpoints. Figure. The two largest shaded areas on the left and right sides of the car represent the areas of the set of bionic 3D viewpoints that are still not covered and become blind spots. The smaller area shaded portions of the four corners of the car represent the areas of the four dead corners of the set of bionic 3D viewpoints that are still uncovered. The bionic 3D viewpoint 1 is disposed at a position below the center of the inner surface of the front windshield of the automobile, and the lenses of the two cameras 22 in the viewpoint face the front direction of the car. The bionic 3D viewpoint 2 is disposed at a position above the center of the inner surface of the window glass of the automobile, and the lenses of the two cameras 22 in the viewpoint face the front and rear directions of the car. The bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 are respectively disposed in the left and right side of the car in the conventional mirror rear view mirror, and the lenses of the camera 22 in the bionic 3D viewpoints 3 and 4 are respectively directed to the left front direction and the left side of the car. In the rear direction, the lenses of the cameras 22 in the bionic 3D viewpoints 5 and 6 are respectively directed to the right front direction and the right rear direction of the automobile. The bionic 3D viewpoints 7 and 8 are respectively disposed on the left and right edges of the roof of the automobile. The range of viewing angles of the bionic 3D viewpoints 7 and 8 is not indicated in FIG. As long as the lenses of the cameras 22 in the bionic 3D viewpoints 7 and 8 are directed toward the left front direction and the right front direction of the car, respectively, it is possible to completely cover the blind areas indicated by the two shaded portions on the left and right sides of the car.

Figure 8a shows a typical scene design for 3D image capture in the original scene space. The two cameras 26 and 27 are at a distance t from each other. The two cameras 26 and 27 are simultaneously turned inward until the center lines of the two cameras 26 and 27 converge on the attention scene 30. This is a typical example of a 3D shot using a convergence method. The scene in front of the scene 30 is referred to as the foreground object 31, and the scene in the rear is referred to as the scene object 32. Concerning the scene, the distances of foreground and foreground objects to cameras 26 and 27 are z _conv , z _near and z _{far , respectively} .

Figure 8b, in a 3D playback space, a flat screen 33 of width W. The horizontal distance between the left and right images projected onto the screen 33 is the parallax P. When the left and right eyes of a person can only see and focus on the left and right images on the screen 33, the 3D images of the left and right images merged in the brain appear behind the screen, called the screen space. When the left and right eyes of a person can only see and focus on the right image and the left image on the screen 33, the 3D images of the left and right images in the brain appear between the viewer and the screen, called the audience space. . According to the geometric relationship shown in Figure 8b, the following relationship is obtained,

Z _c =Z _D xT/(TP) (1)

Among them, Z _c - the distance between the audience's eyes and the two image convergence points on the screen

Z _D – the distance between the eyes of the audience and the screen

T – the distance between the eyes of the audience

P–parallax, the horizontal distance between the left and right images appearing on the screen

d(P)=P _max –P _min =TxZ _D (1/Z _cnear –1/Z _cfar ) (2)

Where: P _max – the maximum parallax that appears on the left and right images on the screen

P _min – the minimum parallax that appears on the left and right images on the screen

_Z cnear - the minimum disparity viewer from the point of convergence (P <0 negative disparity, the viewer space)

Z _cfar – the distance from the audience to the maximum parallax convergence point (P>0 positive parallax, screen space)

Definition, P _rel =d(P)/W

Where: P _rel – the maximum parallax over the width of the unit screen

W–width of screen 30

Figure 9a shows that when capturing 3D images in an original scene space, the two cameras 26 and 27 are simultaneously turned inward until the centerlines of the two cameras 26 and 27 converge onto the subject of interest 30. The centerlines of the two cameras 26 and 27 are imaged directly at the center point of the imaging chips 28 and 29 of the two cameras 26 and 27 at the point of convergence on the subject of interest 30. Figure 9b shows the two cameras 26 and 27 being rotated outwardly from the convergence mode shown in Figure 9a until the centerlines of the two cameras 26 and 27 are parallel to each other. The 3D shooting method is changed from the convergence method to an example of the parallel method. At this time, the image forming position on the imaging chips 28 and 29 of the two cameras 26 and 27 at the convergence point on the subject 30 is no longer the midpoint, but leaves the horizontal and horizontal points opposite to each other in the imaging chip. Distance h on. At this time, if the imaging chips 28 and 29 of the cameras 26 and 27 are respectively translated by a distance h toward the opposite and horizontal directions of each other, the result of the shooting using the parallel method is exactly the same as that of the shooting using the convergence method. This method of parallel shooting of 3D images obtained by translating the two imaging chips 28 and 29 to obtain the same effect as the convergence shooting is called a parallel convergence equivalent method. According to the geometric relationship shown in Figure 9b, we get the following relationship,

d=txfx(1/z _conv -1/z)=h–txf/z (3)

Where d – the parallax of any point in the original scene space on the two camera imaging chips

h – horizontal offset of the two camera imaging chips in opposite directions to each other

T–distance between two parallel camera centerlines

f – equivalent focal length of the camera lens

Z–the distance from any point in the original scene space to the camera

z _conv – the distance to the scene or the equivalent convergence point to the camera when shooting in parallel

The following formula is derived according to formula (3);

d(d)=d _max -d _min =txf(1/z _near –1/z _far ) (4)

Where: d _max – the maximum parallax of the left and right images on the two camera imaging chips

d _min – the minimum parallax between the left and right images on the two camera imaging chips

z _near – the distance from the foreground object 31 to the camera

z _far - 32 from the scene to the camera

Definition, d _rel =d(d)/w

Where: d _rel – the maximum parallax over the width of the unit imaging chip

W–camera imaging chip width

_Let d _rel =P _rel

Push: t=[(Z _D /Axf)x(1/Z _cnear –1/Z _cfar )/(1/z _near –1/z _far )]xT (5)

Among them: A – screen magnification W/w

Equation (5) shows that the distance between the center lines of the two cameras in 3D shooting is different from the distance between the eyes of the human eye.

Similarly, P = (W / w) xd = Axd and substituted into the formula (1) to get the following formula:

Z _c =(Z _D xT)/(TP)=(Z _D xT)/(T-Axd)

=(Z _D xT)/[Axtxf-(Axh-T)xz]xz (6)

Equation (6) shows that there is no linear relationship between Z _c and z. The ideal image is any point in the original scene space. A line and a face correspond to the only point in the 3D image reconstruction space, a line and a face. In order to make two images with different viewing angles captured in one original scene space ideally imaged in 3D playback space without distortion and distortion, the only condition is to make the mathematical relationship between the corresponding points in the two spaces. Become a linear relationship. In the formula (6), the condition that the linear relationship between Z _c and z holds is that

Axh–T=0 or h=T/A

Formula (6) is simplified into the following formula,

Z _c =(Z _D xT)/(Axtxf)xz (7)

Equation (7) shows that two images with different viewing angles obtained at any point in the original scene space have a unique corresponding point in the 3D playing space, and two images with different viewing angles achieve convergence at the corresponding points.

Conclusion: When shooting 3D images in parallel, the bionic 3D image reconstruction technology requires that the imaging chips 28 and 29 of the camera be moved horizontally in opposite directions to each other before shooting or after the normal shooting is completed. During post-production, the left and right images are horizontally translated by a distance h from each other in opposite directions. Through the above method, the parallel method can also obtain a more ideal effect than the convergence method for 3D shooting when shooting a 3D image, which not only conforms to the manner and habit of the person's eyes, but also has no distortion.

According to the geometric relationship shown in Fig. 9b and the formula (5), the following relationship is obtained,

z _conv =(txf)/2h=txfxA/2T

=(Z _D /2)x(1/Z _cnear –1/Z _cfar )/(1/z _near –1/z _far ) (8)

Because z _far >>1

So 1/z _far <<1

Simplify equation (8) to the following equation,

(1/Z _cnear –1/Z _cfar )=txfxA/(TxZ _D xz _near )

By definition, the maximum parallax angle d(a _max ) is, see Figure 8b

d(a _max )=a ₂ -a ₁ =tan ^-1 (T/Z _cnear )–tan ^-1 (T/Z _cfar )

=T(1/Z _cnear –1/Z _cfar )

=(txfxA/Z _D )x1/z _near (9)

Conclusion: Equation (9) yields the theoretical limit of the maximum parallax angle a _max . The widely used empirical formula a _max of 0.02 is a relatively conservative empirical safety value. When the safety value of the maximum viewing angle difference is increased, the 3D effect seen and perceived by the human eyes will be closer to the true stereoscopic depth relationship in the original scene, but at the same time the focus of the human eye is increased. influences.

One of the core ideas of bionic 3D image reconstruction technology is to correctly represent the stereoscopic depth relationship in the original scene space in 3D image reconstruction. Equation (7) shows that there is a mathematical relationship between the two spaces, and the geometry of an object in each original scene space can be correctly restored in a certain scale in the 3D play space without distortion. Equation (8) shows that the value of h is properly selected and the position of the focus of the scene in the 3D play space is set on the 3D screen. At this time, the following relationship holds in the ideal space (the following relationship is passed) Equation (7) can also be proved)

z _conv /Z _D =z _near /Z _cnear =z _far /Z _cfar (10)

Bringing the formula (8) into the formula (10) gives the maximum amount of screen output.

Z _cnear =Z _D xz _near /z _conv

=2TxZ _D xz _near /(txfxA) (11)

Equation (11) shows that the maximum amount of screen is related to the position of the close-up object in the original scene, in addition to the camera parameters. In fact, the excessive amount of screen output in the audience space will not only make the audience feel unreal, but also cause one of the reasons for the viewer's eye fatigue and physical discomfort. The most effective way to control the maximum amount of screen shot is to control the position of the close-up object with a certain amount of playback screen.

Figure 10 shows the first design of the left and right bionic 3D viewing mirrors of the car. The bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 are respectively disposed in the left and right side of the car in the conventional mirror mirrors. In Fig. 10, only the structure and implementation method of the bionic 3D viewfinder lens on the left side of the car are shown. The bionic view mirror on the right side of the car is a mirror image on the left side, and the structure and implementation method of the two are completely the same. The left side conventional mirror mirror housing 34, the mirror lens 35, the mirror rotation mechanism 37, the bionic 3D viewpoints 3 and 4, and the bionic 3D viewpoint support structure 38. The bionic 3D viewpoint 3 is fixed to the surface of the rear view mirror front housing 34 and a support structure 38 inside the rear view mirror, the intersection of the center line of the two cameras 22 with the outer surface of the objective lens of the camera 22 and the outer surface of the objective lens. The midpoint recombination is located on the same tangent plane. The bionic 3D viewpoint 4 is mounted on the support structure 38 in the rear view mirror, wherein the objective lens of the camera 22 needs to be sufficiently far from the mirror lens 35 that the inner surface of the camera objective lens and the mirror lens 35 in the bionic 3D viewpoint 4 are not adjacent to each other. Any contact will occur. When the driver adjusts the working angle and direction of the mirror lens 35, the rotating mechanism 37 rotates the mirror lens 35 in the up and down and left and right directions. The bottom of the mirror lens 35 has a horizontally-shaped rectangular transparent portion 36 having a mirror coating on the back side. However, the inner and outer surfaces of the transparent portion 36 are plated with a clear coating, and the transparent glass surface coated with the transparent coating has a self-cleaning function of repelling water, mist and foreign matter. The two cameras 22 in the bionic 3D viewpoint 4 capture the outside image through the transparent portion 36 at the bottom of the mirror lens 35.

Figure 11 shows a second design of the left and right bionic 3D viewing mirrors of the car. The bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 are mounted under the outer casing 34 of the left and right side of the conventional mirror rear view mirror, respectively. The mirror lens of the mirror mirror is 35. Figure 11 shows only the structure and implementation method of the bionic 3D viewfinder on the left side of the car. The bionic 3D viewfinder on the right side of the car is a mirror image on the left side. The structure and implementation of the two are identical.

Figure 12 shows a third design for the left and right bionic 3D viewing mirrors of the car. The automotive bionic 3D viewing mirror bracket 39 is a brand new automotive mirror technology and device. The bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 are mounted on the left and right bionic 3D viewing mirror holders 39 of the automobile, respectively. Fig. 12 only shows the structure and implementation method of the bionic 3D viewfinder and the mirror support on the left side of the car. The bionic 3D view mirror support on the right side of the car is a mirror image on the left side, and the structure and implementation method of the two are completely the same. As shown in Fig. 12a, the mirror bracket 39 is hidden in the mirror holder base 40 before the vehicle is started, and the mirror bracket is fixed to the vehicle body 41. Figure 12b shows the mirror mount 39 rotated from the mirror mount base 40 to the working position when the vehicle is activated. In the working position, the centerlines of the two cameras 22 of the bionic 3D viewpoints 3 and 4 mounted on the mirror bracket are located on the same horizontal plane. As shown in Figure 12c, when the car engine is turned off, the mirror mount 39 will automatically rotate and return to the mirror mount base 40. At this time, the four bionic 3D viewpoints 3, 4, 5 and 6 on the mirror holder 39 and a total of eight cameras 22 and lenses therein are all hidden in the automobile body or the mirror holder base 40.

Figure 13 shows a naked eye 3D rearview mirror in an automobile. The in-vehicle naked-eye 3D rearview mirror is composed of a planar backing plate 42, one or more planar naked-eye 3D screens 43, and a coated anti-glare glass 44. The back panel 42 of the naked eye 3D rear view mirror is coupled to a rear view mirror bracket 45. The anti-glare glass 44 is disposed on the outside of the naked-eye 3D screen 43, and the back surface is coated with a coating. The coating only allows light to penetrate from one direction to the outside. The driver can see the content played in the naked eye 3D screen 43 through the anti-glare glass. A switch 46 is provided on the rearview mirror. When the driver turns off the switch 46, the naked eye 3D rearview mirror becomes a conventional interior mirror rearview mirror.

Figure 14 shows the design and operation of a car 3D smart center 47. The 3D smart center 47 is a computer mainly comprising: a main board 48, an operating system 49, a storage unit 50, a wireless communication module 51 with a SIM card slot, a central control unit 52, a touch screen 67 and all located in a black dotted frame. Device. The car bionic 3D viewpoint is divided into three groups, namely, bionic 3D viewpoints 1, 3 and 5, bionic 3D viewpoints 2, 4 and 6, and bionic 3D viewpoints 7 and 8. The images obtained by the bionic 3D viewpoints in the three groups are respectively entered into the bionic 3D image reconstruction technology unit 53 according to the respective groups, and the parameters are adjusted by the built-in bionic 3D image reconstruction and processing software, and then the image clip is entered. Splicing and 3D format conversion unit 54. The images in the three groups are edited in 54 respectively, the process of splicing and 3D illustrations, and finally become a single image with four super wide images, the front full image, the rear full The image of the territory, the entire left image of the left rear and the whole image of the right rear. The ISP image processing chip 55 disposed on the image processing main board 57 processes each image mainly including: algorithm, optimization, brightness, hue and white balance, and directly enters the encoder 56 for compression and encoding. The logic processor in the central control unit 52 will deliver the four images in time to a set of naked-eye 3D screens 16, 20 and 21 and the screen main board 58 of the in-vehicle naked-eye 3D rearview mirror 43 according to a pre-designed program. After being decoded by the decoder 60, it is played in a set of naked-eye 3D screens 16, 20 and 21 and an in-vehicle naked-eye 3D rearview mirror 43. The main board 59 of the heads up display is connected to the main board of the 3D smart center 47. The 3D navigation map from the built-in wireless communication module 51 or the driver's mobile phone 61 directly enters the bionic 3D image reconstruction technology unit 53, and the driver can set whether or not to perform the mapping from the 3D navigation map through the built-in wireless communication module or the mobile phone. The correction, if not required, proceeds directly through unit 53 to a set of naked-eye 3D screens 16, 20 and 21. The simplified 3D navigation guides from the built-in wireless communication module 51 or the mobile phone 61, telephone, WeChat, SMS, video, music, entertainment and other content directly enter the 3D smart center motherboard 48 and enter the head-up display 9 and one The group is played in the naked eye 3D screen 16, 20 or 21. The sound signal from the stereo two-channel microphone 62 in the car is converted and processed by a software package. Information from the car OBD63 plug is transferred to a software package via a dedicated data line, and the data is read by software. The data detected by the range finder 64 directly enters the data processor 65 attached to the range finder. All of the above-mentioned processed microphone 62 sound signals, the OBD63 data and the rangefinder 64 data are directly transferred to the 3D smart center main board 48 and then delivered to the car speaker, the head up display 9 and a set of naked-eye 3D screens 16, 20 and Played in 21. The 3D smart center power supply 66 is responsible for providing all of the power to all of the above devices.

Figure 15 shows the design and operational flow chart of an automotive 3D smart center 47. Compared with Figure 14 above, there is no car networking part in Figure 15.

Claims (10)

1. 3D image acquisition and naked-eye 3D head-up display system for automobiles or mobile devices, including a heads-up display, a set of naked-eye 3D screens, a set of bionic 3D viewpoints, a 3D smart center, left and right side bionic 3D viewpoints Mirror and a naked eye 3D rearview mirror in the car;

   The head-up display is composed of a transparent plexiglass of a flat or curved shape and a flexible or non-flexible transparent screen of a flat or curved shape adhered to the inner surface thereof, and the transparent screen may be a conventional 2D screen or a naked eye 3D. The screen, the head-up display is connected to a base, and the angle between the head-up display and the base is adjustable;

   The set of naked-eye 3D screens adopts one of two different design modes; the first mode is a naked-eye 3D table mode, and the naked-eye 3D table is a backplane or a shape of a flat or curved shape. The bent flat-shaped back plate and one or more flat or curved shaped flexible or non-flexible naked-eye 3D screens fixed on the inner surface of the back plate, the back plate may be directly fixed on the base or connected by a bracket On the base, the base is fixed on the center console of the car, the driver can adjust the position and direction of the backboard and the screen through the bracket; the second mode is the naked-eye 3D console mode, and the naked-eye 3D console is composed of multiple planes or A flexible or non-flexible naked-eye 3D screen composed of curved shapes, multiple screens can be stitched together into a continuous set of screens or set at different positions in the car according to different display contents or purposes, used in the naked-eye 3D console mode. The number, arrangement, location and arrangement of screens are different for each different brand and model of car or mobile device;

   The bionic 3D viewpoint is a device having a camera or two cameras capable of acquiring 3D images; setting a set of bionic 3D viewpoints at a plurality of different positions on a car or a mobile device to collect 3D around the car and in a specific direction image;

   The 3D Smart Center is an independent operating system that processes, controls, and manages images from each bionic 3D viewpoint, automotive OBD and rangefinder, applications and information in the handset, and processes The rear image and information are respectively output to the head-up display, a set of naked-eye 3D screens, and a computer played in the naked-eye 3D rearview mirror by the central control unit according to a preset program, mainly including the heads connected on the main board of the computer. a display, a set of naked-eye 3D screens, and a motherboard for the naked-eye 3D rearview mirror screen, an image processor, a central control unit, an encoder and decoder, a hand touch screen, and a wireless communication module with a communication chip slot;

   The left and right bionic 3D viewing mirrors of the car are arranged in a plurality of bionic 3D viewpoints in the left and right side of the car in a conventional mirror rearview mirror or under the mirror mirror housing or in a bionic 3D view. A car image acquisition device on a mirror mount or on a car body; two images with different viewing angles captured by a bionic 3D viewpoint in a bionic 3D viewfinder, processed by bionic 3D image reconstruction technology and other image software, the driver The relative distance between the car that is driving and the other cars or objects or pedestrians appearing in the adjacent lanes and the rear lane can be determined by a set of naked-eye 3D screens and 3D images played in the naked-eye 3D rearview mirrors;

   The naked eye 3D rear view mirror of the automobile is a flexible or non-flexible naked-eye 3D screen composed of a flat or curved shape back plate and one or more flat or curved shapes fixed on the inner surface of the back plate, and one outside the screen. The back is coated with a coated anti-glare glass.
2. The 3D image capturing system and the naked eye 3D head-up display system of claim 1 or 2, wherein a polarizing film is added between the screen of the head-up display and the transparent plexiglass or the outer surface of the transparent plexiglass. Reduce the intensity of sunlight and the effects of various reflections on the image on the screen.
3. The car or mobile device 3D image capturing and naked eye 3D head-up display system according to claim 1, wherein in a bionic 3D viewpoint having two cameras, the center lines of the two cameras are located on the same horizontal plane and Parallel to each other, the parallel spacing between the two centerlines is 5–80 mm.
4. The car or mobile device 3D image capturing and naked eye 3D head-up display system according to claim 1, wherein each of the bionic 3D viewpoints is marked with a different number or name according to different positions set on the car or the mobile device; For the car, set the bionic 3D viewpoint 1 on the inner surface of the front windshield of the car or in front of the car. At the position of the face, the camera lens is facing the front of the car; the bionic 3D viewpoint 2 is placed on the inner surface of the rear window glass of the car or at the position of the rear cover of the car, and the camera lens is directed toward the front of the car; the bionic 3D viewpoint 3 will be And 4 and bionic 3D viewpoints 5 and 6 are respectively placed in the left and right side of the car in the traditional mirror mirror or under the mirror mirror or on the left and right bionic 3D view mirror brackets or on the left side of the car and On the right side of the body, the camera lenses in the bionic 3D viewpoints 3 and 4 are respectively oriented toward the left front direction and the left rear direction of the car, and the camera lenses in the bionic 3D viewpoints 5 and 6 respectively face the right front direction and the right rear direction of the car; Viewpoints 7 and 8 are respectively placed on the left and right side of the car or on the roof of the car, with the camera lenses facing the left and right directions of the car, respectively.
5. A 3D image capturing system and a naked eye 3D head-up display system for a car or a mobile device according to claims 1 and 4, wherein the left and right bionic 3D viewing mirrors of the automobile adopt one of the following four different design schemes. The first solution is to set the bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 respectively in the left and right side of the car in the traditional mirror mirror; the second option is to bionic 3D viewpoints 3 and 4. And the bionic 3D viewpoints 5 and 6 are fixed under the left side and right side of the conventional mirror mirror housing, respectively; the third option is to set the bionic 3D viewpoints 3 and 4 and the bionic 3D viewpoints 5 and 6 respectively in a new design. The retractable car on the left and right side of the bionic 3D view mirror bracket, the mirror bracket can be set at different positions of the car, when the car starts, the left and right mirror brackets of the car will automatically move or rotate In the normal working position, when the car is parked and turned off, it will be automatically retracted into the car body or the mirror bracket base. The fourth solution is to install 3D viewpoints 3, 4, 5 and 6 on the left and right sides of the car. Side body or car Different locations on the top.
6. The 3D image capturing system and the naked eye 3D head-up display system of the automobile or the mobile device according to claim 1, wherein a switch is arranged on the naked eye 3D rearview mirror in the automobile, and when the switch is turned off, the naked eye 3D rear view in the vehicle is provided. The mirror becomes a traditional interior mirror rearview mirror.
7. A 3D image capturing and naked eye 3D head-up display system and a 3D image processing method according to claims 1, 3, 4 and 5, wherein the 3D intelligent center is processed by a separate operating system and built-in image processing. Software, two images with different viewing angles from each bionic 3D viewpoint through a plurality of image processing chips, screen and display boards, encoders and decoders, hand touch screens and central control unit connected to the motherboard Processing, mainly including: bionic 3D image reconstruction, translation, editing, stitching, 3D format conversion, algorithm, optimization, brightness, hue, white balance, encoding, decoding, output and archiving; through a wireless communication with wireless communication chip The APP software pre-installed in the module or in the mobile phone connects various applications in the mobile phone, navigation map, telephone, WeChat, SMS, voice control and other information and content and processes the information and content; through built-in signal processing software Information from automotive OBDs and rangefinders is processed and processed.
8. A 3D image capturing and naked eye 3D head-up display system and a 3D image processing method according to claims 1, 3, 4, 5 and 7, wherein the 3D smart center is for each of the two from the bionic 3D viewpoint Bionic 3D image reconstruction is performed on images with different viewing angles; the bionic 3D image reconstruction technology mainly includes: first, each of the two images with different viewing angles collected at any point in the original scene space along the horizontal and each other The opposite direction makes a translation h = T / A distance, so that the two images after translation have a unique corresponding point in the 3D playback space and the two images achieve convergence at the corresponding point; the second is through Appropriately select the h value and adjust the position of the scene in front of the car, and set the two convergence views of the scene in the 3D playback space to the corresponding convergence point position on the 3D playback screen. the maximum amount of the camera lens screen nearest foreground object in the 3D space are playing _{Z cnear = 2TxZ D xz near /} (txfxA); a third 3D image using a biomimetic Theoretical maximum angle of view reconstruction difference _{d (a max) = (txfxA} / Z D) x1 / z near replace conventional empirical formulas, the relations between the different depth of the stereoscopic object in the original scene in 3D space to play a similar relationship stereoscopic depth The fourth is to include at least one or part of the components on the car or the mobile device as a reference in the image captured by the partial bionic 3D viewpoint, and the driver can refer to the reference object in the 3D image played in the set of naked eye 3D screens. Determine the relative distance between the car you are driving and the other cars or objects or pedestrians that appear in the adjacent lanes and rear lanes.
9. The 3D image capturing and naked eye 3D head-up display system and the 3D image processing method of the automobile or mobile device according to claims 1, 4, 5, 7 and 8, wherein the 3D navigation map can be produced by the following four different methods. Method production; the first method is to take a 3D shot of the camera against a scaled urban model from multiple different directions and angles, using a miniature camera or two miniature cameras that can capture 3D images to simulate the way people drive. Every street, road, plaza, intersection, parking lot, tunnel, overpass and transportation hub in the city model is 3D shot in different directions; the second method is to encourage every driver who participates in the sharing data plan to drive The car captures the real street scene in the car's driving route through the bionic 3D viewpoints 1, 3 and 5 and saves the image in the 3D smart center. When the driver arrives at the destination to stop and turn off the fire, the 3D smart center will automatically search for the wireless hotspot. And automatically upload all collected data to the cloud; the third method is to use 2D to 3D technology to 2D navigation The urban roads, highways and highways in the figure are numbered according to different roads. Different road colors and different road directions are layered and separated according to different three-dimensional depths. For each different direction in the transportation hub or overpass The roads are separated by roads of different heights according to the true scale; the fourth method is to make a short version of the 3D navigation guide, which is a new navigation guide with perspective and 3D visual effects; The 3D navigation map data and images are reconstructed by bionic 3D image reconstruction technology, post-production of images, sharing the data obtained by the above different production methods, and summarizing to form the final bionic 3D navigation map and the simplified 3D navigation guide.
10. A 3D image capturing and naked eye 3D head-up display system for a car or a mobile device according to claims 1, 7 and 9, wherein the 3D smart center provides three different navigation modes; the first mode is a simplified 3D navigation. The guidance mode is used in the head-up display; the second mode is the augmented reality navigation mode of the real 3D street view navigation combined with the simplified 3D navigation guide; the third mode is the 3D navigation map mode, and the 3D navigation map can be through the above claim 9. The 3D navigation map obtained by the bionic 3D navigation map obtained by the first three production methods or other production methods; the driver can switch between the second and third 3D navigation modes by voice control at any time.

Images