JP2019047296A - Display device, display method, control program, and electronic mirror system - Google Patents

Abstract

To provide a display device that displays a captured image while giving a sense of depth to a user while moving using a single-eye imaging device.SOLUTION: A display device 1 includes: a frame image setting unit 53 for setting a frame image to be displayed around a video according to a result of analyzing a video captured while moving a single-eye imaging device 2 by a video analysis unit 52; and a display control unit 54 for displaying the video together with the frame image on a display unit 40. The frame image has a predetermined pattern for giving a user a sense of depth with respect to the video.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display device or the like that displays an image captured while moving using a monocular imaging device.

  An electronic mirror system has already been known that projects an image obtained by imaging the surroundings of a vehicle on a display device in the vehicle (for example, Patent Document 1).

JP, 2009-83618, A (April 23, 2009 release)

  However, although the prior art as described above can display an image obtained by imaging the surroundings of the vehicle on a display device in the vehicle, the displayed image is different from the mirror image reflected in the mirror, and there is no sense of depth. It is an image of a dimension. Therefore, there is a problem that the driver who has used the mirror image reflected on the door mirror feels a sense of discomfort about the image.

  Some electronic mirrors display 3D video (stereoscopic video). 3D images have a natural sense of depth. Therefore, the driver can easily intuitively understand the position and the speed of the object (visual object) displayed on the display screen. In addition, the 3D image displayed on the far side of the display screen as seen from the driver is far from the driver's viewpoint as compared to the 2D image. Therefore, it is easy for the driver to move his eyes from the front of the vehicle to the display screen and to recognize the visual target displayed on the display screen.

  However, in order to display 3D images, the electronic mirror as described above needs to be equipped with a display device compatible with 3D display, a binocular camera, and a dedicated camera (eye tracking) that detects the driver's gaze state. Because there is, it costs more. In addition, since a conventional electronic mirror combines two captured images captured by a twin-lens camera by parallax mapping processing, it takes time to display a 3D image on the display screen of the electronic mirror. Since the driver operates the vehicle while watching the 3D video displayed on the display screen, it is a problem that the timing for displaying the 3D video on the display screen is delayed. Furthermore, the range in which the driver can recognize 3D video as a stereoscopic video is limited.

  One aspect of the present invention has been made in view of the above problems, and an object thereof is to display an image captured while moving using a monocular imaging device so as to give a sense of depth to a user It is providing a display apparatus.

  In order to solve the above-mentioned subject, a display concerning one mode of the present invention is a display which displays an image picturized moving, using an image pick-up device of a single eye, and an image analysis part which analyzes the image A frame image setting unit configured to set a frame image to be displayed around the video according to a result analyzed by the video analysis unit; and a display control unit configured to display the video together with the frame image on the display unit. The frame image is configured to have a predetermined pattern for giving the user a sense of depth to the video.

  A display method according to an aspect of the present invention is a display method by a display device that displays an image captured while moving using a single-eye imaging device, comprising: an image analysis step of analyzing the image; The frame image setting step of setting a frame image to be displayed around the video according to the analysis result in the analysis step, and the display control step of displaying the video together with the frame image on the display unit, The frame image has a predetermined pattern for giving the user a sense of depth with respect to the image.

  According to one aspect of the present invention, it is possible to provide a display device capable of displaying an image captured while moving so as to give the user a sense of depth with a configuration using a single-eye imaging device.

It is a block diagram showing an example of the principal part composition of the electronic mirror system concerning Embodiment 1 of the present invention. FIG. 2 is a schematic view of the inside of a vehicle equipped with the electronic mirror system shown in FIG. It is a schematic diagram at the time of displaying the combination of a daytime image and the frame image which distribute | arranged the gradation of black by the display apparatus which concerns on Embodiment 1 of this invention. It is a schematic diagram at the time of displaying the combination of the daytime image | video and the frame image which distribute | arranged the black thin wire on the background of white by the display apparatus which concerns on Embodiment 1 of this invention. It is a schematic diagram at the time of displaying the combination of the daytime image | video and the frame image which arranged the radial thick black line on the white background by the display apparatus which concerns on Embodiment 1 of this invention. It is a schematic diagram at the time of displaying the combination of a daytime image | video and the frame image which arranged the radial white thick line on the black background by the display apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows an example of the process which the display apparatus which concerns on Embodiment 1 of this invention performs. It is a schematic diagram which shows another example of the combination of the captured image and frame image which are displayed by the display apparatus which concerns on Embodiment 1 of this invention, (a) displayed the combination of the image and the black frame image. In the case of (b) displaying the combination of the video and the frame image of white only, (c) the display of the combination of the video and the frame image on which the predetermined pattern is arranged, and (d) the video and The case where the combination with the frame image in which the predetermined | prescribed texture was distribute | arranged is each shown is shown. It is a flowchart which shows an example of the process which the display apparatus which concerns on Embodiment 2 of this invention performs. It is a schematic diagram which shows an example of the combination of the imaging imaging | video used for estimation of a moving speed in the display apparatus which concerns on Embodiment 3 of this invention, (a) shows the image at a certain time and the outline image produced | generated from the said imaging, (B) shows a video after a predetermined time has passed since (a) and an outline image generated from the video, and (c) shows a video after a predetermined time has passed from (b) and an outline image generated from the video. The display apparatus which concerns on Embodiment 3 of this invention WHEREIN: It is a schematic diagram which shows an example of the combination of the captured image according to moving speed, and a frame image, (a) is an image when moving speed is low, and a frame image Indicates a combination of (B) shows a combination of video and frame image when the moving speed is medium, and (c) shows a combination of video and frame image when the moving speed is high. It is a schematic diagram which shows an example of the suitable combination of the frame image by the difference of a captured image in the display apparatus which concerns on Embodiment 4 of this invention, (a) distributes the radial black thick line on the background of a day image and a white. FIG. 14B is a schematic view of a combination of a night picture and a frame picture in which a thick radial black line is arranged on a white background. , (C) is a schematic view of a combination of a daytime image and a frame image with a thick white radial line on a black background, and (d) is a night image and a radial pattern on a black background It is a schematic diagram at the time of displaying the combination with the frame image which arranged the white thick line. It is the table | surface which put together the difference of the legibility with respect to the combination shown to (a)-(d) of FIG. It is a schematic diagram which shows the specific example of the method of calculating the brightness | luminance of imaging | video in the display apparatus which concerns on Embodiment 4 of this invention, (a) is an imaging | video and frame image which are displayed on a display part when imaging is a daytime image. 7B shows a series of flows for setting an image to be displayed on the display unit and a frame image when the image is a night image. The display apparatus which concerns on Embodiment 5 of this invention WHEREIN: It is a schematic diagram which shows an example which sets a frame image according to the depth center of a captured image, (a) is a display when a depth center exists in the center of an image. A series of flows which set up a picture and a frame picture which are displayed on a part are shown, and (b) shows a picture which sets up a picture and a frame picture which are displayed on a display part when a depth center exists in positions other than the center of a picture. It shows the flow. It is the block diagram which illustrated the composition of the computer which can be used as a display of the above-mentioned each embodiment.

Embodiment 1
Hereinafter, an embodiment of the present invention will be described in detail with reference to FIGS. Although an electronic mirror system is explained below using an example applied to a car, it does not need to be limited to a car.

(Configuration of electronic mirror system)
The configuration of the electronic mirror system 100 according to the first embodiment will be described with reference to FIGS. 1 and 2.

  The electronic mirror system 100 is a system installed in a car or the like, and includes a display device 1 and a monocular imaging device 2. According to the example of FIG. 2, instead of the door mirror, the two imaging devices 2a and 2b can capture the right rear and the left rear of the vehicle and can display the captured video (hereinafter referred to as video) on the display devices 1a and 1b. Configuration.

  The display device 1 displays an image captured by the imaging device 2 (2a and 2b). The display device 1 (1a and 1b) displays an image together with a frame image for giving a sense of depth. Here, the frame image is an image displayed around an image on which a predetermined pattern is arranged. Preferably, the predetermined pattern is a pattern that causes the user to feel a sense of depth when viewing the predetermined pattern. The display device 1 displays a video captured while moving and displays it in combination with a frame image, thereby giving the user who has seen the video and the frame image a sense of depth as if the video with depth is viewed. it can. The display device 1 may be, for example, a liquid crystal monitor mounted on a car and optionally switchable with a car navigation function or the like. According to the example of FIG. 2, the display device 1 can display at least one of the images captured by each of the two imaging devices 2. Details of the display device 1 will be described later.

  The imaging device 2 is a monocular imaging device that captures an image with one lens. The imaging device 2 is, for example, a digital camera capable of transmitting a captured video to the display device 1. Communication between the imaging device 2 and the display device 1 may be performed by wire embedded in a car or may be performed by wireless communication.

(Detailed configuration of display device)
The detailed configuration of the display device 1 will be described with reference to FIG. The display device 1 includes a communication unit 10, an input unit 20, a storage unit 30, a display unit 40, and a control unit 50. The control unit 50 further includes an image acquisition unit 51, an image analysis unit 52, a frame image setting unit 53, and a display control unit 54.

  The communication unit 10 communicates with the imaging device 2, receives video information on a video captured by the imaging device 2, and transmits the video information to the video acquisition unit 51.

  The input unit 20 receives a user's input, and is configured of, for example, a touch panel integrally configured with the display unit 40. The user can use the input unit 20 to switch the display content of the display unit 40, and the like.

  The storage unit 30 stores various information handled by the display device 1. For example, the storage unit 30 stores a frame image to be displayed together with an image captured by the imaging device 2 by the display unit 40.

  The control unit 50 integrally controls the respective units of the display device 1.

  The video acquisition unit 51 receives video information from the imaging device 2 via the communication unit 10, and transmits a video to the video analysis unit 52 and the display control unit 54.

  The video analysis unit 52 analyzes the video received from the video acquisition unit 51, and transmits the analysis result to the frame image setting unit 53. The analysis includes, for example, detection of luminance of an image, center of depth, movement speed of a car, and the like.

  The frame image setting unit 53 uses the analysis result received from the video analysis unit 52 and the information stored in the storage unit 30 to set a frame image to be combined with the video. For example, the frame image setting unit 53 sets a frame image that greatly differs in color from the video so as to emphasize the video. The frame image setting unit 53 transmits the set frame image to the display control unit 54.

  The display control unit 54 causes the display unit 40 to display the video received from the video acquisition unit 51 in combination with the frame image received from the frame image setting unit 53. For example, the display control unit 54 causes the display unit 40 to display the image superimposed on the center portion of the frame image.

(Example of combination of image and frame image)
In the present embodiment, an example of a combination of a video and a frame image which the display control unit 54 causes the display unit 40 to display will be described using FIGS. 3 to 6. 3 to 6 are schematic views showing a combination of a daytime image 41a which is a daytime image and frame images 42a to 42d different from each other, which are displayed on the display device 1.

  FIG. 3 is a schematic view when a combination of a daytime image 41a and a frame image 42a in which a black gradation is arranged is displayed on the display unit 40. As shown in FIG. When the user looks at the entire display unit 40, the user has the illusion of the daytime image 41a as if it were a video with depth due to the visual effect given by the gradation of gradation. That is, the user recognizes the daytime image 41a as an image with a sense of depth.

  FIG. 4 is a schematic view in the case where a combination of a daytime image 41 a and a frame image 42 b in which radial black thin lines are arranged on a white background is displayed on the display unit 40. Similar to FIG. 3, the user recognizes the daytime video 41a as a video with a sense of depth by the visual effect given by the radial black thin line.

  FIG. 5 is a schematic view in the case where a combination of a daytime image 41 a and a frame image 42 c in which a radial thick black line is arranged on a white background is displayed on the display unit 40. Similar to FIGS. 3 and 4, the user recognizes the daytime video 41a as a video with a sense of depth by the visual effect given by the thick radial black line.

  FIG. 6 is a schematic view in the case where the combination of a daytime image 41a and a frame image 42d in which a thick radial white line is arranged on a black background is displayed on the display unit 40. Similar to FIGS. 3 and 4, the user recognizes the daytime video 41a as a video with a sense of depth by the visual effect given by the thick radial white line.

(Flow of processing)
In the present embodiment, the flow of processing performed by the display device 1 will be described with reference to FIG. FIG. 7 is a flowchart showing an example of the flow of processing performed by the display device 1 according to the present embodiment. The display device 1 repeats a series of processes shown in FIG. 7 in real time.

  First, the video acquisition unit 51 acquires video information from the imaging device 2 via the communication unit 10, and transmits the acquired video to the video analysis unit 52 and the display control unit 54 (S1). Next, the video analysis unit 52 analyzes the video received from the video acquisition unit 51, and transmits the analysis result to the frame image setting unit 53 (S2: video analysis step).

  The frame image setting unit 53 acquires a frame image according to the analysis result from the storage unit 30 using the analysis result received from the video analysis unit 52, and sets it as a frame image to be displayed together with the video (S3: frame image Setting step). Then, the display control unit 54 displays the video captured by the imaging device 2 and the frame image received from the video acquisition unit 51 on the display unit 40 (S4: display control step).

  By the above processing, the display device 1 can display an image to the user together with the frame image having the predetermined pattern. Thus, the user can view the image in a state of having a sense of depth by the predetermined pattern provided in the frame image. Therefore, it is effective in providing a display which can be displayed so that a user can give a feeling of depth to a user by composition which used an image pick-up device of a single eye, and moved.

  In addition, the process which the display apparatus 1 which concerns on this embodiment performs does not need to be limited to FIG. For example, if the frame image set by the frame image setting unit 53 in S3 is the same as the frame image being displayed on the display unit 40, the display control unit 54 updates only the video in S4. It is also good.

  Further, the video displayed by the display device 1 may be switchable to other than the video captured while the imaging device 2 is moving. For example, the map information displayed by the car navigation function may be displayed together with the frame image.

  All of the frame images 42a to 42d shown in FIG. 3 to FIG. 6 are configured to have a pattern indicating a change from the video toward the outer periphery of the frame image. However, the predetermined pattern provided on the frame image may be any configuration as long as the user can view the image so that the image has an illusion of depth.

(Another example of combination of image and frame image)
Another example of the predetermined pattern provided in the frame image will be described with reference to FIG. FIG. 8 is a schematic view showing another example of the combination of the video and the frame image displayed on the display device 1.

  FIG. 8A is a schematic view in the case where the combination of the daytime image 41a and the black black frame image 42e is displayed on the display unit 40. FIG.

  (B) of FIG. 8 is a schematic view in the case where a combination of a daytime image 41a and a frame image 42f of only white color is displayed on the display unit 40.

  FIG. 8C is a schematic view in the case where a combination of a daytime image 41a and a frame image 42g in which a predetermined pattern is arranged is displayed on the display unit 40. The predetermined pattern may be colored, and may be pink, for example. Also, the predetermined pattern may be, for example, a brick pattern.

  FIG. 8D is a schematic view in the case where a combination of a daytime image 41a and a frame image 42h in which a predetermined texture is arranged is displayed on the display unit 40. The predetermined pattern may be colored, and may be, for example, brown.

  The frame image setting unit 53 can also set frame images 42 e to h shown in (a) to (d) of FIG. 8 as frame images according to the analysis result. Since none of the frame images has a pattern indicating a change from the video toward the outer periphery of the frame images, it is difficult for the frame images 42e to 42h to give the user a sense of depth alone. However, since there are individual differences in human visual characteristics, when the frame images 42e to 42h are set as frame images and displayed together with the video, some users may feel a sense of depth with respect to the video.

Second Embodiment
Embodiment 2 of the present invention will be described below using FIGS. 1, 6, 8 and 9. In addition, about the member which has the same function as the member demonstrated in the said embodiment for convenience of explanation, the same code | symbol is appended, and the description is not repeated.

(Configuration of electronic mirror system)
The configuration of the electronic mirror system 100 according to the second embodiment will be described with reference to FIG.

  In the present embodiment, the basic configuration of the electronic mirror system 100 is the same as that of the first embodiment, but the partial configuration of the display device 1 is different. The display device 1 is different from the frame image displayed with the image captured by the imaging device 2 in that the frame image is set according to the tendency of the visual characteristic of the user who views the image. More specifically, the frame image setting unit 53 sets the frame image based on the analysis result in the video analysis unit 52 and the tendency of the user who views the video to easily feel a sense of depth. In addition, although the information regarding the tendency of a user's visual characteristic may be hold | maintained how, it is suitable to be preset by the memory | storage part 30, for example.

  Specific examples of the combination of frame images according to the tendency of the user's visual characteristics will be described below.

(A combination of frame images according to the user's visual characteristics)
The combination of the image and the frame image that most produces a sense of depth when viewed is largely dependent on the visual characteristics of the individual. Therefore, there is a possibility that a combination with a frame image that gives a certain sense of depth most for a particular video may be a combination that does not have a sense of depth for another user. Similarly, with respect to a specific frame image, a combination with a video that gives the most a sense of depth to one user may be a combination that does not have a sense of depth at all for another user.

  For example, with regard to the combination of the daytime image 41a and the frame image 42d in which a radial white thick line is arranged on a black background, many people tend to feel a sense of depth, as shown in FIG. On the other hand, although there are not many combinations of the daytime image 41a shown in (c) of FIG. 8 and the frame image 42g in which a pink brick pattern is arranged, some people feel a sense of depth.

  Furthermore, even with the same pattern, the strength of remembering the sense of depth differs depending on the color of the frame image and the like. For example, in the frame image 42g in which the pink brick pattern shown in FIG. 8C is arranged, if the brick pattern is pink, the user feels a sense of depth if the color is another color. There is a possibility not to feel much.

  From this, it is preferable to set the frame image based on not only the video but also the visual characteristics of the user who uses the display device 1. That is, it is preferable that the frame image setting unit 53 set, for each user who uses the display device 1, a frame image along a tendency that the user can easily feel a sense of depth. Note that any configuration may be used as long as the frame image setting unit 53 can refer to the tendency of the user to easily sense the sense of depth. For example, it is suitable for each user by registering in advance the tendency that the user tends to feel a sense of depth by using the input unit 20 or the display unit 40 and selecting the user who uses the apparatus when the use of the display apparatus 1 is started. It is possible to display a combination of an image and a frame image.

(Flow of processing)
In the present embodiment, the flow of processing performed by the display device 1 will be described with reference to FIG. FIG. 9 is a flowchart illustrating an example of the flow of processing performed by the display device 1.

  The process performed by the display device 1 in the present embodiment is basically the same as that of the first embodiment except that the process of S5 is performed instead of S3.

  After S2, in addition to the analysis result received from the video analysis unit 52, the frame image setting unit 53 sets the frame image according to the tendency of the frame image in which the user currently using the display device 1 can easily feel a sense of depth. To do (S5). After S5, the display control unit 54 executes the process of S4 as in the first embodiment.

  By the above processing, the display device 1 can set a frame image according to the tendency of the visual characteristic of the user who uses the display device 1 and can display the frame image together with the image.

Third Embodiment
The third embodiment of the present invention will be described below using FIGS. 1, 7, 10 and 11. In addition, about the member which has the same function as the member demonstrated in the said embodiment for convenience of explanation, the same code | symbol is appended, and the description is not repeated.

(Configuration of electronic mirror system)
The configuration of the electronic mirror system 100 according to the third embodiment will be described with reference to FIG.

  In the present embodiment, the basic configuration of the electronic mirror system 100 is the same as that of the above-described embodiments, but a partial configuration of the display device 1 is different. The display device 1 analyzes the magnitude of change per hour of the image captured by the imaging device 2, estimates the moving speed of the moving object equipped with the imaging device 2, and determines the moving speed of the moving object estimated. The difference is that the size of the display area of the frame image is changed and displayed. More specifically, the display control unit 54 changes the height or width of the display area of the frame image according to the moving speed of the moving object obtained as a result of analyzing the video by the video analysis unit 52. Display images and picture frames. As the frame image, a specific frame image may be set according to the moving speed of the image, or may be changed as appropriate. Further, the mobile body may be the automobile used in each of the above-described embodiments, but it may be movable as long as the imaging device 2 is mounted. For example, the mobile may be a motorcycle, a ship, an airplane, and so on. In the following description, as in the above embodiments, the case where the mobile unit is an automobile will be described.

(Relationship between moving speed of car and field of vision)
A person's visual field has a characteristic that it narrows as the moving speed increases. For example, the field of view of a person in a stationary state is a wide angle of view, for example, about 200 degrees in the horizontal direction, but the field of view of a person moving at 40 km per hour is narrowed to about 100 degrees in the lateral direction. Furthermore, it is said that it will be narrowed to 30 degrees when it reaches 130 km / hr. Therefore, when a video captured while moving is displayed together with the frame image, the user narrows the field of view according to the movement, and as a result, the frame image may be largely deviated from the user's field of view. If the frame image greatly deviates from the user's field of view, the user does not obtain a sense of depth from the frame image, so it is necessary to change the size of the display area of the frame image according to the moving speed of the car estimated from the video. There is.

(Moving speed estimation method based on video)
A specific example of a method of estimating the moving speed of a car based on a video captured by the imaging device 2 will be described with reference to FIG. FIG. 10 is a schematic view showing an example of a combination of images used to estimate the moving speed in the display device 1 according to the present embodiment, and (a) is an outline image generated from the image 41d and the image 41d at a certain point in time. 44 d is shown. (B) shows an image 41 e after a predetermined time has passed since (a) and an outline image 44 e generated from the image 41 e, and (c) shows an image 41 f after a predetermined time has passed from (b) and the image 41 f An outline image 44f is shown.

  FIG. 10A shows a video 41 d showing the rear of a car and a contour video 44 d extracted from the contour of an object shown in the video 41 d, captured by the imaging device 2 at a certain point in time. The image analysis unit 52 uses the contour image 44 d to select an appropriate feature to analyze the moving speed. In the illustrated example, the front portion of the track located on the center left side of the contour image 44d is selected as the feature.

  (B) of FIG. 10 shows an image 41e showing the rear of the car and an outline image 44e of which an outline is extracted from the image 41e, which is imaged by the imaging device 2 after a predetermined time has elapsed from (a) of FIG. The image analysis unit 52 calculates the moving speed of the vehicle equipped with the imaging device 2 (and the display device 1) from the change in the position and the size of the front part of the track selected as the feature in FIG. Do.

  (C) of FIG. 10 shows a video 41 f showing the rear of the car and a contour video 44 f of which the contour is extracted from the video 41 f captured by the imaging device 2 after a predetermined time has elapsed from (b) of FIG. And the image | video analysis part 52 calculates the moving speed of the motor vehicle carrying the imaging device 2 (and the display apparatus 1) similarly to the time of (b) of FIG. That is, the video analysis unit 52 calculates the moving speed from the video captured at predetermined time intervals.

  The display control unit 54 according to the present embodiment changes the size of the display area of the captured image displayed on the display unit 40 based on the moving speed of the vehicle estimated by the above method.

(Combination of image and frame image according to movement speed)
In the present embodiment, the display control unit 54 changes and displays the size of the display area of the frame image in accordance with the moving speed of the vehicle estimated by the method described with reference to FIG. An example of the combination of the video and the frame image according to the moving speed will be described with reference to FIG. FIG. 11 is a schematic view showing an example of a combination of a video and a frame image according to the moving speed in the display device 1, and (a) shows a combination of the video and the frame image when the moving speed is low. (B) shows a combination of video and frame image when the moving speed is medium, and (c) shows a combination of video and frame image when the moving speed is high.

  (A) of FIG. 11 shows a combination of an image and a frame image when the estimated moving speed of the vehicle is low. At low speeds, the viewing angle of a person is relatively wide, so even if the display area of the frame image is reduced, it is considered to be sufficiently recognized. Therefore, by making the display area of the frame image smaller and making the display area of the video larger, it is possible to make the video easier to view and allow the user to feel a sense of depth.

  (B) of FIG. 11 shows a combination of an image and a frame image when the estimated moving speed of the vehicle is medium. At a medium speed, the viewing angle of a person is narrower than at a low speed, so the display area of the frame image is made larger than in the case of FIG. As a result, even when the moving speed is medium, the user can be made to feel a sense of depth.

  (C) of FIG. 11 shows the combination of the image and the frame image when the estimated moving speed of the automobile is high. The display area of the frame image is made larger than in the cases of (a) and (b) in FIG. 11 because the viewing angle of a person is the narrowest at high speeds compared to those at low and medium speeds. As a result, even when the moving speed is high, the user can feel a sense of depth.

  As described above, by combining the image according to the moving speed and the frame image, it is possible to cause the user to have a sufficient sense of depth regardless of the size of the moving speed.

(Flow of processing)
In the present embodiment, the flow of processing performed by the display device 1 will be described. In addition, since the flow of the process in this embodiment is the same as FIG. 7 in the said Embodiment 1, it demonstrates using FIG.

  As described above, the process performed by the display device 1 in the present embodiment is basically the same as that in the first embodiment. The difference is in the following three points. The first point is that the video analysis unit 52 estimates the moving speed of the vehicle by the method as described with reference to FIG. 10 in S2, and transmits the moving speed to the frame image setting unit 53 as an analysis result. is there. The second point is that the frame image setting unit 53 sets a frame image in S3 and transmits the moving speed of the vehicle received from the video analysis unit 52 together with the set frame image to the display control unit 54. The third point is that the display control unit 54 determines the size of the display area of the captured image on the display unit 40 according to the moving speed of the car received from the frame image setting unit 53 in S4, and displays the image together with the video It is a point that

  By the above processing, the display device 1 can change and display the size of the captured image according to the moving speed of the vehicle. In addition, if the display control unit 54 can be used to determine the size of the display area of the captured image, the moving speed of the vehicle does not have to be estimated from the video. For example, the display control unit 54 may acquire the value indicated by the speedometer of the automobile, and use the acquired value to determine the size of the display area of the captured image.

  Further, according to the example of FIG. 2, since the imaging device 2 is disposed outside the vehicle, it is easier to acquire position information by communication with the outside than the display device 1. Therefore, the imaging device 2 may be configured to acquire position information regarding a position at which the image has been captured and to transmit the position information to the display device 1 together with the captured image. At this time, the display device 1 may estimate the moving speed of the vehicle using the received position information.

Embodiment 4
Embodiment 4 of the present invention will be described below with reference to FIGS. In addition, about the member which has the same function as the member demonstrated in the said embodiment for convenience of explanation, the same code | symbol is appended, and the description is not repeated.

(Configuration of electronic mirror system)
The configuration of the electronic mirror system 100 according to the fourth embodiment will be described with reference to FIG.

  In the present embodiment, the basic configuration of the electronic mirror system 100 is the same as that of the above-described embodiments, but a partial configuration of the display device 1 is different. The display device 1 is different in that a frame image is set and displayed according to the luminance of the image captured by the imaging device 2. More specifically, the video analysis unit 52 calculates the overall luminance of the video, and the frame image setting unit 53 sets a frame image according to the calculated overall luminance.

(Combination of frame images according to the brightness of the image)
Regarding the frame image, in each of the above embodiments, the frame image that gives the user a sense of depth when displayed in combination with the daytime image 41a has been described. However, depending on the combination of the video and the frame image, the boundary between the video and the frame image may be difficult to see, and the display may be difficult for the user to feel a sense of depth. FIG. 12 is a schematic view showing an example of an appropriate combination of frame images due to differences in images in the display device 1 according to the present embodiment. (A) of FIG. 12 is a schematic view in the case of displaying a combination of a daytime image 41a and a frame image 42c in which a radial thick black line is arranged on a white background. (B) of FIG. 12 is a schematic view in the case of displaying a combination of a night image 41b indicating night and a frame image 42c in which a thick radial line is arranged on a white background. FIG. 12C is a schematic view when a combination of a daytime image 41a and a frame image 42d in which a thick radial white line is arranged on a black background is displayed. FIG. 12D is a schematic view in the case of displaying a combination of a night image 41b and a frame image 42d in which a thick radial white line is arranged on a black background. Moreover, the table | surface which put together the difference of the legibility with respect to the combination shown to (a)-(d) of FIG. 12 is shown in FIG. In the illustrated example, each record corresponds to each diagram of FIG. 12, and the item name “figure” corresponds to the figure number. The item name “video” indicates which of the day video 41 a and the night video 41 b the video to be displayed is. The item name “frame image” indicates which of the frame image to be displayed is a frame image 42c in which a thick radial black line is arranged on a white background or a frame image 42d in which a thick thick radial white line is arranged on a black background. It shows. The item name “judgement” indicates the result of judging whether the combination of “image” and “frame image” is easy to see, “o” indicates that it is easy to see, and “Δ” indicates that it is hard to see There is.

  In (a) of FIG. 12, the daytime image 41 a is an image with relatively high luminance, and the frame image 42 c is also an image with relatively high luminance. In FIG. 12, (a) corresponds to the record in which the item name “figure” in FIG. 13 is “12 (a)”, and in the illustrated example, the record in which the item name “figure” is “12 (a)” The item name “decision” is “Δ”. That is, the combination of the daytime image 41a and the frame image 42c is a combination that is difficult for the user to see. For example, the boundary between the daytime image 41a and the frame image 42c is difficult for the user to check. Therefore, it can be said that it is not preferable to combine the frame image 42c with the daytime image 41a.

  In (b) of FIG. 12, the night image 41 b is an image with relatively low luminance, while the frame image 42 c is an image with relatively high luminance. In FIG. 12, (b) corresponds to the record whose item name "figure" is "12 (b)" in FIG. 13, and in the illustrated example, the record whose item name "figure" is "12 (b)" The item name “judgement” is “o”. That is, the combination of the night image 41b and the frame image 42c is a combination that is easy for the user to see. For example, the boundary between the night image 41 b and the frame image 42 c is easy for the user to check. In addition, the night image 41b is illuminated by the frame image 42c, and the night image 41b can be easily viewed. Therefore, it is preferable to combine the frame image 42c with the night image 41b.

  In (c) of FIG. 12, the daytime image 41 a is an image with relatively high luminance, while the frame image 42 d is an image with relatively low luminance. Item (c) in FIG. 12 corresponds to the record whose item name “figure” in FIG. 13 is “12 (c)”, and in the illustrated example, the record whose item name “figure” is “12 (c)” The item name “judgement” is “o”. That is, the combination of the daytime image 41a and the frame image 42d is a combination that is easy for the user to view. For example, the boundary between the daytime image 41a and the frame image 42d is easy for the user to check. Further, since the reflection of the light emitted by the daytime image 41a in the frame image 42d is suppressed, the daytime image 41a is easily viewed. Therefore, it can be said that it is preferable to combine the frame image 42d with the daytime image 41a.

  In (d) of FIG. 12, the night image 41 b is an image with relatively low luminance, and the frame image 42 d is also an image with relatively low luminance. Item (d) in FIG. 12 corresponds to the record whose item name “figure” in FIG. 13 is “12 (d)”, and in the illustrated example, record where item name “figure” is “12 (d)” The item name “decision” is “Δ”. That is, the combination of the night image 41b and the frame image 42d is a combination that is difficult for the user to see. For example, it is difficult for the user to confirm the boundary between the night image 41b and the frame image 42d. Therefore, it can be said that it is not preferable to combine the frame image 42d with the night image 41b.

  From these facts, it can be said that the frame image is preferably set in consideration of the luminance of the video.

(Specific example of calculation method of luminance)
In the present embodiment, a specific example of a method in which the video analysis unit 52 calculates the luminance of a video will be described with reference to FIG. (A) of FIG. 14 shows a series of flows of setting the video and the frame image to be displayed on the display unit 40 when the video is a daytime video. (B) of FIG. 14 shows a series of flows of setting the video and the frame image to be displayed on the display unit 40 when the video is a night video.

  In the example of FIG. 14A, when the video analysis unit 52 receives the day video 41a as a video from the video acquisition unit 51, histogram analysis is performed on the brightness of the day video 41a to show the peak of the brightness and the distribution of the brightness. A histogram 43a is generated. The image analysis unit 52 further calculates the overall brightness of the day image 41a from the histogram 43a.

  Next, the frame image setting unit 53 sets a frame image 42d in which a thick radial line is arranged on a black background based on the overall brightness of the day image 41a received from the video analysis unit 52.

  Then, the display control unit 54 causes the display unit 40 to display the daytime image 41 a and the frame image 42 d. In this manner, the frame image can be set and displayed according to the brightness of the video.

  In the example of FIG. 14B, when the video analysis unit 52 receives the night video 41b as a video from the video acquisition unit 51, the histogram 43b related to the luminance of the night video 41b is generated, and the histogram 43b generates the night video 41b. Calculate the overall brightness. The calculation method may be the same as that shown in FIG.

  Next, the frame image setting unit 53 sets a frame image 42c in which a radial thick black line is arranged on a white background based on the entire luminance of the night image 41b.

  Then, the display control unit 54 causes the display unit 40 to display the night image 41 b and the frame image 42 c. In this manner, the frame image can be set and displayed according to the brightness of the video.

  The information on the luminance of the video calculated by the video analysis unit 52 and used for setting the frame image by the frame image setting unit 53 is not the overall luminance as long as it indicates the characteristic on the luminance of the video. Good. For example, the frame image setting unit 53 may set the frame image based on the peak value of luminance.

  In addition, if the frame image setting unit 53 can set a frame image in which the border with the video can be easily viewed, the video analysis unit 52 may not generate the information on the luminance of the video. For example, the frame image setting unit 53 may receive the value of the illuminance detected by the illuminance sensor using an illuminance sensor (not shown), and set the frame image according to the value of the illuminance.

Fifth Embodiment
Embodiment 5 of the present invention is described below using FIGS. 1, 3, 4 and 15. In addition, about the member which has the same function as the member demonstrated in the said embodiment for convenience of explanation, the same code | symbol is appended, and the description is not repeated.

(Configuration of electronic mirror system)
The configuration of the electronic mirror system 100 according to the fifth embodiment will be described with reference to FIGS.

  In the present embodiment, the basic configuration of the electronic mirror system 100 is the same as that of the above-described embodiments, but a partial configuration of the display device 1 is different. The display device 1 analyzes the depth center so that the user can obtain a sense of depth with respect to the depth center of the image captured by the imaging device 2, and the depth center corresponds to the origin of the plurality of radial lines or the origin of the gradation. The point of setting the frame image is different. More specifically, the video analysis unit 52 analyzes the depth center of the video, and the frame image setting unit 53 sets the depth center to the origin of a plurality of radial lines in the frame image such as the frame image 42b of FIG. This is the origin of gradation in a frame image such as the frame image 42a of FIG. Note that, for a frame image whose origin is the depth center of the image, for example, a frame image in which the origin is set at the center of the screen or a frame image in which the origin is set at the left of the screen center A plurality of frame images in which the origin is set at a position may be created in advance and stored in the storage unit 30.

(A concrete example of the analysis method of the depth center)
In the present embodiment, a specific example of a method in which the video analysis unit 52 analyzes the depth center of a video will be described with reference to FIG. FIG. 15 is a schematic view showing an example of setting a frame image according to the depth center of a captured image in the display device 1 according to the present embodiment. (A) of FIG. 15 shows a series of flows for setting the video and the frame image to be displayed on the display unit 40 when the depth center exists at the center of the video. (B) of FIG. 15 shows a series of flows for setting the video and the frame image to be displayed on the display unit 40 when the depth center is present at a position other than the center of the video.

  In the example of (a) of FIG. 15, when the video analysis unit 52 receives the video 41a, the contour video 44a is generated, and the depth center of the contour video 44a is analyzed. For example, the image analysis unit 52 selects a position where the outlines are dense in the outline image 44a and the outlines extend radially therefrom as the depth center.

  Next, the frame image setting unit 53 selects from the storage unit 30 a frame image 42d in which the origins of the plurality of radial lines overlap with the depth center selected by the video analysis unit 52 and displays the frame image 42d together with the video 41a. Set

  Then, the display control unit 54 causes the display unit 40 to display the image 41 a and the frame image 42 d in combination. In this manner, the position of the center of the image depth can be analyzed, and an appropriate frame image can be set according to the analyzed position.

  In the example of (b) of FIG. 15, when the video analysis unit 52 receives the video 41 c, the contour video 44 c is generated, and the depth center of the contour video 44 c is analyzed. The analysis method may be the same as (a) in FIG.

  Next, the frame image setting unit 53 selects from the storage unit 30 a frame image 42i in which the origins of a plurality of radial lines overlap with the depth center selected by the video analysis unit 52 and sets it as a frame image to be displayed together with the video 41c. Do.

  Then, the display control unit 54 causes the display unit 40 to display the video 41 c and the frame image 42 i in combination. In this manner, the position of the center of the image depth can be analyzed, and an appropriate frame image can be set according to the analyzed position.

  In the above description, the frame image setting unit 53 is configured to select a frame image in which the center of the image depth and the origin overlap from a plurality of images. However, the present invention is not limited to this. For example, the frame image setting unit 53 may be configured to generate a frame image in which a plurality of radial lines are arranged such that the depth center selected by the video analysis unit 52 is the origin.

Sixth Embodiment
In the above embodiments, an example in which one display device 1 is used has been described, but a part of the functions of the display device 1 may be provided in a server that can communicate with the display device 1. That is, the present invention can also be realized as an electronic mirror system including a display device, an imaging device, and a server.

Seventh Embodiment
Each block of the display device 1 may be realized by a logic circuit (hardware) formed in an integrated circuit (IC chip) or the like, or may be realized by software. In the latter case, the display device 1 can be configured using a computer (electronic computer) as shown in FIG.

  FIG. 16 is a block diagram illustrating the configuration of a computer 910 that can be used as the display device 1 of each of the embodiments. The computer 910 includes an arithmetic unit 912 connected to one another via a bus 911, a main storage unit 913, an auxiliary storage unit 914, an input / output interface 915, and a communication interface 916. Arithmetic unit 912, main storage unit 913, and auxiliary storage unit 914 may each be, for example, a processor (for example, CPU: Central Processing Unit), RAM (random access memory), or hard disk drive. Connected to the input / output interface 915 are an input device 920 for the user to input various information to the computer 910 and an output device 930 for the computer 910 to output various information to the user. The input device 920 and the output device 930 may be built in the computer 910 or may be connected (externally connected) to the computer 910. For example, the input device 920 may be a camera and a button, a touch sensor or the like, and the output device 930 may be a display or the like. In addition, a device having both the functions of the input device 920 and the output device 930, such as a touch panel in which a touch sensor and a display are integrated, may be applied. The communication interface 916 is an interface for the computer 910 to communicate with an external device.

  The auxiliary storage device 914 stores various programs for operating the computer 910 as the display device 1. Then, the arithmetic unit 912 expands the above program stored in the auxiliary storage unit 914 onto the main storage unit 913 and executes an instruction included in the program, thereby causing the computer 910 to be a unit included in the display unit 1. Make it work. Note that the recording medium for recording information such as a program included in the auxiliary storage device 914 may be a computer readable “non-temporary tangible medium”, for example, a tape, a disk, a card, a semiconductor memory, a programmable logic It may be a circuit or the like. Further, as long as the computer can execute the program recorded on the recording medium without expanding the program on the main storage device 913, the main storage device 913 may be omitted. Note that each of the above devices (the arithmetic unit 912, the main storage unit 913, the auxiliary storage unit 914, the input / output interface 915, the communication interface 916, the input unit 920, and the output unit 930) may be one. There may be more than one.

  Also, the program may be acquired from the outside of the computer 910, and in this case, it may be acquired via any transmission medium (a communication network, a broadcast wave, etc.). The present invention can also be realized in the form of a data signal embedded in a carrier wave in which the program is embodied by electronic transmission.

[Modification]
In each of the embodiments described above, the frame image setting unit 53 may be configured to allow the user to register in advance what frame image to set. Further, the frame image is not limited to a still image, and may be, for example, a moving image.

  In each of the above-described embodiments, the frame image may be displayed in any manner as long as it can be displayed on the display unit 40 together with the video and the user can be given a sense of depth with respect to the video. For example, the size, the brightness, the design, the pattern, the position on the display unit 40, and the like of the display area may be changed and displayed.

  In each of the above embodiments, the information referred to by the frame image setting unit 53 for setting the frame image may be any information as long as the information can be acquired from the analysis result by the video analysis unit 52 or the like. . For example, the frame image setting unit 53 may set the frame image with reference to information such as user settings, ambient illumination estimated from the brightness of the image, the speed of the car, and the origin position. Furthermore, the frame image setting unit 53 may set the frame image based on a combination of two or more of the pieces of information.

[Summary]
A display device (1) according to aspect 1 of the present invention is a display device for displaying a video captured while moving using a single-eye imaging device (2), and a video analysis unit (52) for analyzing the video And a frame image setting unit (53) for setting a frame image to be displayed around the video according to the result analyzed by the video analysis unit, and displaying the video together with the frame image on the display unit (40) And a display control unit (54), wherein the frame image is provided with a predetermined pattern for giving the user a sense of depth to the video.

  According to the above configuration, it is possible to display an image together with a frame image having a predetermined pattern. As a result, the user can view the image in a state of having a sense of depth by the predetermined pattern provided in the frame image. Therefore, it is possible to provide a display device that displays a video captured while moving using a single-eye imaging device so as to give the user a sense of depth.

  In the display device (1) according to aspect 2 of the present invention, in the aspect 1, the predetermined pattern is a plurality of radial lines extending from the video toward the outer periphery of the frame image, or the frame image from the video The gradation may change in gradation toward the outer periphery of the image.

  According to the above configuration, the user can feel a sense of depth with respect to the image based on the frame image provided with the radial line or the gradation.

  In the display device (1) according to aspect 3 of the present invention, in the aspect 2, the video analysis unit (52) analyzes the depth center of the video, and the frame image setting unit (53) is the depth center May be set as the origin of the plurality of radial lines or the origin of the gradation.

  According to the above configuration, it is possible to display a frame image having radial lines or gradations set with the depth center of the image as the origin together with the image. Thus, the origin of the sense of depth estimated by the user from the predetermined pattern of the frame image can be matched with the origin of the depth of the actual video, so that the sense of depth without discomfort to the user can be provided. .

  In the display device (1) according to aspect 4 of the present invention, in any one of the aspects 1 to 3, the display control unit (54) performs the movement according to the moving speed of the movable body on which the imaging device is mounted. The size of the display area of the frame image may be changed and displayed.

  According to the above configuration, the video and the frame image can be displayed in a size according to the moving speed of the moving object. As a result, for example, when the moving speed is large, the display area of the frame image can be large, that is, the image can be small. When the moving speed is small, the display area of the frame image can be small, that is, the image can be large. By displaying an image with a size according to the moving speed, it is possible to give an appropriate sense of depth even when the user's view is narrowed according to the moving speed.

  In the display device (1) according to aspect 5 of the present invention, in the aspect 4, the video analysis unit (52) is also configured to estimate the moving speed of the mobile mounted with the imaging device from the video. Good.

  According to the above configuration, it is possible to change and display the size of the display area of the frame image using the estimated moving speed of the moving object.

  In the display device (1) according to aspect 6 of the present invention, in any of the aspects 1 to 5, the video analysis unit (52) analyzes the overall luminance of the video, and the frame image setting unit (53) The frame image may be set according to the overall brightness.

  According to the above configuration, the video can be displayed together with the frame image set according to the overall brightness of the video. For example, if the video is a daytime video, a low luminance frame image can be set, and if the video is a nighttime video, a high luminance frame image can be set. Thereby, the brightness of the entire display content can be adjusted to make it easy to check the video.

  In the display device (1) according to the seventh aspect of the present invention, in any one of the first to sixth aspects, the frame image setting unit (53) responds to the tendency of the visual characteristic of the user set in advance for each user. The frame image may be set.

  According to the above configuration, the display device can display the video together with the frame image set according to the tendency of the visual characteristic of the user. As a result, with regard to the image displayed on the display device, it is possible to give each user a sense of depth regardless of differences in appearance due to individual differences.

  A display method according to aspect 8 of the present invention is a display method by a display device (1) that displays a video captured while moving using a single-eye imaging device (2), and a video analysis step of analyzing the video (S2), a frame image setting step (S3) for setting a frame image to be displayed around the video according to the analysis result in the video analysis step, and displaying the video together with the frame image on the display unit And a display control step (S4), wherein the frame image is provided with a predetermined pattern for giving the user a sense of depth to the video. According to the above-mentioned method, the same operation and effect as in the above-mentioned aspect 1 are exerted.

  An electronic mirror system (100) according to aspect 9 of the present invention is an electronic mirror system that displays an image captured while moving using a single-eye imaging device (2), and the display apparatus (1) according to aspect 1 And a monocular imaging device for capturing the image and transmitting the image to the display device. According to the above-mentioned composition, the same operation effect as the above-mentioned mode 1 is produced.

  The display device 1 according to each aspect of the present invention may be realized by a computer. In this case, the display device 1 is used as a computer by operating the computer as each unit (software element) included in the display device 1. The control program of the display device 1 to be realized and the computer readable recording medium recording the same also fall within the scope of the present invention.

  The present invention is not limited to the above-described embodiments, and various modifications can be made within the scope of the claims, and embodiments obtained by appropriately combining the technical means disclosed in the different embodiments. Is also included in the technical scope of the present invention. Furthermore, new technical features can be formed by combining the technical means disclosed in each embodiment.

Reference Signs List 100 electronic mirror system 1 display device 10 communication unit 20 input unit 30 storage unit 40 display unit 41a day image 41b night image 42a frame image 42b frame image 42c frame image 42d frame image 42e frame image 42f frame image 42g frame image 42h frame image 50 Control unit 51 Image acquisition unit 52 Image analysis unit 53 Frame image setting unit 54 Display control unit 2 Imaging device

Claims (10)

A display device for displaying a captured image while moving using a single-eye imaging device,
A video analysis unit that analyzes the video;
A frame image setting unit configured to set a frame image to be displayed around the video according to a result analyzed by the video analysis unit;
And a display control unit that causes the display unit to display the video together with the frame image.
The display device is characterized in that the frame image has a predetermined pattern for giving a user a sense of depth to the video. The predetermined pattern is a plurality of radial lines extending from the video toward the outer periphery of the frame image, or a gradation in which shading changes from the video toward the outer periphery of the frame image. The display device according to 1. The video analysis unit analyzes the depth center of the video,
The display device according to claim 2, wherein the frame image setting unit sets the depth center as an origin of the plurality of radial lines or an origin of the gradation. The display control unit according to any one of claims 1 to 3, wherein the display control unit changes the size of the display area of the frame image according to the moving speed of the moving object mounted with the imaging device. The display device according to item 1. The display apparatus according to claim 4, wherein the video analysis unit estimates a moving speed of a moving body mounted with the imaging device from the video. The video analysis unit analyzes the overall brightness of the video,
The display device according to any one of claims 1 to 5, wherein the frame image setting unit sets the frame image according to the overall luminance. The display device according to any one of claims 1 to 6, wherein the frame image setting unit sets the frame image according to the tendency of the visual characteristic of the user set in advance for each user. . A display method by a display device for displaying a captured image while moving using a single-eye imaging device,
A video analysis step of analyzing the video;
A frame image setting step of setting a frame image to be displayed around the video according to the analysis result in the video analysis step;
And d) displaying the video together with the frame image on a display unit.
The display method is characterized in that the frame image is provided with a predetermined pattern that gives a user a sense of depth to the video. An electronic mirror system for displaying a captured image while moving using a monocular imaging device, comprising:
A display device according to claim 1;
An electronic mirror system comprising: a monocular imaging device for capturing the image and transmitting the image to the display device.   A control program for causing a computer to function as the display device according to claim 1, wherein the control program causes the computer to function as the video analysis unit, the frame image setting unit, and the display control unit.

Images