JP2019092094A - Image generation device, display system, and program - Google Patents

Abstract

To provide an image generation device, a display system, and a program which can obtain an image changing in accordance with a peripheral environment of a moving body.SOLUTION: The image generation device includes: an information generation unit (a parameter generation unit) that generates environment information (an example of a complexity parameter) indicating a degree of change in a peripheral environment of a moving body; and an image generation unit that controls an image processing according to the environment information and generates an image that changes according to the environment information.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image generation device, a display system, and a program.

  Patent Document 1 is a program for converting a mechanical original image to be input into a pictorial image, and is characterized in that the original image and the intended picture of the original image are felt. There is disclosed a conversion program to a pictorial image, characterized in that it has a synthesizing step of synthesizing with a filter image created with a predetermined gray scale pattern.

  Patent Document 2 includes a processor, storage means in which programs and data are stored, and image output means, and the data includes a subject image taken by a camera and a screen tone, and the program In an image processing apparatus for converting a subject image into cartoonishness with respect to a processor, the program causes the processor to (a) convert the subject image into a gray scale image, and (b) to convert the gray scale image into a first image. (B) generating a screen tone cutout image of pixels of the screen tone corresponding to the positions of the white pixels of the screen tone; An outline image obtained by binarizing with a second threshold smaller than a threshold of 1 to generate a binary image, and (d) inverting the edge image of the gray scale image (E) generating a first composite image in which the screen tone cutout image, the binary image, and the outline image are product-synthesized, and generating an image including a part or all of the first composite image An image processing apparatus is disclosed that is supplied to the image output unit as a toned image.

  Patent Document 3 stores data of a group of images for combination used for combination arbitrarily selected by the user from a group of continuously shot images obtained by photographing an object and a group of moving image constituting images forming a moving image. Image combining means for combining, image extracting means for extracting an image of start of operation of the subject, image in action, image of ending of action from the image group for combining, and a plurality of images extracted by the image extracting means Extracted image combining means for combining images, display means for displaying images, display control means for displaying a combined extracted image combined by the extracted image combining means, and a user selecting a predetermined rectangular area in the combined extracted image by the user A rectangular area input unit used for image processing; a rectangular area adjustment unit for adjusting a rectangular area selected by the user to a rectangular area matching the size of the printing paper; Rectangular area synthesizing means for synthesizing the inside of the rectangular area adjusted by the rectangular area adjusting means in the image, pictorial tone conversion means for converting the rectangular area synthetic image obtained by synthesizing the rectangular area synthesizing means into a pictorial tone; An image processing apparatus is disclosed, comprising: printing means for printing the pictorial image generated by the pictorial conversion means on the printing paper.

  Patent Document 4 relates to a method of converting the style of an original image, wherein at least a part of the image is subjected to an image conversion process that expresses the movement of a fluid. A conversion method is disclosed.

JP 2002-298136 A JP, 2014-102596, A JP 2012-129669 A JP, 2013-191114, A

  The conventional image processing apparatus or the like does not relate to an image reflecting the surrounding environment of a moving object.

  An object of the present invention is to provide an image generation device, a display system, and a program that can obtain an image that changes in accordance with the surrounding environment of a mobile body.

  In order to achieve the above object, an image generation apparatus according to the present invention is an image generation unit that generates environment information that indicates the degree of change in the surrounding environment of a mobile object, and controls image processing according to the environment information. And an image generation unit that generates the image generation unit.

  A display system of the present invention is a display system provided with the image generation device of the present invention and a display device for displaying information based on the image transmitted from the image generation device.

  A program according to the present invention includes a computer as an information generation unit that generates environment information that indicates the degree of change in the surrounding environment of a mobile object, and an image generation unit that controls image processing according to the environment information to generate an image. It is a program to make it work.

  According to the present invention, it is possible to provide an image generation device, a display system, and a program that can obtain an image that changes in accordance with the surrounding environment of a moving object.

It is a block diagram showing an outline of composition of an image generation device concerning an embodiment of the invention. The image taken by the imaging unit is shown. (A) is a front view of the image for obtaining the complexity parameter 1, (B) is a front view of the image for obtaining the complexity parameter 2, (C) is an image frame divided in FIG. 5 (B) It is a sequence diagram of the number. It is a speed- complexity parameter characteristic view. FIG. 10 is a front view of an image for obtaining a complexity parameter 4 showing an image taken by the imaging unit. FIG. 1 is a block diagram showing an example of an electrical configuration of an image generation apparatus according to a first embodiment. It is a schematic diagram which shows the outline | summary of operation | movement of the image generation apparatus which concerns on 1st Embodiment. It is a block diagram showing an example of composition of an information processor concerning a 1st embodiment. It is a block diagram which shows the specific example of a structure of the information processing part which concerns on 1st Embodiment. It is a flow chart which shows a flow of a program which performs "image generation processing" concerning a 1st embodiment. It is a flowchart which shows the flow of the program which performs "the effect process" of FIG. It is a figure for explaining image processing. (A) shows a captured camera image, (B) shows an outline image, (C) shows an outline image in which the density of the outline is lightly changed, (D) shows a background image, and (E) shows a composite image . It is a figure for explaining image processing. (A) shows a captured camera image, (B) shows an outline image, (C) shows an outline image in which the density of the outline is changed to a high density, (D) shows a background image, and (E) shows a composite image . It is a figure which shows an example of the gradation image produced | generated in 2nd Embodiment. It is a schematic diagram which shows the outline | summary of operation | movement of the image generation apparatus which concerns on 2nd Embodiment. It is a block diagram which shows an example of a structure of the information processing part which concerns on 2nd Embodiment. It is a flow chart which shows a flow of a program which performs "image generation processing" concerning a 2nd embodiment. (A)-(D) are figures which show the example 1 of the gradation image corresponding to a camera imaging | video. It is a figure which shows an example of the figure used for production | generation of a moving image in 3rd Embodiment. It is a figure which shows an example of the movement path | route of a figure. It is a figure which shows another example of the movement path | route of a figure. It is a schematic diagram which shows the outline | summary of operation | movement of the image generation apparatus which concerns on 3rd Embodiment. It is a block diagram which shows an example of a structure of the information processing part which concerns on 3rd Embodiment. It is a block diagram showing an example of composition of a display system concerning a 4th embodiment. It is a schematic diagram which shows an example of the display apparatus which concerns on 4th Embodiment. It is a schematic diagram which shows another example of the display apparatus which concerns on 4th Embodiment. (A)-(D) are figures which show the example 2 of the gradation image corresponding to a camera imaging | video.

  Hereinafter, an example of an embodiment of the present invention will be described in detail with reference to the drawings.

<Overview of the Invention>
In this specification, an example in which the image generation device according to the present invention is applied to an image generation device mounted on a vehicle which is an example of a “moving object” will be described. FIG. 1 is a block diagram showing an outline of a configuration of an image generation apparatus according to an embodiment of the present invention.

  The image generation apparatus 10 includes a parameter generation unit 12 that generates a complexity parameter, and an image generation unit 14 that generates an image that changes according to the complexity parameter. Here, the “complexity parameter” is an example of “environment information” that represents the degree of change in the surrounding environment of the vehicle. The greater the degree of change in the surrounding environment of the vehicle, the more complex the surrounding environment of the vehicle. The types of complexity parameters and the method of acquisition will be described later.

  The parameter generation unit 12 acquires situation information according to the situation of the surrounding environment of the vehicle from the environment detection sensor installed in the vehicle. The parameter generation unit 12 generates a complexity parameter according to the acquired situation information. Examples of the environment detection sensor include an image sensor such as a camera, a light sensor, a temperature sensor such as a radiation thermometer, and a position sensor such as a car navigation system including a GPS. In the present embodiment, an example will be described in which the image generation device 10 includes an imaging unit (camera) that images the periphery of a vehicle.

  The image generation unit 14 controls image processing in accordance with the complexity parameter, generates an image that changes in accordance with the complexity parameter, and outputs image data. Image processing is controlled according to the complexity parameter. For example, the image generation unit 14 may generate a new image by controlling an effect applied to the camera image according to the complexity parameter. In addition, for example, when generating a moving image by moving a graphic, the image generation unit 14 may generate a new image by controlling the color or the movement of the graphic in accordance with the complexity parameter.

<Complexity parameter>
Next, types of complexity parameters and acquisition methods will be described.
As described above, the imaging unit (camera) captures an image of the periphery of the vehicle while the vehicle is traveling or stopping. The complexity parameters are generated from images of the surroundings of the taken vehicle. Hereinafter, a plurality of complexity parameters (complexity parameters 1 to 4) will be described. In the present embodiment, an example in which the complexity parameter 4 is used will be described, but other complexity parameters may be used, and a plurality of complexity parameters may be used in combination.

  FIG. 2 is a diagram showing an image captured by the imaging unit. 2 (A) is a front view of an image for obtaining complexity parameter 1, FIG. 2 (B) is a front view of an image for obtaining complexity parameter 2, and FIG. 2 (C) is FIG. 2 (B). It is an arrangement | sequence figure of the number which specifies the divided | segmented image frame.

(Complexity parameter 1)
The complexity parameter 1 is information for monitoring the degree of change of the entire image. When generating the complexity parameter 1, the black and white data generated as basic processing is acquired (see FIG. 2A), and the difference (absolute value) of the image information preceding and succeeding in time is calculated (difference calculation). In accordance with the traveling (movement), the average value when the difference calculation is repeated several tens of times (for example, about 60 times) is calculated. By this complexity parameter 1, it is possible to recognize a change in the entire field of view of the imaging unit in the peripheral environment of the vehicle.

(Complexity parameter 2)
The complexity parameter 2 is information for focusing on the degree of change of the left and right ends of the image. Here, "left and right" means "right" and "left" with respect to the traveling direction of the vehicle. When generating the complexity parameter 2, black and white data (absolute value) generated as basic processing is acquired, and the acquired black and white data image is divided into a plurality of image frames. For example, as shown in FIG. 2C, the image is divided into 9 × 5 and each image frame is numbered from upper left to lower right.

  Among the divided image frames, the image frames corresponding to the left and right ends of the image, No. 1 in FIG. 21 and No. The average value of the absolute value of the difference with No. No. 25 and No. Extract the average value of the absolute value of the difference from 30. And No. 21 and No. The average value of the absolute value of the difference with No. 26, and No. No. 25 and No. The average of the absolute value of the difference with 30 is calculated (average value calculation), and the average value calculation is repeated several dozen times (for example, about 40 times) according to the travel (movement) of the vehicle. Calculate the value. This complexity parameter 2 makes it possible to recognize changes in the side of the vehicle, in particular in the environment around the vehicle.

(Complexity parameter 3)
The complexity parameter 3 is information for monitoring the presence or absence of an obstacle existing in the vicinity of the vehicle. FIG. 3 is a speed-complexity parameter characteristic diagram. The characteristic curve shown in FIG. 3 represents the relationship between the above-mentioned complexity parameter 2 and the speed. Hereinafter, it is referred to as “characteristic curve of complexity parameter 2”. When generating the complexity parameter 3, a reference line A (see FIG. 3) is set for the characteristic curve of the complexity parameter 2.

  Here, it is assumed that the complexity parameter 2 includes a component that varies linearly with the velocity. The reference line A shown in FIG. 3 represents a linearly changing component. The difference between the characteristic curve C of complexity parameter 2 and the reference line A (see shaded area D in FIG. 3) is complexity parameter 3. Similarly, a reference line may be set for the characteristic curve of complexity parameter 1, and the difference between the characteristic curve of complexity parameter 2 and reference line A may be determined to generate complexity parameter 3. The difference between the characteristic curve of complexity parameter 1 and the reference line may be compared with the difference between the characteristic curve of complexity parameter 2 and the reference line, and the larger difference may be used as the complexity parameter 3.

  For example, when the vehicle is traveling on a road with few obstacles, the actual velocity-complexity parameter characteristic curve is the characteristic curve B, and there is no element beyond the reference line A.

  On the other hand, when traveling on a road with many obstacles, the speed-complexity parameter 1 and 2 characteristic curve is the characteristic curve C, and there is an element exceeding the reference line A (see shaded area D in FIG. 3). ). As a situation where this hatched area D occurs, the vehicle is in the site of the parking lot, the road is narrow, the telegraph pole and the vending machine are installed protruding to the road side, and there are many people passing by, etc. It can be mentioned.

(Complexity parameter 4)
The complexity parameter 4 is information for focusing on the degree of change in the central part of the image. Here, the “central portion” is a central portion when the image is divided into three in the height direction of the vehicle. FIG. 4 shows an image captured by the imaging unit, and is a front view of the image for obtaining the complexity parameter 4.

  When generating the complexity parameter 4, black and white data (absolute value) generated as basic processing is acquired, and the acquired black and white data image is divided into a plurality of areas in the height direction. For example, as shown in FIG. 7, it is divided into 1 × 3.

  From the divided regions, the difference (absolute value) of the difference in image information that temporally precedes and follows in the image of the central region is calculated (difference calculation), and the difference calculation is performed according to the travel (movement) of the vehicle. The average value when it is repeated several dozen times (for example, about 60 times) is calculated. This complexity parameter 4 makes it possible to recognize, among other things, changes in the front center of the vehicle in the surroundings of the vehicle.

First Embodiment
An image generation apparatus according to the first embodiment will be described. Each of the electrical configuration, operation, and functional configuration of the image generation apparatus will be described. The electrical configuration is common to each embodiment. The operation and functional configuration of the image generation apparatus are specified for each embodiment.

(Electrical configuration)
First, the electrical configuration of the image generation apparatus will be described.
FIG. 5 is a block diagram showing an example of the electrical configuration of the image generation apparatus according to the embodiment of the present invention. As shown in FIG. 5, the image generation apparatus 10 includes an information processing unit 30 that executes the function of the “image generation unit 14” shown in FIG. 1. The information processing unit 30 is configured as a computer that performs various calculations. The information processing unit 30 includes a central processing unit (CPU) 30A, a read only memory (ROM) 30B, a random access memory (RAM) 30C, a non-volatile memory 30D, and an input / output unit (I / O) 30E. Is equipped.

  The CPU 30A, the ROM 30B, the RAM 30C, the memory 30D, and the I / O 30E are each connected via a bus 30F. For example, the CPU 30A reads a program stored in the ROM 30B and executes the program using the RAM 30C as a work area. Further, to the I / O 30E, a photographing unit 20, an operation unit 32, a display unit 34, a communication unit 36, a storage unit 38, an image output unit 40, and a music output unit 42 are connected.

  The imaging unit 20 is configured by an imaging device such as a camera installed in a vehicle. The imaging unit 20 acquires a visible image. The imaging unit 20 may acquire an infrared image or a distance image in addition to the visible image. The imaging unit 20 captures, for example, a landscape in front of or in the side of the vehicle while the vehicle is traveling. The photographing unit 20 outputs the image data of the photographed image to the information processing unit 30.

  The operation unit 32 includes a switch, a button, and the like, and receives an operation from a passenger. The display unit 34 includes a display and the like, and displays various information to the passenger. As a configuration having both functions of the operation unit 32 and the display unit 34, such as a touch panel, an operation from the passenger may be received, and various information may be displayed to the passenger.

  The communication unit 36 is an interface for communicating with an external device via a wired or wireless communication line. For example, it functions as an interface for communicating with an external device such as a computer connected to a network such as a LAN (Local Area Network) or the Internet. The storage unit 38 is an external storage device such as a hard disk.

  The image output unit 40 outputs an image by displaying or projecting the image generated by the information processing unit 30. The image output unit 40 projects an image at a position viewed by the passenger in the vehicle, for example, in the peripheral vision of the passenger.

  The music output unit 42 outputs music based on the music information. The music output unit 42 outputs, for example, music according to the color of the image output from the image output unit 40. As will be described later, "red" induces a warm sensation, and "blue" induces a cool sensation, etc., and induces a sensation according to the color of the image. In the case of a color image that induces a warm sensation, a warm sound effect such as the sound of a fireplace may be output together. In the case of a color image that induces a cool sensation, cool sound effects such as water and frog sounds may be output together.

  In the present embodiment, a program for executing an "image generation process" described later is stored in the ROM 30B. The storage area of the program is not limited to the ROM 30B. The various programs may be stored in other storage devices such as the memory 30D and the storage unit 38, or may be acquired from an external device via the communication unit 36.

  In addition, various drives may be connected to the information processing unit 30. The various drives are devices that read data from a computer-readable portable recording medium such as a CD-ROM or a USB (Universal Serial Bus) memory, or write data to the recording medium. When various drives are provided, the program may be recorded on a portable recording medium and read and executed by the corresponding drive.

(Outline of operation)
Next, the operation of the image generation apparatus will be described.
FIG. 6 is a schematic view showing an outline of the operation of the image generation apparatus according to the first embodiment. As shown in FIG. 6, in the first embodiment, a complexity parameter is generated from a camera image obtained by photographing the periphery of a vehicle. The camera image is subjected to an effect according to the generated complexity parameter to generate a new image.

  In the first embodiment, a new image is automatically created by the daily act of getting on the vehicle and traveling. Copyright-free images are created endlessly. The created image can also be secondarily used, for example, printed out on a medium such as clothes, bags, and notebooks.

  It is known that, by applying an effect to an image, it is possible to convert the style, such as a painting style or a cartoon style. In this embodiment, by applying an effect according to the complexity parameter to the camera image, a new image representing “dramatic” according to the scene encountered by the passenger is generated.

(Configuration of image generation apparatus)
Next, the functional configuration of the image generation apparatus will be described.
FIG. 7 is a block diagram showing an example of the configuration of the information processing unit according to the first embodiment. As shown in FIG. 7, the information processing unit 30 includes a memory 50, a parameter generation unit 52, and an image processing unit 54. An image around the vehicle captured by the imaging unit 20 is referred to as “first image”, and the image data is referred to as “first image data”.

  The first image data acquired from the imaging unit 20 is stored in the memory 50. The parameter generation unit 52 acquires the first image data from the memory 50, and generates the complexity parameter 4 from the first image data. The image processing unit 54 acquires the first image data from the memory 50.

  The image processing unit 54 performs image processing on the first image to generate a second image. The first image is subjected to an effect according to the complexity parameter to generate a second image. The image data of the second image is referred to as "second image data". The information processing unit 30 outputs the second image data.

Next, a specific example of the configuration of the image generation apparatus will be described.
FIG. 8 is a block diagram showing a specific example of the functional configuration of the information processing unit according to the first embodiment. Similar to the information processing unit 30 illustrated in FIG. 7, the information processing unit 30 includes a memory 50, a parameter generation unit 52, and an image processing unit 54. The image processing unit 54 includes an outline extraction unit 56, an outline color / density change unit 58, an image combining unit 60, and a background extraction unit 62. In the specific example, the procedure of the image processing performed by the image processing unit 54 will be described.

  The contour extraction unit 56 grayscales the first image, which is a camera image, and generates a grayscale image represented in black and white. The outline extraction unit 56 generates an outline image from the gray scale image. For example, edge detection is performed on a grayscale image, and the result of the edge detection is binarized to generate an edge image. The black and white of the edge image is inverted to generate a first contour image.

  The outline color / density change unit 58 sets the color of the outline of the first outline image according to the complexity parameter 4. By changing the color of the outline, a change is given to the second image generated by the information processing unit 30.

  The outline color / density changing unit 58 changes the density of the outline of the first outline image according to the complexity parameter 4 to generate a second outline image. The greater the value of the complexity parameter, the greater the degree of change in the surrounding environment of the vehicle. Therefore, the greater the value of the complexity parameter, the denser the contour. The density of the outline becomes thicker as the outline becomes thicker.

  The background extraction unit 62 separates the background image from the first image which is a camera image. For example, by combining a plurality of images whose imaging times are before and after, a moving object disappears and a background image is separated. Here, "moving object" refers to an object that appears in the field of view of the imaging unit or disappears in the field of view. The image combining unit 60 combines the background image and the second outline image to generate a combined image (second image). For example, for each pixel, the sum of the pixel value of the background image and the pixel value of the second contour image is obtained.

  The functions of the contour extraction unit 56, the contour color / density change unit 58, the image combining unit 60, and the background extraction unit 62 are executed by an effector that applies an effect to the image. As an effector which performs outline extraction, "SKETCHER" etc. are mentioned. Examples of the effector that changes the color and density of the outline include "2TONER". As an effector which extracts a background image, "SLIDER" etc. are mentioned. "SLIDER" erases a moving object by superimposing a plurality of delayed images.

  In the above example, the effect is applied to the entire image to control the degree of application, but a part of the image may be applied to the effect to control the size of the partial area to which the effect is applied.

(program)
Next, a program for executing the “image generation process” will be described.
FIG. 9 is a flow chart showing the flow of a program for executing the “image generation process” according to the first embodiment. FIG. 10 is a flowchart showing the flow of a program for executing the “effect process” of FIG. The “image generation process” and the “effect process” are executed by the CPU 30A of the image generation apparatus 10 when an instruction to start the execution of the process is issued.

  First, in step 100, the first image data stored in the memory is acquired. Next, at step 102, a complexity parameter 4 is generated from the first image data. Next, in step 104, “effect processing” is performed on the first image to generate a second image. Next, at step 106, the second image data corresponding to the second image is output, and the routine is ended.

  In the effect processing, first, at step 200, a first outline image is generated by extracting the outline of the first image. Next, at step 202, at least one of the color and the density of the outline of the first outline image is changed according to the complexity parameter 4. A second contour image is generated from the first contour image. Next, in step 204, the background image is separated from the first image. Next, in step 206, the second contour image and the background image are synthesized, and the routine is ended.

(Image processing)
FIGS. 11A to 11E are diagrams for explaining the procedure of image processing. 11 (A) is a photographed camera image, FIG. 11 (B) is a first outline image, FIG. 11 (C) is a second outline image in which the density of the outline is lightly changed, FIG. 11 (D) Shows a background image, and FIG. 11E shows a composite image.

  The camera image (first image) shown in FIG. 11A is an image taken in a situation where there is not much change in the environment around the vehicle. If the change in the surrounding environment of the vehicle is small, the complexity parameter 4 becomes smaller. As the complexity parameter 4 decreases, as shown in FIG. 11C, in the second contour image, the density of the contour decreases.

  On the other hand, as a result of the background image being separated from the camera image (first image), the building and the like on the side are eliminated in the background image shown in FIG. The second outline image shown in FIG. 11C is synthesized with the background image shown in FIG. 11D to generate a synthesized image (second image) shown in FIG.

  12A to 12E are diagrams for explaining the procedure of the image processing. 12 (A) is a captured camera image, FIG. 12 (B) is a first outline image, FIG. 12 (C) is a second outline image in which the density of the outline is changed to be dark, FIG. 12 (D) Shows a background image, and FIG. 12 (E) shows a composite image.

  The camera image (first image) shown in FIG. 12A is an image captured in a situation where there are many changes in the surrounding environment of the vehicle, such as many trees in the surrounding environment of the vehicle. When there are many changes in the surrounding environment of the vehicle, the complexity is increased, and the complexity parameter 4 is increased. As the complexity parameter 4 increases, as shown in FIG. 12 (C), the density of the outline becomes dense in the second outline image. The color of the outline is, for example, a color representative of the first image, such as the color of a tree (green).

  On the other hand, as a result of the background image being separated from the camera image (first image), the specific tree shape disappears in the background image shown in FIG. 12 (D). The second outline image shown in FIG. 12C is synthesized with the background image shown in FIG. 12D to generate a synthesized image (second image) shown in FIG. In the second image, the green outline is clearly visible.

  By changing the density of the outline according to the complexity parameter 4, the second image changes according to the change of the surrounding environment of the vehicle. Contours appear more and more, giving a "dramatic" feel to the viewer of the second image, when there is little change, and when a curved and complex place is reached. According to the change in the surrounding environment of the vehicle, the image is toned. The dramatically changing image gives the viewer a feeling of coming to a special place and adds value to the everyday activities of driving a vehicle.

Second Embodiment
The second embodiment is the same as the first embodiment in that the complexity parameter 4 is generated from a camera image obtained by photographing the periphery of a vehicle. In the second embodiment, a band-like figure is moved in one direction at a speed according to the generated complexity parameter 4 to generate a gradation image (moving image). At this time, color information is extracted from the first image, and the color of the strip-shaped figure is changed according to the extracted color information.

  Also in the second embodiment, a new image is automatically created by the daily action of getting on the vehicle and traveling. Copyright-free images are created endlessly. The created image can also be used secondarily.

  Also in the second embodiment, an image around the vehicle captured by the imaging unit 20 is referred to as a “first image”, and the image data thereof is referred to as “first image data”. Also, a moving image generated by the information processing unit 30 is referred to as a “gradation image”.

(Gradation image)
Here, the gradation image will be described.
FIG. 13 is a view showing an example of a gradation image generated in the second embodiment. The gradation image is generated by moving a strip-shaped graphic (hereinafter referred to as a “strip”) 82 from one end of the screen 80 to the other end. In the illustrated example, the band 82 appears at the upper end of the screen 80, moves in the arrow direction from the upper end to the lower end of the screen 80, and disappears when the lower end of the screen 80 is reached.

  For example, the coordinates of each corner of the screen 80 are the upper left end (0, 0), the upper right end (448, 0), the lower left end (0, 469), and the lower right end (448, 469). The coordinates of each corner of the band 82 are the upper left end (0, y), the upper right end (448, y), the lower left end (0, y + 20), and the lower right end (448, y + 20). The length of the band 82 is "449 (= 469-20)", and the width of the band 82 is "20".

  As described above, the position coordinates of the band 82 are represented by the value of "y". In the illustrated example, the value of y varies from 0 to 449. For example, the position coordinates of the band 82 may be changed according to the count value of the counter. The count value is incremented by one each time a signal is input to the counter. The interval at which the signal is applied to the counter is changed according to the complexity parameter 4 using a delay device or the like. The moving speed of the band 82 changes according to the complexity parameter 4.

  If the complexity parameter 4 is large (a change in the surrounding environment is large), the interval between signals entering the counter is shortened to speed up the moving speed of the band 82. If the complexity parameter 4 is small (the change in the surrounding environment is small), the interval between signals entering the counter is increased to slow the moving speed of the band 82.

  The color of the band 82 is a color extracted from a camera image (first image) obtained by photographing the periphery of the vehicle. The color of the band 82 may be a representative color that represents all or part of the first image. For example, the first image is divided into three in the height direction of the vehicle, and the representative color is extracted from the central portion. The sky is reflected at the top of the first image and the color of the sky is at the center, and the color of the road and bonnet is at the bottom of the first image. When the color of the central portion of the first image is used, the change in the surrounding environment of the vehicle is more easily reflected in the color.

  In the present embodiment, representative colors are extracted from the central portion. When the pixel value of each pixel in the central portion of the first image is divided into each color of RGB, the ratio of each color of RGB can be obtained. A color obtained by mixing the RGB colors at the obtained ratio is set as a representative color of the central portion of the first image. The obtained representative color is a color representing the environment around the vehicle, which is obtained from the camera image (first image). As the surrounding environment of the vehicle changes, the color of the band 82 also changes accordingly.

  In addition, the color of the band 82 may be a corrected color obtained by correcting the representative color with the “separating color”. There are various colors in the central part of the first image. The greater the variation in pixel value of each pixel, the closer the representative color is to gray. In order to make the color of the band 82 brighter, the representative color may be corrected using the alternative color.

  The color to be added is seven colors of red, yellow, green, light blue, blue and purple. The color having the largest number of color components in the representative color extracted from the central portion of the first image is specified as the “supplementary color” for the representative color. Then, the correction color in which the representative color of the central portion of the first image and the specified “foreground color” are mixed is set as the color of the band 82.

  A gradation image is generated by moving the color band 82 representing the environment around the vehicle from the upper end to the lower end of the screen 80 at a moving speed according to the complexity parameter 4. The surrounding environment of the vehicle is represented by the color and movement of the band 82.

  Gradation images are, for example, projected into the vehicle as so-called ambient lighting (ambient lighting). The gradation image is projected to a portion such as a floor or a side lower side in the vehicle, which is a "peripheral view" when the passenger is facing the front. The floor in the vehicle is a portion of the floor under the driver's seat, the passenger's seat, and the rear seat. The side lower side in a vehicle is a door lower side etc.

  By projecting an image whose color changes in accordance with the surrounding environment of the vehicle into the vehicle, the outside and the inside of the vehicle are connected by colors, and the awareness of the passenger is directed to the surrounding environment of the vehicle. For example, when a yellowish-green gradation image is projected, it becomes easy to look at rice fields such as rice fields. In this way, passengers are sent to notice the surrounding environment.

  I feel flicker when the color of the whole screen changes a lot. Flickering also interferes with driving. If it is a gradation image, the color of part of the screen will gradually change and you will not feel the flicker. The gradation image changes in accordance with the surrounding environment of the vehicle, and thus helps to provide an image moving by the vehicle. In addition, according to the gradation image, there is an effect that the user feels shaking and does not get bored.

  In the above example, the color to be changed changes in accordance with the representative color representing the surrounding environment of the vehicle, but the color to be changed may be fixed to a specific color. For example, in the case of inducing a warm sensation, the opposite color is "red", and in the case of inducing a cool sensation, the opposite color is "blue". A gradation image based on a specific color induces a specific sensation using psychological effects.

  In summer, when the inside of the vehicle is very hot, the driver gets into the vehicle and for a while, a gradation image based on "blue" which induces a cool feeling is projected on the inside of the vehicle to face the passenger. You may also output cool sound effects such as water and frog sounds. On the contrary, in winter, a gradation image based on "red" which induces a warm feeling is projected on the inside of the vehicle to face the passenger. In winter, warm sound effects such as the sound of a fireplace may be output together.

(Outline of operation)
Next, the operation of the image generation apparatus will be described.
FIG. 14 is a schematic view showing an outline of the operation of the image generation apparatus according to the second embodiment. As shown in FIG. 14, in the second embodiment, a complexity parameter is generated from a camera image (first image) obtained by imaging the periphery of a vehicle. The shape of the figure to be moved (the length and width of the band), the initial value of the movement path, and the initial value of the movement speed are set.

  Color information is extracted from the first image, and the extracted color information is set as the color of the band. As described above, the color of the band is set to the representative color or the correction color. A band of the representative color or the correction color is moved at a speed according to the generated complexity parameter 4 to generate a "gradation image" separately from the first image. Here, changing the moving speed of the figure is referred to as "effect".

  In the second embodiment, image processing is controlled so that the strip-shaped figure moves at a moving speed according to the complexity parameter 4. Thereby, a gradation image according to the change of the surrounding environment of the vehicle is generated. The greater the change in the surrounding environment of the vehicle, the faster the moving speed of the figure, and the coarser the gradation. The gradation image abstracted from the camera image expresses the atmosphere of the environment around the vehicle.

  The gradation image is projected into the vehicle, but the rider does not notice flicker when viewed. The gradation image is made up of colors representing the surrounding environment of the vehicle, thereby making the passenger aware of the surrounding environment. The gradation image changes in accordance with the surrounding environment of the vehicle, and thus helps to provide an image moving by the vehicle.

(Configuration of image generation apparatus)
Next, the functional configuration of the image generation apparatus will be described.
FIG. 15 is a block diagram showing an example of the configuration of an information processing unit according to the second embodiment. As shown in FIG. 15, the information processing unit 30 includes a memory 50, a parameter generation unit 52, a motion control unit 64, a color information extraction unit 66, a color setting unit 68, and a moving image generation unit 70. The first image data acquired from the imaging unit 20 is stored in the memory 50. The parameter generation unit 52 acquires the first image data from the memory 50, and generates a complexity parameter from the first image data.

  The moving image generation unit 70 sets the shape (length and width) of the band, the initial value of the movement path, and the initial value of the movement speed. The color information extraction unit 66 extracts a representative color from the central portion of the first image. In addition, if necessary, the representative color is corrected by the "separating color" to obtain a corrected color. The color setting unit 68 sets the color of the band to the representative color or the correction color.

  The motion control unit 64 obtains a moving speed according to the complexity parameter 4. When the motion control unit 64 generates the gradation image by the moving image generation unit 70, the moving image is moved so that the color band set by the color setting unit 68 moves at a moving speed according to the complexity parameter 4 The generation unit 70 is controlled. The moving image generator 70 generates a gradation image. The information processing unit 30 outputs image data representing a gradation image.

(program)
Next, a program for executing the “image generation process” will be described.
FIG. 16 is a flowchart showing the flow of a program for executing the “image generation process” according to the second embodiment. The “image generation process” is executed by the CPU 30A of the image generation apparatus 10 when an instruction to start image generation is issued.

  First, in step 300, the first image data stored in the memory is acquired. Next, in step 302, color information is extracted from the first image data. Next, in step 304, the color of the band is set from the extracted color information. The color of the band is set to the representative color or the correction color.

  Next, in step 306, it is determined whether to start moving the band. When the previous band reaches the bottom of the screen and disappears, the next band appears at the top of the screen and starts moving. The determination in step 306 is repeated until the movement of the band is started. If the movement of the band is to be started, the process proceeds to step 308, and at step 308, the latest first image data is acquired.

  Next, at step 310, the complexity parameter 4 is generated from the latest first image data. Next, at step 312, from the complexity parameter 4, the moving speed of the band is calculated. Next, in step 314, the moving image generation unit is controlled so that the band moves downward at the calculated moving speed.

  Next, in step 316, it is determined whether or not the movement of the band is completed. If the movement of the band is completed, the process proceeds to step 318. If the movement of the band has not ended, the process returns to step 308, and the procedure from step 308 to step 316 is repeated. The latest first image data is sequentially acquired, a complexity parameter 4 is generated, a moving speed according to the complexity parameter 4 is calculated, and the band is moved at the calculated moving speed.

  Next, in step 318, it is determined whether the image generation has ended. If the image generation is completed, the routine ends. If the image generation has not been completed, the process returns to step 300 and the procedure from step 300 to step 318 is repeated. Sequentially, the color of the band is set, a new band of the set color appears, and it moves from the top to the bottom of the screen.

(Specific example 1)
Next, specific example 1 of the gradation image will be described. Each of FIGS. 17A to 17D is a view showing a first specific example of the gradation image corresponding to the camera image. The left side is a camera image, and the right side is a corresponding gradation image. The greater the change in the surrounding environment of the vehicle, the faster the moving speed of the figure, and the coarser the gradation.

  A gradation image shown in FIG. 17A is obtained for the camera image shown in FIG. The change in the surrounding environment of the vehicle in this example is "medium", and the density of gradation is "medium".

  In the camera image shown in FIG. 17 (B), the same surrounding environment continues in the tunnel, so there is less change in the environment around the vehicle than the camera image shown in FIG. 17 (A). The band moves slowly when there is little change in the surrounding environment of the vehicle. Therefore, in the gradation image shown in FIG. 17 (B), the gradation becomes finer than the gradation image shown in FIG. 17 (A).

  In the camera image shown in FIG. 17C, there is a barrier around the vehicle, and the change in the surrounding environment of the vehicle is larger than that of the camera image shown in FIG. When the change in the surrounding environment of the vehicle is large, the band moves fast. For this reason, in the gradation image shown in FIG. 17 (C), the gradation becomes rougher than that of the gradation image shown in FIG. 17 (A).

  In the camera image shown in FIG. 17 (D), the left side does not change, but the right side changes constantly, so the change in the surrounding environment of the vehicle becomes moderate as in the camera image shown in FIG. 17 (A). For this reason, in the gradation image shown in FIG. 17 (D), the density of gradation is medium as in the case of the gradation image shown in FIG. 17 (A).

(Specific example 2)
Next, specific example 2 of the gradation image will be described. In this example, the psychological effect of color is used to make it feel cool when it is hot and warm when it is cold. Each of FIGS. 26A to 26D is a view showing a second specific example of the gradation image corresponding to the camera image. The left side is a camera image, and the right side is a corresponding gradation image.

  In the camera image shown in FIG. 26A, since the vehicle enters the stop state from the moving state, the environment around the vehicle changes significantly. The band moves slowly when the surrounding environment of the vehicle changes significantly. Therefore, in the gradation image shown in FIG. 26A, the gradation becomes rough.

  In the camera images shown in FIGS. 26 (B) and 26 (C), the change in the surrounding environment of the vehicle is monotonous. The less the change in the environment around the vehicle, the faster the band will move. Therefore, in the gradation image shown in FIG. 26 (B) and FIG. 26 (C), the gradation becomes finer.

  In the camera image shown in FIG. 26 (D), the surrounding environment of the vehicle is somewhat complicated. Therefore, in the gradation image shown in FIG. 26D, the gradation becomes rough.

  Further, in the gradation images of FIG. 26A to FIG. 26D, the sense of warm or cool and the image of water or flame are expressed by colors based on a specific color. It has a blue and red gradation, and has an image effect with color.

  The gradation image of a specific color, which induces a specific sensation such as warm or cool, induces a specific sensation, and when it is displayed at the foot of the passenger, the swaying movement of the gradation image causes the water to shake. It feels more effective by overlapping with the image that is swaying with fire and flames.

  Figures 26 (A) and (B) represent the image of water in blue. The floor and sides in the vehicle become blue and feel cool. It is effective to combine it with a cool sound. 26 (C) and (D) represent the image of fire in blue. The floor and sides in the vehicle turn red and feel warm. It is effective to combine it with the sound of fire and hot air.

Third Embodiment
The third embodiment is the same as the first embodiment in that the complexity parameter 4 is generated from a camera image obtained by photographing the periphery of a vehicle. In the third embodiment, a graphic of a predetermined shape is moved at a speed according to the generated complexity parameter 4 to generate a new moving image. The shape of the figure is not limited to "strip". The color of the figure is also set arbitrarily. Here, changing the moving speed of the figure is referred to as "effect".

  Also in the third embodiment, a new moving image is automatically created by the daily action of getting on the vehicle and traveling. Copyright-free images are created endlessly. The created image can also be used secondarily.

  Also in the third embodiment, an image around the vehicle taken by the imaging unit 20 is referred to as a “first image”, and the image data is referred to as “first image data”. In addition, a moving image generated by the information processing unit 30 is referred to as a “graphic movement image”.

(Figure moving image)
In the third embodiment, a figure of a predetermined shape is moved along a predetermined movement path to generate a figure movement image. A closed movement route is set as the predetermined movement route. The figure repeatedly moves along the movement path. The plane including the movement path may be rotated. The movement route can be set arbitrarily. The following describes the shape and movement of the figure. Moreover, a specific figure is called an "object."

  The shape of the object, the initial value of the movement path, and the initial value of the movement speed are set. In the virtual space, a moving image is generated that represents a situation in which an object of a predetermined shape is repeatedly moved along a predetermined movement path at a predetermined movement speed.

  FIG. 18 is a view showing an example of a figure (object) used to generate a moving image in the third embodiment. In the illustrated example, a plurality of annular objects 71 are illustrated, but the number of objects 71 may be one. The shape of the object 71 is not particularly limited, and may be a two-dimensional shape such as a polygonal shape or a strip shape, or may be a three-dimensional shape such as a columnar shape or a conical shape.

  The shape of the object 71 is preferably different depending on the viewing direction. For example, a cylinder, a torus, or a shape that changes between a top view and a side view is desirable. When the object is moved according to the complexity parameter, the angle of the surface having the movement path of the object changes so that it looks different.

  FIG. 19 is a view showing an example of a movement path of a figure. FIG. 20 is a view showing another example of the movement path of the figure. For example, in the example of the movement route of FIG. 19, FIG. 20, the object 71 is a point (1) to a point (2), a point (2) to a point (3), a point (3) to a point (4), a point (4). Move from point 4) to point (1). The object 71 moves at a speed according to the complexity parameter 4.

  Also, the size of the object 71 may be changed according to the complexity parameter 4. For example, the size of the object 71 is changed so that the larger the value of the complexity parameter 4, the larger the object.

  Further, as in the second embodiment, the color of the object 71 may be a representative color that represents all or part of the first image, or a corrected color obtained by correcting the representative color with a substitute color. The color to be changed may be a specific color that induces a specific sense, such as “red” for warm and “blue” for cool.

(Image generation device)
Next, an image generation apparatus according to the third embodiment will be described.
FIG. 21 is a schematic view showing an outline of the operation of the image generation apparatus according to the third embodiment. As shown in FIG. 21, in the third embodiment, a complexity parameter 4 is generated from a camera image obtained by photographing the periphery of a vehicle. The shape of the figure, the initial value of the movement path, and the initial value of the movement speed are set. A figure of a predetermined shape is moved at a speed according to the generated complexity parameter 4 to generate a "figure movement image" separately from the camera image. Here, changing the moving speed of the figure is referred to as "effect".

  FIG. 22 is a block diagram showing an example of the configuration of an information processing unit according to the third embodiment. As shown in FIG. 22, the information processing unit 30 includes a memory 50, a parameter generation unit 52, a motion control unit 64, and a moving image generation unit 70. The first image data acquired from the imaging unit 20 is stored in the memory 50. The parameter generation unit 52 acquires the first image data from the memory 50, and generates the complexity parameter 4 from the first image data.

  The moving image generation unit 70 sets the shape of the object, the initial value of the movement route, and the initial value of the movement speed. The motion control unit 64 obtains a moving speed according to the complexity parameter 4. The motion control unit 64 controls the moving image generation unit 70 so that the object moves at the obtained moving speed when the moving image generation unit 70 generates a moving image. The moving image generation unit 70 generates a figure movement image. The information processing unit 30 outputs image data representing a figure movement image.

  In the third embodiment, image processing is controlled so that the object moves at a moving speed according to the complexity parameter 4. Thereby, the figure movement image according to the change of the surrounding environment of a vehicle is produced | generated. The atmosphere of the surrounding environment of the vehicle is expressed by the figure movement image abstracted from the camera image.

  Similar to the gradation image, the figure movement image is projected into the vehicle, but the rider does not notice flicker when viewed. In addition, since the graphic movement image changes in accordance with the surrounding environment of the vehicle, it helps to give an image of the movement by the vehicle.

Fourth Embodiment
The fourth embodiment is an embodiment according to a display system.
FIG. 23 is a block diagram showing an example of the configuration of a display system according to the fourth embodiment. As shown in FIG. 23, the display system 100 includes an image generation device 10 and a display device 90. The display device 90 is disposed, for example, at a place away from the image generation device 10 such as a home. The display device 90 is disposed, for example, at the home of the passenger A or the like. The image generation device 10 includes a photographing unit 20, an information processing unit 30, and a communication unit 36. The display device 90 includes a communication unit 92, a display control unit 94, and a display unit 96.

  The configuration of the image generation device 10 may be the same as that of the image generation device according to the second embodiment or the third embodiment. The information processing unit 30 acquires the first image data from the imaging unit 20, and generates the complexity parameter 4 from the first image data. The information processing unit 30 changes according to the complexity parameter 4 and generates an “environmental image” representing the surrounding environment of the vehicle. The communication unit 36 transmits the image data of the environmental image to the display device 90.

  When the configuration of the image generation device 10 is the same as that of the image generation device according to the second embodiment, the gradation image generated by moving the band is the “environmental image”. When the configuration of the image generation apparatus 10 is the same as that of the image generation apparatus according to the third embodiment, a graphic movement image generated by moving a graphic having a predetermined shape is an “environmental image”.

  The display unit 96 of the display device 90 is controlled by the display control unit 94 to display an image. The communication unit 36 is an interface for communicating with an external device via a wired or wireless communication line. The communication unit 36 receives image data of an environmental image from the image generation device 10. The display control unit 94 controls the display unit 96 so that the environment image is displayed on the display unit 96. On the display unit 96, an environment image representing the environment around the vehicle is displayed.

  The passenger A of the vehicle on which the image generation device 10 is mounted sees the same video as the video (first image) of the environment around the vehicle captured by the imaging unit 20. The other person B who is at a distant place sees the environment image displayed on the display unit 96 of the display device 90.

  The environment image is displayed on the display unit 96 of the display device 90, so that the state of the environment around the vehicle on which the passenger A is riding is transmitted to the other person B who is at a distant place. It is transmitted to the other person B by the color and movement in the environment image. The other person B can infer the state of the vehicle or the passenger A.

  It is easier to convey the situation of the surrounding environment of the vehicle to the other person B when displaying the abstracted environmental image than displaying the first image as it is. Abstraction reduces the burden of looking at it consciously. In addition, abstraction makes it possible to protect the privacy of the passenger A and the third party reflected in the camera image.

  FIG. 24 is a schematic view showing an example of a display device according to the fourth embodiment. FIG. 25 is a schematic view showing another example of the display device according to the fourth embodiment. In FIG. 24 and FIG. 25, the display device 90 is a lighting object. The lighting object changes the color and brightness of lighting based on the environmental image to display information based on the environmental image.

  It is an image in which the color and movement of the surrounding environment of the vehicle are wirelessly transmitted to the lighting object seen by the other person B at home. As in a car, the color does not change rapidly, so it does not attract attention, but you can feel its presence and movement somehow. The abstracted display leads to the effect of imagining the other party.

<Modification>
The configurations of the image generation apparatus, the display system, and the program described in the above embodiments are merely examples, and it goes without saying that the configurations may be changed without departing from the scope of the present invention.

  Although the above-mentioned embodiment explained the example which projects a gradation picture and a graphics movement picture on the inside of vehicles, it may display using a light emitting element or an electric bulletin board.

  In the above embodiment, the example using the complexity parameter 4 has been described, but instead of the complexity parameter 4, any of the complexity parameters 1 to 3 may be used.

  In the above embodiment, an example has been described in which the sound effects corresponding to the color of the image are output together. However, the sound effects may be output according to the effect of the effect, and music information is generated from the complexity parameter May be For example, MIDI note signals are generated according to the complexity parameter using "software for MIDI signal conversion", and music information is generated from the note signals using "software for generating music information from MIDI signals" Generate The music output unit 42 shown in FIG. 5 plays music corresponding to the generated music information.

  Here, MIDI (Musical Instrument Digital Interface) is an international standard for communicating music information by digital signals. The MIDI signal is information of the MIDI standard, and is information of a standard MIDI file format (SMF). In "MIDI signal conversion software", the basic frequency of sound is "note number", sound pressure level is "velocity level", timbre is "number representing instrument name", and time change is "note on / note off" Are each converted to data representing "."

DESCRIPTION OF REFERENCE NUMERALS 10 image generation device 12 parameter generation unit 14 image generation unit 20 imaging unit 30 information processing unit 32 operation unit 34 display unit 36 communication unit 38 storage unit 40 image output unit 42 music output unit 50 memory 52 parameter generation unit 54 image processing unit 56 Outline extraction unit 58 Density changing unit 60 Image combining unit 62 Background extraction unit 64 Motion control unit 66 Color information extraction unit 68 Color setting unit 70 Moving image generation unit 71 Object 80 Screen 82 Band 90 Display 92 Communication unit 94 Display control unit 96 Display unit 100 Display system

Claims (11)

An information generation unit that generates environment information that indicates the degree of change in the surrounding environment of the mobile object;
An image generation unit configured to generate an image by controlling image processing according to the environment information;
An image generation device equipped with The image generation unit performs image processing on a first image obtained by photographing the periphery of a moving object to generate a second image.
The image generation apparatus according to claim 1. In the image processing,
Generating a first contour image in which the contours of the first image are extracted;
Generating a background image by extracting a background portion of the first image;
Generating a second outline image in which the density of the outline of the first outline image is changed according to the environment information;
Combining the second outline image and the background image to generate a second image;
The image generation apparatus according to claim 2. As the degree of change of the surrounding environment of the mobile object represented by the environment information is larger, the density of the outline is increased.
The image generation apparatus according to claim 3. The image generation unit generates a moving image in which a graphic of a predetermined shape moves at a speed according to the environment information.
The image generation apparatus according to claim 1. The moving image is a moving image in which a strip-shaped figure moves in a predetermined direction.
The image generation apparatus according to claim 5. The moving speed of the figure is made faster as the degree of change of the surrounding environment of the moving body represented by the environment information is larger.
An image generation apparatus according to claim 5 or 6. A color extraction unit for extracting color information of a representative color representing all or a part of each pixel value of the first image obtained by photographing the periphery of the mobile object;
The image generation unit sets the color of the figure as the representative color.
The image generation apparatus according to any one of claims 5 to 7. The representative color is a representative value of each pixel value of the central portion when the first image is divided into three in the height direction of the moving body.
The image generation apparatus according to claim 8. An image generation apparatus according to any one of claims 1 to 9;
A display device for displaying information based on the image transmitted from the image generation device;
Display system equipped with Computer,
An information generation unit that generates environment information that indicates the degree of change in the surrounding environment of the mobile object;
An image generation unit that generates an image by controlling image processing according to the environment information;
Program to function as.