JPH09289613A - Image conversion method and its device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a plane image obtained by directing a virtual sight line vector of the user in an optional direction from a source image obtained from one exposure through the use of one camera. SOLUTION: A read section 6 reads a still image from a camera 1 or a moving image from a video camera 4, a field angle calculation section 7 and a visual field range calculation section 8 calculate a field angle and the range of visual field segmented from the image based on a direction and zooming designated by the user, and a coordinate conversion section 9 relates all points in the range of visual field to correspond to the read image. The attribute of picture elements at each point is mapped to a point corresponding to the range of visual field as corresponding transformation, a visual field range display section 10 obtains the image signal subject to coordinate transformation as a display signal for a CRT 11 and the plane image in an optional direction and at the visual field range is obtained through one exposure by one camera or a frame image.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image conversion method and an apparatus for displaying a still image or a moving image in a visual field range specified by a user with respect to input image information in an image information processing apparatus. Is.

[0002]

2. Description of the Related Art Conventionally, in order to obtain a plane image obtained by directing a virtual line-of-sight vector in an arbitrary direction within a certain range around a virtual viewpoint of a user set at a certain place, There are two ways.

(A) A virtual perspective of a user is set so that a camera equipped with a lens having a wide angle of view such that the screen distortion peculiar to a wide-angle lens can be ignored covers the entire range in which the virtual visual field of the user moves. A method of arranging multiple cameras in the place where is placed and taking a picture, and cutting out the image in the direction set by the user from the taken images.

(B) One camera is photographed a plurality of times in a plurality of directions so as to cover the entire range in which the user's virtual visual field moves, and from this photographed image, the image in the direction set by the user How to cut out.

[0005]

A camera equipped with a lens having a wide angle of view such that the screen distortion peculiar to the conventional (A) wide-angle lens is negligible covers the entire range in which a user's virtual visual field moves. In such a method of arranging a plurality of cameras at a position where the user's virtual viewpoint is placed, there is a problem in terms of cost that a plurality of cameras are required for the camera.

[0006] (B) In the method of shooting one camera in a plurality of directions so as to cover the entire range in which the user's virtual visual field moves, a plurality of times is taken to obtain one image. Exposure is required. Moreover, there is a problem in that basically a moving image cannot be taken because it requires multiple exposures.

Further, in both (A) and (B), since it is necessary to join a plurality of images obtained after photography so that the joints cannot be seen, it is necessary to carefully set the camera before photography. There is a problem that must be done. In addition, there is a problem that a live image cannot be basically used as a source because a process of connecting the plurality of images is necessary after shooting.

An object of the present invention is to provide an image conversion method for obtaining a plane image obtained by directing a virtual line-of-sight vector of a user in an arbitrary direction from a source image obtained by one exposure using one camera, and To provide a device.

Another object of the present invention is to obtain a plane image obtained by using a moving image using one camera, in particular, a live image as a source image and arranging a virtual line-of-sight vector of a user in an arbitrary direction. An object is to provide an image conversion method and an apparatus.

[0010]

[Means for Solving the Problems]

(First invention) In order to achieve the above object, the image conversion method of the present invention prepares an image photographed by using a lens having an angle of view of a certain angle and capable of projecting the entire field of view, and Determine the starting point and direction of the line-of-sight vector that reaches toward the area, determine an arbitrary field of view centered on the line from which the line-of-sight vector reaches the above image, and from the starting point of this line-of-sight vector to the above field of view The present invention is characterized in that a plane image obtained in the visual field range when the above image is viewed through the range is generated from the above image.

According to the above-described image conversion method, this image is captured based on a still image or a moving image captured by a camera equipped with a lens having an angle of view such as an all-round fisheye lens and capable of capturing the entire field of view. Set or calculate the starting point (viewpoint) and direction that are arbitrarily set in the specified area, and set or calculate the viewing range that matches the direction of the viewing vector from the starting point, and calculate the coordinates of each point in this viewing range. By generating a plane image by coordinate conversion for converting each point coordinate on the image, the virtual line-of-sight vector of the user can be obtained by using a still image by one exposure by one camera or a moving image by a video camera as a source. A plane image oriented in an arbitrary direction can be obtained.

Further, the starting point arbitrarily set in the area where the image is photographed includes placing the virtual viewpoint of the user at a position different from the position where the camera is placed at the time of photographing. In this case, The range of movement of the virtual visual field can be made wider than when the virtual viewpoint of the user is placed at the same position as the place where the user is placed.

(Second invention) In the first invention, the image conversion method of the present invention is characterized in that the starting point of the line-of-sight vector is regarded as having been photographed when viewed from the image. It is characterized by what you are doing.

The present invention is a case in which the starting point of the line-of-sight vector is made to coincide with the point considered to have taken the image.

(Third Invention) In the above first or second invention, the image conversion method of the present invention is such that the above image has an angle of view of 180 degrees in all directions and can capture the entire field of view. It is characterized in that it is a still image or a moving image taken by using a periscopic lens.

The present invention is a case where the input image is a still image or a moving image photographed by using the all-round fisheye lens.

(Fourth Invention) In order to achieve the above object, the image conversion apparatus of the present invention is based on an image reading unit for reading a photographed still image or a moving image, and a direction and zooming specified by a user. The calculation means for calculating the angle of view and the visual field range of the image cut out from the image, and which coordinates in the visual field range all the points correspond to in the image read by the image reading unit are obtained, and The present invention is characterized by including a coordinate conversion unit that moves the attribute of a pixel to a corresponding point in the visual field range, and a visual field range display unit that obtains an image signal that is the coordinate conversion result of this coordinate conversion unit as an image display signal.

The present invention is a configuration of an image conversion apparatus used in the image conversion methods of the above-mentioned first to third inventions, wherein the calculation means allows the user to determine the direction of the line-of-sight vector and the focal length of the virtual camera. The virtual angle of view is obtained by specifying, and the virtual field of view of the user to be finally displayed is obtained from this virtual angle of view and the direction of the line-of-sight vector. The coordinate conversion is performed to transfer the pixel attribute in association with the read image.

[0019]

FIG. 1 is an apparatus configuration diagram showing an embodiment of the present invention. In the figure, as an example of inputting a source image, a still image is photographed by a still image camera 1, and then a still image 2 printed on photographic paper is converted into an image signal by an image scanner 3 or a moving image. The image is of a system in which the video signal captured by the video camera 4 is directly read. However, the image input method is of course not limited to these methods.

Both the still image camera 1 and the video camera 4 are equipped with an all-round fisheye lens, and in this embodiment, the axis of the all-round fisheye lens is set in the zenith direction. Furthermore, in the present embodiment, the case where a still image is input will be described, but if the process described here can be performed at high speed, it can be expanded to the case where a moving image is input as it is.

In FIG. 1, a portion indicated by a broken line block 5 is an image conversion device. The image reading unit 1 reads images taken by the cameras 1 and 4 which are external devices. In parallel with this image reading, the user specifies the direction (virtual line-of-sight vector) he or she wants to see and zooming (focal length f (mm) of the virtual camera).

In zooming, the user may directly specify the value of the apex angle (hereinafter referred to as a virtual angle of view) of the visual field cone that the user virtually has, but here, the user is a virtual one. The value of the focal length of the camera is designated, and the angle-of-view calculation unit 7 uses this value to obtain the virtual angle of view.

Using the virtual line-of-sight vector designated by the user and the virtual angle of view calculated by the angle-of-view calculation unit 7, the visual-field range calculation unit 8 finally displays the virtual visual field of the user on the CRT. (A rectangle inscribed in the field cone is hereinafter referred to simply as a virtual field) is determined.

The coordinate conversion unit 9 finds which coordinates in the source image read by the image reading unit 6 correspond to all the points in the virtual visual field obtained by the visual field range calculation unit 8. Coordinate conversion is performed and RGB of that point is converted.
The attribute of the pixel represented by CMYK or the like (hereinafter referred to as color information) is moved to the corresponding point in the virtual visual field. That is, a plane image is generated.

The coordinate conversion by the coordinate conversion unit 9 is performed a necessary and sufficient number of times by calculating the virtual visual field range by the field angle calculation unit 7 and the visual field range calculation unit 8 in advance. Generate a planar image.

The field-of-view range display unit 10 outputs an image signal as a result of the color information of all the points in the virtual field of view being transferred from the source image by the coordinate conversion unit 9 as a C image display device.
The image signal of RT11 is used, and the image of the visual field range is displayed on the CRT11.

FIG. 2 shows definitions of terms relating to the viewpoint on the user side. Point A indicates the user's viewpoint, arrow B indicates the direction in which the user wants to look, that is, the line-of-sight vector,
The angle C indicates the angle of view. If the direction of the line-of-sight vector of the arrow B and the angle of view of the angle C are determined, the entire visual field range of the user is determined,
It becomes the field cone D.

Since the portion displayed on the CRT is usually a rectangle, an inscribed rectangle is cut out from this field cone D. That is the visual field range indicated by the rectangle E. A vector from the viewpoint of the point A to an arbitrary point within the field of view of the rectangle E is the line of sight indicated by the arrow F.

Therefore, in the image conversion device shown by the broken line block 5, the image reading section 6 reads an image photographed by using a lens having an angle of view such as an all-round fisheye lens and capable of projecting the entire field of view, and this image is read. The starting point (viewpoint) and the angle of view of the line-of-sight vector arriving toward the inside of the area are set, and from this setting, the angle-of-view calculation unit 7 and the field-of-view range calculation unit 8 arbitrarily set the line of sight as the center By calculating the field of view range of the camera and generating a plane image obtained when the image is viewed through the field of view from the starting point of the line-of-sight vector by the coordinate conversion unit 9. It is possible to obtain a plane image in which the virtual line-of-sight vector of the user is directed in an arbitrary direction for each frame image of a still image or a moving image by one exposure.

Here, the setting of the starting point of the virtual line-of-sight vector is not limited to the method of making it coincide with the point at which the image is photographed, but it is also possible to obtain a planar image in which the virtual visual field range is expanded by moving the starting point. Regarding these image conversions, FIG. 3 and FIG.
This will be described below with reference to FIG.

An image photographed with the all-round fish-eye lens facing the zenith is placed on the y-axis at the origin of the visual field hemisphere G as shown in FIG. 3, and all the objects in the visual field mapped to the visual field hemisphere G are mapped. The image is projected on the film surface H. Here, the y-axis is directed in the zenith direction, and the x-axis is directed in the direction perpendicular to the paper surface.

Therefore, the image conversion is the simplest.
Put the virtual viewpoint A of the user at the origin of the same field hemisphere G,
The coordinates of the intersection J of the visual field vector I and the visual field hemisphere G from the origin toward an arbitrary point inside the virtual visual field are obtained, and the intersection J is obtained.
It is possible to generate a two-dimensional image by converting the coordinates of the coordinates on the film surface H and using the resulting color information of the coordinates on the film surface H as the color information of the corresponding points inside the virtual visual field.

As a result, in the configuration of FIG. 1, the image read by the reading unit 6 is within a range photographed on the basis of an image obtained by one exposure using one camera equipped with an all-round fisheye lens. A virtual image of the user's virtual line-of-sight vector can be obtained in a planar image, and a camera setting with a high degree of freedom can be set as the shooting position and direction of the camera.

Further, even when a moving image taken by using a video camera or the like, particularly a live image is used as a source, the image can be taken by one camera whose setting is fixed,
It is possible to obtain a plane image in which the virtual visual field vector is oriented in an arbitrary direction without connecting the frame images.

Next, in the method of matching the starting point of the virtual line-of-sight vector with the point at which the image is captured, the virtual visual field cannot be moved within the horizontal plane on which the viewpoint is placed, that is, below the xz plane. This means that the line-of-sight vector must always be directed upward, although there is a degree of difference depending on the size of the virtual angle of view, and the virtual visual field range of the user is limited.

A method of expanding this virtual visual field range will be described. FIG. 4 shows a visual field G _A that extends the range in which the virtual holiday field can be moved below the xz plane so that the virtual line-of-sight vector can still be directed in the horizontal direction even at the maximum virtual angle of view that is set. There is. The field of view G _B is a hemisphere onto which the film surface H taken is projected. Field G _B is moved below the center of the field of view G _A a virtual field center by an amount field can cover all the range that can be moved G _A (origin).

For the coordinate conversion, an intersection J _A between the virtual visual field vector I _A and the visual field G _A that goes from the virtual visual point placed at the origin to an arbitrary point inside the virtual visual field, and the intersection J _A and the visual field. G
Obtain the intersection J _B of the straight line L and the field G _B connecting the origin of _B,
The color information of the coordinates on the film surface H corresponding to the intersection J _B is
Color information of an arbitrary point inside the desired virtual visual field is used.

As described above, by performing the coordinate transformation by moving the center of the visual field, it becomes possible to include the virtual visual field below the yz plane, and the virtual visual point of the user is set at the position where the camera is placed. The range of movement of the virtual visual field can be made wider than in the case where it is placed.

[0039]

As described above, the present invention is based on a still image or a moving image taken by a camera equipped with a lens having a certain angle of view such as an all-round fisheye lens and capable of capturing the entire field of view. Set the start point (viewpoint) and direction to be set arbitrarily within the area of, and determine the view range that matches the direction of the view vector from the start point, and convert each point coordinate within this view range to each point coordinate on the image Since the plane image is generated by the coordinate conversion, the plane image in which the virtual line-of-sight vector of the user is directed in an arbitrary direction from the still image by one exposure of one camera or the moving image by the video camera as the source. There is an effect that can be obtained.

As a result, the setting of the camera can be freely performed, and even if the visual field range is changed, the joining process of the images is not required, and the planar image without the joining portion can be obtained.

If the user's virtual viewpoint is placed at a position different from the position at which the camera was placed at the time of shooting, the starting point arbitrarily set in the image area is the same position as the position at which the camera is placed. Therefore, the moving range of the virtual visual field can be wider than that when the virtual viewpoint of the user is placed.

[Brief description of drawings]

FIG. 1 is a configuration diagram of an image conversion apparatus showing an embodiment of the present invention.

FIG. 2 is a diagram explaining the meaning of words in the image conversion method of the present invention.

FIG. 3 is a diagram for explaining the simplest coordinate conversion in a coordinate conversion method and device according to the present invention.

FIG. 4 is a diagram for explaining coordinate conversion when the starting point in the coordinate conversion method and apparatus according to the present invention is lowered from the origin.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 1 ... Camera 3 ... Image scanner 4 ... Video camera 5 ... Image conversion device 6 ... Image reading unit 7 ... View angle calculation unit 8 ... View range calculation unit 9 ... Coordinate conversion unit 10 ... View range display unit 11 ... CRT

Claims (4)

[Claims] 1. An image taken using a lens having a certain angle of view and capable of projecting the entire field of view is prepared, and the starting point and the direction of the line-of-sight vector reaching toward the area of this image are determined. Determine an arbitrary visual field range around the line where the line-of-sight vector from the starting point reaches the above image, and when the above-mentioned image is seen from the starting point of this line-of-sight vector through the above visual field range, An image conversion method characterized in that the obtained plane image is generated from the above image. 2. The image conversion method according to claim 1, wherein the starting point of the line-of-sight vector is the same as the point at which the image is considered to be photographed when viewed from the image. 3. The image is a still image or a moving image captured by using an all-round fisheye lens having an angle of view of 180 degrees in all directions and capable of capturing the entire field of view. Alternatively, the image conversion method described in 2. 4. An image reading unit for reading a photographed still image or a moving image, calculation means for calculating an angle of view and a field of view range of an image cut out from the image based on a direction and zooming designated by a user, and A coordinate conversion unit that obtains which coordinates in the image read by the image reading unit all the points in the visual field range correspond to, and transfers the pixel attributes of each point to the corresponding points in the visual field range. An image conversion apparatus, comprising: a field-of-view range display unit that obtains an image signal as a result of coordinate conversion by the unit as an image display signal.