KR100703713B1 - 3D mobile devices capable offer 3D image acquisition and display - Google Patents

Abstract

1. TECHNICAL FIELD OF THE INVENTION

The present invention relates to a three-dimensional mobile device capable of three-dimensional image acquisition and display.

2. The technical problem to be solved by the invention

An object of the present invention is to provide a three-dimensional mobile device that allows a user to obtain a three-dimensional image anytime, anywhere, and display the three-dimensional image in three dimensions by using a mobile device.

3. Summary of Solution to Invention

The present invention provides a three-dimensional mobile device capable of acquiring and displaying a three-dimensional image, comprising: wireless communication means for converting and outputting an audio signal received from a mobile communication network into an audible sound, and converting and transmitting an audio signal received from a microphone; A first photographing means and a second photographing means, which are provided in a mobile terminal to photograph and acquire left and right images of a subject, respectively, and which acquire images by using a CMOS image sensor (CIS); Photographing control means for controlling the vergence points of the first photographing means and the second photographing means; Image acquisition means for processing and outputting images received from the first and second photographing means; Image information storage means for storing image information processed by the image acquisition means; And display means for multiplexing and displaying the left / right image received from the image acquisition means in three dimensions.

4. Important uses of the invention

The present invention is used in a three-dimensional mobile system.

Mobile, Stereoscopic, Camera, 3D Display, Binocular

Description

3D mobile devices capable offer 3D image acquisition and display}             

1 is a configuration diagram of an embodiment of a three-dimensional mobile device according to the present invention;

2 is a detailed configuration diagram of an embodiment of an image acquisition unit according to the present invention;

3 is a view for explaining a movement and rotation function for setting the vergence point of the first camera 110 and the second camera,

4 is a diagram for explaining a left / right image conversion process in an image codec according to an embodiment of the present invention;

5 is a view showing a multiplexed image according to an embodiment of the present invention;

6 is a configuration diagram of an embodiment of a display unit according to the present invention.

* Explanation of symbols for the main parts of the drawings

ANT: Antenna MIC: Microphone

SPK: Speaker 10: Wireless

20: voice processing unit 30: key input unit

40: main controller 50: main memory

110: first camera 120: second camera

130: camera control unit 140: image acquisition unit

150: display unit 160: video memory

170: portable memory

The present invention relates to a three-dimensional mobile terminal capable of obtaining and displaying a three-dimensional image, and more particularly, to a three-dimensional mobile device that allows a user to obtain and observe a three-dimensional image in the mobile terminal.

In general, the biggest factor that humans feel in three dimensions is the effect of spatial differences on the left and right retinas caused by the left and right eyes looking at an object in one direction. By using this effect, different images, i.e., stereoscopic images (three-dimensional images), may be displayed in both left and right eyes so that the user can observe three-dimensional images in three dimensions.

The conventional stereoscopic image display method is divided into a method of observing a 3D image wearing glasses and a method of observing a 3D image without wearing glasses.

A method of observing a 3D image by wearing glasses is a method using polarized glasses. In a large space such as a theater, an observer may wear a pair of polarized glasses and observe a stereoscopic image (3D image) of two eyes.

On the other hand, the most typical method of viewing without wearing glasses is a method of viewing a stereoscopic image by attaching a lenticular plate representing only vertical parallax to an image display device, and furthermore, a multi-view or ultra multi-view IP expressing vertical and horizontal parallax. There is a method using a multi-view image display mask such as an (Integral Photography) plate, a cross lenticular plate or a square lens plate. This method of viewing without wearing glasses is a method of observing a 3D image at a position where a few observers are determined, and a method of observing a 3D image by attaching a parallax barrier or a lenticular to an image display plate is mainly used. Used.

A mobile device that has a communication function and an information processing function and secures mobility, such as a conventional mobile phone or a personal digital assistant (PDA), acquires a 2D image by attaching a single camera, and uses a 2D image display device such as an LCD. Display a two-dimensional image.

Therefore, the conventional mobile device cannot obtain a 3D image due to a single camera, and a 2D display device cannot display a 3D image to a user in a three-dimensional sense.

Accordingly, the present invention has been proposed to solve the above problems, a three-dimensional mobile device that allows a user to obtain a three-dimensional image anytime, anywhere using a mobile device, and display the three-dimensional image in three dimensions The purpose is to provide.

Other objects and advantages of the present invention can be understood by the following description, and will be more clearly understood by the embodiments of the present invention. Also, it will be readily appreciated that the objects and advantages of the present invention may be realized by the means and combinations thereof indicated in the claims.

The present invention for achieving the above object is a three-dimensional mobile terminal capable of obtaining and displaying a three-dimensional image, converts and outputs an audio signal received from the mobile communication network to an audible sound, and converts and transmits the audio signal received from the microphone Wireless communication means for; A first photographing means and a second photographing means, which are provided in a mobile terminal to photograph and acquire left and right images of a subject, respectively, and which acquire images by using a CMOS image sensor (CIS); Photographing control means for controlling the vergence points of the first photographing means and the second photographing means; Image acquisition means for processing and outputting images received from the first and second photographing means; Image information storage means for storing image information processed by the image acquisition means; And display means for multiplexing and displaying the left / right image received from the image acquisition means in three dimensions.

The above objects, features and advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings, whereby those skilled in the art may easily implement the technical idea of the present invention. There will be. In addition, in describing the present invention, when it is determined that the detailed description of the known technology related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description thereof will be omitted. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a configuration diagram of an embodiment of a three-dimensional mobile device according to the present invention.

As shown in FIG. 1, the three-dimensional mobile device according to the present invention is an antenna (ANT), a wireless unit 10, a voice processing unit 20, and a microphone (MIC) as components for performing general communication and information processing related functions. ), A speaker (SPK), a key input unit 30, the main control unit 40 and the main memory 50, the first camera 110, the second as a component that performs functions related to 3D image processing The camera 120 may include a camera controller 130, an image acquisition unit 140, a display unit 150, and an image memory 160, and may further include a portable memory 170.

In the configuration for performing general communication and information processing related functions of the three-dimensional mobile device according to the present invention, the main controller 40 performs the overall control operation of the three-dimensional mobile device.

The wireless unit 10 controls the transmission and reception of voice and data from the mobile communication network under the control of the main control unit 40, and the voice processing unit 20 receives the voice received from the wireless unit 10 under the control of the main control unit 40. The data is converted into an audible sound through the speaker SPK, and is output. The voice signal received from the microphone MIC is converted into data and output to the wireless unit 10.

The key input unit 30 includes a plurality of numeric keys and function keys, and outputs key input data corresponding to a key pressed by the user to the main controller 40, and the main memory 50 processes the 3D mobile device. Store various data.

On the other hand, in the configuration for performing a function related to the three-dimensional image processing of the three-dimensional mobile device, the two cameras (110, 120) each includes a CMOS image sensor (CIS) to obtain an image, the camera controller 130 The two cameras 110 and 120 are controlled.

The image acquisition unit 140 processes the image obtained from the CIS, and the display unit 150 displays the image received from the image acquisition unit 140 in 2D or 3D using a TN panel attached to the LCD.

The image memory 160 stores image-related data processed by the image acquisition unit 140, and the portable memory 170 can be attached to or detached from a 3D mobile device, thereby storing and carrying image data according to a user's needs. Make sure

2 is a detailed block diagram of an embodiment of an image acquisition unit according to the present invention.

As shown in FIG. 2, the image acquisition unit 140 according to the present invention includes an image signal processor or image sensor processor (ISP) 141, a stereo image processing unit 142, and an image processor to process an image received from a CIS. The video codec 143 is included.

The ISP 141 controls the shutter speed, focus, zoom, and aperture of the camera when acquiring images using CIS, and synchronizes two CISs when acquiring three-dimensional images using two CIS. Make the shooting environment of CIS the same.

The stereo image processor 142 receives a stereo image through two CISs, a camera controller 130, and an ISP 141, and processes and corrects problems such as color balance and mismatch of pixel units between left and right images. To be output directly to the display unit 150 or to the image codec 143.

The image codec 143 receives the 3D image corrected by the stereo image processing unit 142, encodes the image, multiplexes the image by the 3D multiplexing method, and stores the image in the image memory 160.

FIG. 3 is a diagram for describing a movement and rotation function for setting a vergence point of the first camera 110 and the second camera.

In the 3D mobile device according to the present invention, CIS of two cameras 110 and 120 is used to acquire a stereo image.

In this case, as shown in FIG. 3, the two cameras 110 and 120 may move horizontally in order to adjust the distance difference between the two CISs according to the distance of the gaze point, and rotate to adjust the gaze point. Also includes.

The process of processing the 3D image by the components of the 3D mobile device according to the present invention will be described as follows.

When acquiring a 2D image, an image is acquired using only one CIS. When acquiring a 3D image composed of left and right images, two CISs are used.

The ISP 141 controls the shutter speed, focus, zoom, and aperture of the camera when acquiring the 2D image using the CIS. In addition, when acquiring a 3D image using two CIS, synchronization between the two CIS is made and the photographing environment of the two CIS is the same.

The camera controller 130 controls two CISs to be horizontally moved and rotated according to an optimal camera arrangement method in order to match a viewing angle for acquiring a comfortable 3D image.

Stereo images acquired through the two CIS, the camera controller 130 and the ISP 141 are input to the stereo image processor 142, thereby correcting problems such as color balance and mismatch of pixel units between the left and right images. Is processed.

Outputs the 3D image corrected by the stereo image processing unit 142 to the screen through the display unit, or encodes the stereo image through the image codec 143 and multiplexes it by a 3D multiplexing method (storage medium (image memory 160)). Or in the portable memory 170).

In this case, when the image obtained by the CIS is displayed in real time, the image is output directly to the display unit 150 without performing stereo image encoding / decoding through the image codec 143.

When displaying the 3D image content encoded in the image codec 143 and stored in the image memory 160, the image codec 143 receives the stored 3D image content through the image memory 160. Decode and output to the display for display.

The display unit 150 displays the input 3D image by multiplexing the left and right images in a pattern suitable for a 3D image display apparatus.

On the other hand, when acquiring a 3D image, the camera controller 130 may use a distance difference between two CISs to obtain an optimal 3D image that may minimize eye fatigue according to a position of an object (subject) to be acquired. It optimizes the image capturing environment of the two CIS.

The image photographed by the above method is transferred to the stereo image processor 142 as described above. In this case, the distance between the two CISs is calculated using Equation 1 below to optimize the distance difference between the two CISs. Calculate and control.

D = nd / f

Here, D is the distance between two CIS, d is the distance between the CIS and the object, n is a multiple of d, f is the focal length of the CIS.

The obtained 3D image is input to the stereo image processor 142 to correct the image. The image obtained due to two CIS positions and angles of view does not have a color and an object on the same horizontal axis. Therefore, in order to solve these problems, the stereo image processor 142 must correct the image.

As described above, when the displayed left / right image is displayed without storing the 2D or 3D image (in real time processing), the corrected left / right image is directly output to the display unit 150 without performing the encoding / decoding process, and the storage is performed. In the case of a two-dimensional image is stored by a storage method suitable for a two-dimensional image, in the case of a three-dimensional image is stored in the image memory 160 by a storage method suitable for a three-dimensional image, at this time The 3D image is composed of left and right images, and the left and right images have a characteristic of having 70% or more overlapping data as shown in FIG. 4.

4 is a diagram illustrating a left / right image conversion process in an image codec according to an embodiment of the present invention.

As shown in FIG. 4, the right region of the left image and the right region of the right image or the left region of the left image and the left region of the right image have high correlation with each other.

Therefore, when storing the 3D image, the left image is fixed, and the right image is rotated in the horizontal direction to make the two images side-by-side images and then compress them. This improves the compression efficiency because there is a continuous characteristic at the interface between the two images.

5 is a view showing an image multiplexed on the display unit according to the present invention.

Left and right images processed by the image acquisition unit 140 are multiplexed and displayed on the display unit 150 as illustrated in FIG. 5. In this case, when the acquired image is a 2D image, the 2D image is displayed in a conventional 2D method.

The display unit 150 according to the present invention may display both 2D and 3D images. That is, the display unit 150 is a two-dimensional and three-dimensional dual-eye display device in which the image display panel is composed of twisted nematic (TN) panels, and the two-dimensional image can be displayed while the TN panel is turned off, and the three-dimensional image is displayed. The TN panel is turned on so that the TN panel functions as a parallax barrier to display a 3D image.

The display unit 150 according to the present invention may be applied to a multi-view image display mask such as an IP (Integral Photography) plate, a lenticular plate, a cross lenticular plate, or a rectangular lens plate to view a 3D image.

6 is a configuration diagram of an embodiment of a display unit according to the present invention.

As shown in FIG. 6, the display unit 150 includes two cameras (a first camera and a second camera) and two speakers (a first speaker and a second speaker) for outputting three-dimensional stereo sound. The left and right images may be multiplexed as shown in FIG. 5 and displayed in three dimensions using a multiview image display mask.

The present invention described above is capable of various substitutions, modifications, and changes without departing from the technical spirit of the present invention for those skilled in the art to which the present invention pertains. It is not limited by the drawings.

The present invention as described above, the user can directly obtain a 3D image in the mobile terminal, there is an effect that can store and display the obtained 3D image.

Claims (7)

A 3D mobile terminal capable of 3D image acquisition and display, Wireless communication means for converting and outputting a voice signal received from a mobile communication network into an audible sound, and converting and transmitting a voice signal received from a microphone; A first photographing means and a second photographing means, which are provided in a mobile terminal to photograph and acquire left and right images of a subject, respectively, and which acquire images by using a CMOS image sensor (CIS); Photographing control means for controlling the vergence points of the first photographing means and the second photographing means; Image acquisition means for processing and outputting images received from the first and second photographing means; Image information storage means for storing image information processed by the image acquisition means; And Display means for multiplexing and displaying the left and right images received from the image acquisition means in three dimensions 3D mobile terminal comprising a. The method of claim 1, The image acquisition means, Image sensor control means for equalizing a photographing environment of the first photographing means and the second photographing means; Stereo image processing means for correcting and outputting left and right images received from the first photographing means and the second photographing means; And Image encoding / decoding means for encoding and outputting the image signal corrected by the stereo image processing means to the image information storage means and decoding the image received from the image information storage means. 3D mobile terminal comprising a. delete The method of claim 1, The display means, 3D mobile terminal, characterized in that for using the multi-view image display mask to display a three-dimensional image. The method according to any one of claims 1 to 4, The photographing control means, 3D mobile terminal, characterized in that the gaze point is set to the subject in the near and far distance by rotating and horizontally moving each of the first and second photographing means. The method of claim 4, wherein The photographing control means, In order to minimize visual fatigue and to obtain an optimal 3D image, the first photographing means and the second photographing means are set by dividing the distance between the cameras from the cameras by the camera focal length. 3D mobile terminal, characterized in that for controlling the means. The method of claim 2, The video encoding / decoding means, 3D mobile terminal, characterized in that for rotating the encoding of one image to increase the compression efficiency.

Images