JP2011525075A - Stereo image generation chip for mobile equipment and stereo image display method using the same - Google Patents

Abstract

  The present invention relates to a stereoscopic image generation chip for a mobile device and a stereoscopic image display method using the same. The present invention is not to display a stereoscopic image using a stereoscopic image generation chip inserted in a mobile device, and to synthesize and store / image process the captured left image and right image in an alternating arrangement. It is possible to improve the quality of the displayed stereoscopic image through the synthesis step in which the left and right images of the stereoscopic image generation chip are alternately arranged in the display stage after performing the storage / image processing in a side-by-side manner. it can. Further, a 3D display of a general 2D image is possible through the above-described stereoscopic image generation chip.

Description

  The present invention relates to a stereoscopic image generation chip using a mobile device and a stereoscopic image display method using the same, and more specifically, by enhancing the stereoscopic image processing function of the stereoscopic image generation chip in the mobile device. The present invention relates to a technique for improving the quality of a stereoscopic image and processing so that a 2D image can be recognized as 3D.

  In general, a method of realizing a stereoscopic image (or 3D image) is implemented by illuminating different images of two human eyes, and the stereoscopic image display apparatus illuminates different images of the two eyes. Depending on whether or not it is necessary to wear separate glasses, the glasses type stereoscopic image display device and the non-glasses type (naked eye type) stereoscopic image display device are roughly classified.

  According to the glasses-type stereoscopic image display device, there is an inconvenience that the observer should wear special glasses. However, according to the non-glasses-type stereoscopic image display device, the screen is directly used without wearing the glasses described above. A stereoscopic image can be felt just by gazing at the camera, and the disadvantages of the glasses-type stereoscopic image display device can be eliminated. For this reason, research on non-glasses-type stereoscopic image display devices has been greatly advanced. Non-glasses type stereoscopic image display apparatuses are roughly classified into a lenticular type apparatus and a parallax-barrier type apparatus.

  Hereinafter, the operation of the parallax-barrier stereoscopic image display apparatus will be described with reference to FIGS.

  FIG. 1 is a cross-sectional view of a stereoscopic image display device using a parallax barrier, and FIG. 2 is a perspective view of the stereoscopic image display device using a parallax barrier.

  As shown in FIGS. 1 and 2, the stereoscopic image display device using the parallax-barrier method has a left-side image (YY ′ direction in FIG. 2) corresponding to the left / right two eyes. L) and the right image (R) are arranged in the horizontal direction (XX ′ direction in FIG. 2) alternately, a display module 10 arranged in the front side, and a bar-shaped blocking film called a barrier 20 directed in the vertical direction ,including. In such a stereoscopic image display device, as shown in FIG. 1, light corresponding to the left image (L) is incident only on the left eye, and light corresponding to the right image (R) is incident only on the right eye. As described above, the display module 10 and the barrier 20 are arranged, and through this, the two divided left and right images (L, R) are observed in a separated state to give a stereoscopic effect.

  In order to implement a stereoscopic image as described above, stereoscopic image content in which left and right images are alternately arranged and stored is necessary. However, at present, such stereoscopic image content is provided only by a specific service provider because equipment for taking a stereoscopic image is expensive and not popularized, and is widely activated by general users. Not.

  Therefore, there is a great expectation for a technique for easily generating a high-quality stereoscopic image using an existing mobile device including a mobile phone. In addition, if the existing 2D image can be adjusted and displayed so that the user feels a certain degree of stereoscopic effect, the demand for 3D technology should increase.

  In addition, in the case of a stereoscopic image, the quality of decoding of the compressed and compressed image is affected by a special processing method for the left and right side images, so a stereoscopic image generation chip or module that matches the characteristics of the stereoscopic image content is required. is there.

Korean Patent Application No. 2005-0127631

  In order to solve the above-described problems, the object of the present invention is to easily produce stereoscopic image content by a general user using a mobile device equipped with two existing camera modules, and at the time of shooting a stereoscopic image, An object of the present invention is to provide a method for producing a higher-quality stereoscopic image by confirming a stereoscopic image that is actually displayed and generating content while adjusting the sense of depth.

  In particular, in the present invention, a stereoscopic image using the above-described mobile device is generated using a stereoscopic image generation chip attached to the mobile device, and the image processing procedure is improved in consideration of characteristics specific to stereoscopic image content. Thus, it is intended to provide a method for easily improving the quality of a displayed stereoscopic image.

  In addition, an attempt is made to provide a method for increasing the utilization of stereoscopic image technology by displaying an existing 2D image so that the user feels a certain degree of stereoscopic effect.

  In one embodiment of the present invention for achieving the above-described object, a method of displaying a stereoscopic image through a stereoscopic image generation chip inserted in a mobile device including a back end capable of image processing and two camera modules. (1) inputting a left image and a right image taken through two camera modules of the mobile device into the stereoscopic image generation chip; (2) inputting the input left image and right image into the stereo image generation chip; Connecting them in a side-by-side manner with a stereoscopic image generation chip and outputting these images to the back end of the mobile device; (3) connected in a side-by-side manner, image processed in the back end. Inputting the left image and the right image from the back end to the stereoscopic image generation chip; (4) through the stereoscopic image generation chip, The left image and the right image are respectively divided into a plurality of vertical images, and the plurality of images obtained by dividing the left image and the plurality of images obtained by dividing the right image are combined in a alternately arranged manner. And (5) A method for displaying a stereoscopic image by displaying the synthesized image through a stereoscopic image display module including a parallax barrier or a lenticular lens array is proposed.

  At this time, before the step (5), (4-1) a plurality of images obtained by dividing the left image and a plurality of images obtained by dividing the right image in the synthesized image, And adjusting the sense of depth by shifting in different directions. Here, “the sense of depth” is assumed to mean the degree to which the user perceives that the image protrudes compared to the screen or the image is located behind the screen.

  The step (3) includes a step of inputting a two-dimensional image image-processed by the back end to the stereoscopic image generation chip, and the step (4) includes the two-dimensional image as the stereoscopic image generation chip. And dividing the plurality of divided images sequentially into a pseudo left image image combination and a pseudo right image image combination, and (4-1) Includes adjusting the sense of depth by shifting the pseudo left image combination and the pseudo right image combination in different directions of left and right directions, and the step (5) includes the step of adjusting the sense of depth. The adjusted pseudo-left image image combination and the pseudo-right image image combination are used as the stereoscopic image data. By setting so as to include the step of displaying through the spray module, the general 2D image can also be a user to a display to feel a stereoscopic effect.

  The step (4-1) includes a plurality of images obtained by dividing the left image according to control information input by a user who has confirmed a stereoscopic image displayed on the liquid crystal of the mobile device through an input device of the mobile device. Alternatively, it is preferable to determine the degree to which the pseudo left image combination and the plurality of images obtained by dividing the right image or the pseudo right image combination are shifted in different directions.

  In the step (2), using the time synchronization generated from the counter of the stereoscopic image generation chip, the input left image and the input left image are synchronized with the shooting time synchronization of the input left image and the right image. The right images can be connected. Specifically, the two camera modules include a pause function including one or more functions of standby or power down. In the step (2), the two cameras If the left image input from the module and the right image are not synchronized, the left image and the right image are paused using the pause function of the two camera modules, and then the synchronization is performed. It may be set to include a step of simultaneously restarting the left image and the right image so as to match.

  On the other hand, in another embodiment of the present invention for solving the above-described problem, a two-dimensional image is divided into a plurality of vertical images, and the divided plurality of images are combined with a pseudo left image image combination. Mapping sequentially as a pseudo right image image combination, adjusting the feeling of depth by shifting the pseudo left image image combination and the pseudo right image image combination in different directions from each other in the left and right directions; and A method for displaying a stereoscopic image of a two-dimensional image including displaying a pseudo left image image combination and a pseudo right image image combination with a sense of depth through a stereoscopic image display module including a parallax barrier or a lenticular lens array is provided.

  At this time, the two-dimensional image is a two-dimensional image input to a stereoscopic image generation chip installed in the mobile device after being subjected to image processing in a back end capable of image processing of the mobile device, The mapping step and the depth adjustment step are performed by the stereoscopic image generation chip.

  The depth adjustment step may include the pseudo left image image combination and the pseudo right image image according to control information input through the input device of the mobile device by a user who has confirmed a stereoscopic image displayed on the liquid crystal of the mobile device. This is done by determining the degree to which the combination is shifted in different directions.

  On the other hand, in still another embodiment of the present invention for solving the above-described problem, a stereoscopic image display function is performed by being inserted into a mobile device including a back end capable of image processing and two camera modules. A stereoscopic image generation chip, a camera input unit for receiving a left image and a right image taken through two camera modules of the mobile device, and a side-by-side display of the input left image and right image. An image connection module for performing processing to connect in a side manner, and a camera output unit for outputting the left image and the right image connected in a side-by-side manner by the image connection module to a back end of the mobile device And the input of the left image and the right image connected in a side-by-side manner, processed by the back end. A left side image and a right side image connected in a side-by-side manner to each of a plurality of vertical images, and a plurality of images obtained by dividing the left side image A synthesis module for synthesizing a plurality of images obtained by dividing the right image alternately, and outputting an image synthesized through the synthesis module to a stereoscopic image display module including a parallax barrier or a lenticular lens array. There is provided a stereoscopic image generation chip for a mobile device including a display output unit.

  At this time, by shifting a plurality of images obtained by dividing the left image and a plurality of images obtained by dividing the right image in the images synthesized by the synthesis module, respectively, in different directions among the left and right directions, A depth feeling adjustment module for adjusting the depth feeling may be further included.

  The display input unit additionally receives an input of a two-dimensional image subjected to image processing at the back end, and the synthesis module divides the two-dimensional image into a plurality of vertical images, and the divided A function of sequentially mapping a plurality of images alternately as a pseudo left image image combination and a pseudo right image image combination is performed, and the depth adjustment module includes the pseudo left image image combination and the pseudo right image image combination, respectively. A function of adjusting a feeling of depth by shifting in different directions among the left and right directions is added, and the display output unit is configured to combine the pseudo left image image whose depth feeling is adjusted by the depth feeling adjustment module. And the pseudo-right image combination with the stereoscopic image It is possible to perform an additional function to be output to the I spray module.

  In addition, the depth adjustment module may include a plurality of images obtained by dividing the left image according to control information input by a user who has confirmed a stereoscopic image displayed on the liquid crystal of the mobile device through an input device of the mobile device, or Depth adjustment is performed by determining the degree to which the pseudo left image combination and the plurality of images obtained by dividing the right image or the pseudo right image combination are shifted in different directions.

  The image connection module further includes a counter for generating time synchronization of the stereoscopic image generation chip, and the image connection module captures the input left image and right image using the time synchronization generated from the counter. The input left image and right image are connected in time synchronization. Specifically, the two camera modules include a pause function including one or more functions of standby or power down, and the image connection module includes the two camera modules. If the left image and the right image received from the camera are not synchronized, the left image and the right image are paused using the pause function of the two camera modules. The left image and the right image are restarted at the same time so that they match, and the left image and the right image are connected in synchronization with the shooting time.

  Finally, in still another embodiment of the present invention for solving the above-described problem, a two-dimensional image is divided into a plurality of images in the vertical direction, and the divided plurality of images are pseudo left-side image images. A combination module that sequentially and alternately maps as a combination and a pseudo right image combination, and a depth that adjusts the sense of depth by shifting the pseudo left image combination and the pseudo right image combination in different directions of the left and right directions. Movement including a feeling adjustment module, and a display output unit that outputs the pseudo left image image combination and the pseudo right image combination in which the sense of depth is adjusted to a stereoscopic image display module including a parallax barrier or a lenticular lens array. 3D image generation chip for equipment To draft.

  At this time, the two-dimensional image is a two-dimensional image input to the stereoscopic image generation chip after being subjected to image processing in a back end capable of image processing of the mobile device.

  In addition, the depth adjustment module may include the pseudo-left image image combination and the pseudo-right side according to control information input by a user who has confirmed a stereoscopic image displayed on the liquid crystal of the mobile device through an input device of the mobile device. The degree of shifting the image / image combination in the different directions is determined and the sense of depth is adjusted.

  According to the embodiment of the present invention as described above, a general user can easily create stereoscopic image content using a mobile device equipped with two existing camera modules, and can actually display a stereoscopic image when shooting. It is possible to produce a higher quality stereoscopic image by confirming the stereoscopic image to be generated and generating content while adjusting the sense of depth.

  In particular, a stereoscopic image using the above-described mobile device is generated using a stereoscopic image generation chip attached to the mobile device, and the characteristics unique to the stereoscopic image content, that is, the left image and the right image are alternately arranged. The quality of the displayed stereoscopic image can be easily improved by improving the image processing procedure in consideration of the characteristic that it must be and the influence upon image processing.

  In addition, the degree of utilization of the stereoscopic image technology can be increased by displaying an existing 2D image so that the user feels a certain degree of stereoscopic effect.

It is sectional drawing of the stereoscopic image display apparatus using a parallax barrier. It is a perspective view of the stereoscopic image display apparatus using a parallax barrier. It is the figure which each showed the left side image and right side image which were image | photographed using two camera modules. It is the figure which showed the image which synthesize | combined the left side image and right side image which were image | photographed using two camera modules with the stereo image generation chip | tip. It is a figure for demonstrating the general method of displaying a stereo image using the stereo image generation chip | tip inserted in the mobile device. FIG. 6 is a diagram for explaining a method of displaying a stereoscopic image using a stereoscopic image generation chip inserted in a mobile device according to an embodiment of the present invention. It is a figure for demonstrating the principle which adjusts the feeling of depth of the image | photographed left side image and right side image with a stereo image generation chip | tip by one Embodiment of this invention. FIG. 6 is a diagram illustrating a method for displaying a two-dimensional image three-dimensionally according to an embodiment of the present invention. It is the conceptual diagram which showed the concept that the stereo image generation chip concerning one Embodiment of this invention processes 2D image and / or 3D image. FIG. 6 is a diagram for explaining a method of synchronizing left and right images taken through two camera modules when there is no memory module according to an embodiment of the present invention. It is a figure for demonstrating the arrangement | positioning relationship in the moving apparatus of the stereo image production | generation chip | tip which concerns on one Embodiment of this invention.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. The detailed description disclosed below in connection with the appended drawings is intended as a description of exemplary embodiments of the invention and is not intended to represent the only embodiments in which the invention may be practiced.

  The following detailed description includes specific details in order to provide a thorough understanding of the present invention. However, one skilled in the art will know that the present invention may be practiced without such specific details. In some instances, well-known structures and devices are omitted or shown in block diagram form around the core functions of each structure and device to avoid obscuring the concepts of the present invention. Further, throughout the present specification, the same constituent elements will be described using the same reference numerals as much as possible.

  FIG. 3 is a diagram illustrating a left image and a right image captured using two camera modules.

  According to one embodiment of the present invention, the left image (L) as shown in (1) of FIG. 3 and ((1) of FIG. The right image (R) as in 2) is taken. On the other hand, a mobile device having an existing image processing function includes only a function of processing an image taken through one camera module through the back-end chipset of the mobile device or simply through the back end. As intended by an embodiment of the present invention, it is impossible to adjust the stereoscopic effect by synthesizing the left image and the right image taken through the two camera modules in synchronism with each other. Therefore, in one embodiment of the present invention, it is possible to insert a stereoscopic image generation chip into such a mobile device and generate a stereoscopic image through this chip.

  On the other hand, the left image (L) and the right image (R) as shown in FIG. 3 are a stop image or a moving image, and correspond to general image content through the respective camera modules. Each camera module can also support CMOS or CCD, which is a matter of choice.

  FIG. 4 is a diagram illustrating an image obtained by combining a left image and a right image captured using two camera modules with a stereoscopic image generation chip.

  Specifically, a left image and a right image captured by two camera modules as shown in FIGS. 3A and 3B are input to a stereoscopic image generation chip and are divided in units of columns in the vertical direction. And are arranged alternately in the horizontal direction. As for the left and right images arranged alternately in this way, only the left image can be seen by the user's left eye and the right image can be seen by the right eye through the display module using the parallax barrier as shown in FIGS. Only becomes visible, and is recognized as a stereoscopic image as a whole.

  In the following, a stereo image is generated by inserting a stereo image generation chip into a mobile device based on the general theory regarding such a stereo image generation method, and taking into consideration the unique characteristics of the stereo image content, a high quality stereo image An improved stereoscopic image generation chip enabling a display and a method using the same will be described. For this purpose, first, a general method for generating a stereoscopic image using a stereoscopic image generation chip inserted in a mobile device and its problem will be described.

  FIG. 5 is a diagram for explaining a general method for displaying a stereoscopic image using a stereoscopic image generation chip inserted in a mobile device.

  Generally, in order to generate a stereoscopic image using a mobile device, the number of camera modules attached to the existing mobile device is expanded to two (401a, 401b), and a stereoscopic image that cannot be processed by the existing mobile device. In order to perform functions such as image composition, it is necessary to insert a stereoscopic image generation chip 402. Of course, it is possible to add the function of the stereoscopic image generation chip 402 to the module corresponding to the existing back end 403 in software format, and the present invention does not exclude this. That is, the method according to each embodiment of the present invention is performed by a stereoscopic image generation chip, or the entire stage or a partial stage is performed by software. However, according to a preferred embodiment of the present invention, the processing speed is increased by performing a hardware process using a separate stereo image generation chip for synthesizing and processing a stereo image, thereby generating a stereo image more stably. Consider how to do.

  First, the two camera modules 401 a and 402 b respectively capture a left image (L) and a right image (R) corresponding to the left and right eyes of a human, and transmit these images to the stereoscopic image generation chip 402. Upon receiving the left image (L) and the right image (R), the stereoscopic image generation chip 402 subsequently combines the left and right images in a horizontal arrangement manner as shown in FIG. ) Can be generated. The composite image (C) generated in this way is subsequently transmitted to the back end 403 of the mobile device.

  The back end 403 of the mobile device that has received the composite image (C) in this way performs various image processing on the composite image (C). Such image processing includes an image compression process, a compressed content storage process, a stored image decoding process, and the like.

  Among these, as the image compression method, various methods are defined by the stop image and the moving image. For the sake of simplicity, the following outlines an example of a compression method for a stop image.

  The compression for the stop image is performed by subdividing the image into small pixels, reading the image data value of each pixel in a predetermined order, and then converting the continuous form of each read data value to a code value determined by a certain standard. It will be compressed by mapping to. At this time, the repetition of “0” has an important meaning in image compression. For example, a form in which “0” is repeated 10 times does not arrange “0” 10 times, but repeats “0”. Image data can be compressed by indicating the number of times as a separate signal value.

  However, as shown in FIG. 4, in the case of stereoscopic image content in which the left image (L) and the right image (R) are alternately repeated, there are a large number of boundary regions between the left image and the right image. The border region will increase the discontinuity of the image. Thus, when the image discontinuity increases, the compression rate in the image compression process decreases, and the capacity of the compressed content increases. Therefore, a storage medium with a large capacity is required for storing stereoscopic image content. In addition, the quality of the compressed image decreases as the correlation degree of each divided pixel decreases.

  Meanwhile, the stereoscopic image content stored in the storage medium is displayed through the display module 404 through a decoding process. Even in such a decoding process, there is a possibility that the quality of the decoded image is deteriorated due to a low correlation between the content data. Therefore, in FIG. 5, the decoded image (C ′) transmitted from the back end 403 to the stereoscopic image display module 404 has a lower quality than the 2D image even under the same conditions.

  Accordingly, in one embodiment of the present invention, the following method is proposed to solve the above-described problem due to the above-described discontinuity or low correlation of the stereoscopic image content transmitted to the back end of the mobile device. .

  FIG. 6 is a diagram illustrating a method of displaying a stereoscopic image using a stereoscopic image generation chip inserted into a mobile device according to an embodiment of the present invention.

  Also in the present embodiment, the two camera modules 401a and 401b mounted on the mobile device respectively capture the left image (L) and the right image (R) and transmit these images to the stereoscopic image generation chip 502 ( S501). However, the stereoscopic image generation chip 502 according to the present embodiment immediately synthesizes the left image (L) and the right image (R) in an iterative form of L, R, L, R,... As shown in FIG. Instead, we propose combining the entire left image and the entire right image in a side-by-side manner. Hereinafter, the “side-by-side” method is defined as a form in which two or more images are directly connected in the left-right direction. When the left image (L) and the right image (R) taken through the two camera modules 401a and 401b are moving images, these images are connected in time synchronization using the counter of the stereoscopic image generation chip 502. This will be explained later. The left image (L) and the right image (R) thus connected in the side-by-side manner are transmitted to the back end 403 of the mobile device (S502). Image processing is performed.

  In the case of stereoscopic image content in which the left image (L) and the right image (R) are connected in a side-by-side manner, the left image (L) and the right image (R) are alternately arranged as shown in FIG. Since the discontinuity of the image data is smaller than that of the stereoscopic image content, a high compression rate can be obtained. As a result, the capacity of the compressed image becomes smaller, and more content can be stored in the storage medium of the limited mobile device. Also, the high degree of correlation of the image data can reduce image data loss during the compression and decoding process.

  On the other hand, in the present embodiment, it is proposed that the stereoscopic image content that has undergone the image processing processes such as compression, storage, and decoding as described above is set so as to be input to the stereoscopic image generation chip 502 again thereafter ( S503). That is, as shown in FIG. 4, contrary to the fact that a 3D image processed in the back end in a general 3D image display process in a mobile device is immediately transmitted to the 3D image display module 404 (that is, 3D). This embodiment proposes that the stereoscopic image generation chip 502 includes a stereoscopic image display module driving function (as opposed to the image display module driving function being in the back end of a mobile device that is not a stereoscopic image generation chip). In addition, the stereoscopic image generation chip 502 according to the present embodiment is required to additionally perform a stereoscopic image content synthesis function as follows.

  That is, the stereoscopic image content (Sc ′) that has been image-processed by the back end 403 and that is connected in a side-by-side format is left-hand side in accordance with the display format of the parallax-barrier or lenticular stereoscopic image display module 404. It is desirable to convert the image and the right image into a form in which they are alternately arranged. Accordingly, the stereoscopic image generation chip 502 according to the present embodiment divides the left image (L) portion and the right image (R) portion of the input stereoscopic image content into a plurality of vertical images, respectively, and the left image is The plurality of divided images and the plurality of images obtained by dividing the right-side image are combined by a method of alternately arranging them in the horizontal direction. The synthesized stereoscopic image content (Ic) is transmitted to the stereoscopic image display module 404, and the parallax-barrier type stereoscopic image display or the lenticular lens array type as described with reference to FIGS. A stereoscopic image display is performed (S504).

  As described above, according to the present embodiment, the capacity of the stereoscopic image content is reduced, and a higher quality stereoscopic image can be displayed. However, the stereoscopic effect recognized by the display means and the user may be different from each other. That is, some users may want to feel a stronger sense of depth, or may want to feel a weaker sense of depth for reasons such as inconvenience in the cognitive process. Here, “the sense of depth” is assumed to mean the degree to which the user perceives that the image protrudes compared to the screen or the image is located behind the screen. Therefore, in a preferred embodiment of the present invention, the user can make the stereoscopic effect more efficient by checking the stereoscopic image and separately adjusting the sense of depth according to each user.

  FIG. 7 is a diagram for explaining the principle of adjusting the sense of depth of the captured left image and right image by a stereoscopic image generation chip according to an embodiment of the present invention.

  As shown in FIG. 7, according to the combination position of the left image and the right image constituting the stereoscopic image, the user can display the corresponding object on the front surface of the screen 401 (for example, the position where the left image and the right image are indicated as “402”). ) Or a rear surface of the screen 401 (for example, a position where the left image and the right image are indicated as “403”). According to one embodiment of the present invention, it is proposed to adjust the stereoscopic effect by the user by adjusting the sense of depth through the relative shift of the left-right arrangement of the left image and the right image synthesized as described above. To do.

  The adjustment of the stereoscopic effect through the synthesis of the left image and the right image and the relative shift is performed in a stereoscopic image processor of the stereoscopic image generation chip according to an embodiment of the present invention, in the configuration of an existing mobile device. In particular, it may be realized through a stereoscopic effect adjustment module of the stereoscopic image processor or by software installed in a mobile device, which will be described later.

  Specifically, when it is recognized that an object is located on the front surface of the screen 401 (a position indicated by “402”) by the arrangement relationship between the synthesized left image and right image, the right image is displayed in the right direction among the left and right directions. If the left image is shifted in the “A” direction and the left image is shifted in the “A ′” direction, which is the left direction, the left and right images are indicated as “402a” by the interval between the left and right images. It is perceived to be in position, and as a result, has the same effect as moving things further away from the screen 401 (ie, the “a” direction). In addition, the right image is shifted in the “A” direction and the left image is “A ′” with respect to an object recognized as being located on the rear surface of the screen 401 (position indicated by “403”) in the same image. If the direction is shifted, the user recognizes that the right image and the right image are located at the position indicated by “403a” according to the interval between the left and right images. a) direction).

  Eventually, when the right image is shifted in the “A” direction and the left image is shifted in the “A ′” direction, the entire object has an effect of moving in the “a” direction. The feeling can be adjusted.

  On the other hand, when it is recognized that an object is located in front of the screen 401 (a position indicated by “402”), the right image is in the “B” direction, which is the left direction, and the left image is “B ′, which is the right direction. ”In the opposite directions, the user will recognize that these images are located at the“ 402b ”position, resulting in the same movement to the screen 401 side (ie, the“ b ”direction). Has the effect of. Also, when it is recognized that an object is located on the rear surface of the screen 401 (the position indicated by “403”), if the right image is shifted in the “B” direction and the left image is shifted in the “B ′” direction, The user recognizes that these images are located at the position “403b”, and as a result, has the same effect as moving in the direction away from the screen 401 (ie, the “b” direction).

  Eventually, when the right image is shifted in the “B” direction and the left image is shifted in the “B ′” direction, the object is moved in the “b” direction as a whole, and the overall stereoscopic effect is increased. Can be adjusted.

  The adjustment of the sense of depth as described in connection with FIG. 7 can be performed while confirming the stereoscopic effect of the stereoscopic image photographed through the liquid crystal of the mobile device in one embodiment of the present invention. A stereoscopic effect required to be different depending on the display module and the user can be adjusted to generate a stereoscopic image.

  In this way, the stored content can be displayed immediately after adjusting the sense of depth, or can be stored again in a state where the sense of depth is adjusted, and can be used immediately in subsequent playback. Through this, the stereoscopic image generation method and the stereoscopic image generation chip according to the present embodiment can display a stereoscopic image optimized in accordance with the conditions of each user.

  In addition, the depth feeling adjustment method as shown in FIG. 7 adjusts the depth feeling of the 2D image arbitrarily as follows, and displays the 2D image so that the user feels a certain degree of stereoscopic effect. Such a technique is described below as still another embodiment of the present invention.

  FIG. 8 is a diagram illustrating a method for three-dimensionally displaying a two-dimensional image according to an embodiment of the present invention.

  The 2D image shown in FIG. 8A is a 2D image that has been image-processed at the back end of the mobile device, an image taken through any one camera module of the mobile device, or other media. This is an image that has been subjected to image processing. Upon receiving such a 2D image input, the stereoscopic image generation chip according to this embodiment first divides the two-dimensional image into a plurality of images in the vertical direction as shown in FIG. Each image portion is sequentially cross-mapped as a left image portion and a right image portion. In the following description, in order to easily distinguish the captured left image and right image alternately from the original form, the 2D image is divided as described above, and the combination mapped to the left image portion is used. It is referred to as “pseudo left image combination” and the combination mapped to the right image portion is referred to as “pseudo right image combination”. The sense of depth is adjusted as shown in FIGS. 8B and 8C by such a pseudo left image combination and a pseudo right image combination.

  FIG. 8B shows that the pseudo left image image combination is relatively shifted in the right direction and the pseudo right image image combination is relatively shifted in the left direction. When shifting the pseudo left image combination in the right direction and shifting the pseudo right image combination in the left direction in this way, in the form shown in FIG. 7, the direction in which the image moves to the rear of the screen (ie, “b ”Direction).

  FIG. 8C shows that the pseudo left image image combination is relatively shifted in the left direction and the pseudo right image image combination is relatively shifted in the right direction. When shifting the pseudo left image image combination in the left direction and shifting the pseudo right image image combination in the right direction in this way, in the form shown in FIG. 7, the direction in which the image moves to the front of the screen (ie, “a ”Direction). By using the pseudo left image image combination and the pseudo right image image combination in which the sense of depth is adjusted in this manner, the pseudo left image image combination can be applied only to the left eye through a parallax-barrier type or lenticular array type stereoscopic image display module. Illuminated, and the pseudo right image combination is illuminated only to the right eye, the user feels a stereoscopic effect.

  Such 2D image stereoscopicization is used in a form independent of or combined with the general 3D image display described above.

  FIG. 9 is a conceptual diagram illustrating a concept of processing a 2D image and / or a 3D image by a stereoscopic image generation chip according to an embodiment of the present invention.

  That is, the stereoscopic image generation chip 502 ′ according to the present invention can receive 3D content or general 2D content in which left and right images are combined in a side-by-side manner from the back end of a mobile device. At this time, the 3D content / 2D content is preferably content that has been subjected to image processing in the back end. Upon receiving such content, the stereoscopic image generation chip 502 ′ performs a process of dividing the left image portion and the right image portion into a plurality of images for 3D content, and arranging them alternately and combining them. For the 2D content, as described with reference to FIG. 8, it is possible to perform a process of mapping a plurality of images divided in the vertical direction into a pseudo left image image combination and a pseudo right image image combination. Accordingly, the image (Ic ′) output to the stereoscopic image display module has a form in which the left image / pseudo left image and the right image / pseudo right image are alternately arranged. As a result, the parallax barrier / The user's left eye and right eye are divided and displayed through a lenticular lens array type display module.

  Meanwhile, in a preferred embodiment of the present invention, a general 2D image and a 3D image can be combined and displayed. That is, in the case of one or more specific objects (Objects), the left image and the right image captured through two camera modules are processed in a combined form, and display is performed so that a sense of depth differs depending on each part of the corresponding object. These objects can be displayed under a general 2D image background. At this time, the 2D image corresponding to the background is also adjusted in depth as described in connection with FIG. 8 as a whole, so that the user can enjoy a full-blown stereoscopic image.

  In the above description, in the parallax barrier type stereoscopic image display module, as shown in FIG. 1 and FIG. 2, the left image and the right image are each divided in vertical columns, and are arranged in the horizontal direction and synthesized. Only the form in which only a stereoscopic image is recognized as a stereoscopic image by the user has been described. However, according to the “cell structure parallax-barrier and stereoscopic image display apparatus using the same (Patent Document 1)” invented by the present inventor and filed by the present applicant, the above-described parallax barrier is a cell system. And can be selectively driven in the horizontal direction or the vertical direction. Therefore, when a stereoscopic image is synthesized according to an embodiment of the present invention, the direction of synthesis need not be limited to one of the horizontal direction and the vertical direction, and the stereoscopic image and the stereoscopic video are screened. 3D images can be displayed by adjusting the barrier direction of the parallax barrier type display module used in the above.

  On the other hand, when the left and right images taken through the two camera modules are moving images, the respective images must be synthesized in time synchronization by the above-described stereoscopic image generation chip. Such time synchronization uses time synchronization generated by the counter of the stereoscopic image generation chip according to an embodiment of the present invention, and when the left and right images are not synchronized, one of the left and right images is stereoscopically displayed. After temporarily storing in the memory of the image generation chip, synthesizing in synchronism, or a pause function including a standby or power down function included in the existing two camera modules Is obtained by suspending all the left and right images and then resuming them simultaneously so that the two synchronizations match, and will be described in more detail below. To do.

  First, the stereoscopic image generation chip according to the embodiment of the present invention may include a memory module for temporarily storing any one of these when the left image and the right image are not synchronized. This memory module can perform a buffer function to temporarily store fast-synchronized images in order to synchronize the left and right images taken through the two camera modules when they are not synchronized. The left and right images can be combined in synchronism with each other.

  On the other hand, in another embodiment of the present invention, as described above, the left and right images can be synchronized by the following method even if the stereoscopic image generation chip does not include a separate memory module.

  FIG. 10 is a diagram for explaining a method of synchronizing left and right images taken through two camera modules when there is no memory module according to an embodiment of the present invention.

  Specifically, the synchronization of the left image and the right image taken through the two camera modules is indicated as “CH1 camera sync” and “CH2 camera sync”, respectively. The process of synchronizing using the pause function is shown.

  The existing camera module can include a pause function such as standby or power off for the convenience of taking an image. Therefore, in one embodiment of the present invention, even if there is no memory module for temporary storage of any one of the left and right images in the stereoscopic image generation chip, the left and right images input from the two camera modules are synchronized. Otherwise, when there is an event in the synchronization of both images, that is, as shown in FIG. 10, when the voltage is converted from “high” to “low” or when it is converted from “low” to “high”. The left and right images can be paused, the two images can be synchronized, and the images can be restarted at the same time.

  As described above, the function of synchronizing the left and right images even when the stereoscopic image generation chip does not include the memory module using the temporary stop function of the two camera modules is described in detail below. However, the present invention is not limited to this as long as the above-described method can be performed.

  On the other hand, the following describes how the stereoscopic image generation chip according to each embodiment of the present invention described above can be arranged in a mobile device.

  FIG. 11 is a diagram for explaining an arrangement relationship in the mobile device of the stereoscopic image generation chip according to the embodiment of the present invention.

  FIG. 11 basically corresponds to each configuration shown in the form of a block diagram in FIG. That is, the two camera modules 401a and 402b are indicated as “CAM1” and “CAM2”, and the stereoscopic image generation chip 502 is indicated as “3D MUXER”. On the other hand, the back end 403 of the mobile device is indicated as “CPU”, and the stereoscopic image display module 404 is indicated as “LCD” and “TN LCD”. In FIG. 11, reference numerals are shown based on such correspondence.

  Hereinafter, a specific operation related to FIG. 11 will be described with reference to FIG. The left image and the right image captured through the two camera modules CAM1 401a and CAM2 401b are input to a 3D MUXER 502 that performs a stereoscopic image generation chip function. At this time, the output signal of the camera is input through the two camera input units CAM1 and CAM2 of the 3D MUXER 502. The 3D MUXER 502 that receives the left image and the right image from the camera module can combine the left and right images in a side-by-side manner. At this time, if each image is a moving image, it is necessary to synchronize. The combined image is input to the back end of the mobile device, that is, the CPU 403 via the camera interface through the camera output unit (CAM OUT) of the 3D MUXER 502.

  Through this, the CPU 403 of the mobile device can perform various image processing of the stereoscopic image content. For example, the CPU 403 can perform procedures such as compression of stereoscopic image content, storage of compressed images, and decoding of stored images. The 3D content subjected to such image processing is input to the 3D MUXER 502 again through a display module input unit, for example, an LCD input unit (LCD IN). In addition, as described above, the 3D MUXER 502 can receive not only 3D content but also general 2D content from the CPU 403 to perform a depth adjustment function in an embodiment of the present invention.

  The 3D MUXER 502 that receives the input of the 3D content / 2D content subjected to the image processing as described above performs a process of synthesizing the 3D content in a form in which the left image and the right image are alternately arranged as described above. For 2D content, as described above, an image can be divided into a plurality of images in the vertical direction, and each can be mapped to a pseudo left image image combination and a pseudo right image image combination. The combined / mapped information is transmitted to a stereoscopic image display module such as an LCD and a TN LCD through a display output unit such as an LCD output unit (LCD OUT).

  FIG. 11 described above shows a specific form of each functional block shown in FIG. 6 so that those skilled in the art can easily carry out the present invention. The scope of the present invention is not necessarily limited to this. . That is, it is obvious to those skilled in the art that each functional block shown in FIG. 6 can be referred to as other parts or names as long as they perform the same or equivalent functions.

  In addition, the stereoscopic image display method according to each embodiment of the present invention as described above is implemented with the following modifications. That is, in another embodiment of the present invention, a left image and a right image captured through two cameras of a mobile device such as a mobile phone or a UMPC are connected in a side-by-side manner by the corresponding mobile device. Later, it may be transmitted to other media and displayed. Hereinafter, in the present embodiment, a device that displays a stereoscopic image is referred to as a “first device”, and a device that captures a left image and a right image and generates content is referred to as a “second device”.

  First, it is desirable that the second device includes two camera modules. The left image and the right image are respectively taken through these two camera modules. Thereafter, the captured left and right images can be processed in a side-by-side manner by the second device, and the second device is the mobile device according to the above-described embodiment of the present invention. The left and right images are connected in a side-by-side manner by the stereoscopic image generation chip.

  Through this, stereoscopic image content is generated, and in the present embodiment, it is proposed to display such stereoscopic image content on another medium (first device). That is, the left and right side images connected in a side-by-side manner by the second device are input to the first device by a wired connection terminal such as a USB or various wireless connection means. The left and right images as the stereoscopic image content input at this time are simply connected in a side-by-side manner by the second device, but image processing such as compression is performed by the second device. It is desirable for the content to be broken. When the second device is a mobile device according to the above-described embodiment, such image processing is performed at the back end of the mobile device.

  As described above, the first device that receives the input of the stereoscopic image content thereafter divides the input left image and the right image into a plurality of vertical images, and the plurality of images obtained by dividing the left image. And a plurality of images obtained by dividing the right-side image can be synthesized by alternately arranging them. Through this, the stereoscopic image content can be displayed through the stereoscopic image display module of the first device that includes the parallax barrier or lenticular lens array to provide a stereoscopic image to the user.

  In addition, the stereoscopic image content input to the first device in the above-described embodiment is adjusted so that the 2D image has a depth feeling described with reference to FIG. It is also the content generated as.

  The detailed description of the preferred embodiments of the present invention disclosed above is provided to enable any person skilled in the art to implement and practice the invention. The above description has been made with reference to the preferred embodiments of the present invention. However, those skilled in the relevant technical field will not be departing from the spirit and scope of the present invention described in the following claims. It will be understood that the present invention can be modified and changed in various ways. For example, in the above description of the present invention, “mobile device” is a concept that includes all mobile devices including mobile phones, PDAs, PMPs, etc., and two camera modules and a back-end chip capable of image processing. As long as the mobile device includes the mobile device, the device is not limited to a specific type of device, and a stereo image can be generated by inserting the stereo image generation chip according to the embodiment of the present invention into any mobile device.

  Accordingly, the present invention is not limited to the embodiments disclosed herein, but provides the widest scope consistent with the principles and novel features disclosed herein.

  The above-described stereoscopic image generation chip for a mobile device and a stereoscopic image generation method using the same according to the present invention are applied to various mobile devices including mobile phones, PDAs, PMPs, etc. Makes it possible to realize a stereoscopic image display of higher quality.

Claims (21)

In a method of displaying a stereoscopic image through a stereoscopic image generation chip inserted in a mobile device including a back end capable of image processing and two camera modules,
(1) inputting a left side image and a right side image taken through two camera modules of the mobile device to the stereoscopic image generation chip;
(2) connecting the input left image and right image in a side-by-side manner by the stereoscopic image generation chip, and outputting these images to the back end of the mobile device;
(3) inputting a left-side image and a right-side image, which are image-processed at the back end, connected in a side-by-side manner, from the back end to the stereoscopic image generation chip;
(4) Through the stereoscopic image generation chip, the left image and the right image are each divided into a plurality of vertical images, and the plurality of images obtained by dividing the left image and the plurality of images obtained by dividing the right image are divided. Synthesizing by alternately arranging the methods,
(5) displaying the synthesized image through a stereoscopic image display module including a parallax barrier or a lenticular lens array;
A stereoscopic image display method including: Before step (5),
(4-1) Depth by shifting a plurality of images obtained by dividing the left image and a plurality of images obtained by dividing the right image in different directions among the left and right directions in the synthesized image. The stereoscopic image display method according to claim 1, further comprising adjusting a feeling. The step (3)
Inputting a two-dimensional image image-processed at the back end to the stereoscopic image generation chip;
In step (4),
Dividing the two-dimensional image into a plurality of vertical images through the stereoscopic image generation chip, and sequentially mapping the divided images as a pseudo left image image combination and a pseudo right image image combination;
The step (4-1)
Adjusting the feeling of depth by shifting the pseudo left image image combination and the pseudo right image image combination in different directions among left and right directions,
In step (5),
3. The stereoscopic image display method according to claim 2, further comprising: displaying the pseudo left image image combination and the pseudo right image image combination with the sense of depth adjusted through the stereoscopic image display module. The step (4-1)
By the control information input by the user who confirmed the stereoscopic image displayed on the liquid crystal of the mobile device through the input device of the mobile device,
A plurality of images obtained by dividing the left image or the pseudo left image image combination, and a plurality of images obtained by dividing the right image or the pseudo right image image combination are determined to be shifted in different directions. The stereoscopic image display method according to claim 2 or 3. In step (2),
The input left image and the right image are connected in synchronization with shooting time synchronization of the input left image and the right image using time synchronization generated from the counter of the stereoscopic image generation chip. The three-dimensional image display method as described. The two camera modules include a pause function including one or more functions of standby or power release,
In step (2),
When the left image and the right image input from the two camera modules are not synchronized, the left image and the right image are paused using the pause function of the two camera modules. The stereoscopic image display method according to claim 5, further comprising the step of simultaneously restarting the left image and the right image so as to be synchronized with each other. Dividing the two-dimensional image into a plurality of vertical images, and sequentially mapping the plurality of divided images as a pseudo left image image combination and a pseudo right image image combination;
Adjusting the feeling of depth by shifting the pseudo left image combination and the pseudo right image combination in the left and right directions, respectively,
And displaying the pseudo left image image combination and the pseudo right image image combination with the sense of depth adjusted through a stereo image display module including a parallax barrier or a lenticular lens array, and a stereoscopic image display method of a two-dimensional image . The two-dimensional image is a two-dimensional image that has been subjected to image processing in a back end capable of image processing of the mobile device and then input to a stereoscopic image generation chip installed in the mobile device,
The stereoscopic image display method of a two-dimensional image according to claim 7, wherein the mapping step and the depth adjustment step are performed by the stereoscopic image generation chip. The depth adjustment step includes
The degree to which the pseudo left image image combination and the pseudo right image combination are shifted in different directions according to control information input by the user who has confirmed the stereoscopic image displayed on the liquid crystal of the mobile device through the input device of the mobile device. The stereoscopic image display method for a two-dimensional image according to claim 7, wherein the stereoscopic image display method is performed. In a stereoscopic image generation chip that performs a stereoscopic image display function inserted in a mobile device including a back end capable of image processing and two camera modules,
A camera input unit for receiving a left image and a right image taken through two camera modules of the mobile device;
An image connection module that performs processing for connecting the input left image and right image in a side-by-side manner;
A camera output unit for outputting the left image and the right image connected in a side-by-side manner by the image connection module to a back end of the mobile device;
A display input unit for receiving an input of a left image and a right image, which are image-processed at the back end, connected in a side-by-side manner, from the back end;
The left image and the right image connected by the side-by-side method are respectively divided into a plurality of vertical images, and the plurality of images obtained by dividing the left image and the plurality of images obtained by dividing the right image are divided. A synthesis module that synthesizes each other in an alternating manner,
A stereoscopic image generation chip for a mobile device, including a display output unit for outputting an image synthesized through the synthesis module to a stereoscopic image display module including a parallax barrier or a lenticular lens array.   In the image synthesized by the synthesis module, a plurality of images obtained by dividing the left image and a plurality of images obtained by dividing the right image are shifted in different directions from each other in the horizontal direction. The stereoscopic image generation chip for a mobile device according to claim 10, further comprising a depth feeling adjustment module for adjusting the depth. The display input unit additionally receives an input of a two-dimensional image subjected to image processing at the back end,
The composition module performs an additional function of dividing the two-dimensional image into a plurality of vertical images and sequentially mapping the divided images as a pseudo left image image combination and a pseudo right image image combination. ,
The depth adjustment module additionally performs a function of adjusting a feeling of depth by shifting the pseudo left image combination and the pseudo right image combination in different directions among the left and right directions,
The display output unit additionally performs a function of outputting the pseudo left image image combination and the pseudo right image image combination whose depth sense is adjusted by the depth sense adjustment module to the stereoscopic image display module. 11. A stereoscopic image generation chip for a mobile device according to 11. The depth adjustment module is
By the control information input by the user who confirmed the stereoscopic image displayed on the liquid crystal of the mobile device through the input device of the mobile device,
A depth is determined by determining a degree to which the plurality of images obtained by dividing the left image or the pseudo left image combination and the plurality of images obtained by dividing the right image or the pseudo right image image combination are shifted in different directions. The three-dimensional image generation chip for mobile devices according to claim 11 or 12, which performs a feeling adjustment. A counter for generating time synchronization of the stereoscopic image generation chip;
The image connection module connects the input left image and the right image in synchronization with photographing time synchronization of the input left image and the right image using time synchronization generated from the counter. A stereoscopic image generation chip for a mobile device described in 1. The two camera modules include a pause function including one or more functions of standby or power release,
The image connection module includes:
When the left image and the right image input from the two camera modules are not synchronized, the left image and the right image are paused using the pause function of the two camera modules. 15. The mobile device stereo image generation according to claim 14, wherein the left image and the right image are restarted at the same time so as to be synchronized, thereby connecting the left image and the right image in synchronization with the shooting time synchronization. Chip. A synthesis module that divides a two-dimensional image into a plurality of vertical images, and sequentially maps the plurality of divided images as a pseudo left image image combination and a pseudo right image image combination;
A depth sensation adjustment module that adjusts the sense of depth by shifting the pseudo left image combination and the pseudo right image combination in different directions among left and right directions;
3D image generation for a mobile device, including a display output unit that outputs the combination of the pseudo left image image and the pseudo right image image with the sense of depth adjusted to a stereo image display module including a parallax barrier or a lenticular lens array. Chip.   The three-dimensional image for a mobile device according to claim 16, wherein the two-dimensional image is a two-dimensional image input to the three-dimensional image generation chip after image processing is performed in a back end capable of image processing of the mobile device. Generate chip. The depth adjustment module is
The pseudo left image image combination and the pseudo right image combination are shifted in different directions according to control information input by a user who has confirmed a stereoscopic image displayed on the liquid crystal of the mobile device through an input device of the mobile device. The three-dimensional image generation chip for mobile devices according to claim 16 or 17, wherein the sense of depth is adjusted by determining a degree. In a method for displaying a stereoscopic image through a first device,
(1) inputting the left image and the right image captured through the two camera modules of the second device into the first device in a state of being connected in a side-by-side manner;
(2) The input left image and the right image are each divided into a plurality of vertical images by the first device, and the plurality of images obtained by dividing the left image and the plurality of right images are divided. A step of synthesizing images of each other alternately,
(3) displaying the synthesized image through a stereoscopic image display module including a parallax barrier or lenticular lens array of the first device;
A stereoscopic image display method including:   The left image and the right image connected in the side-by-side manner are input to the first device through a connection unit between the first device and the second device. The three-dimensional image display method as described. The first device is a mobile device,
The stereoscopic image display method according to claim 20, wherein the connecting means includes a wired connecting means and a wireless connecting means.

Images