JP2012175685A - Information processing program, imaging apparatus, imaging method, and imaging system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a correction method for allowing a user to simply adjust an installation position (optical position, etc.) of a stereo camera when the installation position of the stereo camera may be deviated due to an external shock, secular change, or the like during use.SOLUTION: A first temporarily captured image captured by a first imaging part constituting a stereo camera and a second temporarily captured image captured by a second imaging part constituting the stereo camera are displayed on a display part. A user moves the first temporarily captured image displayed on the display part to adjust the display position of the first temporarily captured image to the display position of the second temporarily captured image. On the basis of the result of the movement, the display positions of the two images constituting a stereo image captured by the stereo camera, so that the stereo image is stereoscopically displayed on the display part.

Description

  The present invention relates to an information processing program, an imaging apparatus, an imaging method, and an imaging system, and more specifically to an information processing program, an imaging apparatus, an imaging method, and an imaging system that perform stereo imaging.

  When generating a stereoscopic image (stereo imaging) with a stereo camera, it is desirable that the optical positions such as the focal lengths and the mounting positions of the two cameras are accurately aligned. However, in reality, the optical positions of both cameras are not necessarily aligned, and therefore two images (stereo images) captured by both cameras are caused by a shift in the optical positions of the stereo cameras. Display error. Therefore, unless this error is corrected and the display positions of both images (stereo images) are appropriately matched, it is not possible to display an optimal stereoscopic image that gives a sense of depth or popping out. For this reason, methods of Patent Document 1 and Patent Document 2 are disclosed as the correction method.

JP-A-10-307352 Japanese Patent Laying-Open No. 2005-017286

  According to the conventional correction method described above, an adjustment image (adjustment pattern) needs to be prepared. Therefore, even if the vendor can make adjustments (correction) at the time of manufacture or shipment, the user can It was difficult to adjust by this method. However, there is a possibility that the installation position (optical position) of the stereo camera may be shifted depending on the impact from the outside during use, aging, etc. In such a case, there is a correction method that the user can easily adjust by himself / herself. Necessary.

  Therefore, the main object of the present invention is that the user composes a stereo image by himself even when the optical position of the image pickup unit (stereo camera) is shifted due to an external impact or a secular change during use. To provide an information processing program or the like that can easily adjust the display positions of two images.

  The present invention employs the following configuration in order to solve the above-described problems.

  An information processing program according to the present invention includes a provisional captured image acquisition unit, a provisional captured image display, a computer of an imaging apparatus including a storage unit that captures a subject with a stereo imaging unit and acquires a stereoscopically captured image. Means, input receiving means, display position changing means, correction amount calculating means, and storage control means. The temporary captured image acquisition means captures an arbitrary imaging target by the stereo imaging unit, sets the captured image by the first imaging unit among the stereo imaging units as the first temporary captured image, and sets the captured image by the second imaging unit as the second. Each is acquired as a provisional captured image. The temporary captured image display means displays the first temporary captured image and the second temporary captured image on the display unit. The input receiving unit receives an input from the user. The display position changing unit changes the display position of at least one of the first temporary captured image and the second temporary captured image based on the predetermined input received by the input receiving unit. The correction amount calculating unit is configured to detect an image captured by the first imaging unit and the second imaging unit based on the displacement of the display position of the first temporary captured image and / or the second temporary captured image changed by the display position changing unit. A correction amount for correcting the display position on at least one display unit is calculated. The storage control unit stores the correction amount calculated by the correction amount calculation unit in the storage unit without associating the correction amount with the first temporary captured image and the second temporary captured image.

  According to this configuration, both of the first temporary captured image and the second temporary captured image acquired by capturing an arbitrary imaging target (subject) by the both imaging units (stereo camera) of the first imaging unit and the second imaging unit. Since adjustment is performed based on the images (both images for adjustment), the user can acquire an adjustment image by capturing a desired imaging target with a stereo camera, and it is necessary to prepare a pattern image for adjustment separately. Absent. Then, based on a predetermined input from the user (instruction for moving at least one of both images for adjustment), the display position of the image for adjustment displayed on the display unit (for example, a display of a stereo camera) is changed. Therefore, the user can move the adjustment image to a desired position while checking the display. Further, a display position when stereoscopically displaying a stereo image (two images captured by the stereo camera) on the display based on the displacement of the display position of the adjustment image changed based on the user instruction is displayed. A correction amount for correction is calculated, and this correction amount is stored in the storage unit. As a result, the user can calculate a correction amount for appropriately adjusting the display position of the stereo image and store it in the storage unit simply by performing simple image adjustment (moving the adjustment image). Can do.

  The computer may further function as correction means and stereoscopic display control means. The correction means determines the display position on at least one display unit of the first image captured by the first imaging unit and the second image captured by the second imaging unit separately from the first temporary captured image and the second temporary captured image. The correction is made based on the correction amount. The stereoscopic display control means causes the first image and the second image to be stereoscopically displayed on the display unit based on the correction result by the correction means.

  According to this configuration, the display positions of the first image and the second image (stereo image) on the display unit (display) are corrected based on the correction amount, and the stereo image is displayed on the display in three dimensions based on the correction result. Visually displayed. As a result, the user can easily adjust the display position of the stereo image by simply performing simple image adjustment (moving the adjustment image), and appropriately display the stereo image on the display. Can be made.

  The calculation of the correction amount by the correction amount calculation means may be performed as follows. That is, the correction amount calculation means calculates the correction amount based only on the displacement of the display position in the vertical direction of the first temporary captured image and / or the second temporary captured image.

  According to this configuration, the displacement when the display position of the first temporary captured image and / or the second temporary captured image (adjustment image) is moved includes the displacement in each direction. It is calculated based only on the displacement of the display position in the vertical direction of the image for adjustment (the y-axis direction shown in FIGS. 1 and 5 to be described later). As a result, a correction amount for correcting the stereo image misalignment (vertical misalignment between the first image and the second image), which is the main cause of being recognized as two misaligned images, is calculated. Further, if the direction in which the user can move the adjustment image is not limited to the vertical direction, the first temporary captured image and the second temporary captured image can be substantially overlapped, resulting in the first The display positions in the vertical direction of the first temporary captured image and the second temporary captured image can be easily aligned (adjusted easily), and an appropriate correction amount can be easily calculated.

  The storage unit may store display parameters for determining display positions on the display unit of the first image and the second image. In this case, the temporary captured image display means displays the first temporary captured image and the second temporary captured image on the display unit based on the display parameters. The correction unit corrects the display position on the display unit of at least one of the first image and the second image based on the display parameter corrected based on the correction amount.

  According to this configuration, the display parameter for determining the display position when displaying the stereo image (first image and second image) on the display unit (display) is stored in the storage unit (for example, main memory). Then, the display position of the stereo image is determined based on the display parameter, and the display position of the stereo image is corrected by correcting the display parameter. Thus, the user can display a stereo image at an appropriate display position in which the correction on the display is reflected (updated).

  The computer may further function as display position determination means and display adjustment parameter calculation means. The display position determining unit determines the display position of the first temporary captured image and / or the second temporary captured image based on the determination input received by the input receiving unit. The display adjustment parameter calculation means calculates a display adjustment parameter for correcting the display position on the display unit of at least one of the first image and the second image. In this case, the storage control means updates the display adjustment parameter as the display parameter and stores it in the storage unit. The display position changing means changes the display position of the first temporary captured image based on the first input received by the input receiving means, and the display position determining means determines the first temporary captured image based on the determination input. The display position is determined as the display position after being changed by the display position changing means. The display adjustment parameter calculation means is a displacement between the display position of the first temporary captured image determined by the display position determination means and the display position of the first temporary captured image before being changed by the display position changing means. The display adjustment parameter is calculated based on the displacement, and the correction unit corrects the display position on the display unit of at least one of the first image and the second image based on the display parameter updated in the storage unit.

  According to this configuration, when the user instructs that the display position of the first provisional captured image is appropriately matched with the display position of the second provisional captured image (that is, the adjustment has been completed) (that is, the determination input is performed). By doing so, the display adjustment parameter is calculated based on the movement information (displacement information of the display position) of the first temporary captured image (one image for adjustment) that has been moved so far. And this display adjustment parameter is updated as a display parameter which determines the display position at the time of displaying a stereo image (a 1st image and a 2nd image) on a display part (display). As a result, the user confirms the shift of the first temporary captured image (one image for adjustment) on the display by himself / herself, and the second temporary captured image and the first temporary captured image (one image for adjustment). Just by aligning (adjusting) the display position of the image (the other image for adjustment), the display parameter value is updated (corrected) to the appropriate parameter (display adjustment parameter) value, and based on the updated display parameter The stereo image can be displayed at an appropriate display position.

  The change of the display position by the display position changing unit, the determination of the display position by the display position determining unit, and the calculation of the display adjustment parameter by the display adjustment parameter calculating unit may be performed as follows. That is, the display position changing unit further changes the display position of the second provisional captured image based on the second input received by the input receiving unit, and the display position determining unit is configured to change the first input based on the determination input. The display positions of the temporary captured image and the second temporary captured image are determined as the display positions after the change by the display position changing unit, and the display adjustment parameter calculating unit displays the second temporary captured image determined by the display position determining unit. The display adjustment parameter is calculated based on the second displacement, which is the displacement between the position and the display position of the second temporary captured image before being changed by the display position changing means, and the first displacement.

  According to this configuration, both the temporary captured images of the first temporary captured image and the second temporary captured image (both images for adjustment) can be moved by the user's instructions (first input and second input). Thus, the user can adjust (adjust) the display positions of both images by moving both images for adjustment. Further, display adjustment parameters are calculated based on the movement (displacement) information of both images. For this reason, it is easier for the user to align the display positions of both images than when the first temporary captured image (one image for adjustment) is moved to match the display positions of both images for adjustment. Based on the result, the stereo image can be displayed at an appropriate display position.

  The calculation of the display adjustment parameter by the display adjustment parameter calculation unit may be performed as follows. In other words, the display adjustment parameter calculation means calculates the display adjustment parameter based on the relative displacement of the first displacement with respect to the second displacement.

  According to this configuration, both the temporary captured images of the first temporary captured image and the second temporary captured image (both images for adjustment) can be moved by a user instruction (first input and second input). The displacement used when calculating the adjustment parameter is a displacement in which the first temporary captured image is moved relative to the second temporary captured image (displacement of the other image with respect to one image for adjustment). As a result, the user can more easily adjust (adjust) the display positions of both images by moving both images when adjusting the display positions of both images for adjustment. Since the calculation of the parameter is sufficient when the second temporary captured image is fixed and only the first temporary captured image is moved (that is, when only one adjustment image is moved), the display adjustment can be easily performed. Parameters can be calculated.

  The calculation of the display adjustment parameter by the display adjustment parameter calculation unit may be performed as follows. That is, the display adjustment parameter calculation means calculates the display adjustment parameter based only on the vertical displacement of the first displacement.

  According to this configuration, the displacement (first displacement) when the first provisional captured image is moved includes the displacement in each direction, but the display adjustment parameter is calculated in the vertical direction (FIGS. 1 and 1 described later). Only the displacement in the y-axis direction indicated by 5 etc. is used. As a result, the positional deviation in the vertical direction of the stereo image (first image and second image), which is the main cause of being recognized as two shifted images, is corrected. Further, even if only the vertical displacement is used for calculating the display adjustment parameter, the direction in which the user can move the first provisional captured image (one image for adjustment) is not limited to the vertical direction. Thus, the user can substantially overlap the first temporary captured image and the second temporary captured image, and as a result, it is easy to align the vertical display positions of the first temporary captured image and the second temporary captured image (adjustment). Easy to do).

  The calculation of the display adjustment parameter by the display adjustment parameter calculation unit may be performed as follows. That is, the display adjustment parameter calculation means calculates the display adjustment parameter based only on the vertical displacement of the first displacement and the second displacement.

  According to this configuration, the displacement (first displacement and second displacement) when the first temporary captured image and the second temporary captured image are moved includes the displacement in each direction. Only the displacement in the vertical direction (the y-axis direction shown in FIGS. 1, 5, etc., which will be described later) of both temporary captured images is used. As a result, the positional deviation in the vertical direction of the stereo image (first image and second image), which is the main cause of being recognized as two shifted images, is corrected. Further, even when only the vertical displacement is used for the calculation of the display adjustment parameter, the direction in which the user can move the first temporary captured image and the second temporary captured image (both images for adjustment) is moved up and down. By being not limited to only the direction, the user can substantially superimpose the first temporary captured image and the second temporary captured image, and as a result, the vertical display position of the first temporary captured image and the second temporary captured image. Is easy to adjust (easy to adjust).

  Storage control by the storage control means, calculation of display adjustment parameters by the display adjustment parameter calculation means, and stereoscopic display control by the stereoscopic display control means may be performed as follows. That is, the storage control unit causes the storage unit to store the display position of the first temporarily captured image changed by the display position changing unit every time the input receiving unit receives the first input. When the input receiving unit receives the switching input, the display adjustment parameter calculating unit displays the display position of the first temporary captured image stored in the storage unit and the first temporary captured image before being changed by the display position changing unit. The display adjustment parameter is calculated based on the third displacement that is a displacement from the display position of the display position, and the stereoscopic display control means displays the first image and the second image on the display unit based on the calculated display adjustment parameter. Display stereoscopically on top.

  According to this configuration, every time the display position of the first provisional captured image (one image for adjustment) is changed by the user's instruction, the display position is stored in the storage unit, and the stored display position is displayed. Based on the calculated display adjustment parameter, a stereo image (first image and second image) is stereoscopically displayed based on the calculated display adjustment parameter. As a result, when the user is adjusting the display position of the first provisional captured image (one image for adjustment) (that is, during adjustment), if the current adjustment is being made, It is possible to actually confirm whether or not the stereoscopic display is performed, and it is possible to confirm whether or not there is no problem with the adjustment before the adjustment is finished (determined).

  Storage control by the storage control means, calculation of display adjustment parameters by the display adjustment parameter calculation means, and stereoscopic display control by the stereoscopic display control means may be performed as follows. That is, the storage control unit further causes the storage unit to store the display position of the second temporary captured image changed by the display position changing unit every time the input receiving unit receives the second input. When the input receiving unit receives the switching input, the display adjustment parameter calculating unit displays the display position of the second temporary captured image stored in the storage unit and the second temporary captured image before the change by the display position changing unit. The display adjustment parameter is calculated based on the fourth displacement and the third displacement, which are displacements with respect to the display position, and the stereoscopic display control means converts the first image and the second image to the calculated display adjustment parameter. Based on this, stereoscopic display is performed on the display unit.

  According to this configuration, every time the display position of the first temporary captured image and the second temporary captured image (both images for adjustment) is changed by a user instruction, the display position is stored in the storage unit, and the storage A display adjustment parameter is calculated based on the displayed display position, and a stereo image (first image and second image) is stereoscopically displayed based on the calculated display adjustment parameter. As a result, when the user is adjusting the display position of the first temporary captured image and the second temporary captured image (both images for adjustment) (that is, during the adjustment) In this way, it is possible to confirm whether or not the stereo image is stereoscopically displayed, and it is possible to confirm whether there is no problem with the adjustment before the adjustment is finished (determined).

  Display of the temporarily captured image by the temporarily captured image display means may be performed as follows. That is, the temporary captured image display means displays at least one of the first temporary captured image and the second temporary captured image in a semi-transparent display and displays the temporary captured image on the display unit. .

  According to this configuration, at least one temporary captured image of the first temporary captured image and the second temporary captured image (both images for adjustment) is displayed semi-transparently, and both images are displayed on the display. Therefore, the user can easily visually recognize the positional deviation between the two images. Furthermore, since both images for adjustment are easily visible, the user changes the display position of the first temporary captured image (one image for adjustment) to change the second temporary captured image (the other image for adjustment). The display position of (image) can be easily adjusted (adjusted) intuitively.

  Display of the temporarily captured image by the temporarily captured image display means may be performed as follows. That is, the temporary captured image display unit displays the first temporary captured image and the second temporary captured image in a plan view on the display unit.

  According to this configuration, the first temporary captured image and the second temporary captured image (both images for adjustment) are displayed in a plan view on the display (that is, both images are visually recognized by the user's right eye and left eye). Therefore, it is easy for the user to visually recognize the positional deviation between the two images for adjustment, and the adjustment of both images is facilitated.

  The storage unit may store in advance an initial parameter that defines a preset initial display position on the display unit of the first image and the second image. In this case, the storage control unit updates the initial parameter as the display parameter and stores it in the storage unit when the input receiving unit receives the initialization input.

  According to this configuration, the value of the display parameter is changed to the value of the initial parameter when the user issues an instruction for initialization (instruction for returning the display position of the stereo image to the initial setting). Thus, even when the user has changed (adjusted) the display position of the stereo image, it is possible to return to the default display position at any time.

  The change of the display position by the display position changing means may be performed as follows. That is, the display position changing unit changes the display position of at least one of the first temporary captured image and the second temporary captured image in the left-right direction and the vertical direction based on a predetermined input.

  This configuration is more effective when the first temporary captured image and the second temporary captured image (both images for adjustment) are displayed in plan view. In general, since a stereoscopically viewable image is an image having parallax, when both images for adjustment are displayed in a plan view, both images are displayed in the left-right direction (the x-axis direction shown in FIGS. 1 and 5 to be described later). ) Is displayed. For this reason, even if the shift due to the optical position of the stereo camera is only the shift in the vertical direction of both images (the y-axis direction shown in FIGS. Displayed out of position. In addition, the magnitude of the horizontal shift differs depending on the distance between the captured subject and the stereo camera. For this reason, it may be difficult for the user to make an adjustment to eliminate the vertical shift between the two images. However, according to this configuration, since at least one of the images for adjustment can also be moved in the left-right direction, the user need not consider the distance between the subject and the stereo camera, and can adjust the image in the up-down, left-right direction The images can be substantially superimposed by moving at least one of the images. As a result, the user can adjust (adjust) the vertical display positions of both images for adjustment.

  The case where the present invention is configured as an information processing program has been described above. However, the present invention may be configured as an imaging device, an imaging system, or an imaging method. Furthermore, the present invention may be configured as a computer-readable recording medium that records the information processing program.

  According to the present invention, even when the optical position of the image pickup unit is shifted due to an external impact or secular change during use, the user can easily display the display positions of the two images constituting the stereo image by himself / herself. It is possible to provide an information processing program and the like that can be adjusted.

Front view of game device 10 in the open state Left side view, front view, right side view, and rear view of game device 10 in the closed state Block diagram showing the internal configuration of the game apparatus 10 The figure which shows an example in which a stereo camera images the to-be-photographed object 50 The figure which shows an example in which an adjustment image is displayed The figure for demonstrating an example of a stereoscopic vision The figure which shows an example of the optical position shift of a stereo camera The figure which shows an example of the shift | offset | difference of the display position of an adjustment image The figure which shows an example when moving the display position of an adjustment image The figure which shows an example when moving the display position of an adjustment image The figure which shows an example of the memory map of the main memory 32 of the game device 10 Example of flowchart of display adjustment processing Example of flowchart of display adjustment processing The figure which shows an example of the shift | offset | difference of the display position of an adjustment image The figure which shows an example of the shift | offset | difference of the display position of an adjustment image

(One embodiment)
Hereinafter, a game apparatus which is an imaging apparatus according to an embodiment of the present invention will be described. Note that the present invention is not limited to such an apparatus, and may be an imaging system that realizes the functions of such an apparatus or an imaging method in such an apparatus. An information processing program executed in a simple apparatus may be used. Furthermore, the present invention may be a computer-readable recording medium on which the information processing program is recorded.

(Appearance structure of game device)
Hereinafter, a game device according to an embodiment of the present invention will be described. 1 and 2 are plan views showing the appearance of the game apparatus 10. The game apparatus 10 is a portable game apparatus, and is configured to be foldable as shown in FIGS. 1 and 2. FIG. 1 shows the game apparatus 10 in an open state (open state), and FIG. 2 shows the game apparatus 10 in a closed state (closed state). FIG. 1 is a front view of the game apparatus 10 in the open state. The game apparatus 10 can capture an image with an imaging unit, display the captured image on a screen, and store data of the captured image. In addition, the game device 10 can be stored in a replaceable memory card or can execute a game program received from a server or another game device, such as an image captured by a virtual camera set in a virtual space. An image generated by computer graphics processing can be displayed on the screen.

  First, an external configuration of the game apparatus 10 will be described with reference to FIGS. 1 and 2. As shown in FIGS. 1 and 2, the game apparatus 10 includes a lower housing 11 and an upper housing 21. The lower housing 11 and the upper housing 21 are connected so as to be openable and closable (foldable).

(Description of lower housing)
First, the configuration of the lower housing 11 will be described. As shown in FIGS. 1 and 2, the lower housing 11 has a lower LCD (Liquid Crystal Display) 12, a touch panel 13, operation buttons 14A to 14L, an analog stick 15, LEDs 16A to 16B, and an insertion. A mouth 17 and a microphone hole 18 are provided. Details of these will be described below.

  As shown in FIG. 1, the lower LCD 12 is housed in the lower housing 11. The number of pixels of the lower LCD 12 may be, for example, 320 dots × 240 dots (horizontal × vertical). Unlike the upper LCD 22 described later, the lower LCD 12 is a display device that displays an image in a planar manner (not stereoscopically viewable). In the present embodiment, an LCD is used as the display device, but other arbitrary display devices such as a display device using EL (Electro Luminescence) may be used. Further, as the lower LCD 12, a display device having an arbitrary resolution can be used.

  As shown in FIG. 1, the game apparatus 10 includes a touch panel 13 as an input device. The touch panel 13 is mounted on the screen of the lower LCD 12. In the present embodiment, the touch panel 13 is a resistive film type touch panel. However, the touch panel is not limited to the resistive film type, and any type of touch panel such as a capacitance type can be used. In the present embodiment, the touch panel 13 having the same resolution (detection accuracy) as that of the lower LCD 12 is used. However, the resolution of the touch panel 13 and the resolution of the lower LCD 12 do not necessarily match. An insertion port 17 (dotted line shown in FIGS. 1 and 2D) is provided on the upper side surface of the lower housing 11. The insertion slot 17 can accommodate a touch pen 28 used for performing an operation on the touch panel 13. In addition, although the input with respect to the touchscreen 13 is normally performed using the touch pen 28, it is also possible to input with respect to the touchscreen 13 not only with the touch pen 28 but with a user's finger | toe.

  Each operation button 14A-14L is an input device for performing a predetermined input. As shown in FIG. 1, on the inner surface (main surface) of the lower housing 11, among the operation buttons 14A to 14L, a cross button 14A (direction input button 14A), a button 14B, a button 14C, a button 14D, A button 14E, a power button 14F, a select button 14J, a HOME button 14K, and a start button 14L are provided. The cross button 14 </ b> A has a cross shape, and has buttons for instructing up, down, left, and right directions. Functions according to a program executed by the game apparatus 10 are appropriately assigned to the buttons 14A to 14E, the select button 14J, the HOME button 14K, and the start button 14L.

  The analog stick 15 is a device that indicates a direction. The analog stick 15 is configured such that its key top slides parallel to the inner surface of the lower housing 11. The analog stick 15 functions according to a program executed by the game apparatus 10. As the analog stick 15, an analog stick that allows analog input by being tilted by a predetermined amount in any direction of up / down / left / right and oblique directions may be used.

  A microphone hole 18 is provided on the inner surface of the lower housing 11. A microphone 42 (see FIG. 3), which will be described later, is provided below the microphone hole 18, and the microphone 42 detects sound outside the game apparatus 10.

  2A is a left side view of the game apparatus 10 in the closed state, FIG. 2B is a front view of the game apparatus 10 in the closed state, and FIG. 2C is a view of the game apparatus 10 in the closed state. FIG. 2D is a right side view, and FIG. 2D is a rear view of the game apparatus 10 in the closed state. As shown in FIGS. 2B and 2D, an L button 14 </ b> G and an R button 14 </ b> H are provided on the upper side surface of the lower housing 11. The L button 14G and the R button 14H can function as shutter buttons (shooting instruction buttons) of the imaging unit, for example. Further, as shown in FIG. 2A, a volume button 14 </ b> I is provided on the left side surface of the lower housing 11. The volume button 14I is used to adjust the volume of a speaker provided in the game apparatus 10.

  As shown in FIG. 2A, a cover portion 11 </ b> C that can be opened and closed is provided on the left side surface of the lower housing 11. A connector (not shown) for electrically connecting the game apparatus 10 and the data storage external memory 45 is provided inside the cover portion 11C. The data storage external memory 45 is detachably attached to the connector. The data storage external memory 45 is used, for example, for storing (saving) data of an image captured by the game apparatus 10.

  As shown in FIG. 2D, an insertion port 11D for inserting the game apparatus 10 and an external memory 44 in which a game program is recorded is provided on the upper side surface of the lower housing 11, and the insertion port Inside the 11D, a connector (not shown) for electrically detachably connecting to the external memory 44 is provided. When the external memory 44 is connected to the game apparatus 10, a predetermined game program is executed.

  Further, as shown in FIGS. 1 and 2C, on the lower side surface of the lower housing 11, the first LED 16A for notifying the user of the power ON / OFF state of the game apparatus 10 and the right side surface of the lower housing 11 Is provided with a second LED 16B for notifying the user of the wireless communication establishment status of the game apparatus 10. The game apparatus 10 can perform wireless communication with other devices, and the second LED 16B lights up when wireless communication is established. The game apparatus 10 has a function of connecting to a wireless LAN, for example, by a method compliant with the IEEE 802.11b / g standard. A wireless switch 19 for enabling / disabling this wireless communication function is provided on the right side surface of the lower housing 11 (see FIG. 2C).

  Although not shown, the lower housing 11 stores a rechargeable battery that serves as a power source for the game apparatus 10, and the terminal is provided via a terminal provided on a side surface (for example, the upper side surface) of the lower housing 11. The battery can be charged.

(Description of upper housing)
Next, the configuration of the upper housing 21 will be described. As shown in FIGS. 1 and 2, the upper housing 21 has an upper LCD (Liquid Crystal Display) 22, an outer imaging unit 23 (an outer imaging unit (left) 23a and an outer imaging unit (right) 23b). The inner imaging unit 24, the 3D adjustment switch 25, and the 3D indicator 26 are provided. Details of these will be described below.

  As shown in FIG. 1, the upper LCD 22 is accommodated in the upper housing 21. The number of pixels of the upper LCD 22 may be, for example, 800 dots × 240 dots (horizontal × vertical). In the present embodiment, the upper LCD 22 is a liquid crystal display device. However, for example, a display device using EL (Electro Luminescence) may be used. In addition, a display device having an arbitrary resolution can be used as the upper LCD 22.

  The upper LCD 22 is a display device capable of displaying a stereoscopically visible image (stereoscopic image). In the present embodiment, the left-eye image and the right-eye image are displayed using substantially the same display area. Specifically, the display device uses a method in which a left-eye image and a right-eye image are alternately displayed in a horizontal direction in a predetermined unit (for example, one column at a time). Or the display apparatus of the system by which the image for left eyes and the image for right eyes are alternately displayed by a time division may be sufficient. Further, in the present embodiment, the display device is capable of autostereoscopic viewing. Then, a lenticular method or a parallax barrier method (parallax barrier method) is used so that the left-eye image and the right-eye image alternately displayed in the horizontal direction appear to be decomposed into the left eye and the right eye, respectively. In the present embodiment, the upper LCD 22 is a parallax barrier type. The upper LCD 22 displays an image (stereoscopic image) that can be stereoscopically viewed with the naked eye, using the right-eye image and the left-eye image. That is, the upper LCD 22 uses the parallax barrier to visually recognize the left-eye image for the user's left eye and the right-eye image for the user's right eye, thereby generating a stereoscopic image (stereoscopic image) that has a stereoscopic effect for the user. Can be displayed. Further, the upper LCD 22 can invalidate the parallax barrier. When the parallax barrier is invalidated, the upper LCD 22 can display an image in a planar manner (in the sense opposite to the above-described stereoscopic view, the planar LCD (This is a display mode in which the same displayed image can be seen by both the right eye and the left eye). Thus, the upper LCD 22 is a display device capable of switching between a stereoscopic display mode for displaying a stereoscopically viewable image and a planar display mode for displaying an image in a planar manner (planar view display). This display mode switching is performed by a 3D adjustment switch 25 described later.

  The outer imaging unit 23 is a general term for two imaging units (23 a and 23 b) provided on the outer surface (back surface opposite to the main surface on which the upper LCD 22 is provided) 21 D of the upper housing 21. In the present embodiment, the outer imaging unit 23 includes two imaging units, an outer imaging unit (left) 23a and an outer imaging unit (right) 23b. The imaging directions of the outer imaging unit (left) 23a and the outer imaging unit (right) 23b are both outward normal directions (z-axis positive direction shown in FIG. 1) of the outer surface 21D. That is, the imaging direction of the outer imaging unit (left) 23a and the imaging direction of the outer imaging unit (right) 23b are parallel. The outer imaging unit (left) 23a and the outer imaging unit (right) 23b can be used as a stereo camera by a program executed by the game apparatus 10. Further, depending on the program, it is possible to use one of the two outer imaging units (23a and 23b) alone and use the outer imaging unit 23 as a non-stereo camera. Further, depending on the program, it is possible to perform imaging with an expanded imaging range by combining or complementarily using images captured by the two outer imaging units (23a and 23b). The outer imaging unit (left) 23a and the outer imaging unit (right) 23b each include an imaging element (for example, a CCD image sensor or a CMOS image sensor) having a predetermined common resolution and a lens. The lens may have a zoom mechanism.

  As shown by the broken line in FIG. 1 and the solid line in FIG. 2B, the outer imaging unit (left) 23a and the outer imaging unit (right) 23b that constitute the outer imaging unit 23 are arranged in the horizontal direction ( They are arranged in parallel with the x-axis direction shown in FIG. That is, the outer imaging unit (left) 23a and the outer imaging unit (right) 23b are arranged so that a straight line connecting the two imaging units is parallel to the horizontal direction of the screen of the upper LCD 22. 1 indicate the outer imaging unit (left) 23a and the outer imaging unit (right) 23b existing on the outer surface opposite to the inner surface of the upper housing 21, respectively. As shown in FIG. 1, when the user views the screen of the upper LCD 22 from the front, the outer imaging unit (left) 23a is positioned on the left side and the outer imaging unit (right) 23b is positioned on the right side. When a program that causes the outer imaging unit 23 to function as a stereo camera is executed, the outer imaging unit (left) 23a captures an image for the left eye that is visually recognized by the user's left eye, and the outer imaging unit (right) 23b A right-eye image that is visually recognized by the user's right eye is captured. The interval between the outer imaging unit (left) 23a and the outer imaging unit (right) 23b is set to be approximately the interval between both eyes of the human, and may be set in a range of 30 mm to 70 mm, for example. In addition, the space | interval of the outer side imaging part (left) 23a and the outer side imaging part (right) 23b is not restricted to this range.

  In the present embodiment, the outer imaging unit (left) 23a and the outer imaging unit (right) 23 are fixed to the housing, and the imaging direction cannot be changed.

  The outer imaging unit (left) 23a and the outer imaging unit (right) 23b are respectively arranged at positions symmetrical from the center with respect to the left-right direction (x-axis direction shown in FIG. 1) of the upper LCD 22 (upper housing 21). . That is, the outer imaging unit (left) 23a and the outer imaging unit (right) 23b are respectively arranged at symmetrical positions with respect to a line dividing the upper LCD 22 into two equal parts. In addition, the outer imaging unit (left) 23a and the outer imaging unit (right) 23b are on the upper side of the upper housing 21 in a state where the upper housing 21 is opened, and on the back side of the upper side of the screen of the upper LCD 22. Placed in. That is, the outer imaging unit (left) 23a and the outer imaging unit (right) 23b are the outer surfaces of the upper housing 21, and when the upper LCD 22 is projected on the outer surface, the upper image 22 is higher than the upper end of the projected upper LCD 22 screen. Placed in.

  As described above, the two image pickup units (23a and 23b) of the outer image pickup unit 23 are arranged at symmetrical positions from the center with respect to the left-right direction of the upper LCD 22, so that when the user views the upper LCD 22 from the front, the outer image pickup is performed. The imaging direction of the unit 23 can be matched with the user's line-of-sight direction. Further, since the outer imaging unit 23 is disposed at a position on the back side above the upper end of the screen of the upper LCD 22, the outer imaging unit 23 and the upper LCD 22 do not interfere inside the upper housing 21. Accordingly, it is possible to make the upper housing 21 thinner than in the case where the outer imaging unit 23 is disposed on the back side of the screen of the upper LCD 22.

  The inner imaging unit 24 is an imaging unit that is provided on the inner side surface (main surface) 21B of the upper housing 21 and uses the inward normal direction (z-axis negative direction shown in FIG. 1) of the inner side surface as the imaging direction. . The inner imaging unit 24 includes an imaging element (for example, a CCD image sensor or a CMOS image sensor) having a predetermined resolution, and a lens. The lens may have a zoom mechanism.

  The 3D adjustment switch 25 is a slide switch, and is a switch used to switch the display mode of the upper LCD 22 as described above. The 3D adjustment switch 25 is used to adjust the stereoscopic effect of the stereoscopically visible image (stereoscopic image) displayed on the upper LCD 22. The slider 25a of the 3D adjustment switch 25 can be slid to an arbitrary position in a predetermined direction (vertical direction), and the display mode of the upper LCD 22 is set according to the position of the slider 25a. Further, the appearance of the stereoscopic image is adjusted according to the position of the slider 25a.

  The 3D indicator 26 indicates whether or not the upper LCD 22 is in the stereoscopic display mode. The 3D indicator 26 is an LED, and lights up when the stereoscopic display mode of the upper LCD 22 is valid. Note that the 3D indicator 26 may be lit only when the upper LCD 22 is in the stereoscopic display mode and a program process for displaying a stereoscopic image is being executed.

  A speaker hole 21 </ b> E is provided on the inner surface of the upper housing 21. Sound from a speaker 43 described later is output from the speaker hole 21E.

(Internal configuration of game device 10)
Next, with reference to FIG. 3, an internal electrical configuration of the game apparatus 10 will be described. FIG. 3 is a block diagram showing an internal configuration of the game apparatus 10. As shown in FIG. 3, in addition to the above-described units, the game apparatus 10 includes an information processing unit 31, a main memory 32, an external memory interface (external memory I / F) 33, an external memory I / F 34 for data storage, data It includes electronic components such as a storage internal memory 35, a wireless communication module 36, a local communication module 37, a real time clock (RTC) 38, an acceleration sensor 39, a power supply circuit 40, and an interface circuit (I / F circuit) 41. These electronic components are mounted on an electronic circuit board and housed in the lower housing 11 (or the upper housing 21).

  The information processing unit 31 is information processing means including a CPU (Central Processing Unit) 311 for executing a predetermined program, a GPU (Graphics Processing Unit) 312 for performing image processing, and the like. The CPU 311 of the information processing unit 31 executes the program stored in a memory (for example, the external memory 44 connected to the external memory I / F 33 or the data storage internal memory 35) in the game apparatus 10, thereby executing the program. The process according to is executed. Note that the program executed by the CPU 311 of the information processing unit 31 may be acquired from another device through communication with the other device. The information processing unit 31 includes a VRAM (Video RAM) 313. The GPU 312 of the information processing unit 31 generates an image in response to a command from the CPU 311 of the information processing unit 31 and draws it on the VRAM 313. Then, the GPU 312 of the information processing unit 31 outputs the image drawn in the VRAM 313 to the upper LCD 22 and / or the lower LCD 12, and the image is displayed on the upper LCD 22 and / or the lower LCD 12.

  A main memory 32, an external memory I / F 33, a data storage external memory I / F 34, and a data storage internal memory 35 are connected to the information processing unit 31. The external memory I / F 33 is an interface for detachably connecting the external memory 44. The data storage external memory I / F 34 is an interface for detachably connecting the data storage external memory 45.

  The main memory 32 is a volatile storage unit used as a work area or a buffer area of the information processing unit 31 (the CPU 311). That is, the main memory 32 temporarily stores various data used for the processing based on the program, or temporarily stores a program acquired from the outside (such as the external memory 44 or another device). In the present embodiment, for example, a PSRAM (Pseudo-SRAM) is used as the main memory 32.

  The external memory 44 is a nonvolatile storage unit for storing a program executed by the information processing unit 31. The external memory 44 is composed of, for example, a read-only semiconductor memory. When the external memory 44 is connected to the external memory I / F 33, the information processing section 31 can read a program stored in the external memory 44. A predetermined process is performed by executing the program read by the information processing unit 31. The data storage external memory 45 is composed of a non-volatile readable / writable memory (for example, a NAND flash memory), and is used for storing predetermined data. For example, the data storage external memory 45 stores an image captured by the outer imaging unit 23 or an image captured by another device. When the data storage external memory 45 is connected to the data storage external memory I / F 34, the information processing section 31 reads an image stored in the data storage external memory 45 and applies the image to the upper LCD 22 and / or the lower LCD 12. An image can be displayed.

  The data storage internal memory 35 is configured by a readable / writable nonvolatile memory (for example, a NAND flash memory), and is used for storing predetermined data and programs. For example, the data storage internal memory 35 stores initial parameters and display adjustment parameters, which will be described later, for adjusting the display positions of the left-eye image and the right-eye image when the outer imaging unit 23 functions as a stereo camera. .

  The wireless communication module 36 has a function of connecting to a wireless LAN by a method compliant with, for example, the IEEE 802.11b / g standard. Further, the local communication module 37 has a function of performing wireless communication with the same type of game device by a predetermined communication method (for example, communication using a unique protocol or infrared communication). The wireless communication module 36 and the local communication module 37 are connected to the information processing unit 31. The information processing unit 31 transmits / receives data to / from other devices via the Internet using the wireless communication module 36, and transmits / receives data to / from other game devices of the same type using the local communication module 37. You can do it.

  An acceleration sensor 39 is connected to the information processing unit 31. The acceleration sensor 39 detects the magnitude of linear acceleration (linear acceleration) along the three-axis (xyz-axis) direction. The acceleration sensor 39 is provided inside the lower housing 11. As shown in FIG. 1, the acceleration sensor 39 is configured such that the long side direction of the lower housing 11 is the x axis, the short side direction of the lower housing 11 is the y axis, and the inner side surface (main surface) of the lower housing 11. With the vertical direction as the z axis, the magnitude of linear acceleration on each axis is detected. The acceleration sensor 39 is, for example, an electrostatic capacitance type acceleration sensor, but other types of acceleration sensors may be used. The acceleration sensor 39 may be an acceleration sensor that detects a uniaxial or biaxial direction. The information processing unit 31 can detect data indicating the acceleration detected by the acceleration sensor 39 (acceleration data) and detect the attitude and movement of the game apparatus 10. In addition to (or instead of) the acceleration sensor 39 described above, another sensor such as an angle sensor or an angular velocity sensor is connected to the information processing unit 31, and the attitude and movement of the game apparatus 10 are detected by this sensor. May be.

  Further, the RTC 38 and the power supply circuit 40 are connected to the information processing unit 31. The RTC 38 counts the time and outputs it to the information processing unit 31. The information processing unit 31 calculates the current time (date) based on the time counted by the RTC 38. The power supply circuit 40 controls power from a power source (the rechargeable battery housed in the lower housing 11) of the game apparatus 10 and supplies power to each component of the game apparatus 10.

  The information processing unit 31 is connected to the LEDs 16 (16A, 16B). Using the LED 16, the information processing unit 31 notifies the user of the power ON / OFF status of the game apparatus 10 and notifies the user of the wireless communication establishment status of the game apparatus 10.

  In addition, an I / F circuit 41 is connected to the information processing unit 31. A microphone 42 and a speaker 43 are connected to the I / F circuit 41. Specifically, a speaker 43 is connected to the I / F circuit 41 via an amplifier (not shown). The microphone 42 detects the user's voice and outputs a voice signal to the I / F circuit 41. The amplifier amplifies the audio signal from the I / F circuit 41 and outputs the audio from the speaker 43. The touch panel 13 is connected to the I / F circuit 41. The I / F circuit 41 includes a voice control circuit that controls the microphone 42 and the speaker 43 (amplifier), and a touch panel control circuit that controls the touch panel. The audio control circuit performs A / D conversion and D / A conversion on the audio signal, or converts the audio signal into audio data of a predetermined format. The touch panel control circuit generates touch position data in a predetermined format based on a signal from the touch panel 13 and outputs it to the information processing unit 31. The touch position data indicates the coordinates of the position where the input is performed on the input surface of the touch panel 13. The touch panel control circuit reads signals from the touch panel 13 and generates touch position data at a rate of once per predetermined time. The information processing unit 31 can know the position where the input is performed on the touch panel 13 by acquiring the touch position data.

  The operation button 14 includes the operation buttons 14 </ b> A to 14 </ b> L and is connected to the information processing unit 31. From the operation button 14 to the information processing section 31, operation data indicating the input status (whether or not the button is pressed) for each of the operation buttons 14A to 14I is output. The information processing unit 31 acquires the operation data from the operation button 14 to execute processing according to the input to the operation button 14.

  The analog stick 15 is connected to the information processing unit 31. Operation data indicating analog input (operation direction and operation amount) to the analog stick 15 is output from the analog stick 15 to the information processing unit 31. The information processing unit 31 acquires operation data from the analog stick 15 to execute processing according to the input to the analog stick 15.

  The lower LCD 12 and the upper LCD 22 are connected to the information processing unit 31. The lower LCD 12 and the upper LCD 22 display images according to instructions from the information processing unit 31 (the GPU 312). In the present embodiment, the information processing section 31 displays a stereoscopic image (a stereoscopically viewable image) on the upper LCD 22.

  Specifically, the information processing unit 31 is connected to an LCD controller (not shown) of the upper LCD 22 and controls ON / OFF (valid / invalid) of the parallax barrier for the LCD controller. When the parallax barrier of the upper LCD 22 is ON, the right-eye image and the left-eye image stored in the VRAM 313 of the information processing unit 31 are output to the upper LCD 22. More specifically, the LCD controller alternately repeats the process of reading pixel data for one line in the vertical direction for the image for the right eye and the process of reading pixel data for one line in the vertical direction for the image for the left eye. Thus, the right-eye image and the left-eye image are read from the VRAM 313. As a result, the right-eye image and the left-eye image are divided into strip-like images in which pixels are arranged vertically for each line, and the striped right-eye image and the left-eye image are alternately displayed. The image arranged on the upper LCD 22 is displayed on the screen. Then, when the user visually recognizes the image through the parallax barrier of the upper LCD 22, the right eye image is visually recognized by the user's right eye and the left eye image is visually recognized by the user's left eye. As a result, a stereoscopically viewable image is displayed on the screen of the upper LCD 22.

  The outer imaging unit 23 and the inner imaging unit 24 are connected to the information processing unit 31. The outer imaging unit 23 and the inner imaging unit 24 capture an image in accordance with an instruction from the information processing unit 31, and output the captured image data to the information processing unit 31.

  The 3D adjustment switch 25 is connected to the information processing unit 31. The 3D adjustment switch 25 transmits an electrical signal corresponding to the position of the slider 25 a to the information processing unit 31.

  The 3D indicator 26 is connected to the information processing unit 31. The information processing unit 31 controls lighting of the 3D indicator 26. For example, the information processing section 31 turns on the 3D indicator 26 when the upper LCD 22 is in the stereoscopic display mode. The above is the description of the internal configuration of the game apparatus 10.

(Shooting adjustment images with a stereo camera)
Next, with reference to FIG. 4 and FIG. 5, the adjustment image capturing by the outer imaging unit 23 functioning as a stereo camera will be described. Here, the adjustment image is an image including an image of an arbitrary subject (for example, the subject 50) and the display position of which is adjusted by an adjustment mechanism described later. In the following description, the outer imaging unit (left) 23a is the left stereo camera 23a (an example of the first imaging unit of the present invention), and the outer imaging unit (right) 23b is the right stereo camera 23b (the second imaging of the present invention). An example of a part). FIG. 4 is a diagram illustrating a state in which the left stereo camera 23a and the right stereo camera 23b capture the subject 50, and FIG. 5 illustrates an adjustment image (left-eye adjustment image including an image of the subject 50) captured by both stereo cameras. 50a (an example of the first provisional captured image of the present invention) and an adjustment image for right eye 50b (an example of the second provisional captured image of the present invention) are displayed on the display screen of the upper LCD 22 in an overlapping manner. .

  As shown in FIG. 4, the left stereo camera 23a, the right stereo camera 23b, and the subject 50 are arranged in the same three-dimensional space as the three-dimensional orthogonal coordinate space (xyz coordinate space) shown in FIG. More specifically, the left stereo camera 23a and the right stereo camera 23b are arranged on the x axis so that the midpoint of the line segment that connects the centers of both stereo cameras is the origin. Also, the z axis is defined in a direction parallel to the imaging directions 60a and 60b (optical axis direction) of both stereo cameras, and the y axis is defined in the upward direction orthogonal to both the x axis and the z axis. . The subject 50 is arranged on the z axis parallel to the imaging direction of both stereo cameras. The left stereo camera 23a captures the subject 50 and generates a left-eye adjustment image 50a, and the right stereo camera 23b captures the subject 50 and generates a right-eye adjustment image 50b. At this time, when the left-eye adjustment image 50a and the right-eye adjustment image 50b including the image of the subject 50 are simultaneously displayed on a predetermined display surface (display screen on the upper LCD 22) arranged on the xy plane, the subjects in both images The display position of 50 is displayed with a predetermined shift in the x-axis direction (see FIG. 5). The amount of deviation is parallax, and by turning on the parallax barrier, the left-eye adjustment image 50a is displayed (visualized) and the right-eye adjustment image 50b is displayed (visualized). ), The subject 50 is visually recognized as a stereoscopic image having a stereoscopic effect (that is, recognized as an image popping forward or an image retracted to the back).

  More specifically, as shown in FIG. 5, the adjustment image displayed on the upper LCD 22 is that the left-eye adjustment image 50a (the display position of the subject 50) is the right-eye adjustment image 50b (the display position of the subject 50). ) Relative to the right direction (x-axis positive direction). In this case, the user recognizes the subject 50 as if it has jumped out from the predetermined display surface by visually recognizing both images by cross-view (see FIG. 6). When the left-eye adjustment image 50a (the display position of the subject 50 in the right eye) is shifted relative to the left direction (x-axis negative direction) relative to the right-eye adjustment image 50b (the display position of the subject 50). The user visually recognizes both images by parallel viewing, thereby recognizing that the subject 50 is retracted from the predetermined display surface, and adjusts the left-eye adjustment image 50a and the right-eye adjustment image 50b (the subject 50 If the display positions of the two images coincide with each other, the user recognizes that the subject 50 is located on a predetermined display surface (not shown). As described above, the right-eye adjustment image that is similarly shifted by a predetermined amount of parallax in the x-axis direction (left-right direction) with respect to the right eye and the left eye (see FIG. 6) that are spaced apart in the x-axis direction (left-right direction). By visually recognizing 50b and the left-eye adjustment image 50a, the user can synthesize both images in the brain and recognize them as a stereoscopic image. That is, the shift (that is, parallax) between the two images in the x-axis direction has a large meaning in generating a stereoscopic image. By the way, since this parallax is different for each individual, the parallax (three-dimensional effect) can be adjusted by the 3D adjustment switch 25. On the other hand, when both images shifted in the y-axis direction (vertical direction) on the predetermined display surface (the display screen of the upper LCD 22) are visually recognized by the respective eyes of the user, the user appropriately views in the brain. It becomes difficult to synthesize as an image, and may be recognized as two shifted images. This is probably because the left and right eyes of the human have the same vertical viewing direction. That is, the shift between both images in the y-axis direction may cause a problem when displaying a stereoscopic image. Hereinafter, the cause of the deviation between both images in the y-axis direction (vertical direction) will be described.

  In general, it is desirable that the left stereo camera 23a and the right stereo camera 23b are precisely aligned in optical positions such as the mounting position and the optical axis. However, even if the optical position is suppressed within a predetermined error at the time of sale, the optical position may deviate beyond the predetermined error due to an external impact or aging change during use. This deviation includes, for example, translational deviations Δx, Δy, Δz parallel to the x-axis, y-axis, z-axis, rotational deviations Δa, Δb, Δc around the x-axis direction, y-axis direction, z-axis direction, or There is a shift due to the difference in the angle of view of the lens (see FIG. 4).

  Here, for example, with respect to the left stereo camera 23a as a reference, when the right stereo camera 23b (its optical position) is shifted relative to the left stereo camera 23a, the two images displayed on the display surface are shifted. Consider. When the right stereo camera 23b undergoes a translational deviation by Δx in the x-axis direction and when a rotational deviation occurs by Δb around the y-axis direction, the right-eye adjustment image 50b causes a deviation in the x-axis direction. When the right stereo camera 23b undergoes a translational shift by Δy in the y-axis direction and when a rotational shift occurs by Δa around the x-axis direction, the right-eye adjustment image 50b causes a shift in the y-axis direction. When the right stereo camera 23b is translated by Δz in the z-axis direction, or when the angle of view is shifted due to the shift of the focal length of the lens, the right-eye adjustment image 50b is shifted to the size displayed on the display surface. (Enlarge or reduce). In addition, when the right stereo camera 23b is rotationally shifted by Δc around the z-axis direction, the right-eye adjustment image 50b is rotationally shifted on the display surface.

  As described above, when recognizing a stereoscopic image, if there is a deviation between both images in the y-axis direction (vertical direction when viewed from the user), the user combines both images in the brain and recognizes them as a stereoscopic image. When trying to do so, it may be recognized as two shifted images, and stereoscopic viewing may become difficult. Therefore, when a deviation occurs in the optical positions of both stereo cameras, adjustment (correction) to eliminate the deviation in the y-axis direction becomes important. However, in the conventional technique for correcting not only the displacement in the y-axis direction but also other displacements, it is necessary to prepare a pattern image for adjustment for the correction, and even if the manufacturer can make adjustments, It was difficult for the user himself to make adjustments. In the following, an adjustment mechanism that can be adjusted by the user himself, which is characteristic in the present embodiment, will be described.

(Characteristic adjustment mechanism of this embodiment)
FIG. 7 shows a state in which the right stereo camera 23b (optical position thereof) is shifted relative to the left stereo camera 23a (optical position thereof). FIG. 7 (1) shows the right stereo camera. FIG. 7B shows a state in which the translation shift is caused by Δy in the positive y-axis direction, and FIG. 7B shows a state in which the right stereo camera 23b is rotationally shifted by Δa clockwise in the positive x-axis direction. Show.

  As shown in (1) of FIG. 7, when the right stereo camera 23b has a translational shift of Δy in the positive y-axis direction with respect to the left stereo camera 23a, the right-eye adjustment image generated by the right stereo camera 23b is generated. 50b is displayed with a predetermined amount shifted in the y-axis negative direction (downward) with respect to the left-eye adjustment image 50a (see FIG. 8). The predetermined amount is an amount determined according to the z coordinate (that is, the depth distance from the stereo camera to the subject 50) where the subject 50 is located and the magnitude of Δy. Similarly, as shown in (2) of FIG. 7, the right stereo camera 23b is also rotated by Δa clockwise in the positive x-axis direction with respect to the left stereo camera 23a. The right-eye adjustment image 50b generated by the camera 23b is displayed with a predetermined amount shifted from the left-eye adjustment image 50a in the negative y-axis direction (downward) (see FIG. 8). The predetermined amount is an amount determined according to the magnitude of the rotation angle Δa.

  As described above, when the right-eye adjustment image 50b is displaced in the y-axis negative direction (downward) with respect to the left-eye adjustment image 50a, the left-eye adjustment image 50a is turned on as a stereoscopic image with the parallax barrier turned on. When the right eye adjustment image 50b is visually recognized by the left eye, the user recognizes it as two shifted images. For this reason, it is necessary to make an adjustment to eliminate the shift in the display positions of both images that have been shifted after purchase. In this embodiment, the user can make this adjustment. Specifically, when the user images an arbitrary subject (subject 50) using a stereo camera, one of the left eye adjustment image 50a and the right eye adjustment image 50b (here, the right eye adjustment image 50b). ) Is translucently displayed and the parallax barrier is turned off, so that both images are displayed as a still image on the display surface in plan view. That is, the user can visually recognize the right-eye adjustment image 50b and the left-eye adjustment image 50a simultaneously as still images with both eyes, and adjust (move) these two still images in the vertical and horizontal directions as will be described later. Thus, it is possible to make an adjustment that eliminates the displacement of the display position. In the present embodiment, a planar view is displayed as a still image so that the user can easily adjust. However, the present invention can also be applied to a moving image (so-called preview image). By the way, since both images are usually images having parallax, both images displayed in plan view are visually recognized as images shifted in the x-axis direction (left-right direction). Here, since the shift in the left-right direction is a shift for generating parallax (giving a stereoscopic effect), it affects the stereoscopic effect when stereoscopically viewed. Has no significant effect. Further, the shift in the left-right direction can be adjusted by, for example, the 3D adjustment switch 25 that adjusts the parallax. On the other hand, if there is a shift in the y-axis direction (vertical direction), the right eye and the left eye are in a state where different heights are visually recognized, which affects the stereoscopic vision itself. Therefore, it is important for the user to make an adjustment to eliminate the shift between both images shifted in the y-axis direction (vertical direction).

  The user matches the display position of the subject 50 in the y-axis direction (vertical direction) in the right-eye adjustment image 50b displayed semi-transparently with the display position of the subject 50 in the y-axis direction in the left-eye adjustment image 50a. Adjustment for eliminating the displacement in the y-axis direction is performed by moving the adjustment image 50b. Since this is a process of adjusting while visually recognizing the deviation of both images in the y-axis direction, the process is intuitive and easy for the user, and can be easily adjusted. Specifically, when an adjustment mode for eliminating image displacement (display adjustment mode) is entered, a display screen for display adjustment is displayed on the lower LCD 12. The user can move the right-eye adjustment image 50b, which is a translucent image, up and down (in the y-axis direction) by touching the up / down button 70 displayed on the lower LCD 12 with the touch pen 28 on this screen (FIG. 5). 9). The user operates the touch pen 28 while observing both images displayed on the upper LCD 22 to adjust the left-eye adjustment image 50a and the right-eye adjustment image 50b so as not to shift in the y-axis direction. As a result, this adjustment result (the amount of displacement in the y-axis direction of the adjustment image 50b for the right eye before adjustment and the adjustment image 50b for the right eye after adjustment) is stored in the main memory 32, and then captured by both stereo cameras. When the stereo images (both images) are stereoscopically displayed, the display position of the stereo image (deviation between both images) is corrected based on the adjustment result. As a result, an appropriate stereoscopic display that the user does not recognize as two shifted images is performed.

  By the way, as described above, both the left-eye adjustment image 50a and the right-eye adjustment image 50b are images that have parallax for stereoscopic display, and thus are shifted in the x-axis direction. As a result, the user cannot make the display position of the subject 50 in both images completely match when attempting to make an adjustment that eliminates the deviation of both images in the y-axis direction. It is difficult to make adjustments that eliminate the displacement in the y-axis direction. In such a case, the right-eye adjustment image 50b displayed semi-transparently may be movable in the x-axis direction (left-right direction). That is, as shown in FIG. 10, the user moves the right-eye adjustment image 50b, which is a translucent image, in the left-right direction (x-axis direction) by touching the left-right button 71 displayed on the lower LCD 12. Can do. Thus, by enabling the right-eye adjustment image 50b to move in both the x-axis direction and the y-axis direction, the user can display the imaging target (subject 50) in the right-eye adjustment image 50b and the left-eye adjustment image 50a. The positions can be substantially overlapped. As a result, the right-eye adjustment image 50b and the left-eye adjustment image 50a are accurately adjusted so that there is no shift in the y-axis direction. In addition, this makes it possible to make adjustments to eliminate the deviation of both images in the y-axis direction regardless of what the subject 50 is, so that the user can capture an arbitrary subject as an adjustment image. Even in this case, the correction of the display positions of both images is performed by adjusting the y-axis direction of the adjustment results (the y-axis of the adjustment image 50b for the right eye before adjustment and the adjustment image 50b for the right eye after adjustment). Based on the amount of displacement in the direction).

(Details of information processing)
Next, a process of performing imaging with a stereo camera (hereinafter referred to as “stereo camera imaging process”) and a process of adjusting the display position (shift between both images) of the stereo image by the adjustment mechanism described above (hereinafter referred to as “display adjustment process”). The details of the information processing performed when it is executed will be described. First, data stored in the main memory 32 during the stereo camera imaging process and the display adjustment process will be described.

(Memory map)
FIG. 11 is a diagram illustrating an example of a memory map of the main memory 32 of the game apparatus 10. As shown in FIG. 11, the main memory 32 includes a program storage area 400 and a data storage area 500. A part of the data in the program storage area 400 and the data storage area 500 is stored in, for example, the external memory 44, and is read and stored in the main memory 32 when the stereo camera imaging process and the display adjustment process are executed.

  The program storage area 400 stores programs such as an imaging processing program 401 and a drawing processing program 402 that execute processing of a flowchart shown in FIG.

  The data storage area 500 stores operation data 501, stereo camera data 502, vertical displacement data 503, initial parameter data 504, display parameter data 505, display adjustment parameter data 506, and the like.

  The operation data 501 is data indicating user operations performed on the operation buttons 14 </ b> A to E and GH, the analog stick 15, the 3D adjustment switch 25, and the touch panel 13. The operation data 501 includes data indicating an operation of moving the right-eye image displayed by the user translucently, data indicating the position of the slider 25a in the 3D adjustment switch 25, and the like.

  The stereo camera data 502 includes the data of each stereo camera described with reference to FIG. 4, that is, the left stereo camera data 502a and the right stereo camera data 502b.

  The left stereo camera data 502a is data related to the left stereo camera 23a that captures a left-eye image to be viewed by the user's left eye, and the position, imaging direction, imaging angle of view, and captured image of the left stereo camera 23a with respect to the upper housing 21. It is the data which shows etc. The data of the captured image captured by the left stereo camera 23a is repeatedly updated every frame (for example, 1/60 seconds).

  The right stereo camera data 502b is data related to the right stereo camera 23b that captures a right-eye image to be viewed by the user's right eye, and the position, imaging direction, imaging angle of view, and captured image of the right stereo camera 23b with respect to the upper housing 21. It is the data which shows etc. Note that captured image data captured by the right stereo camera 23b is repeatedly updated every frame (for example, 1/60 seconds).

  Based on the operation data 501 indicating that the user has moved the right-eye adjustment image 50b displayed semi-transparently for adjustment, the vertical displacement data 503 indicates which of the right-eye adjustment images 50b is in the vertical direction (y-axis direction). It is the data which shows whether it was displaced only (displacement amount).

  Initial parameter data 504 is data indicating initial parameters for setting an initial display position of stereo images (both images) captured by both stereo cameras. The initial parameter is determined according to the optical position of the stereo camera at the time of shipment, and the display position of the stereo image corresponding to the state at the time of shipment is determined based on the initial parameter. The initial parameter is calculated using, for example, a pattern image for adjustment known at the factory. The initial parameters are stored in the internal data storage memory 35. When the game apparatus 10 is turned on, the data stored in the internal data storage memory 35 is read into the main memory 32, and the internal data storage memory is stored. The initial parameters stored in 35 are stored as initial parameter data 504.

  The display parameter data 505 is data indicating parameters for determining the display position of the stereo image. When the game apparatus 10 is turned on, the data stored in the data storage internal memory 35 is read into the main memory 32, and the display adjustment parameters described later are stored in the data storage internal memory 35. Adjustment parameters are stored as display parameter data 505. On the other hand, when the display adjustment parameter is not stored in the data storage internal memory 35, the initial parameter stored in the data storage internal memory 35 is stored as display parameter data 505.

  The display adjustment parameter data 506 is data indicating display adjustment parameters for adjusting the display position of the stereo image captured by both stereo cameras. This display adjustment parameter is calculated based on the vertical displacement data 503 described above. When the user adjusts the display position of the stereo image (that is, the display adjustment parameter is calculated and registered), the display adjustment parameter is also stored in the data storage internal memory 35. As a result, when the game apparatus 10 is powered on, the data in the internal data storage memory 35 is read into the main memory 32, and the display adjustment parameters stored in the internal data storage memory 35 are displayed as described above. Stored as parameter data 505.

(Stereo camera imaging processing)
Next, a stereo camera imaging process executed by the game apparatus 10 will be briefly described. When the power of the game apparatus 10 is turned on, the CPU 311 of the game apparatus 10 executes a startup program stored in the data storage internal memory 35 or the like, thereby initializing each unit such as the main memory 32. . Then, the imaging processing program 401 stored in the external memory 44 and the data stored in the data storage internal memory 35 are read into the main memory 32, and the imaging processing program 401 is executed by the CPU 311.

  The processing executed by the game apparatus 10 includes a stereo camera imaging process (stereo camera imaging mode) and a display adjustment process (display adjustment mode). When the stereo camera imaging mode is selected, based on an instruction from the CPU 311 The GPU 312 first acquires display parameters by referring to the display parameter data 505. Next, the GPU 312 refers to the stereo camera data 502, and acquires the left-eye image captured by the left stereo camera 23a and the right-eye image captured by the right stereo camera 23b. The GPU 312 stereoscopically displays the acquired stereo image (the left-eye image and the right-eye image) on the display screen of the upper LCD 22 in which the parallax barrier is turned on based on the acquired display parameter. Note that the stereoscopic effect (stereoscopic condition) when stereoscopically displayed is adjusted by the 3D adjustment switch 25. When the shutter button (L button 14G and R button 14H) is pressed, the CPU 311 stores the stereoscopically displayed stereo image in the data storage internal memory 35 or the data storage external memory 45.

  On the other hand, when the display adjustment mode is selected, the GPU 312 performs processing for changing the above-described display parameters based on an instruction from the CPU 311 or the CPU 311. As described above, in the stereo camera imaging process, the stereo image is displayed on the display screen of the upper LCD 22 based on the display parameters. That is, the display position of the left-eye image and the right-eye image is determined by the display parameter. Therefore, the display position of the stereo image that is shifted due to the shift of the optical position of the stereo camera is corrected by correcting (changing) the display parameter. Hereinafter, display adjustment processing executed in this display adjustment mode will be described in detail with reference to FIGS.

(Display adjustment processing)
12 and 13 are an example of a flowchart of the display adjustment process executed by the GPU 312 based on the instruction from the CPU 311 or the CPU 311.

  First, in step S1, the CPU 311 refers to the operation data 501 and determines whether or not there is predetermined input information for returning the stereo image display adjustment to the initial setting. If the result of this determination is YES, the process moves to step S21, and if it is NO, the process moves to step S2.

  In step S <b> 2, the GPU 312 refers to the stereo camera data 502 based on an instruction from the CPU 311, and the left-eye image and right image captured by the left stereo camera 23 a that is updated every frame (for example, 1/60 seconds). The right eye image captured by the stereo camera 23b is acquired. Thereafter, the process proceeds to step S3.

  In step S3, the GPU 312 displays one of the left-eye image and right-eye image acquired in step S2 on the display screen of the upper LCD 22 (planar display). As a result, the user can view the scene captured by the non-stereo camera in real time on the display screen of the upper LCD 22. Thereafter, the process proceeds to step S4.

  In step S4, the CPU 311 refers to the operation data 501 and determines whether there is predetermined input information indicating that the user has pressed the shutter button (L button 14G and R button 14H). If the result of this determination is YES, the process proceeds to step S6, and if NO, the process returns to step S2 (ie, the processes of steps S2 and S3 are performed until input information indicating that the shutter button has been pressed is received. repeat).

  In step S5, the CPU 311 causes the GPU 312 to translucently process one of the left-eye image and the right-eye image acquired in step S2 (here, the right-eye image), and then use both images as adjustment images (that is, The left-eye image is used as the left-eye adjustment image 50a, the right-eye image is used as the right-eye adjustment image 50b), and stored (saved) in the data storage area 500 (stored as stereo camera data 502). Specifically, the GPU 312 changes the setting of the transparency of the right-eye adjustment image 50b, and even if the right-eye adjustment image 50b and the left-eye adjustment image 50a overlap, the visibility of both images is ensured. In this way, the transparency of the right-eye adjustment image 50b is set. Thereafter, the process proceeds to step S6.

  In step S6, the GPU 312 refers to the stereo camera data 502, and displays the adjustment image (both the left-eye adjustment image 50a and the right-eye adjustment image 50b) stored in step S5 in plan view. Specifically, the GPU 312 refers to the display parameter data 505 to acquire a display parameter for adjusting the display position of the stereo image. As described above, when the user has already adjusted the display position of the stereo image, the value of the display parameter is the value of the display adjustment parameter. Otherwise, the value of the display parameter is the initial parameter. It becomes the value of. Then, the GPU 312 displays the left-eye adjustment image 50a and the translucent right-eye adjustment image 50b in a plan view on the display screen of the upper LCD 22 based on the acquired display parameters (the parallax barrier is turned off by the CPU 311). As a result, both images are displayed in a plan view on the display screen). Thereafter, the process proceeds to step S7.

  In step S <b> 7, the CPU 311 refers to the operation data 501 and determines whether there is predetermined input information for moving the right-eye adjustment image 50 b displayed semi-transparently from the user. Specifically, the CPU 311 determines whether or not the user has touched the up / down button 70 or the left / right button 71 (see FIG. 9 or 10) displayed on the lower LCD 12 with the touch pen 28 or the like. If the result of this determination is YES, the process moves to step S8, and if it is NO, the process moves to step S10 (see FIG. 13).

  In step S <b> 8, the CPU 311 moves the display position of the right-eye adjustment image 50 b displayed semi-transparently on the GPU 312 based on the operation data 501. Specifically, based on an instruction from the CPU 311, the GPU 312 moves the display position of the right-eye adjustment image 50 b upward when the user touches the upper button of the up / down button 70, for example. When the left button of the left / right button 71 is touched, the display position of the right-eye adjustment image 50b is moved to the left. As shown in FIG. 9, the lower LCD 12 specifically displays the movement amount according to the touch of the user's up / down button 70 (for example, 15 mm), and the GPU 312 can adjust the right-eye adjustment image 50b based on this movement amount. Is moved upward or downward. For this reason, it becomes easy for the user to finely adjust the vertical movement of the right-eye adjustment image 50b. In this embodiment, the vertical movement amount is displayed on the lower LCD 12, but the present invention is not limited to such a form. Thereafter, the process proceeds to step S9.

  In step S9, the CPU 311 determines the latest position of the right-eye adjustment image 50b and the right-eye adjustment image 50b displayed in step S6 based on the movement instruction data of the right-eye adjustment image 50b from the user included in the operation data 501. The deviation (displacement amount) in the vertical direction (y-axis direction) from the display position is updated (stored) as the vertical displacement data 503. Thereafter, the process proceeds to step S10 (see FIG. 13).

  In step S10 (see FIG. 13), the CPU 311 refers to the operation data 501 and determines whether or not there has been a predetermined input instructing stereoscopic display from the user. Specifically, the CPU 311 determines whether or not the user has touched the 3D display button 72 (see FIG. 9 or 10) displayed on the lower LCD 12 with the touch pen 28 or the like. If the result of this determination is YES, the process moves to step S11, and if it is NO, the process moves to step S16.

  In step S <b> 11, the CPU 311 calculates display adjustment parameters based on the vertical displacement data 503, and updates (stores) the calculated display adjustment parameters as display adjustment parameter data 506. Thereafter, the process proceeds to step S12.

  Here, calculation of the display adjustment parameter executed by the CPU 311 will be described. As described above, the display position of the stereo image is corrected by correcting (changing) the display parameter. Further, the relationship between the display parameter and the display position of the stereo image is given by a known linear transformation or the like. Therefore, if how much the display position of the stereo image should be changed is calculated, how much the display parameter should be changed is calculated by linear transformation. Therefore, the CPU 311 moves the right-eye adjustment image 50b in the vertical direction in order to adjust the vertical display position of the adjustment image (the subject 50) in order to adjust the vertical display position of the stereo image. It is calculated from the vertical displacement amount (vertical displacement data 503). Then, the CPU 311 calculates how much the display parameter is changed (parameter displacement amount) by linear conversion based on the result, and sets the display adjustment parameter to correct the display parameter based on the parameter displacement amount. calculate.

  In step S12, the GPU 312 refers to the stereo camera data 502 based on an instruction from the CPU 311 to update the left-eye image captured by the left stereo camera 23a and the right-eye image captured by the right stereo camera 23b, which are updated every frame. Acquire images. Thereafter, the process proceeds to step S13.

  In step S <b> 13, the GPU 312 corrects the display position of both the images acquired in step S <b> 12 based on the display adjustment parameter and displays them stereoscopically. Specifically, the GPU 312 acquires display adjustment parameters for correcting the display position of the stereo image by referring to the display adjustment parameter data 506. Then, the GPU 312 stereoscopically displays both images on the display screen of the upper LCD 22 based on the acquired display adjustment parameter. Note that this stereoscopic display is realized by the CPU 311 turning on the parallax barrier so that the right-eye image is visually recognized by the user's right eye and the left-eye image is visually recognized by the user's left eye. This allows the user to actually check how both images are stereoscopically displayed when the display positions of the images captured by the left stereo camera 23a and the right stereo camera 23b are corrected. It can be confirmed whether there is no problem with the correction. Thereafter, the process proceeds to step S14.

  In step S <b> 14, the CPU 311 refers to the operation data 501 and determines whether there is predetermined input information for instructing the end of stereoscopic display from the user. If the result of this determination is YES, the process proceeds to step S15, and if NO, the process returns to step S12 (that is, steps S12 and S13 are repeated until input information for instructing the end of stereoscopic display is received. ).

  In step S15, the GPU 312 displays the adjustment image again in plan view. Specifically, the GPU 312 again displays the adjustment image that has been moved by the GPU 312 in step S7 on the display screen of the upper LCD 22 based on the adjustment image data and the operation data 501 stored as the stereo camera data 502. Are displayed in a plan view (that is, the adjustment image is again displayed in a plan view at the display position immediately before switching to the stereoscopic display). Thereafter, the process proceeds to step S16.

  In step S <b> 16, the CPU 311 refers to the operation data 501 and determines whether there is predetermined input information for instructing the end of adjustment (movement) of the adjustment image from the user. If the result of this determination is YES, the process proceeds to step S17. If NO, the process returns to step S7 (see FIG. 12), and again whether or not there has been input information indicating an instruction to move the adjustment image from the user. judge.

  In step S <b> 17, the CPU 311 refers to the operation data 501 and determines whether there is a predetermined input for instructing the end of the display adjustment mode from the user. If the result of this determination is YES, the process moves to step S18, and if it is NO, the process moves to step S20.

  In step S <b> 18, the CPU 311 calculates display adjustment parameters based on the vertical displacement data 503. If the display adjustment parameter calculated in step S11 is not changed thereafter, the display adjustment parameter value calculated in step S18 matches the display adjustment parameter value stored as the display adjustment parameter data 506. . Thereafter, the process proceeds to step S19.

  In step S19, the CPU 311 updates (stores) the display adjustment parameter calculated in step S18 as display parameter data 505, and simultaneously stores the display adjustment parameter in the data storage internal memory 35. Thus, even if the user turns off the power of the game apparatus 10, the display adjustment parameter saved in the data saving internal memory 35 is stored as the display parameter data 505 in the main memory 32 when the power is turned on next time. The adjustment parameter is always reflected in the stereo camera imaging process. Then, the CPU 311 clears (resets) the adjustment image data stored as the stereo camera data 502, clears (resets) the vertical displacement data 503 and the display adjustment parameter data 506, and ends the display adjustment process.

  On the other hand, in step S20, the CPU 311 clears (resets) the adjustment image data stored as the stereo camera data 502, and clears (resets) the vertical displacement data 503 and the display adjustment parameter data 506. Thereafter, the process returns to step S1 (that is, when it is difficult to understand an image (subject) determined by the user as an adjustment image, a process of determining a captured image of another subject as an adjustment image again is started). .

  Returning to FIG. 12, in step S <b> 21, the CPU 311 refers to the operation data 501 and determines whether or not there is a predetermined input instructing stereoscopic display from the user. Specifically, the CPU 311 determines whether or not the user has touched the 3D display button 72 (see FIG. 9 or 10) displayed on the lower LCD 12 with the touch pen 28 or the like. If the result of this determination is YES, the process moves to step S22, and if it is NO, the process moves to step S25.

  In step S <b> 22, the GPU 312 refers to the stereo camera data 502 based on an instruction from the CPU 311, and the left-eye image captured by the left stereo camera 23 a and the right-eye captured by the right stereo camera 23 b are updated every frame. Acquire images. Thereafter, the process proceeds to step S23.

  In step S <b> 23, the GPU 312 displays the two images acquired in step S <b> 22 stereoscopically by returning the display position to the initial setting based on the initial parameters. Specifically, the GPU 312 acquires initial parameters for returning the stereo image display position to the initial setting by referring to the initial parameter data 504. Then, the GPU 312 stereoscopically displays both images on the display screen of the upper LCD 22 based on the acquired initial parameters. Note that this stereoscopic display is realized by the CPU 311 turning on the parallax barrier so that the right-eye image is visually recognized by the user's right eye and the left-eye image is visually recognized by the user's left eye. Thus, the user can actually confirm how both images are stereoscopically displayed when the display positions of the images captured by the left stereo camera 23a and the right stereo camera 23b are returned to the initial settings. You can check whether there is no problem even if you return to the initial settings. Thereafter, the process proceeds to step S24.

  In step S24, the CPU 311 refers to the operation data 501 and determines whether there is predetermined input information for instructing the end of stereoscopic display from the user. If the result of this determination is YES, the process proceeds to step S25, and if NO, the process returns to step S22 (that is, the processes of steps S22 and S23 are repeated until input information instructing the end of stereoscopic display is received. ).

  In step S <b> 25, the CPU 311 refers to the operation data 501 and determines whether there is a predetermined input for instructing the end of the display adjustment mode from the user. If the result of this determination is YES, the process proceeds to step S26, and if NO, the process returns to step S1 (that is, the display adjustment process is resumed).

  In step S <b> 26, the CPU 311 updates (stores) the initial parameter stored as the initial parameter data 504 as the display parameter data 505, and stores the display adjustment parameter when the data storage internal memory 35 stores the display adjustment parameter. The displayed adjustment parameter is cleared (reset), and the adjustment process is terminated. As a result, even when the user turns off the power of the game apparatus 10, when the power is turned on next time, only the initial parameters are stored in the data storage internal memory 35. Therefore, the initial parameters are the display parameter data 505. The initial parameters are always reflected during the stereo camera imaging process.

  As described above, according to the present embodiment, when the user manually adjusts the display position of the adjustment image (step S1: NO), both images obtained by imaging an arbitrary imaging target (subject) with both stereo cameras. Since the (stereo image) is acquired and displayed in plan view (steps S2 and S3), the user selects a desired image while viewing the image displayed in plan view and presses the shutter button (step S4). Thus, the adjustment image can be displayed in a plan view on the display screen (steps S5 and S6). Therefore, the user does not need to prepare a specific pattern image for adjustment separately. In addition, among the adjustment images, the right-eye adjustment image 50b is displayed semi-transparently (step S5), so that the user can easily visually recognize the positional deviation between the right-eye adjustment image 50b and the left-eye adjustment image 50a. By giving an instruction to move the right eye adjustment image 50b (step S7: YES), the display position of the right eye adjustment image 50b is easily adjusted (adjusted) to the display position of the left eye adjustment image 50a. (Step S8). At this time, of the movement information of the right-eye adjustment image 50b, only the movement information (displacement amount) in the vertical direction used for calculation of the display adjustment parameter is stored (step S9), but the user adjusts the right-eye adjustment. The image 50b can be moved vertically and horizontally. For this reason, the user can move the right-eye adjustment image 50b and substantially overlap the left-eye adjustment image 50a, and as a result, the vertical display positions of both images can be easily aligned. The vertical displacement is updated every time there is a movement instruction from the user, and if there is a stereoscopic display instruction from the user (step S10: YES), the display adjustment parameter is calculated based on this displacement. Then, a stereo image is stereoscopically displayed based on the calculated display adjustment parameter (steps S12 and S13). As a result, when the user adjusts the display position of the right-eye adjustment image 50b (that is, during adjustment), the user can actually see how the stereo image is stereoscopically displayed with the current adjustment. It is possible to confirm whether there is no problem with the adjustment before the adjustment is finished (determined). Then, when the user instructs that the adjustment is finished (step S17: YES), the display adjustment parameter is calculated based on the vertical displacement of the right-eye adjustment image 50b that has been moved so far (step S17). S18) This display adjustment parameter is updated as a display parameter for determining the display position of the stereo image (step S19). Thus, the user can set an appropriate parameter (display adjustment parameter) as the display parameter only by adjusting (adjusting) the display position of the adjustment image while checking the display screen by himself / herself. When the user wants to return the display position of the stereo image to the initial setting (step S1: YES), the initial parameter is set as the display parameter (step S26), so the user changes the display position of the stereo image ( It is possible to return to the default display position at any time even when the adjustment is performed. As described above, when the display parameter is updated (set) in the display adjustment mode (display adjustment process) and then the stereo imaging is performed in the stereo imaging mode (stereo imaging process), the display parameter updated in the display adjustment process is updated. An appropriate stereo image that is read and corrected in the vertical direction that causes the user to recognize it as two shifted images is stereoscopically displayed.

(Modification)
In the above embodiment, the adjustment image 50b for the right eye is adjusted in the x-axis direction (in the x-axis direction (in order to eliminate the shift in the vertical direction (y-axis direction)) of the adjustment images (the left-eye adjustment image 50a and the right-eye adjustment image 50b). It is also possible to move in the left and right direction. However, instead of this, or at the same time, the right-eye adjustment image 50b may be rotatable around the z-axis direction (on the xy plane) (see (2) in FIG. 14). Specifically, as shown in FIG. 14 (1), when the right stereo camera 23b is rotationally displaced by Δc around the z-axis direction, the right-eye adjustment image 50b is rotationally displaced on the display surface. Wake up (see (2) in FIG. 14). In this case, when the user tries to make an adjustment to eliminate the deviation of both images in the y-axis direction, the display position of the subject 50 in both images cannot be completely matched, so depending on the image (subject), It is difficult to make an adjustment that eliminates the deviation in the y-axis direction accurately. Therefore, as shown in (3) of FIG. 14, the user touches the rotation button 73 displayed on the lower LCD 12 so that the right-eye adjustment image 50b, which is a translucent image, is moved around the z-axis direction (xy). Can be rotated). Thus, by rotating the right-eye adjustment image 50b, the user can substantially overlap the display position of the imaging target (subject 50) in the right-eye adjustment image 50b and the left-eye adjustment image 50a. As a result, the right-eye adjustment image 50b and the left-eye adjustment image 50a are accurately adjusted so that there is no shift in the y-axis direction. In this case, in step S9 in FIG. 12, data indicating the rotational displacement is also stored in the data storage area 500. In steps S11 and S18 in FIG. 13, display adjustment parameters based on the rotational displacement are also calculated. Is done. That is, the rotational deviation of the stereoscopic images (both images) displayed stereoscopically is also corrected based on the calculated display adjustment parameter.

  In the above embodiment, the right-eye adjustment image 50b can be moved in the x-axis direction (left and right direction) and / or can be rotated around the z-axis direction (on the xy plane). However, instead of this, or at the same time, the right-eye adjustment image 50b may be zoom adjustable (see (2) in FIG. 15). Specifically, as shown in (1) of FIG. 15, when the right stereo camera 23b causes a translational shift by Δz with respect to the z-axis direction, the right-eye adjustment image 50b shifts in size on the display surface. Wake up (see (2) in FIG. 15). In this case, when the user tries to make an adjustment to eliminate the deviation of both images in the y-axis direction, the display position of the subject 50 in both images cannot be completely matched, so depending on the image (subject), It is difficult to make an adjustment that eliminates the deviation in the y-axis direction accurately. Therefore, as shown in FIG. 15 (3), the user touches the zoom button 74 displayed on the lower LCD 12 to zoom (enlarge or reduce) the right eye adjustment image 50b, which is a translucent image. Can do. Thus, by zooming the right-eye adjustment image 50b, the user can substantially superimpose the display position of the imaging target (subject 50) in the right-eye adjustment image 50b and the left-eye adjustment image 50a. As a result, the right-eye adjustment image 50b and the left-eye adjustment image 50a are accurately adjusted so that there is no shift in the y-axis direction. In this case, data indicating the zoom amount is also stored in the data storage area 500 in step S9 in FIG. 12, and display adjustment parameters based on the zoom amount are also calculated in steps S11 and S18 in FIG. Is done. That is, the deviation of the size of the stereo image (both images) displayed stereoscopically is corrected at the same time based on the calculated display adjustment parameter.

  Further, in the above embodiment, correction due to the shift of the angle of view of the stereo camera is not mentioned. However, when the angle of view is shifted due to the shift of the focal length of the right stereo camera 23b, the size of the right-eye adjustment image 50b is shifted. Also in this case, the right-eye adjustment image 50b may be zoom-adjustable by the user as described above.

  Further, in the above embodiment, every time the user gives an instruction to move the right-eye adjustment image 50b (step S7: YES in FIG. 12), the amount of movement in the vertical direction (y-axis direction) of the display position of the right-eye adjustment image 50b. (Displacement amount) is updated (stored) as vertical displacement data 503 (step S9 in FIG. 12). However, instead of this, in step S9, the CPU 311 updates the latest display position of the right-eye adjustment image 50b as predetermined data in the data storage area 500 every time the user gives an instruction to move the right-eye adjustment image 50b ( It may be stored. In this case, when there is a stereoscopic display instruction from the user (step S10: YES in FIG. 19), in step S11, the CPU 311 displays the latest display position of the right-eye adjustment image 50b updated in step S9, and in step S6. Display adjustment parameters are calculated based on the amount of vertical displacement from the display position of the displayed right-eye adjustment image 50b.

  In the above-described embodiment, the right-eye adjustment image 50b displayed semi-transparently can be moved by a user instruction. However, instead of this, the adjustment image 50a for the left eye that is not translucently displayed may be movable.

  In the above embodiment, the right-eye adjustment image 50b is displayed semi-transparently. However, if the visibility of the right-eye adjustment image 50b and the left-eye adjustment image 50a is ensured, both images are semi-transparent. It may be displayed in a transparent manner, or both images may not be displayed in a semi-transparent manner (normally displayed).

  In the above embodiment, only the right-eye adjustment image 50b displayed semi-transparently can be moved by the user's instruction, but both the right-eye adjustment image 50b and the left-eye adjustment image 50a can be moved by the user's instruction. It may be. In this case, the vertical displacement data 503 may be determined based on the displacement amount in which the right-eye adjustment image 50b is displaced in the vertical direction (y-axis direction) relative to the left-eye adjustment image 50a.

  In the above-described embodiment, the right-eye adjustment image 50b is shifted based on the optical position shift of the right stereo camera 23b with respect to the left stereo camera 23a (that is, the amount of displacement of the right-eye adjustment image 50b with respect to the left-eye adjustment image 50a). The display position was corrected. However, instead of this, optical positions serving as predetermined references are set for both stereo cameras (that is, display positions serving as predetermined references are set for both images), and both images can be moved so that both images can be moved. The display positions of both images may be corrected based on a deviation (displacement amount) from a predetermined reference position.

  Moreover, in the said embodiment, in order to acquire an adjustment image, either one of the image for left eyes and the image for right eyes shall be displayed by planar view (step S3 of FIG. 12). However, instead of this, the image displayed on the display screen of the upper LCD 22 to obtain the adjustment image may be a stereoscopic image. That is, in step S3, the GPU 312 may stereoscopically display both images acquired in step S2 based on the display parameters. In this case, the user determines the stereoscopic image as an adjustment image by selecting a desired imaging target while visually recognizing the stereoscopic image displayed on the upper LCD 22 and pressing the shutter button.

  In the above-described embodiment, stereoscopic display is possible even when the display parameter is set to the initial parameter (returned) (step S1: YES) in the display adjustment process (see FIG. 12) (steps S21 to S21). S24). However, when the display parameter is returned to the initial parameter, the stereoscopic display may not be performed. That is, the processes in steps S21 to S24 may be omitted.

  In the above-described embodiment, the display adjustment mode can be freely switched from the stereo camera imaging mode, and the display adjustment mode may be set to automatically return (switch) to the stereo camera imaging mode when the display adjustment mode ends.

  In the above-described embodiment, the right-eye adjustment image 50b can be moved by touching a button on the touch panel displayed on the display screen of the lower LCD 12. However, instead of this, the right-eye adjustment image 50b may be movable by using the operation buttons 14A to 14L.

  In the above embodiment, the stereoscopic image displayed on the upper LCD 22 has been described as an image that can be stereoscopically viewed with the naked eye. However, the upper LCD 22 only needs to display an image that can be viewed stereoscopically. For example, an image that can be viewed stereoscopically by the user wearing stereoscopic glasses (that is, an image for the left eye and an image for the right eye are time-divided. Alternately displayed images) may be displayed.

  In the above embodiment, the imaging direction of the outer imaging unit (left) 23a and the imaging direction of the outer imaging unit (right) 23b are assumed to be parallel, and the subject is imaged by the parallel method. However, the present invention is not limited to this. For example, the imaging directions of the two imaging units may cross each other instead of being parallel, and the subject may be imaged by the intersection method.

  In the above embodiment, the storage destination of the display adjustment parameter is the data storage internal memory 35. However, instead of this, the display adjustment parameter storage destination may be an external (removable) nonvolatile memory or the like. Further, various parameter data and the like referred to in the display adjustment processing (see FIGS. 12 and 13) are referred to or updated on the main memory 32. Instead, they are referred to on the cache memory. Alternatively, it may be updated.

  Moreover, although the said embodiment applies this invention to the game device 10, this invention is not limited to the game device 10. FIG. For example, the present invention can be applied to a portable information terminal such as a cellular phone, a simple cellular phone (PHS), and a PDA. The present invention can also be applied to stationary game machines, personal computers, and the like.

  Moreover, in the said embodiment, although the process mentioned above is performed with the one game device 10, you may share the said process with the some apparatus which can communicate by wire or radio | wireless.

  In the above embodiment, the shape of the game apparatus 10 and the various operation buttons 14 provided thereon, the shape, number, and installation position of the touch panel 13 are merely examples, and other shapes, numbers, and It goes without saying that the present invention can be realized even at the installation position. In addition, the processing order, setting values, values used for determination, and the like used in the information processing described above are merely examples, and other orders and values may be used without departing from the scope of the present invention. It goes without saying that can be realized.

  In addition, various information processing programs executed in the game device 10 of the above embodiment are not only supplied to the game device 10 through a storage medium such as the external memory 44 but also supplied to the game device 10 through a wired or wireless communication line. May be. The program may be recorded in advance in a non-volatile storage device (such as the internal data storage memory 35) inside the game apparatus 10. In addition to the nonvolatile memory, the information storage medium for storing the program includes CD-ROM, DVD, or similar optical disk storage medium, flexible disk, hard disk, magneto-optical disk, magnetic tape, etc. There may be. The information storage medium that stores the program may be a volatile memory that temporarily stores the program.

  Although the present invention has been described in detail above, the above description is merely illustrative of the present invention in all respects and is not intended to limit the scope thereof. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention.

10 Game device 11 Lower housing 12 Lower LCD
13 Touch Panel 14 Operation Buttons 15 Analog Stick 16 LED
21 Upper housing 22 Upper LCD
23 Outside imaging unit 23a Outside imaging unit (left)
23b Outside imaging unit (right)
24 Inner imaging unit 25 3D adjustment switch 26 3D indicator 28 Touch pen 31 Information processing unit 32 Main memory 50 Subject 50a Adjustment image for left eye 50b Adjustment image for right eye 311 CPU
312 GPU
313 VRAM

Claims (18)

An information processing program executed by a computer of an imaging apparatus including a storage unit that captures a subject with a stereo imaging unit and obtains a stereoscopically captured image,
The computer,
An arbitrary imaging target is imaged by the stereo imaging unit, an image captured by the first imaging unit among the stereo imaging units is used as a first temporary captured image, and an image captured by the second imaging unit is used as a second temporary captured image. Provisional captured image acquisition means for acquiring;
Temporary captured image display means for displaying the first temporary captured image and the second temporary captured image on a display unit;
Input receiving means for receiving input from the user;
Display position changing means for changing a display position of at least one of the first temporary captured image and the second temporary captured image based on a predetermined input received by the input receiving means;
Based on the displacement of the display position of the first temporary captured image and / or the second temporary captured image changed by the display position changing means, at least of the images captured by the first imaging unit and the second imaging unit Correction amount calculating means for calculating a correction amount for correcting the display position on one display unit;
An information processing program causing a correction amount calculated by the correction amount calculation unit to function as a storage control unit that stores the correction amount in the storage unit without associating the correction amount with the first temporary captured image and the second temporary captured image. The computer,
A display position on a display unit of at least one of the first image captured by the first imaging unit and the second image captured by the second imaging unit separately from the first temporary captured image and the second temporary captured image. Correction means for correcting based on the correction amount;
The information processing program according to claim 1, further causing the first image and the second image to further function as a stereoscopic display control unit that stereoscopically displays the first image and the second image on the display unit based on a correction result by the correction unit.   3. The information processing according to claim 1, wherein the correction amount calculating unit calculates the correction amount based only on a vertical display position displacement of the first temporary captured image and / or the second temporary captured image. program. The storage unit stores display parameters for determining display positions of the first image and the second image on the display unit,
The temporary captured image display means displays the first temporary captured image and the second temporary captured image on a display unit based on the display parameter,
The correction means corrects a display position on the display unit of at least one of the first image and the second image based on a display parameter corrected based on the correction amount. The information processing program described. The computer,
Display position determining means for determining a display position of the first temporary captured image and / or the second temporary captured image based on a determination input received by the input receiving means;
Further functioning as display adjustment parameter calculation means for calculating a display adjustment parameter for correcting a display position on the display unit of at least one of the first image and the second image,
The storage control means updates the display adjustment parameter as the display parameter and stores it in the storage unit,
The display position changing means changes the display position of the first provisional captured image based on the first input received by the input receiving means,
The display position determining means determines the display position of the first provisionally captured image based on the determination input as the display position after being changed by the display position changing means,
The display adjustment parameter calculating means is a displacement between the display position of the first temporary captured image determined by the display position determining means and the display position of the first temporary captured image before being changed by the display position changing means. Based on a certain first displacement, the display adjustment parameter is calculated,
The information according to claim 4, wherein the correction unit corrects a display position on the display unit of at least one of the first image and the second image based on the display parameter updated in the storage unit. Processing program. The display position changing means further changes the display position of the second temporary captured image based on the second input received by the input receiving means,
The display position determining means determines the display positions of the first temporary captured image and the second temporary captured image based on the determination input as the display positions after being changed by the display position changing means,
The display adjustment parameter calculation means is a displacement between the display position of the second temporary captured image determined by the display position determination means and the display position of the second temporary captured image before being changed by the display position changing means. The information processing program according to claim 5, wherein the display adjustment parameter is calculated based on a certain second displacement and the first displacement.   The information processing program according to claim 6, wherein the display adjustment parameter calculation unit calculates the display adjustment parameter based on a relative displacement of the first displacement with respect to the second displacement.   The information processing program according to claim 5, wherein the display adjustment parameter calculation unit calculates the display adjustment parameter based only on a vertical displacement of the first displacement.   The information processing program according to claim 6 or 7, wherein the display adjustment parameter calculation unit calculates the display adjustment parameter based only on a vertical displacement of the first displacement and the second displacement. The storage control unit causes the storage unit to store the display position of the first temporary captured image changed by the display position changing unit every time the input receiving unit receives the first input,
When the input receiving means receives a switching input,
The display adjustment parameter calculation means is a displacement between the display position of the first temporary captured image stored in the storage unit and the display position of the first temporary captured image before being changed by the display position changing means. Based on the three displacements, the display adjustment parameter is calculated,
10. The stereoscopic display control unit according to claim 5, wherein the stereoscopic display control unit causes the first image and the second image to be stereoscopically displayed on the display unit based on the calculated display adjustment parameter. Information processing program. The storage control unit further causes the storage unit to store the display position of the second temporary captured image changed by the display position changing unit every time the input receiving unit receives the second input.
When the input receiving means receives a switching input,
The display adjustment parameter calculation means is a displacement between a display position of the second temporary captured image stored in the storage unit and a display position of the second temporary captured image before being changed by the display position changing means. Calculating the display adjustment parameter based on four displacements and the third displacement;
The information processing program according to claim 10, wherein the stereoscopic display control unit causes the first image and the second image to be stereoscopically displayed on the display unit based on the calculated display adjustment parameter.   The temporary captured image display means displays at least one of the first temporary captured image and the second temporary captured image as a semi-transparent display and displays the temporary captured image on the display unit. The information processing program according to any one of claims 1 to 11.   The information processing program according to any one of claims 1 to 12, wherein the temporary captured image display unit displays the first temporary captured image and the second temporary captured image in a plan view on a display unit. The storage unit stores in advance an initial parameter that defines a preset initial display position on the display unit of the first image and the second image,
14. The information according to claim 5, wherein the storage control unit updates the initial parameter as the display parameter and stores it in the storage unit when the input receiving unit receives an initialization input. Processing program.   15. The display position changing unit according to claim 1, wherein the display position changing unit changes a display position of at least one of the first temporary captured image and the second temporary captured image in a horizontal direction and a vertical direction based on the predetermined input. An information processing program according to any one of the above. An imaging device including a storage unit that captures a subject with a stereo imaging unit and obtains a stereoscopically captured image,
An arbitrary imaging target is imaged by the stereo imaging unit, an image captured by the first imaging unit among the stereo imaging units is used as a first temporary captured image, and an image captured by the second imaging unit is used as a second temporary captured image. Provisional captured image acquisition means for acquiring;
Temporary captured image display means for displaying the first temporary captured image and the second temporary captured image on a display unit;
Input receiving means for receiving input from the user;
Display position changing means for changing a display position of at least one of the first temporary captured image and the second temporary captured image based on a predetermined input received by the input receiving means;
Based on the displacement of the display position of the first temporary captured image and / or the second temporary captured image changed by the display position changing means, at least of the images captured by the first imaging unit and the second imaging unit Correction amount calculating means for calculating a correction amount for correcting the display position on one display unit;
An imaging apparatus comprising: a storage control unit that stores the correction amount calculated by the correction amount calculation unit in the storage unit without associating the correction amount with the first temporary captured image and the second temporary captured image. An imaging system including a storage unit that captures a subject with a stereo imaging unit and obtains a stereoscopically captured image,
An arbitrary imaging target is imaged by the stereo imaging unit, an image captured by the first imaging unit among the stereo imaging units is used as a first temporary captured image, and an image captured by the second imaging unit is used as a second temporary captured image. Provisional captured image acquisition means for acquiring;
Temporary captured image display means for displaying the first temporary captured image and the second temporary captured image on a display unit;
Input receiving means for receiving input from the user;
Display position changing means for changing a display position of at least one of the first temporary captured image and the second temporary captured image based on a predetermined input received by the input receiving means;
Based on the displacement of the display position of the first temporary captured image and / or the second temporary captured image changed by the display position changing means, at least of the images captured by the first imaging unit and the second imaging unit Correction amount calculating means for calculating a correction amount for correcting the display position on one display unit;
An imaging system comprising: a storage control unit that stores the correction amount calculated by the correction amount calculating unit in the storage unit without associating the correction amount with the first temporary captured image and the second temporary captured image. An imaging method for capturing a subject with a stereo imaging unit and acquiring a stereoscopically captured image,
An arbitrary imaging target is imaged by the stereo imaging unit, an image captured by the first imaging unit among the stereo imaging units is used as a first temporary captured image, and an image captured by the second imaging unit is used as a second temporary captured image. A provisional captured image acquisition step to acquire;
A temporary captured image display step of displaying the first temporary captured image and the second temporary captured image on a display unit;
An input receiving step for receiving input from the user;
A display position changing step for changing a display position of at least one of the first temporary captured image and the second temporary captured image based on the predetermined input received by the input receiving step;
Based on the displacement of the display position of the first temporary captured image and / or the second temporary captured image changed by the display position changing step, at least of the images captured by the first imaging unit and the second imaging unit A correction amount calculating step for calculating a correction amount for correcting the display position on one display unit;
An imaging method comprising: a storage control step of storing the correction amount calculated in the correction amount calculation step in a storage unit of the imaging apparatus without associating the correction amount with the first temporary captured image and the second temporary captured image.

Images