JP5965434B2 - Image display system, lighting system, information processing apparatus, and control program - Google Patents

Description

The present invention relates to an image display system and the like, and more particularly to an image display system and the like that display an image on a display screen and give a user a further visual effect in addition to the image.

2. Description of the Related Art Conventionally, there is an image display system that displays television broadcast images, game images, and the like on a television receiver (hereinafter simply referred to as “TV”). For example, Patent Literature 1 discloses a game system in which a game operation is performed by moving a controller that a user has in his hand, and a game image that changes according to the game operation is displayed on a television. In this game system, a device (marker unit) that emits infrared light is installed around the television as an accessory device of the game device body. The marker unit is used by the game device to calculate the movement of the controller. The game device calculates the movement of the controller based on the position of the marker unit (infrared light) in the image captured by the camera of the controller. To do.

JP 2008-125614 A

Conventional image display systems only display television images, game images, and the like on the television screen. Therefore, a visual effect can be given to the user only with an image displayed on the screen of the television, and there is a limit to the presence and power of the image. In addition, the above Patent Document 1
As in the game system of FIG. 1, there are accessory devices installed around the television, but there is no accessory device that gives a visual effect to the user together with the television.

Therefore, an object of the present invention is to provide an image display system or the like (an image display system, a lighting system, an information processing apparatus, and a control program) that can give a user a further visual effect in addition to an image displayed on a screen. Is to provide.

  The present invention employs the following configurations (1) to (9) in order to solve the above problems.

(1)
The present invention is an illumination system including an information processing apparatus capable of communicating with a server via a network and an illumination apparatus capable of receiving a signal from the information processing apparatus. The information processing apparatus includes a receiving unit and a light projection control unit. The receiving means receives instruction data for each channel indicating an instruction to cause the lighting device to emit light in accordance with an image and / or sound of a television broadcast. The light projection control means outputs a control signal for controlling the light projection of the lighting device to the lighting device based on the received instruction data. The illumination device projects visible light according to the control signal. The information processing apparatus can instruct a broadcast receiving apparatus that receives a television broadcast and displays an image on a screen. In response to the input of the channel by the user, the information processing apparatus issues an instruction to select the one input channel to the broadcast receiving apparatus, and transmits instruction data corresponding to the input one channel to the channel. A request is made to the server in response to the input, and the instruction data is received from the server.

The “information processing apparatus” is a concept including an arbitrary information processing apparatus that can communicate with a server and can control a lighting device. That is, the “information processing apparatus” is a game apparatus in the embodiment described later, but is not limited to an apparatus capable of executing game processing, and may be a personal computer, for example.
The above “illumination device” is only required to be able to communicate with the information processing device, and in the embodiment described later, it is connected by wire, but even if it communicates with the display control device wirelessly by radio waves, infrared rays, or the like. Good.
The “instruction data” may be any data as long as the image and / or sound of the television broadcast and the light emission of the lighting device have a certain relationship. For example, an instruction may be given to change the lighting state in response to a scene change in a program, or an instruction may be provided to change the lighting state in accordance with music flowing in the program. May be. Further, the light emission of the lighting device does not need to be strictly synchronized with the image and / or sound.
The processing of the “light projection control means” corresponds to the processing of steps S5 and S14 in the embodiment described later, and the “control signal” is light emission control data in the embodiment described later. Any content may be used as long as it is a signal (data) indicating various instructions for controlling the control.

According to the configuration of (1) above, when the television program is viewed, the illumination device is installed around the television, so that the illumination effect by the projection of the illumination device can be added to the image of the television program. it can. Therefore, a further visual effect can be given to the user in addition to the image of the television program, and the power and presence of the television program can be improved.

(2)
The information processing apparatus may be connectable to a broadcast receiving apparatus that receives a television broadcast and displays an image on a screen. The information processing apparatus further includes game processing means and game display control means. The game processing means executes a predetermined game process. The game display control means displays a game image corresponding to the game process on the screen of the broadcast receiving device. At this time, when a game image is displayed on the screen of the broadcast receiving device, the light projection control unit controls the light projection of the lighting device based on the game process.

In the embodiment described later, the “broadcast receiving device” is the television 2, but the function of receiving television broadcasting (tuner) and the function of displaying images (display device) may be configured separately.
The above “game process” is the process of step S3 in the embodiment described later, but any specific content of the game process may be used.
The process of the “game display control unit” is the process of step S4 in the embodiment described later, but any specific display content may be used as long as it is a process of displaying an arbitrary game image. .

According to the configuration of (2) above, when the broadcast receiving apparatus displays a game image, a lighting effect corresponding to the game image is performed by the lighting apparatus. Therefore, the lighting system can add a further visual effect not only to the TV program but also to the game image, and can improve the power and realism of the game.

(3)
The illumination system may further include an infrared light emitting device that emits infrared light and an input device that includes an imaging unit capable of detecting infrared light. At this time, the game processing means executes game processing based on the position of the infrared light imaged by the imaging means.

The above “infrared light emitting device” is an infrared LED (markers 6R and 6 in the embodiments described later).
In addition to L), it is a concept including various light-emitting devices that emit infrared light.
The above-mentioned “input device” is the controller 5 in the embodiment described later. However, unlike the controller 5, the “input device” is not necessarily provided with the operation button or the acceleration sensor 37, and may be provided with at least an imaging unit. The “imaging means” is a CMOS in the embodiment described later.
Although it is a solid-state image sensor such as a sensor or a CCD sensor, any image sensor (imaging device) that can detect infrared light may be used.

According to the configuration of (3) above, since predetermined information processing is executed with the position of the infrared light imaged by the imaging means as an input, the user can input a game by operating the input device including the imaging means. It can be performed.

(4)
The lighting device may further include an audio output unit. At this time, the instruction data further indicates an instruction for the sound output means to output sound in accordance with the image and / or sound of the television broadcast. The light projection control unit further controls the sound output of the sound output device based on the received instruction data.

The “sound output unit” is a concept including an arbitrary device having a function of outputting sound in addition to the speaker 112 in an embodiment described later.

According to the configuration of (4) above, the lighting system can provide an auditory effect in addition to giving a visual effect to the user using the lighting device 9.

(5)
The lighting device may be installed so as to emit visible light toward the rear of a display screen on which an image of a television broadcast is displayed.

In addition, the above-mentioned “emitting visible light toward the rear of the display screen” does not mean, in a strict sense, “emitting in a direction perpendicular to the display screen”. , ”Is pointing to the rear side rather than the direction parallel to the display screen. For example, as in an embodiment to be described later, the case where the light is emitted slightly toward the rear of the display screen is also included in the case of “emitting visible light toward the rear of the display screen”.

According to the configuration of (5) above, light from the lighting device is projected onto a wall surface or the like behind the display screen. Therefore, from the viewpoint of a user located in front of the display screen, light appears to be projected around the display screen (FIG. 15), and an illumination effect is effectively added to the displayed image. Can do.

(6)
The lighting device may include a plurality of light emitting elements and a condensing lens. The plurality of light emitting elements are arranged in a line in a predetermined direction and emit light of different colors. The condenser lens is disposed at a position that transmits visible light from the plurality of light emitting elements, and focuses the light only in a direction perpendicular to a predetermined direction.

In the embodiment described later, the “plurality of light emitting elements” is a concept including two or more arbitrary light emitting elements in addition to the three LEDs 75 to 77. Further, the “condensing lens” may be a lens having a curved surface, such as a cylindrical lens, in addition to the planar Fresnel lens in the embodiment described later.

According to the configuration of (6) above, since the direction in which each light emitting element is arranged and the focusing direction of the condenser lens are perpendicular, the light from each light emitting element is not focused in the arrangement direction, and the arrangement direction It is focused only in the direction perpendicular to. As a result, the light from each light emitting element is mixed and projected without causing a shift, so that the positions of the projected light in the short direction can be matched, and the projected light can be matched. You can look beautiful.

(7)
The illuminating device may emit a plurality of visible lights each having a longitudinal section of light.

In the embodiment to be described later, a condensing lens and a diffusion sheet are used to emit light (linear light) having a cross section of light in a longitudinal shape, but the linear light is created by any method. Also good. For example, the linear light may be created using only a condensing lens or may be created using a slit. Further, the illumination device may project laser light as linear light.

According to the configuration of (7) above, since the light projecting means projects a plurality of linear lights, the light projecting pattern can be increased as compared with the case of projecting only one light, and the illumination effect is improved. Variations can be increased.

(8)
The lighting device may include a light emitting unit and a condenser lens. The light emitting unit emits visible light. The condensing lens is disposed at a position where visible light from the light emitting unit is transmitted, and focuses the light only in a predetermined direction.

The “light emitting unit” is a color LED module in the embodiment described later, but may emit a monochromatic light or may be composed of one LED.

According to the configuration of (8) above, the lighting device can easily emit linear light by using the condenser lens.

(9)
The lighting device may further emit visible light having a wider cross section than the visible light in a direction overlapping with the plurality of visible light.

The above-mentioned “visible light having a wider cross section than visible light” corresponds to background light (FIG. 15) in the embodiment described later, but the shape of the cross section and the color of light are whatever. Good.

According to the configuration of (9) above, a plurality of linear light beams and light having a wider cross section than the light beams are projected onto the wall surface around the illumination device. According to this, since two types of light are projected, the number of projection patterns can be increased as compared with the case where one type of light is projected, and variations in illumination effects can be increased.

In addition, this invention may be provided with the form of the information processing apparatus as described in said (1)-(9). Further, the present invention may be provided in the form of an image display system including the lighting system and a broadcast receiving device that can be connected to an information processing device and receives a television broadcast. Furthermore, the present invention may be provided in the form of a control program that causes a computer of any information processing apparatus to function as each of the means (1) to (9).

According to the present invention, light (visible light) is projected around the screen by disposing the illumination device around the screen on which the television program is displayed, so that the television program image is illuminated with light. Effects can be added and further visual effects can be given to the user in addition to the image.

External view of game system 1 The block diagram which shows the connection relation of each apparatus contained in the game system 1 The block diagram which shows the structure of each apparatus of the game system 1 The perspective view which shows the external appearance structure of the controller 5 The perspective view which shows the external appearance structure of the controller 5 The figure which shows the internal structure of the controller 5 The figure which shows the internal structure of the controller 5 Block diagram showing the configuration of the controller 5 External view of lighting device 9 The perspective view which shows the main structures in the illuminating device 9 Diagram showing internal structure of linear optical module The figure which shows the light radiate | emitted from a linear optical module The figure which shows an example of the linear light projected when three LED75-77 shifts | deviates from the axis | shaft L, and is arrange | positioned. Three views showing the arrangement of each optical module The figure which shows the linear light and background light which were projected on the back surface of the television 2 by the illuminating device 9 The figure which shows the main data memorize | stored in the main memory of the game device 3 Main flowchart showing the flow of processing executed in the game apparatus 3 when executing the game Main flowchart showing the flow of processing executed in game device 3 when an image of a television program is displayed The figure which shows the flow of the data between the game device 3 and the other apparatus at the time of performing the process shown in FIG. A flowchart showing data reception processing of the input / output processor 11a in the sleep mode

[Overall configuration of game system]
Hereinafter, a game system 1 which is an example of an image display system according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is an external view of the game system 1. In addition, FIG.
These are block diagrams which show the connection relationship of each apparatus contained in the game system 1. FIG. In FIG. 1, a game system 1 includes a television receiver (hereinafter simply referred to as “TV”) 2,
A game apparatus 3, an optical disk 4, a controller 5, a marker apparatus 6, and an illumination apparatus 9 are included. The game system 1 executes game processing in the game device 3 based on a game operation using the controller 5 and displays a game image or the like obtained as a result of the game processing on the television 2.

An optical disk 4 that is an example of an information storage medium that can be used interchangeably with the game apparatus 3 is detachably inserted into the game apparatus 3. The optical disc 4 stores a game program to be executed on the game apparatus 3. An insertion slot for the optical disk 4 is provided on the front surface of the game apparatus 3. The game apparatus 3 executes a game process by reading and executing a game program stored in the optical disc 4 inserted into the insertion slot. The controller 5 is an input device that provides the game device 3 with operation data indicating the content of the operation performed on the own device. As shown in FIG. 2, the controller 5 and the game apparatus 3 are connected by wireless communication. In the present embodiment, for example, Bluetooth (registered trademark) technology is used for wireless communication between the controller 5 and the game apparatus 3. In other embodiments, the controller 5 and the game apparatus 3 may be connected by wire.

As shown in FIGS. 1 and 2, a television 2 (including a speaker 2a), which is an example of a display device, is connected to the game apparatus 3 via a connection cord. The television 2 displays a game image obtained as a result of the game process executed in the game device 3. The TV 2 is a speaker 2a
The speaker 2a outputs game sound obtained as a result of the game process.
Further, the television 2 receives the television broadcast and displays the program image on the screen, and outputs the program sound from the speaker 2a.

A marker device 6 is installed around the television 2 (upper side of the screen in FIG. 1). Although details will be described later, the user can perform a game operation to move the controller 5, and the marker device 6 is used for the game device 3 to detect the movement of the controller 5. The marker device 6
Two markers 6R and 6L are provided at both ends. Marker 6R (same for marker 6L)
Is one or more infrared LEDs (Light Emitting Diodes), and outputs infrared light toward the front of the television 2. As shown in FIG. 2, the marker device 6 is connected to the game device 3, and lighting of each infrared LED included in the marker device 6 is controlled by the game device 3. In addition, although the marker apparatus 6 represents the aspect installed in the television 2 in FIG. 1, the position and direction which install the marker apparatus 6 are arbitrary.

In addition, a lighting device 9 is installed around the television 2 (upper side of the screen in FIG. 1). The lighting device 9 is a device that outputs visible light for the purpose of giving the user a further visual effect (lighting effect) in addition to the image displayed on the television 2. As shown in FIG. 2, the lighting device 9 is connected to the game device 3, and light emission of the lighting device 9 is controlled by the game device 3.

The position and orientation for installing the lighting device 9 are arbitrary. However, in the present embodiment, it is assumed that the lighting device 9 projects visible light on the rear wall surface (such as a house wall or a curtain) of the television 2 and shows the user the light hit the wall surface (see FIG. 15). Therefore, it is preferable that the illumination device 9 is installed so as to emit visible light toward the rear of the television 2. In FIG. 1, the illumination device 9 is installed on the marker device 6. In another embodiment, the lighting device 9 may be installed directly on the television 2 or may be installed on a pedestal on which the television 2 is placed and at a position behind the television 2. . In addition, a member that can be hooked on the back side of the television 2 may be provided in the lighting device 9, and the lighting device 9 may be disposed by being hooked on the back side of the television 2.

Moreover, in other embodiment, you may comprise so that the illuminating device 9 and the marker apparatus 6 can be attached or detached. Furthermore, at this time, when the illumination device 9 and the marker device 6 are mounted, the marker device 6 can be mounted so that the illumination device 9 emits visible light to the opposite side of the direction in which the infrared light is emitted. Preferably there is. According to this, since infrared light can be emitted toward the front of the television 2 and visible light can be emitted toward the rear of the television 2, it can be applied to the rear wall of the television 2 as in the present embodiment. This is effective when it is assumed that visible light from the illumination device 9 is projected.

In the present embodiment, the marker device 6 and the illumination device 9 are separate, but in other embodiments, the marker device 6 and the illumination device 9 may be integrated. That is, you may make it include the infrared LED of the marker apparatus 6 and the optical module of the illuminating device 9 in a single housing | casing. When it is assumed that visible light from the illumination device 9 is projected onto the wall surface behind the television 2 as in the present embodiment, the illumination device is directed to the opposite side of the direction in which the marker device 6 emits infrared light. It is preferable to attach the marker device 6 and the illumination device 9 in the casing so that the light 9 emits visible light. According to this, infrared light can be emitted toward the front of the television 2 and visible light can be emitted toward the rear of the television 2.

As shown in FIG. 2, the game apparatus 3 is connected to the server 110 via the network 111.
And can communicate with the server 110 via the network 111. The network 111 may be any communication network in which the server 110 and a plurality of terminals (game devices) can communicate, and may be the Internet, a LAN, or a telephone line. Server 110
Instruction data indicating an instruction to cause the lighting device 9 to emit light is transmitted to the game apparatus 3. Although details will be described later, the server 110 transmits instruction data to the game apparatus 3 in order to cause the lighting device 9 to emit light in accordance with an image or sound of a television broadcast program. The server 110 is composed of one or more arbitrary computer devices. The network 111 may be connected to another game device of the same type as the game device 3 or another information processing device having the same function.
10 may transmit the instruction data to the other game device and the other information processing device.

[Internal configuration of game device 3]
Next, the internal configuration of the game apparatus 3 will be described with reference to FIG. FIG. 3 is a block diagram showing the configuration of each device of the game system 1. The game apparatus 3 includes a CPU 10, a system LSI 11, an external main memory 12, a ROM / RTC 13, a disk drive 14, an AV-IC 15 and the like.

The CPU 10 executes a game process by executing a game program stored on the optical disc 4, and functions as a game processor. The CPU 10 is connected to the system LSI 11. In addition to the CPU 10, an external main memory 12, a ROM / RTC 13, a disk drive 14, and an AV-IC 15 are connected to the system LSI 11. The system LSI 11 performs processing such as control of data transfer between components connected thereto, generation of an image to be displayed, and acquisition of data from an external device. The internal configuration of the system LSI will be described later. The volatile external main memory 12 stores a program such as a game program read from the optical disc 4 or a game program read from the flash memory 17, or stores various data. Used as a work area and buffer area. The ROM / RTC 13 includes a ROM (so-called boot ROM) in which a program for starting the game apparatus 3 is incorporated, and a clock circuit (RTC: Real Time Clock) that counts time. The disk drive 14 reads program data, texture data, and the like from the optical disk 4 and writes the read data to an internal main memory 11e or an external main memory 12 described later.

The system LSI 11 includes an input / output processor (I / O processor) 11a, GP
U (Graphics Processor Unit) 11b, DSP (Digital)
l Signal Processor) 11c, VRAM 11d, and internal main memory 11e are provided. Although not shown, these components 11a to 11e are connected to each other by an internal bus.

The GPU 11b forms part of the drawing means, and the graphics command (
The image is generated according to the drawing command. The VRAM 11d is data necessary for the GPU 11b to execute the graphics command (data such as polygon data and texture data).
Remember. When an image is generated, the GPU 11b creates image data using data stored in the VRAM 11d.

The DSP 11c functions as an audio processor, and generates sound data using sound data and sound waveform (tone color) data stored in the internal main memory 11e and the external main memory 12.

The image data and audio data generated as described above are read out by the AV-IC 15. The AV-IC 15 outputs the read image data to the television 2 via the AV connector 16, and outputs the read audio data to the speaker 2 a built in the television 2. As a result, an image is displayed on the television 2 and a sound is output from the speaker 2a.

The input / output processor 11a transmits / receives data to / from components connected to the input / output processor 11a or downloads data from an external device. Input / output processor 11
a is a flash memory 17, a wireless communication module 18, and a wireless controller module 1.
9, connected to the expansion connector 20 and the memory card connector 21. An antenna 22 is connected to the wireless communication module 18, and an antenna 23 is connected to the wireless controller module 19.

The input / output processor 11 a is connected to the network via the wireless communication module 18 and the antenna 22, and can communicate with other game devices connected to the network and various servers including the server 110. The input / output processor 11a periodically accesses the flash memory 17 to detect the presence / absence of data that needs to be transmitted to the network. Send. Further, the input / output processor 11a receives data transmitted from other game devices and data downloaded from the download server via the network, the antenna 22 and the wireless communication module 18, and receives the received data in the flash memory 17. Remember. The CPU 10 executes the game program to thereby execute the flash memory 17.
The data stored in is read out and used in the game program. In the flash memory 17, in addition to data transmitted and received between the game apparatus 3 and other game apparatuses and various servers, save data (game result data or intermediate data) of the game played using the game apparatus 3 May be stored.

The input / output processor 11a receives operation data transmitted from the controller 5 via the antenna 23 and the wireless controller module 19, and receives the internal main memory 11e.
Alternatively, it is stored (temporarily stored) in the buffer area of the external main memory 12.

Further, an expansion connector 20 and a memory card connector 21 are connected to the input / output processor 11a. The expansion connector 20 is a connector for an interface such as USB or SCSI, and connects a medium such as an external storage medium, a peripheral device such as another controller, or a wired communication connector. By connecting, communication with the network can be performed instead of the wireless communication module 18. In the present embodiment, the extension connector 20 is used to connect the lighting device 9 to the game device 3. The memory card connector 21 is a connector for connecting an external storage medium such as a memory card. For example, the input / output processor 11a can access an external storage medium via the expansion connector 20 or the memory card connector 21 to store data in the external storage medium or read data from the external storage medium.

The game apparatus 3 includes a power button 24, a reset button 25, and an eject button 2.
6 is provided. The power button 24 and the reset button 25 are connected to the system LSI 11. When the power button 24 is turned on, the game apparatus 3 passes through an AC adapter (not shown).
Power is supplied to each of the components. When the reset button 25 is pressed, the system L
SI11 restarts the startup program of the game apparatus 3. The eject button 26 is connected to the disk drive 14. When the eject button 26 is pressed, the optical disk 4 is ejected from the disk drive 14.

In the present embodiment, when the power button 24 is turned on, the system LSI 11
A mode in which power is supplied to each component of the game apparatus 3 via an AC adapter (not shown) and a normal energization state is set (referred to as “normal mode”) is set. On the other hand, when the power button 24 is turned off, the system LSI 11 is in a mode in which power is supplied to only some components of the game apparatus 3 and power saving control is performed to minimize power consumption (hereinafter “sleep”). Mode)). In this embodiment, when the sleep mode is set, the system LSI 11 includes the input / output processor 11a, the flash memory 17, and the external main memory 1.
2, ROM / RTC 13 and wireless communication module 18, wireless controller module 1
An instruction to stop power supply is given to the components other than 9. Therefore, this sleep mode is a mode in which no application is executed by the CPU 10. However,
Even in the sleep mode, the game apparatus 3 can receive data from the outside, and data transmitted from other game apparatuses or download servers is stored in the flash memory 17.

Note that power is supplied to the system LSI 11 even in the sleep mode. However, in the sleep mode, the system LSI 11 stops supplying clocks to the components of the GPU 11b, the DSP 11c, and the VRAM 11d. As a result, power consumption is reduced by not driving these components. Although not shown, a fan for exhausting the heat of the IC such as the CPU 10 and the system LSI 11 to the outside is provided inside the housing of the game apparatus 3. In the sleep mode, this fan is also stopped.

Further, the game apparatus 3 switches the normal mode and the sleep mode to the controller 5.
This can be done by remote control by pressing the power button. Note that when the switching by the remote operation is not performed, the power supply to the wireless controller module 19 may not be performed in the sleep mode. Further, the game apparatus 3 may be able to be set not to use the sleep mode according to a user instruction. When the sleep mode is not used, when the power button 24 is turned off, the power supply to all circuits is completely stopped.

The lighting device 9 is detachably connected to the game device 3 via the expansion connector 20. The illumination device 9 is a device that emits visible light for the purpose of giving the user a further visual effect in addition to the image displayed on the television 2. As shown in FIG. 3, the lighting device 9 includes a connector 27, a microcomputer 28, each optical module 29, and a speaker 112. The connector 27 is detachably connected to the expansion connector 20 of the game apparatus 3 via a cable (not shown). Note that the communication method between the game device 3 and the lighting device 9 may be any method, and in other embodiments, communication may be performed wirelessly.

The microcomputer 28 is connected to the connector 27, and each optical module 29 is connected to the microcomputer 28. Each optical module 29 is a plurality of light emitting devices that emit visible light. In the present embodiment, each optical module 29 is composed of a plurality of color LED modules capable of emitting a plurality of types of colors. The detailed configuration of each optical module 29 will be described later. The microcomputer 28 is a circuit that controls the light emission of each optical module 29. The game apparatus 3 transmits data indicating the color to be emitted by each light module 29 to the lighting apparatus 9. The microcomputer 28 acquires the data transmitted from the game apparatus 3 via the connector 27 and controls the light emission of each optical module 29 according to the data. The microcomputer 28 is connected to a speaker 112 which is an audio output means. In addition to the above data, the game apparatus 3 transmits data indicating a sound to be output by the speaker 112. The microcomputer 28 acquires data indicating sound via the connector 27 and causes the speaker 112 to output sound according to the data. The lighting device 9 may be supplied with power through an AC adapter (not shown) (independent of the game device 3), or may be supplied with power from the game device 3.

[Configuration of controller 5]
Next, the controller 5 will be described with reference to FIGS. FIG. 4 is a perspective view showing an external configuration of the controller 5. FIG. 5 is a perspective view showing an external configuration of the controller 5. FIG. 4 is a perspective view of the controller 5 as viewed from the upper rear side, and FIG. 5 is a perspective view of the controller 5 as viewed from the lower front side.

4 and 5, the controller 5 has a housing 31 formed by plastic molding, for example. The housing 31 has a substantially rectangular parallelepiped shape whose longitudinal direction is the front-rear direction (the Z-axis direction shown in FIG. 4), and is a size that can be gripped with one hand of an adult or a child as a whole. A user (player) can perform a game operation by pressing a button provided on the controller 5 and moving the controller 5 itself to change its position and posture.

The housing 31 is provided with a plurality of operation buttons. As shown in FIG. 4, on the upper surface of the housing 31, a cross button 32a, a first button 32b, a second button 32c, an A button 3
2d, a minus button 32e, a home button 32f, a plus button 32g, and a power button 32h are provided. In the present specification, the upper surface of the housing 31 on which these buttons 32a to 32h are provided may be referred to as a “button surface”. On the other hand, as shown in FIG. 5, a recess is formed on the lower surface of the housing 31, and a B button 32i is formed on the rear inclined surface of the recess.
Is provided. A function corresponding to the game program executed by the game apparatus 3 is appropriately assigned to each of the operation buttons 32a to 32i. The power button 32h is for remotely turning on / off the main body of the game apparatus 3. The home button 32 f and the power button 32 h are embedded in the upper surface of the housing 31. This can prevent the player from pressing the home button 32f or the power button 32h by mistake.

A connector 33 is provided on the rear surface of the housing 31. The connector 33 is used for connecting another device to the controller 5. Further, locking holes 33 are provided on both sides of the connector 33 on the rear surface of the housing 31 in order to prevent the other devices from being easily detached.
a is provided.

A plurality (four in FIG. 4) of LEDs 34a to 34d are provided behind the upper surface of the housing 31. Here, the controller type (number) is assigned to the controller 5 to distinguish it from other main controllers. The LEDs 34a to 34d are used for the purpose of notifying the player of the controller type currently set in the controller 5 and notifying the player of the remaining battery level of the controller 5. Specifically, when a game operation is performed using the controller 5, any one of the plurality of LEDs 34a to 34d is turned on according to the controller type.

Further, the controller 5 has an imaging information calculation unit 35 (FIG. 7). As shown in FIG. 5, a light incident surface 35a of the imaging information calculation unit 35 is provided on the front surface of the housing 31. The light incident surface 35a is made of a material that transmits at least infrared light from the markers 6R and 6L.

A sound release hole 31a is formed between the first button 32b and the home button 32f on the upper surface of the housing 31 for emitting sound from a speaker 49 (FIG. 6) built in the controller 5 to the outside.

Next, the internal structure of the controller 5 will be described with reference to FIGS. 6 and 7. FIG.
FIG. 7 is a diagram showing the internal structure of the controller 5. FIG. 6 is a perspective view showing a state in which the upper housing (a part of the housing 31) of the controller 5 is removed. FIG. 7 is a perspective view showing a state where the lower casing (a part of the housing 31) of the controller 5 is removed. The perspective view shown in FIG. 7 is a perspective view of the substrate 30 shown in FIG.

In FIG. 6, a substrate 30 is fixed inside the housing 31, and operation buttons 32 a to 32 h, LEDs 34 a to 34 d, an acceleration sensor 37, an acceleration sensor 37,
An antenna 45, a speaker 49, and the like are provided. These are connected to a microcomputer (microcomputer) 42 (see FIG. 7) by wiring (not shown) formed on the substrate 30 or the like. In the present embodiment, the acceleration sensor 37 is disposed at a position shifted from the center of the controller 5 with respect to the X-axis direction. This makes it easier to calculate the movement of the controller 5 when the controller 5 is rotated about the Z axis. The acceleration sensor 37 is disposed in front of the center of the controller 5 in the longitudinal direction (Z-axis direction). Further, the controller 5 functions as a wireless controller by the wireless module 44 (FIG. 7) and the antenna 45.

On the other hand, in FIG. 7, an imaging information calculation unit 35 is provided at the front edge on the lower main surface of the substrate 30. The imaging information calculation unit 35 includes an infrared filter 38 and a lens 3 in order from the front of the controller 5.
9, an image sensor 40 and an image processing circuit 41 are provided. These members 38 to 41 are respectively attached to the lower main surface of the substrate 30.

Further, the microcomputer 42 and the vibrator 48 are provided on the lower main surface of the substrate 30. The vibrator 48 is, for example, a vibration motor or a solenoid, and is connected to the microcomputer 42 by wiring formed on the substrate 30 or the like. The controller 48 is vibrated by the operation of the vibrator 48 according to the instruction of the microcomputer 42. As a result, a so-called vibration-compatible game in which the vibration is transmitted to the hand of the player holding the controller 5 can be realized. In the present embodiment, the vibrator 48 is disposed slightly forward of the housing 31. That is, by arranging the vibrator 48 on the end side of the center of the controller 5, the entire controller 5 can be vibrated greatly by the vibration of the vibrator 48. The connector 33 is attached to the rear edge on the lower main surface of the substrate 30. 6 and 7, the controller 5 includes a crystal resonator that generates a basic clock of the microcomputer 42, an amplifier that outputs an audio signal to the speaker 49, and the like.

Note that the shape of the controller 5, the shape of each operation button, the number of acceleration sensors and vibrators and the installation positions, etc. shown in FIGS. However, the present invention can be realized. In the present embodiment, the imaging direction by the imaging unit is the positive Z-axis direction, but the imaging direction may be any direction. That is, the position of the imaging information calculation unit 35 in the controller 5 (the light incident surface 3 of the imaging information calculation unit 35).
5a) may not be the front surface of the housing 31, and may be provided on another surface as long as light can be taken from the outside of the housing 31.

FIG. 8 is a block diagram showing the configuration of the controller 5. The controller 5 includes an operation unit 3
2 (each operation button 32a-32i), the connector 33, the imaging information calculating part 35, the communication part 36,
And an acceleration sensor 37. The controller 5 transmits data indicating the details of the operation performed on the own device to the game apparatus 3 as operation data.

The operation unit 32 includes the operation buttons 32 a to 32 i described above, and the operation buttons 32 a to 3.
Operation button data indicating an input state for 2i (whether or not each of the operation buttons 32a to 32i has been pressed) is output to the microcomputer 42 of the communication unit 36.

The imaging information calculation unit 35 is a system for analyzing the image data captured by the imaging unit, discriminating a region having a high luminance in the image data, and calculating a center of gravity position, a size, and the like of the region. Since the imaging information calculation unit 35 has a sampling period of, for example, about 200 frames / second at the maximum, it can track and analyze even a relatively fast movement of the controller 5.

The imaging information calculation unit 35 includes an infrared filter 38, a lens 39, an imaging element 40, and an image processing circuit 41. The infrared filter 38 passes only infrared rays from the light incident from the front of the controller 5. The lens 39 collects the infrared light transmitted through the infrared filter 38 and makes it incident on the image sensor 40. The image sensor 40 is a solid-state image sensor such as a CMOS sensor or a CCD sensor, for example, and receives the infrared light collected by the lens 39 and outputs an image signal. Here, the marker 6 of the marker device 6 disposed in the vicinity of the display screen of the television 2.
R and 6L are composed of infrared LEDs that output infrared light toward the front of the television 2.
Therefore, by providing the infrared filter 38, the image sensor 40 receives only the infrared light that has passed through the infrared filter 38 and generates image data. Therefore, the markers 6R and 6
L images can be taken more accurately. Hereinafter, an image captured by the image sensor 40 is referred to as a captured image. Image data generated by the image sensor 40 is processed by the image processing circuit 41. The image processing circuit 41 calculates the position of the imaging target (markers 6R and 6L) in the captured image. The image processing circuit 41 sets the coordinates indicating the calculated position to the communication unit 3.
6 to the microcomputer 42. The coordinate data is transmitted to the game apparatus 3 as operation data by the microcomputer 42. Hereinafter, the coordinates are referred to as “marker coordinates”. Since the marker coordinates change corresponding to the direction (tilt angle) and position of the controller 5 itself, the game apparatus 3 can calculate the direction and position of the controller 5 using the marker coordinates.

In other embodiments, the controller 5 may not include the image processing circuit 41, and the captured image itself may be transmitted from the controller 5 to the game apparatus 3. At this time, the game apparatus 3 may have a circuit or a program having the same function as the image processing circuit 41, and may calculate the marker coordinates.

The acceleration sensor 37 detects the acceleration (including gravity acceleration) of the controller 5, that is, detects the force (including gravity) applied to the controller 5. The acceleration sensor 37 detects the value of the acceleration (linear acceleration) in the linear direction along the sensing axis direction among the accelerations applied to the detection unit of the acceleration sensor 37. For example, in the case of a multi-axis acceleration sensor having two or more axes, the component acceleration along each axis is detected as the acceleration applied to the detection unit of the acceleration sensor. For example, the triaxial or biaxial acceleration sensor may be of the type available from Analog Devices, Inc. or ST Microelectronics NV. The acceleration sensor 37 is, for example, a capacitance type acceleration sensor, but other types of acceleration sensors may be used.

In the present embodiment, the acceleration sensor 37 has a vertical direction (Y-axis direction shown in FIG. 4), a horizontal direction (X-axis direction shown in FIG. 4), and a front-back direction (Z-axis direction shown in FIG. 4) with reference to the controller 5. ) Linear acceleration is detected in each of the three axis directions. Since the acceleration sensor 37 detects acceleration in a linear direction along each axis, the output from the acceleration sensor 37 is 3
It represents the linear acceleration value of each axis. That is, the detected acceleration is expressed as a three-dimensional vector (ax, ay, az) in an XYZ coordinate system (controller coordinate system) set with the controller 5 as a reference. Hereinafter, a vector having the respective acceleration values related to the three axes detected by the acceleration sensor 37 as components is referred to as an acceleration vector.

Data indicating the acceleration detected by the acceleration sensor 37 (acceleration data) is output to the communication unit 36. The acceleration detected by the acceleration sensor 37 changes in accordance with the direction (tilt angle) and movement of the controller 5 itself, so that the game apparatus 3 can calculate the direction and movement of the controller 5 using the acceleration data. it can. In the present embodiment, the game apparatus 3 determines the attitude (tilt angle) of the controller 5 based on the acceleration data. That is, the acceleration sensor 37 is used as a sensor that outputs data for determining the tilt angle of the controller 5.

Note that, based on the acceleration signal output from the acceleration sensor 37, the processor (for example, the CPU 10) of the game apparatus 3 or the processor (for example, the microcomputer 42) of the controller 5.
Those skilled in the art can easily understand from the description of the present specification that further information regarding the controller 5 can be estimated or calculated (determined) by the processing of the computer. For example, when processing on the computer side is executed on the assumption that the controller 5 on which the acceleration sensor 37 is mounted is stationary (that is, the processing is executed assuming that the acceleration detected by the acceleration sensor is only gravitational acceleration). When the controller 5 is actually stationary, it can be determined whether or not the attitude of the controller 5 is inclined with respect to the direction of gravity based on the detected acceleration. Specifically, whether or not the controller 5 is inclined with respect to the reference depending on whether or not 1G (gravity acceleration) is applied, based on the state in which the detection axis of the acceleration sensor 37 is directed vertically downward. It is possible to know how much it is inclined with respect to the reference according to its size. Further, in the case of the multi-axis acceleration sensor 37, it is possible to know in detail how much the controller 5 is inclined with respect to the direction of gravity by further processing the acceleration signal of each axis. . In this case, the processor
The tilt angle of the controller 5 may be calculated based on the output from the acceleration sensor 37,
The tilt direction of the controller 5 may be calculated without calculating the tilt angle. Thus, by using the acceleration sensor 37 in combination with the processor, the tilt angle or posture of the controller 5 can be determined.

On the other hand, when it is assumed that the controller 5 is in a dynamic state (a state in which the controller 5 is moved), the acceleration sensor 37 detects an acceleration corresponding to the movement of the controller 5 in addition to the gravitational acceleration. Therefore, the movement direction of the controller 5 can be known by removing the gravitational acceleration component from the detected acceleration by a predetermined process. Even if it is assumed that the controller 5 is in a dynamic state, from the detected acceleration,
By removing the acceleration component according to the motion of the acceleration sensor by a predetermined process, it is possible to know the inclination of the controller 5 with respect to the direction of gravity. In other embodiments,
The acceleration sensor 37 is a built-in processing device or another type of dedicated device for performing predetermined processing on the acceleration signal before outputting the acceleration signal detected by the built-in acceleration detection means to the microcomputer 42. A processing device may be provided. A built-in or dedicated processing device converts the acceleration signal into a tilt angle (or other preferred parameter) if, for example, the acceleration sensor 37 is used to detect static acceleration (eg, gravitational acceleration). It may be a thing.

In this embodiment, for example, a capacitance type acceleration sensor is used as a sensor that outputs a value that changes in accordance with the movement of the controller. However, other types of acceleration sensors or gyro sensors are used. May be. However, the acceleration sensor detects acceleration in a linear direction along each axis, whereas the gyro sensor detects angular velocity associated with rotation. In other words, when a gyro sensor is employed instead of the acceleration sensor, the nature of the detected signal is different, and therefore both cannot be easily replaced. Therefore, when calculating the attitude (tilt angle) using a gyro sensor instead of the acceleration sensor, for example, the following changes are made. Specifically, the game apparatus 3 initializes the posture value in the detection start state. Then, the angular velocity data output from the gyro sensor is integrated. Further, using the integration result, the amount of change in posture is calculated from the initialized posture value. In this case, the calculated posture is represented by an angle.

As already described, when the inclination angle (posture) is calculated by the acceleration sensor, the inclination angle is calculated using the acceleration vector. Therefore, the calculated tilt angle can be expressed as a vector, and the absolute direction can be calculated without performing initialization. In the case of using an acceleration sensor and the case of using a gyro sensor. Is different. In addition, since the property of the value calculated as the tilt angle also differs depending on whether it is an angle or a vector as described above, when replacing the acceleration sensor with the gyro sensor, the data of the tilt angle is included. Even for this, it is necessary to perform predetermined conversion.

The communication unit 36 includes a microcomputer 42, a memory 43, a wireless module 44, and an antenna 45.
Is included. The microcomputer 42 controls the wireless module 44 that wirelessly transmits data acquired by the microcomputer 42 to the game apparatus 3 while using the memory 43 as a storage area when performing processing. The microcomputer 42 is connected to the connector 33.

Data output from the operation unit 32, the imaging information calculation unit 35, and the acceleration sensor 37 to the microcomputer 42 is temporarily stored in the memory 43. These data are transmitted to the game apparatus 3 as the operation data. That is, the microcomputer 42 outputs the operation data stored in the memory 43 to the wireless module 44 when the transmission timing to the wireless controller module 19 of the game apparatus 3 arrives. The wireless module 44 is, for example, Bluetooth
Using a technology of oth (Bluetooth) (registered trademark), a carrier wave of a predetermined frequency is modulated with operation data, and the weak radio signal is radiated from the antenna 45. That is, the operation data is modulated by the wireless module 44 into a weak radio signal and transmitted from the controller 5. The weak radio signal is received by the wireless controller module 19 on the game apparatus 3 side. By demodulating and decoding the received weak radio signal, the game apparatus 3 can acquire operation data. And CPU10 of the game device 3 performs a game process based on the acquired operation data and a game program. Note that the wireless transmission from the communication unit 36 to the wireless controller module 19 is sequentially performed at predetermined intervals, but the game processing is generally performed in units of 1/60 seconds (one frame time). Therefore, it is preferable to perform transmission at a period equal to or shorter than this time. The communication unit 36 of the controller 5 outputs each operation data to the wireless controller module 19 of the game apparatus 3 at a rate of once every 1/200 seconds, for example.

By using the controller 5, the player can perform an operation of tilting the controller 5 to an arbitrary tilt angle in addition to the conventional general game operation of pressing each operation button. In addition, according to the controller 5, the player can also perform an operation of instructing an arbitrary position on the screen by the controller 5 and an operation of moving the controller 5 itself.

[Configuration of Lighting Device 9]
Next, the configuration of the illumination device 9 will be described with reference to FIGS. FIG. 9 is an external view of the illumination device 9. FIG. 10 is a perspective view showing a main configuration in the illumination device 9. In FIG. 9, the illumination device 9 includes a housing 51, a cover 52, a shaft head 53, a support shaft 54, a support member 55, and eight optical modules 61 to 68. The illuminating device 9 emits visible light, and projects visible light on the rear wall of the television 2 to give the user a further visual effect in addition to the image displayed on the television 2. . As described above, the lighting device 9 includes the speaker 112, but the speaker 112 is not shown in FIG. The arrangement position of the speaker 112 in the lighting device 9 is arbitrary, and is appropriately attached to an empty space in the housing 51.

As shown in FIG. 9, the casing 51 has an open top surface (a surface on the y-axis positive direction side shown in FIG. 9).
Inside the casing 51, there are a support shaft 54, a support member 55, and eight optical modules 61 to 68.
Is installed. On the support member 55, eight optical modules 61 to 68 (each optical module shown in FIG. 3) are attached. Although details will be described later, each of the optical modules 61 to 68 is a member that emits visible light. A transparent cover 52 is attached to the opening above the support member 55 and each of the optical modules 61 to 68. Therefore, the light emitted from each of the optical modules 61 to 68 passes through the cover 52 and is emitted to the outside of the housing 51. In another embodiment, the lighting device 9 may be configured without the cover 52, and the emission surfaces of the optical modules 61 to 68 may be exposed to the outside.

As shown in FIG. 10, the support member 55 is connected to the support shaft 54 at both ends thereof (end portions in the x-axis direction shown in FIG. 10). The support shaft 54 is inserted into a hole provided on the side surface of the housing 51. Accordingly, the support member 55 is rotatably supported by the casing 51 (around the x axis). The support shaft 54 is connected to the shaft head 53 on the outside of the housing 51.
Therefore, the user can rotate the supporting member 55 (change the inclination) by rotating the shaft head 53. As described above, since the illumination device 9 has the tilt mechanism including the support member 55 and the support shaft 54, the user can change the tilt of the support member 55 to change the emission direction of each of the optical modules 61 to 68. Can be changed.

Next, the internal structure of the optical module will be described. Among the optical modules 61 to 68, seven optical modules 61 to 67 arranged in substantially the same direction (exactly different).
Emits light ("linear light" to be described later) having a cross section of light having a longitudinal shape (linear shape). These 7
The two optical modules 61 to 67 have the same configuration. On the other hand, the optical module 6
8 emits light having a wider cross section than the light from the optical modules 61 to 67 ("background light" to be described later). Hereinafter, for the purpose of distinguishing between the optical modules 61 to 67 and the optical module 68, the former is sometimes referred to as a “linear optical module” and the latter is sometimes referred to as a “background light module”.

FIG. 11 is a diagram illustrating an internal structure of the linear optical module. In FIG. 11, the linear optical module 61 includes a casing 71, a color LED module 72, a condenser lens 73, and a diffusion sheet 74. In addition, in FIG. 11, although the internal structure of the linear optical module 61 is shown,
The internal configurations of the other linear optical modules 62 to 67 are the same as those in FIG.

As shown in FIG. 11, the casing 71 has a box shape with an upper surface opened. A color LED module 72 is attached to the substrate (bottom) in the housing 71. The color LED module 72 can emit light of a plurality of types of colors. In the present embodiment, the color LED module 72 includes a red LED 75, a green LED 76, and a blue LED 77, and each LED
The microcomputer 28 (FIG. 2) appropriately adjusts the light emission intensities of 75 to 77 (for example, 256 levels each) so that light can be emitted in a desired color. The three LEDs 75 to 77 are arranged in a row (in FIG. 11, in a row along the axis L parallel to the a-axis direction).

A condensing lens 73 is installed above the color LED 72. In FIG. 11, the condenser lens 73 is attached to the opening of the housing 71. In the present embodiment, the condenser lens 73 focuses light only in one direction. In the present embodiment, the focusing direction of the condenser lens 73 (the direction in which the lens has a curvature) is the c-axis direction shown in FIG. 11 (see the arrow shown in FIG. 11). In the present embodiment, in order to reduce the size of the optical module 61, the Fresnel lens (
Linear Fresnel lens) is used. In another embodiment, a convex cylindrical lens may be used as the condensing lens 73 instead of the linear Fresnel lens.

A diffusion sheet 74 is attached on the upper side of the condenser lens 73. The diffusion sheet 74 diffuses light only in a direction perpendicular to the focusing direction of the condenser lens 73, that is, in the a-axis direction. In other embodiments, a concave cylindrical lens may be used instead of the diffusion sheet 74.

FIG. 12 is a diagram illustrating light emitted from the linear optical module. As shown in FIG.
Light from the color LED module 72 is condensed in the c-axis direction by the condenser lens 73 and diffused in the a-axis direction by the diffusion sheet 74. As a result, the cross-sectional shape of the light emitted from the linear optical module becomes a long shape (linear) that is long in the a-axis direction and short in the c-axis direction, as shown in FIG. Here, the “light cross-sectional shape” refers to a shape when the light is applied to a surface perpendicular to the traveling direction of the light. Hereinafter, light having a cross-sectional shape of a longitudinal shape (linear shape) is referred to as “linear light”.

In the present embodiment, the focusing direction of the condenser lens 73 (arrow shown in FIG. 11) is perpendicular to the arrangement direction of the three LEDs 75 to 77 (direction of the axis L shown in FIG. 11).
That is, the three LEDs 75 to 77 are arranged side by side along the axis L parallel to the a-axis direction, whereas the condenser lens 73 is arranged such that the focusing direction is the c-axis direction perpendicular to the a-axis direction. Installed. This mixes the three lights from the three LEDs 75-77 without any deviation,
This is because the linear light projected on the wall surface is clearly displayed (a single color is displayed). Here, if the three LEDs 75 to 77 are arranged so as to be shifted with respect to the axis L, the LEDs 75 to 7 are arranged.
The light emitted from 7 through the condenser lens 73 is shifted in the c-axis direction. FIG.
These are figures which show an example of the linear light projected when three LED75-77 is shifted | deviated from the axis | shaft L and is arrange | positioned. In the above case, the three lights from the LEDs 75 to 77 are shifted with respect to the c-axis direction and projected onto the wall surface. As a result, as shown in FIG. In the region 82 at the end with respect to the short direction of the linear light, the colors of the three lights are not mixed. On the other hand, in the present embodiment, the LEDs 75 to 77 are arranged in a direction perpendicular to the focusing direction of the condensing lens 73 so that the positions of the projected lights in the short direction coincide with each other. You can see the shining light cleanly.

In the present embodiment, the condensing lens 73 and the diffusion sheet 74 are used to project linear light onto the wall surface. Here, in other embodiments, the linear light may be generated by any method. For example, the linear light may be created using only the condensing lens 73 without using the diffusion sheet 74, or may be created using a slit instead of the condensing lens 73 and the diffusion sheet 74. Further, the linear light modules 61 to 67 may realize linear light by projecting laser light instead of light from the LED.

The background light module 68 emits light (background light) having a wider cross section than the light from the linear light modules 61 to 67. In this embodiment, the background light module 68 has a housing and a color L.
It is a structure provided with an ED module, and does not have a condensing lens and a diffusion sheet. The color LED module of the background light module 68 has three LEDs of red, green, and blue, but these LEDs have higher luminance than the LEDs used in the linear light modules 61 to 67. And what emits light in a wide angle is used. In the present embodiment, the background light module 68 emits light with an intensity smaller than the intensity of the light of the linear light modules 61 to 67 so that the linear light is more conspicuous than the background light. Conversely, if the light intensity of the background light module 68 is made higher than the light intensity of the linear light modules 61 to 67, the background light can be made conspicuous. Further, only one of background light and linear light may be projected. In other embodiments, a condensing lens may be used in the background light module 68 as in the linear light modules 61 to 67, or a condensing lens and a diffusion sheet may be used. When the condenser lens is used, the converging direction is the linear optical modules 61 to 6.
7 is the same as the focusing direction. In addition, a diffusion sheet that diffuses light in both the vertical and horizontal directions is used.

Next, with reference to FIG. 14, arrangement | positioning of each optical module 61-68 in the illuminating device 9 is demonstrated. FIG. 14 is a trihedral view showing the arrangement of the optical modules. 14A is a view of the optical modules 61 to 68 viewed from the y-axis positive side (upper side), and FIG. 14B is a view of the optical modules 61 to 68 viewed from the z-axis negative side (front side). 68 is a view of the optical modules 61 to 68 as viewed from the negative side of the x-axis. 14 shows a state in which the emission direction of the optical module 64 is vertically upward for the purpose of facilitating the understanding of the description.
Each of the optical modules 61 to 6 is configured so that light is emitted from the illuminating device 9 obliquely upward to the rear.
8 is inclined and arranged.

The linear light modules 61 to 68 emit respective linear lights toward the obliquely upward rear side of the illumination device 9 so as to be radially centered on the illumination device 9. Specifically, (
As shown in FIGS. a and b, the linear optical modules 61 to 67 are arranged side by side in the x-axis direction (left-right direction), and are symmetrical with respect to the yz plane with the linear optical module 64 as the center. Be placed. Further, when viewed from the upper side (y-axis positive side) as shown in FIG. 14A, each of the linear optical modules 61 to 67 has a longitudinal direction (linear light) of the emission surface of each of the linear optical modules 61 to 67. (Longitudinal direction) of the lighting device 9 is arranged so as to have a substantially radial orientation centered on a predetermined position behind the lighting device 9. That is, when viewed from the upper side, each of the linear optical modules 61 to 67 is disposed so that the rear end (the end on the z-axis positive side) is inclined inward as the module is disposed on the outer side. Furthermore, when viewed from the front side as shown in FIG. 14B, each of the linear optical modules 61 to 67 has the emission direction of each of the linear optical modules 61 to 67 centered on a predetermined position below the illumination device 9. It arrange | positions so that it may become substantially radial. That is, when viewed from the front side, each of the linear optical modules 61 to 61
67 is arranged so that the module arranged on the outside is inclined such that the emission direction faces the outside.

The background light module 68 emits background light in a direction overlapping with each linear light emitted from the linear light modules 61 to 67. Specifically, as illustrated in FIG. 14A, the background light module 68 is disposed approximately at the center of each of the linear light modules 61 to 67 with respect to the x-axis direction (left-right direction). Further, the background light module 68 is arranged behind the respective linear light modules 61 to 67 in the z-axis direction (front-rear direction). Furthermore, as illustrated in FIG. 14C, the background light module 68 is disposed so as to be inclined so that the emission direction is slightly downward from the linear light module 64. In other embodiments, the background light module 68 may have an emission direction upward from the linear light module 64 or may be in substantially the same direction.

FIG. 15 is a diagram showing linear light and background light projected onto the back surface of the television 2 by the lighting device 9. In FIG. 15, an area 91 is an area where light from the linear optical module 61 is projected, an area 92 is an area where light from the linear optical module 62 is projected, and an area 93 is linear light. This is a region where light from the module 63 is projected, and a region 94 is the linear optical module 6.
4 is a region where light from the linear optical module 65 is projected, a region 95 is a region where light from the linear optical module 65 is projected, and a region 96 is a region where light from the linear optical module 66 is projected. The region 97 is a region where light from the linear optical module 67 is projected, and the region 98 is a region where light from the linear optical module 68 is projected. As shown in FIG. 15, in this embodiment, since light is projected on the back of the television 2, when viewed from the front side of the television 2, it looks as if decoration around the display screen is given by light. An effective lighting effect (decorative effect) can be imparted to the screen. In addition, if the light from the lighting device 9 is emitted toward the user (toward the front), only one point of the lighting device 9 appears to be shining for the user, which is effective as a decoration effect on the display screen. Will be small. On the other hand, in this embodiment, the light of the illumination device 9 is emitted backward to project light onto the wall surface so that the light hitting the wall surface is shown to the user. Therefore, it is possible to show the user light over a wider range than the display screen of the television 2, and a high decoration effect can be obtained.

By arranging each of the linear optical modules 61 to 67 as shown in FIG. 14, as shown in FIG. Linear light can be projected. According to this embodiment, since the illuminating device 9 projects a plurality of linear lights, it is possible to increase the light projection pattern and increase the variation of the illumination effect as compared with the case where only one light is projected. be able to.

In general, behind the TV 2, there is a flat wall as well as a wall with a concave angle (typically when the TV 2 is placed in the corner of the room) There may be cases where the wall surface is uneven, such as when a curtain is present. When the wall surface is uneven as described above, the projected light may be distorted due to the unevenness. For example, when horizontally long light (light whose cross section is long in the horizontal direction) is projected onto a wall surface with an uneven surface, the projected light does not appear as a single line. In addition, for example, even if a plurality of lights are emitted so as to be arranged horizontally, if the projection surface is uneven, the projected lights will not be arranged horizontally. On the other hand, in this embodiment, since the illuminating device 9 projects a plurality of linear lights so as to be radial, distortion of the linear lights due to the unevenness of the projection surface behind the television 2 is not conspicuous. Even if the wall surfaces are different, the same radial linear light can be projected. For example, even if a plurality of linear lights are emitted toward the corner of the room,
Compared to the case of emission on a flat surface, the radial lines are projected straight only by the way the radial lines are opened, so that the visual discomfort is less. Further, since the surface of the curtain or the like generally has most unevenness in the horizontal direction and does not form so much unevenness in the vertical direction, similarly, light is projected while the appearance of linear light is close to linear. be able to.

As shown in FIG. 15, in this embodiment, the background light module 68 is arranged as shown in FIG. 14, so that linear light and background light can be projected on the wall surface.
Therefore, the illumination device 9 can control not only the color of the linear light but also the color of the background thereof. Therefore, the illumination pattern can be increased as compared with the case of controlling only the linear light. The variation of the effect can be increased. In the present embodiment, the background light module 6
Since the intensity of the light No. 8 is set to be smaller than the intensity of the light of the linear light modules 61 to 67, the region where both the background light and the linear light are projected looks like the color of the linear light. That is, the wall surface appears to be projected with linear light superimposed on the background light.

Note that the arrangement of the optical modules 61 to 68 illustrated in FIG. 14 is an example, and the linear optical modules 61 to 67 have a radial shape with the illumination device 9 approximately at the center with respect to the rear wall of the television 2. As long as it is arranged so as to project linear light, it may be arranged in any way. Moreover, in other embodiment, each linear light module 61-67 may be arrange | positioned so that each linear light may be projected side by side in parallel, for example, and each linear light may represent a predetermined shape. May be arranged. On the other hand, the background light module 68 may be arranged in any manner as long as the background light module 68 is arranged to emit light in a direction overlapping with the light of the linear light modules 61 to 67. In another embodiment, the background light module 68 may be arranged so as to emit light in a direction overlapping with the light of some of the linear optical modules 61 to 67.

[Processing of Game Device 3 Regarding Control of Lighting Device 9]
Next, the process performed in the game apparatus 3 is demonstrated centering on the process regarding control with respect to the illuminating device 9. FIG. In the present embodiment, the game device 3 displays a game image on the television 2 and adds a lighting effect using the lighting device 9 when executing the game process according to the game program stored in the optical disc 4. In addition, the game apparatus 3 adds an illumination effect using the illumination device 9 when the television 2 displays an image of a television broadcast program. Hereinafter, the process performed by the game apparatus 3 will be described focusing on the control of the lighting device 9 when the game is executed and the control of the lighting device 9 when the image of the television program is displayed.

(Processing of game device 3 when executing a game)
Hereinafter, processing of the game apparatus 3 when executing the game will be described. First, main data used in the processing will be described with reference to FIG. FIG. 16 is a diagram illustrating main data stored in the main memory (the external main memory 12 or the internal main memory 11e) of the game apparatus 3. As shown in FIG. 16, a game program 101, operation data 102, and processing data 106 are stored in the main memory of the game apparatus 3. In addition to the data shown in FIG. 16, the main memory stores data necessary for the game process, such as image data of various objects appearing in the game and data indicating various parameters of the object.

Part or all of the game program 101 is read from the optical disc 4 and stored in the main memory at an appropriate timing after the game apparatus 3 enters the normal mode described above. The game program 101 includes a program for executing game processing and a program for controlling light emission of the lighting device 9 in accordance with the game processing (step S5 described later).

The operation data 102 is operation data transmitted from the controller 5 to the game apparatus 3. As described above, since the operation data is transmitted from the controller 5 to the game apparatus 3 at a rate of once every 1/200 second, the operation data 102 stored in the main memory is updated at this rate. The operation data 102 includes operation button data 103 and marker coordinate data 104.
, And acceleration data 105 are included.

The operation button data 103 is data indicating an input state for each of the operation buttons 32a to 32i. That is, the operation button data 103 indicates whether or not each of the operation buttons 32a to 32i is pressed.

The marker coordinate data 104 is data indicating coordinates calculated by the image processing circuit 41 of the imaging information calculation unit 35, that is, the marker coordinates. The marker coordinates are expressed in a two-dimensional coordinate system for representing a position on a plane corresponding to the captured image. Note that 2 depending on the image sensor 40.
When two markers 6R and 6L (infrared light) are imaged, two marker coordinates are calculated. On the other hand, if either one of the markers 6R and 6L is not located within the imageable range of the image sensor 40, only one marker is imaged by the image sensor 40, and only one marker coordinate is calculated. In addition, the markers 6R and 6L are within the image capturing range of the image sensor 40.
If neither of these is located, the marker is not imaged by the image sensor 40, and the marker coordinates are not calculated. Accordingly, the marker coordinate data 104 may indicate two marker coordinates, may indicate one marker coordinate, or may indicate that there is no marker coordinate.

The acceleration data 105 is data indicating the acceleration (acceleration vector) detected by the acceleration sensor 37. Here, the acceleration data 105 indicates a three-dimensional acceleration vector having acceleration components in the directions of the three axes of XYZ shown in FIG.

The processing data 106 is data used in a game process (FIG. 17) described later. The processing data 106 includes game data 107 and light emission control data 108. The game data 107 is game parameter data, and is game parameter data that affects the control of the lighting device 9. For example, the game data 107 may be data indicating parameters of characters appearing in the game, or data indicating the position of a cursor displayed on the screen.

The light emission control data 108 is data for controlling the light emission of the lighting device 9. In the present embodiment, the light emission control data 108 indicates the color and intensity of light that each optical module 29 of the illumination device 9 should emit. Specifically, for example, the emission intensity of each of red, green, and blue LEDs is shown in 256 levels. Although details will be described later, the light emission control data 108 is transmitted to the lighting device 9 and acquired by the microcomputer 28, and the microcomputer 28 controls each optical module 29 according to the light emission control data 108.

Next, details of processing performed in the game apparatus 3 will be described with reference to FIG. FIG.
These are flowcharts which show the flow of the process performed in the game device 3 when performing a game. When an instruction to start a game is input from the user to the game apparatus 3 in the normal mode, the CPU 10 of the game apparatus 3 executes a startup program stored in a boot ROM (not shown). Each unit is initialized.
Then, the game program stored on the optical disc 4 is read into the main memory, and the CP
Execution of the game program is started by U10. The flowchart shown in FIG. 17 is a flowchart showing a process performed after the above process is completed.

First, in step S1, the CPU 10 executes an initialization process relating to the game.
In this initialization process, the values of various parameters used in the game process are initialized, a virtual game space is constructed, and player objects and other objects are placed at initial positions in the game space. After step S1, the processing loop of steps S2 to S6 is repeatedly executed while the game is executed. One processing loop is 1
It is executed once per frame time (for example, 1/60 seconds).

In step S2, the CPU 10 acquires operation data. That is, operation data transmitted from the controller 5 is received via the wireless controller module 19. Then, the operation button data, marker coordinate data, and acceleration data included in the received operation data are stored in the main memory. Step S3 is followed by Step S3
The process is executed.

In step S3, the CPU 10 executes a game process based on the operation data acquired in step S2. Specifically, for example, a process of controlling the action of the game character based on the operation data, or a process of calculating the position of the cursor displayed on the screen based on the operation data (particularly, for example, the marker coordinate data 104) Or execute. At this time, game data 107 obtained as a result of the game process is stored in the main memory. Following step S3, the process of step S4 is executed.

In step S <b> 4, the CPU 10 displays a game image corresponding to the game process executed in step S <b> 3 on the screen of the television 2. Specifically, CPU 10 (and GPU 11b
) Reads out the game data 107 from the main memory, generates a game image based on the game data 107 and the like, and displays it on the screen. The game image may be, for example, an image of a game space including a game character, or may include a cursor image superimposed on the image. In step S4, the CPU 10 (and the DSP 11c) generates a game sound corresponding to the game process based on the game data 107 and the like, and outputs it from the speaker 2a. Note that the game sound may be BGM during the game, a sound effect of the game, a voice of a game character, or the like. Following step S4, the process of step S5 is executed.

In step S <b> 5, the CPU 10 controls light emission by the lighting device 9. That is,
The CPU 10 reads the game data 107 from the main memory, and generates light emission control data 108 indicating the color and intensity of light that each light module 29 should emit based on the game data 107. The generated light emission control data 108 is stored in the main memory, and is transmitted to the lighting device 9 via the expansion connector 20 by the input / output processor 11a. The microcomputer 28 of the lighting device 9 that has received the light emission control data 108 controls the light emission of each optical module 29 according to the light emission control data 108. Note that the time interval for transmitting the light emission control data 108 to the lighting device 9 may be any amount, but in this embodiment, the game image update interval (1/60 seconds).
Is the same. As described above, by repeatedly transmitting the light emission control data 108, the game apparatus 3 can change the light projection by the lighting apparatus 9, for example, depending on the game situation, game image, game sound, game operation, and the like. The lighting effect can be given to the user. Following step S5, the process of step S6 is executed.

In step S5, the CPU 10 determines the light emission state of the illumination device 9 according to the game process in step S3. For example, the light emission state of the lighting device 9 is determined based on the game situation, game image, game sound, game operation, and the like. Specific examples of the control of the lighting device 9 include the following.
(1) Control example according to game image (game situation) The CPU 10 changes the light emission state (light intensity, color, light emission pattern, etc.) of the lighting device 9 according to the change of the game image (game situation) on the screen. Change. For example, in a fighting game, the light emitting state of the lighting device 9 may be changed in response to the attack of the character hitting another character, or the lighting device 9 in response to a bullet hitting the target in the shooting game.
The light emission state may be changed. According to this, an effective lighting effect can be added to the game image by adding the lighting effect according to the change of the game image.
(2) Control example according to game operation CPU10 changes the light emission state of the illuminating device 9 according to a user's game operation. Specifically, the CPU 10 may cause each of the optical modules 61 to 68 to emit light or change the color of light in response to the user pressing an operation button on the controller 5.
When the position of the cursor on the screen is controlled according to the operation of the controller 5,
CPU10 may change the light emission state of each optical module 61-68 according to the position of the cursor in a screen. For example, a region obtained by dividing the screen into seven in the horizontal direction is a linear optical module 6.
1 to 67 one by one, and only the linear optical module corresponding to the region including the cursor position may be caused to emit light.
(3) Control example according to game sound CPU10 changes the light emission state of the illuminating device 9 according to music, such as BGM which flows in a game. Specifically, the light modules 61 to 68 are blinked according to the rhythm of the music, or the light emission pattern and / or the light emission color are changed according to the pitch (pitch) of the output sound. Further, in a performance game in which music is played according to a game operation by the user, the lighting device 9 may be controlled in accordance with the music to be played. According to this, the user can enjoy music played by his / her own operation not only by hearing but also visually.

In step S6, the CPU 31 determines whether or not to end the game. The determination in step S6 is, for example, whether the game is cleared, whether the game is over,
This is performed depending on whether or not the user gives an instruction to stop the game. If the determination result of step S6 is negative, the process of step S2 is executed again. Thereafter, the processing loop of steps S2 to S6 is repeatedly executed until it is determined in step S6 that the game is to be ended. On the other hand, when the determination result of step S6 is affirmative, the CPU 31 ends the game process shown in FIG. This is the end of the description of the game process.

As described above, according to the present embodiment, a game image is displayed on the television 2, and a lighting effect is added to the wall surface behind the television 2 by light projection by the lighting device 9. According to this, since a further visual effect can be given to the user in addition to the game image, the power and sense of presence of the game can be further improved.

(Processing of game device 3 when an image of a television program is displayed)
Next, with reference to FIG. 18 and FIG. 19, processing of the game apparatus 3 when an image of a television program is displayed will be described. FIG. 18 is a main flowchart showing a flow of processing executed in the game apparatus 3 when an image of a television program is displayed. In addition, FIG.
FIG. 19 is a diagram showing a data flow between the game apparatus 3 and another apparatus when the process shown in FIG. 18 is executed.

In the present embodiment, the game apparatus 3 stores an application for watching TV.
When an instruction to start the application is input by the user in the normal mode, the game apparatus 3 starts executing the application. That is, after each unit such as the main memory is initialized, the program of the application is read into the main memory, and the CPU 10 starts executing the program. The flowchart shown in FIG. 18 is a flowchart showing a process performed after the above process is completed. In addition,
In FIG. 18, only the processing related to the control of the lighting device 9 among the processing by the application is described, but the application displays, for example, an EPG (electronic program guide) or an instruction to the television 2 (channel selection). Or a function of changing the volume, etc.).

In the present embodiment, before the processing shown in FIG. 18 is executed (that is, before the application start instruction is given), the user turns on the television 2 to display a program of a desired channel. Let me let you.

In FIG. 18, first, in step S <b> 11, the CPU 10 receives an input of a channel displayed on the television 2 (allows the user to input a channel). That is, C
The PU 10 acquires operation data by the same process as in step S2, and accepts a user instruction to input a channel. At this time, the user inputs a channel displayed on the television 2. Note that any interface may be used for allowing the user to input a channel. For example, the CPU 10 may assign a channel to each button of the controller 5 in advance and cause the user to press the button corresponding to the channel to be input. According to this, it is possible to input the channel while the television program image is displayed on the television 2. Further, for example, the CPU 10 may switch the display of the image of the television program once to display the input image on the television 2 and allow the user to input the channel using the image. Specifically, the CPU 10 uses the EPG
The image may be displayed on the television 2 and the channel may be specified by allowing the user to select a desired program. The CPU 10 stores data indicating the input channel in the main memory as channel data. Step S12 is followed by Step S12
The process is executed.

If the game apparatus 3 can give an instruction to the television 2, step S
11, the CPU 10 may instruct the television 2 to select a channel input by the user. According to this, it is not necessary for the user to set the channel of the television 2 to a desired channel in advance, and the channel instruction for both the television 2 and the game apparatus 3 can be performed by a single operation. As a method for the game device 3 to instruct the television 2, a method of causing the marker device 6 to output a signal similar to the infrared signal output from the remote controller of the television 2 can be considered. The infrared signal output from the marker device 6 is reflected directly or reflected on the wall surface around the television 2 and received by the infrared light receiving unit of the television 2, so that the game device 3 gives an instruction to the television 2. It can be carried out.

In step S <b> 12, the CPU 10 makes a request (acquisition request) for acquiring the instruction data described above to the server 110. Specifically, the CPU 10 reads the channel data from the main memory, generates data indicating an acquisition request (acquisition request data), and transmits the acquisition request data to the server 110 via the network 111. The acquisition request data includes data indicating the transmission destination of the instruction data (indicating information capable of identifying the game apparatus 3) and data indicating the channel input in step S11. Step S
Next to 12, the process of step S13 is executed.

As shown in FIG. 19, when the acquisition request data is transmitted from the game apparatus 3 to the server 110 (step S21), the server 110 transmits instruction data to the game apparatus 3 as a transmission source (step S22). . The instruction data is data indicating an instruction to cause the lighting device 9 to emit light as described above. In the present embodiment, the instruction data indicates the color of light that each light module 29 of the illumination device 9 should emit, and more specifically, indicates the light emission intensity of each of the red, green, and blue LEDs in 256 levels. . In the present embodiment, the instruction data is repeatedly transmitted at predetermined time intervals (for example, at 1/60 second intervals in accordance with the frame rate of television images).
Therefore, the server 110 can dynamically change the light emission state of the lighting device 9 (in other words, the lighting effect by the lighting device 9) by repeatedly transmitting the instruction data. In another embodiment, the server 110 may transmit a plurality of instruction data to the game apparatus 3 together.

The server 110 stores different instruction data for each TV broadcast channel, and transmits instruction data corresponding to the channel indicated by the received acquisition request data. Therefore, the server 110 can cause the lighting device 9 to emit light with a different light emission pattern for each channel (that is, for each program), and can provide different lighting effects for each channel.
In another embodiment, the server 110 transmits instruction data for all channels,
The game apparatus 3 that has received each instruction data may select and use the necessary instruction data (the channel instruction data input in step S11). In this case, the acquisition request data may not include data indicating the channel.

Returning to the description of FIG. 18, in step S <b> 13, the CPU 10 receives instruction data from the server 110. That is, since the instruction data transmitted from the server 110 is stored in the flash memory 17 via the antenna 22 and the wireless communication module 18, C
The PU 10 acquires instruction data from the flash memory 17. Steps S13 to S
Seventeen processing loops are executed at a rate of once per time interval that matches the transmission interval (for example, 1/60 second interval) of the instruction data. Accordingly, in step S <b> 13, the CPU 10 sequentially receives instruction data sequentially transmitted from the server 110. Following step S13, the process of step S14 is executed.

In step S14, the CPU 10 controls the light emission by the illumination device 9 according to the instruction data received in step S13. Specifically, first, the CPU 10 generates light emission control data based on the instruction data. In the present embodiment, the instruction data indicates the color of light that each light module 29 of the lighting device 9 should emit, whereas the light emission control data indicates the color and intensity of light that each light module 29 should emit. Indicates. Therefore, the CPU 10 generates light emission control data by adding information indicating the light intensity to the instruction data. Here, it is assumed that the intensity of light to be added is preset by the user. That is, in the present embodiment, the color and pattern emitted by the lighting device 9 are determined by the server 110, and the intensity of the emitted light can be adjusted as appropriate by the user. Thus, the user can adjust the lighting device 9 so as to have an appropriate light intensity according to his / her preference or according to the environment around the television 2.

In other embodiments, the instruction data may indicate only the pattern that the lighting device 9 emits light, and the color and intensity of the light emission may be determined by the user in the game device 3. In other embodiments, the instruction data and the light emission control data may have the same content. That is, the light emission pattern, light color, and light intensity of the illumination device 9 may be determined by the instruction data.

After generating the light emission control data as described above, the CPU 10 next causes the lighting device 9 to emit light by transmitting the light emission control data to the lighting device 9. The process for transmitting the light emission control data is the same as the process in step S5. Following step S14, the process of step S15 is executed.

As a result of the process in step S14, the light emission control data is transmitted from the game apparatus 3 to the illumination apparatus 9 as shown in FIG. 19 (step S23). The microcomputer 28 of the lighting device 9 that has received the light emission control data controls the light emission of each optical module 29 according to the light emission control data. Since the instruction data is repeatedly transmitted from the server 110 to the game apparatus 3, and the processing loop of steps S13 to S17 is repeatedly performed in accordance with the transmission of the instruction data, the light emission control data is transmitted to the lighting apparatus 9 as shown in FIG. It is sent repeatedly. Therefore, the game apparatus 3 can dynamically change the light projection by the lighting device 9 and can provide the lighting effect by the lighting device 9 so as to change according to the content of the television program or the content of the image.

The specific contents such as the light emission pattern indicated by each instruction data transmitted from the server 110 may be anything as long as it corresponds to the image and / or sound of the television program. Each instruction data may be, for example, data that changes the light emission state every time the scene of the TV program changes, or may change the light emission state according to the image, or the program The light emission state may be changed in accordance with the BGM or sound effect.

Returning to FIG. 18, in step S <b> 15, the CPU 10 acquires operation data.
The process of step S15 is the same as step S2 (FIG. 17) described above. Step S1
Next to 5, the process of step S16 is executed.

In step S16, the CPU 10 determines whether or not an instruction to change the channel has been issued based on the operation data acquired in step S15. When the user changes the channel of the television 2, the user also instructs the game apparatus 3 to change the channel. Any interface for allowing the user to change the channel may be used. For example, the same method as the interface in step S11 may be used. When a user instruction for changing a channel is issued, the CPU 10 stores data indicating the input channel in the main memory as new channel data.
If the determination result of step S16 is affirmative, the process of step S17 is executed. On the other hand, when the determination result of step S16 is negative, a process of step S18 described later is executed.

In step S <b> 17, the CPU 10 makes a request (change request) for changing the channel of the instruction data to be transmitted to the server 110. Specifically, the CPU 10 reads the channel data from the main memory, and indicates the change request (change request data).
And the change request data is transmitted to the server 110 via the network 111. The content of the change request data is the same as the acquisition request data, and includes data indicating the transmission destination of the instruction data and data indicating the channel after the change in step S16. After step S17, the process of step S13 is executed again.

In FIG. 19, when the change request data is transmitted from the game apparatus 3 to the server 110 (step S24), the server 110 transmits instruction data to the game apparatus 3 that is the transmission source (step S25). Here, the instruction data to be transmitted is instruction data corresponding to the channel indicated by the change request data. Also in the case of step S25, the instruction data is repeatedly transmitted at predetermined time intervals as in the case of step S22.

The CPU 10 of the game apparatus 3 performs steps S13 and S14 after step S17.
Execute the process again. Therefore, the CPU 10 sequentially receives the instruction data sequentially transmitted from the server according to the change request data (step S13), and transmits the light emission control data according to the received instruction data to the lighting device 9, thereby the lighting device 9. 9 is controlled (step S14, step S26). So if a user changes channels,
The lighting effect by the lighting device 9 is added with the contents corresponding to the changed channel.

Returning to the description of FIG. 18, in step S <b> 18, the CPU 10 determines whether or not the user has given an instruction to end the application of the lighting effect by the lighting device 9. This determination is made based on the operation data acquired in step S15 and stored in the main memory. Specifically, the CPU 10 refers to the operation button data 103 included in the operation data, and determines whether or not a predetermined button of the controller 5 has been pressed. In addition, the instruction | indication which complete | finishes provision of a lighting effect may be an instruction | indication which complete | finishes an application, for example, and the instruction | indication which complete | finishes the drive of the illuminating device 9 separately from completion | finish of an application. If the determination result of step S18 is affirmative, the process of step S19 is executed. On the other hand, when the determination result of step S18 is negative, the process of step S13 is executed again.

In step S <b> 19, the CPU 10 makes a request (end request) to end transmission of instruction data to the server 110. Specifically, the CPU 10 generates data indicating an end request (end request data) and transmits the data to the server 110 via the network 111. In addition,
The end request data only needs to include data indicating the request source of the end request (indicating information capable of identifying the game apparatus 3). After the end of step S19, the CPU 10 ends the process shown in FIG.

As shown in FIG. 19, when the end request data is transmitted from the game apparatus 3 to the server 110 (step S27), the server 110 finishes transmitting the instruction data to the game apparatus 3 that is the transmission source. . Above, description of the process of the game device 3 when the image of a television program is displayed is complete | finished.

As described above, according to the processing shown in FIGS. 18 and 19, the game apparatus 3 can add a lighting effect to the image of the TV program using the lighting apparatus 9, and the power of the TV program can be increased. And the realism can be improved. In the above embodiment, the server 110
Since different instruction data is provided for each television broadcast channel and each time, the game apparatus 3 that has received the instruction data may provide a lighting effect corresponding to the contents of each television broadcast program. it can.

In the above embodiment, the game apparatus 3 which is an example of the information processing apparatus according to the present invention has a process of adding a lighting effect to the game image during the execution of the game and the TV program while viewing the TV program. Both the processing for adding the lighting effect to the image are performed. Here, in other embodiments, the information processing apparatus may be capable of executing only one of the processes. Note that when only the process of adding the lighting effect to the image of the television program can be executed, the information processing apparatus does not need to be electrically connected to the television.

[Other embodiments]
The above-described embodiment is an example for carrying out the present invention. In other embodiments, the present invention can be implemented with, for example, the configuration described below.

(Variation regarding synchronization between TV broadcasting and lighting of lighting device)
In the above embodiment, in steps S22 and S25, the server 110 transmits instruction data in accordance with a real-time television program. That is, the server 110 transmits the instruction data at a timing assuming that the instruction data is received by the game apparatus 3 and used immediately. Therefore, the game apparatus 3 has a timing for displaying an image of a television program,
Processing to synchronize with the timing for causing the lighting device 9 to emit light is not executed. Therefore,
In the present embodiment, there may be a slight difference between the timing for displaying the image of the television program and the timing for causing the lighting device 9 to emit light. However, if the projection of the illumination device 9 changes according to the image (and / or sound) of the television program, the effect of the illumination device 9 can be obtained sufficiently, so even if there is a slight difference between the two timings. There seems to be no problem. In order to reduce the above-described deviation as much as possible, the server 110 takes into consideration the communication time between the server 110 and the game apparatus 3, the time required for the TV 2 to decode the image of the TV program, and the like. You may adjust the timing which transmits instruction data.

As described above, in the present embodiment, the game apparatus 3 does not perform a synchronization process between the timing for displaying an image of a television program and the timing for causing the lighting device 9 to emit light. However, in other embodiments, the synchronization processing may be performed on the game apparatus 3 side. For example, time data may be included in the instruction data, and the game apparatus 3 may use the instruction data in synchronization with time information included in broadcast data transmitted in digital broadcasting. That is, the game apparatus 3 uses the time data included in the data of the television broadcast and the time data included in the instruction data, so that the display timing of the image of the TV program and the lighting device are matched so that the time data of both matches. 9 is synchronized with the projection timing. According to this, the game apparatus 3 can synchronize more accurately.

(Modified example of instruction data transmission timing)
In the above embodiment, the server 110 transmits instruction data corresponding to a real-time (currently broadcast) television program. However, in other embodiments, the server 110 transmits instruction data corresponding to a television program before the start. You may make it transmit. In this case, the server 110 transmits instruction data corresponding to a television program in a predetermined time zone (after the current time) for each channel. Each instruction data includes time data indicating the time when the instruction data should be used. The game apparatus 3 receives the instruction data and stores it in the main memory or the like. When there is an input of the channel in step S11, the CPU 10 reads out instruction data of the input channel and the time data indicating the current time from the main memory, and the lighting device according to the instruction data 9 is controlled. According to this, the server 110 can transmit the instruction data to the game apparatus 3 in advance.

Further, when the server 110 transmits instruction data corresponding to the TV program before the start,
The game apparatus 3 may execute the process of receiving instruction data even in the sleep mode. When the game apparatus 3 executes the instruction data reception process in the sleep mode, the reception process may be executed at regular time intervals. In the above case, the game apparatus 3 may receive the instruction data together with the EPG data in the application, for example.

(Modification regarding sound output of lighting device 9)
In the above embodiment, the process in which the game device 3 controls the light projection by the lighting device 9 has been described. However, in another embodiment, the game device 3 further controls the sound output by the speaker 112 of the lighting device 9. Also good. That is, the instruction data may include data (sound instruction data) indicating an instruction to output sound to the speaker 11 in accordance with the television broadcast image and / or sound. The sound instruction data may be, for example, data indicating a sound waveform or MIDI data. The CPU 10 causes the speaker 112 to output sound according to the received instruction data. Specifically, the CPU 10 generates sound control data for controlling the sound to be output by the speaker 112 based on the received instruction data. Then, in addition to the light emission control data, sound control data is further transmitted to the lighting device 9. The speaker 112 outputs sound according to the received sound control data. According to this, the game apparatus 3 can give a user an auditory effect in addition to the visual effect by the lighting device 9. In addition,
The contents of the sound instruction data (and sound control data) may be anything, but in view of adding a production effect to the image and sound of a television program, the sound instruction data is a television program. It is preferable to indicate an instruction to output a sound different from the voice. In addition to adding production audio to individual scenes, it also outputs audio that is compatible with stereophonic sound that is not included in TV programs, and replaces it with TV audio or matches it with TV audio. May be used. Therefore, a plurality of speakers 112 may be provided.

Further, the process of controlling the speaker 112 of the lighting device 9 may be executed not only when a television program is displayed (FIG. 18) but also when a game image is displayed (FIG. 17).
When displaying a game image, the sound control data is appropriately generated by the CPU 10 so that a sound corresponding to the game process is output.

(Modification in which the lighting device 9 is driven according to an image other than the game image)
In the above embodiment, the case where the lighting device 9 adds the lighting effect when the game image is displayed on the screen of the television 2 and when the image of the television program is displayed has been described. The displayed image is not limited to a game image or a television program image, and may be a moving image or a still image other than the game image. For example, the game apparatus 3 may acquire an image from another external game apparatus or server via a network and add the lighting effect by the lighting device 9 when displaying the image on the television 2. In this case, the game apparatus 3 may acquire data for controlling the light emission of the lighting device 9 (the light emission control data 108) together with the image,
You may acquire from the apparatus and server different from the acquisition source of an image. Further, the game apparatus 3 may automatically create the light emission control data from the acquired image according to a predetermined algorithm.

(Modification in which the lighting device 9 emits light for notification to the user)
In addition to the purpose of adding a further visual effect to the image displayed on the television 2, the lighting device 9 may be used for the purpose of notifying the user. In the present embodiment, the game apparatus 3 can operate in the sleep mode described above. Since the CPU 10 does not operate in the sleep mode, game processing and game image display are not executed, but communication with an external device (another game device or server device) is executed via the network by the input / output processor 11a. Is done. For example, in the sleep mode, the game apparatus 3 transmits / receives a message created by the user (in an e-mail format) to / from another game apparatus or receives a game program or video data from the server apparatus. To do. In this modified example, when the game apparatus 3 receives a message from another game apparatus in the sleep mode, the game apparatus 3 causes the lighting device 9 to emit light to notify the reception. Details will be described below.

FIG. 18 is a flowchart showing data reception processing of the input / output processor 11a in the sleep mode. In the sleep mode, the input / output processor 11a operates as shown in FIG.
Is executed at a predetermined timing (for example, once every predetermined time). FIG. 18 shows processing in the sleep mode, but the input / output processor 11a communicates with an external device even in the normal mode.

In step S31, the input / output processor 11a accesses a mail server that stores and manages messages, and checks whether there is a message addressed to itself (game device 3). In subsequent step S32, the input / output processor 11a determines whether or not there is a received message as a result of the confirmation in step S31. If there is a received message, the process of step S33 is executed. On the other hand, when there is no received message, the input / output processor 11a.
Ends the data reception process shown in FIG.

In step S33, the input / output processor 11a receives a message from the mail server. In other words, the input / output processor 11 a accesses the mail server and stores the received message in the flash memory 17. Following step S33, the process of step S34 is executed.

In step S34, the input / output processor 11a starts light emission by the illumination device 9. Specifically, the input / output processor 11 a creates the light emission control data and transmits it to the lighting device 9 via the expansion connector 20. The microcomputer 28 of the lighting device 9 that has received the light emission control data controls the light emission of each optical module 29 according to the light emission control data. Thereafter, the input / output processor 11a continuously emits light by the lighting device 9 by repeatedly transmitting the light emission control data. Therefore, the user can know that the message has arrived by the light emission by the lighting device 9. After the above step S34, the input / output processor ends the data reception process shown in FIG.

In step S34, the input / output processor 11a may change the light emission state (light intensity, color, light emission pattern, etc.) of the illumination device 9 according to the data reception state. For example, the light emission state of the lighting device 9 may be changed according to the number of received messages, the source of the message, and the type of received data (whether it is a message or data related to a game). According to this, the user can know information of received data, such as what kind of data is received, from the light emission state of the lighting device 9.

As described above, according to the modified example, the game apparatus 3 can notify the user of message reception by causing the lighting device 9 to emit light in response to reception of the message. In the above modification, the game apparatus 3 causes the lighting device 9 to emit light in response to receiving the message. However, in another embodiment, in addition to the message data or the message data Instead of this, the lighting device 9 may emit light in response to reception of other data (for example, game program or video data transmitted from the server).

Moreover, in the said modification, the game device 3 shall perform the light emission operation | movement of the illuminating device 9 according to having received data only at the time of sleep mode, However, In other embodiment, the said light emission operation | movement is performed. It may be executed even in the normal mode.

(Modification in which control of lighting device 9 is changed according to brightness)
In other embodiments, the game system 1 may change the light emission state of the lighting device 9 in accordance with ambient brightness. Specifically, the game system 1 includes a sensor (for example, an illuminance sensor) that detects ambient brightness in addition to the configuration shown in FIG.
You may change the light emission state of the illuminating device 9 according to the detection result of the said sensor. Note that the sensor is preferably configured separately from the game apparatus 3 and can communicate with the game apparatus 3 in a wired or wireless manner. The sensor is preferably installed around the television 2. For example, when it is determined that the surroundings of the game system 1 are relatively bright from the detection result of the sensor, the game apparatus 3 controls the light emission state of the lighting device 9 to be relatively bright, and the game system 1 When it is determined that the surroundings of the lighting device 9 are relatively dark, the light emitting state of the lighting device 9 may be controlled to be relatively dark. In step S14, the CPU
10 may determine the light intensity based on the detection result of the sensor. According to the above modification, the game system 1 is difficult to see the light on the wall surface projected by the lighting device 9 when, for example, the surroundings are bright, or is projected by the lighting device 9 when the surroundings are dark. It is possible to prevent the light on the wall surface from being too bright. In addition to sensors that detect brightness, for example, a human sensor is provided to detect whether or not there are people in the surroundings. You may make it do.

(Examples of other game systems)
In the above embodiment, the game system 1 is configured to include the marker device 6 as an accessory device to be set around the television 2. Here, in another embodiment, the attached device may be, for example, the following device. That is, the accessory device is the television 2
It may be a camera that images the front of the camera. At this time, the game apparatus 3 performs predetermined information processing using an image captured by the camera. As predetermined information processing, for example, a process for controlling a game according to a captured user or a position of an input device, a process for processing and displaying a captured image, a part of a captured image ( For example, a game process using the user's face part) as a game image is conceivable.

Further, the accessory device may be a device that outputs a predetermined signal by radio waves or ultrasonic waves, instead of the marker device 6 that outputs infrared light. At this time, the controller 5 as an input device detects the predetermined signal, and the game apparatus 3 executes predetermined information processing based on the detection result. For example, the game apparatus 3 calculates the position of the controller 5 or the like from the detection result, and performs game processing using the calculated position as a game input as predetermined information processing. Conversely, a sensor that detects the predetermined signal by the input device emitting the predetermined signal may be provided in the accessory device.

As described above, the present invention can be used for, for example, an image display system that displays an image of a television program, for example, for the purpose of giving a user a further visual effect in addition to the image displayed on the screen. It is.

DESCRIPTION OF SYMBOLS 1 Game system 2 Television 2a Speaker 3 Game apparatus 4 Optical disk 5 Controller 6 Marker apparatus 9 Illumination apparatus 10 CPU
11a Input / output processor 28 Microcomputer 29 Each optical module 40 Imaging device 61-67 Linear optical module 68 Background light module 71 Case 72 Color LED module 73 Condensing lens 74 Diffusion sheet 75-77 LED
110 server 112 speaker

Claims (12)

An illumination system including an information processing device capable of communicating with a server via a network, and an illumination device capable of receiving a signal from the information processing device,
The information processing apparatus includes:
Receiving means for receiving, from the server, instruction data for each channel indicating an instruction to cause the illumination device to emit light in accordance with an image and / or sound of a television broadcast;
Based on the received instruction data, provided with a light projection control means for outputting a control signal for controlling the light projection of the lighting device to the lighting device,
The lighting device projects visible light according to the control signal,
The information processing apparatus can instruct a broadcast receiving apparatus that receives a television broadcast and displays an image on a screen;
The information processing apparatus in response to an input of a channel by the user, along with an instruction to select a single channel which is input to the broadcast receiving apparatus, the instruction data corresponding to one channel input A lighting system that makes a request to the server according to a channel input and receives the instruction data from the server. Game processing means for executing predetermined game processing;
Game display control means for displaying a game image corresponding to the game processing on the screen of the broadcast receiving device;
The lighting system according to claim 1, wherein when a game image is displayed on a screen of the broadcast receiving device, the light projecting control unit controls light projection of the lighting device based on the game process. An infrared light emitting device that emits infrared light; and
And an input device including an imaging unit capable of detecting infrared light,
The image display system according to claim 2, wherein the game processing unit executes the game processing based on a position of the infrared light captured by the imaging unit. The lighting device further includes an audio output means,
The instruction data further indicates an instruction to cause the audio output means to output sound in accordance with an image and / or sound of a television broadcast,
The lighting system according to any one of claims 1 to 3, wherein the light projection control unit further controls the sound output of the sound output unit based on the received instruction data.   The illumination system according to any one of claims 1 to 4, wherein the illumination device is installed so as to emit visible light toward a rear of a display screen on which an image of the television broadcast is displayed. The lighting device includes:
A plurality of light emitting elements arranged in a line in a predetermined direction and emitting light of different colors;
6. The light-emitting device according to claim 1, further comprising a condensing lens that is disposed at a position that transmits visible light from the plurality of light-emitting elements and focuses light only in a direction perpendicular to the predetermined direction. The lighting system described in.   The image display system according to claim 1, wherein the illumination device emits a plurality of visible lights each having a longitudinal section of light. The lighting device includes:
A light emitting unit that emits visible light;
The illumination system according to claim 7, further comprising: a condensing lens that is disposed at a position that transmits visible light from the light emitting unit and focuses light only in a predetermined direction.   The image display system according to claim 7, wherein the illumination device further emits visible light having a wider cross section than the visible light in a direction overlapping the plurality of visible light. An information processing device capable of communicating with a server via a network and capable of transmitting a signal to a lighting device that projects visible light,
Receiving means for receiving, from the server, instruction data for each channel indicating an instruction to cause the illumination device to emit light in accordance with an image and / or sound of a television broadcast;
Based on the received instruction data, provided with a light projection control means for outputting a control signal for controlling the light projection of the lighting device to the lighting device,
The information processing apparatus can instruct a broadcast receiving apparatus that receives a television broadcast and displays an image on a screen;
The information processing apparatus in response to an input of a channel by the user, along with an instruction to select a single channel which is input to the broadcast receiving apparatus, the instruction data corresponding to one channel input An information processing apparatus that makes a request to the server in response to a channel input and receives the instruction data from the server. An image display system including a broadcast receiving device for receiving a television broadcast, an information processing device connectable to the broadcast receiving device, and an illumination device capable of receiving a signal from the information processing device,
The broadcast receiving device displays a received television broadcast image on a screen,
The information processing apparatus includes:
Receiving means for receiving instruction data for each channel indicating an instruction to cause the lighting device to emit light in accordance with the image and / or sound of the television broadcast from a server via a network;
Based on the received instruction data, provided with a light projection control means for outputting a control signal for controlling the light projection of the lighting device to the lighting device,
The lighting device projects visible light according to the control signal,
The information processing apparatus can instruct a broadcast receiving apparatus that receives a television broadcast and displays an image on a screen;
The information processing apparatus in response to an input of a channel by the user, along with an instruction to select a single channel which is input to the broadcast receiving apparatus, the instruction data corresponding to one channel input An image display system that makes a request to the server according to a channel input and receives the instruction data from the server. A control program executed in a computer of an information processing apparatus capable of communicating with a server via a network and capable of transmitting a signal to an illumination apparatus that projects visible light,
The information processing apparatus can instruct a broadcast receiving apparatus that receives a television broadcast and displays an image on a screen;
Receiving means for receiving, from the server, instruction data indicating an instruction to cause the lighting device to emit light according to an image and / or sound of a television broadcast;
Based on the received instruction data, a light projection control means for outputting a control signal for controlling the light projection of the lighting device to the lighting device;
In response to an input of a channel by the user, along with an instruction to select a single channel which is input to the broadcast receiving apparatus, in accordance with instruction data corresponding to one channel input to the input of the channel A control program for making a request to the server and causing the computer to function as channel changing means for receiving the instruction data from the server.

Images