JP2003198457A - Radio system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To make image information and sound information, etc., from the view point of a player or the like into easy-to-see powerful information corresponding to the movements of the player and into interesting and enjoyable information and to deliver the information to a viewer. <P>SOLUTION: The radio system is provided with an image pickup means 100 mounted on the body of a key person of an event performed in a prescribed space by one, two or more members belonging to a plurality of groups, a radio transmission means 200 for radio-transmitting the image information of the image pickup means, a radio reception means 400 for radio-receiving the image information from the image pickup means, and an image information processing means 600 for performing an image processing corresponding to the movement of the body of the key person on the basis of the image information received in the reception means. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a wireless system for wirelessly transmitting and receiving image information and sound information from a camera or the like attached to the body of a sports player, a theater performer, or the like.

[0002]

2. Description of the Related Art Conventionally, video and audio information in events such as sports and plays has been from the viewpoint of looking at players and performers. For example, Japanese Patent Laid-Open No. 2000-33308
In the one disclosed in Japanese Patent No. 0, an image in which a moving subject in a sports match or the like is photographed by a camera, the photographed object is identified, the position of the photographed object is detected, and the name of the athlete is superimposed and displayed. Processing is taking place.

By the way, it has been difficult to obtain how the player perceives the surrounding information and how he or she feels it. For that reason, information from the viewpoint of players and performers, and information about the environment in which the players are placed has great value as content. In particular, the value is even greater because the viewers have a sense of participation that wants to share the same information as the players.

For delivering image information and sound information to the user from the viewpoint of the player or performer, for example, the image of a camera mounted on a race car or ski in the F1 race or ski jump competition is delivered to the user as it is. is there.

Further, as a device for imitating the sensation of a marathon runner, the physical condition information such as the surrounding image, sound, temperature, runner's heart rate, respiration, etc. disclosed in Japanese Patent Application Laid-Open No. 2000-342713 is displayed on the treadmill. There are also things that give a simulated experience to the runners running on the treadmill by distributing it to the surrounding area.

[0006]

However, since the images captured by the camera are relatively simple and do not support various movements of the player and the performer, the above-mentioned problems can be solved.
For example, there is a problem that it is difficult to apply to popular sports such as ball games. For example, when the body moves rapidly or moves greatly, the image often shakes so much that the viewer cannot see it. Due to such problems, there has been no means until now to attach a camera directly to the player or performer to make the information such as image information from the viewpoint of the player easy to see, or to deliver it to the user as interesting and fun information. There wasn't.

When a camera is directly attached to a player or the like, it is considered necessary to wirelessly transmit and receive image information and sound information from the camera so as not to disturb the action of the wearer. In this case, when cameras are attached to a plurality of persons and image information is wirelessly transmitted and received, there is a problem that radio frequencies overlap.

Therefore, the present invention has been made in view of such a problem, and an object of the present invention is to attach a camera directly to a player or performer so that image information from another player's point of view or another camera. An object of the present invention is to provide a wireless system capable of preventing overlapping of wireless frequencies when wirelessly transmitting and receiving image information and the like from.

[0009]

In order to solve the above-mentioned problems, according to one aspect of the present invention, each image pickup means respectively attached to the bodies of a plurality of persons having an event, and image information of each image pickup means. When wirelessly transmitting and receiving between the wireless transmitting means for wirelessly transmitting the image information, the wireless receiving means for wirelessly receiving the image information from the imaging means, and the wireless transmitting means and the wireless receiving means. And a frequency conversion means for converting each of the radio frequencies into a different radio frequency.

In the present invention, image pickup means such as cameras are attached to the bodies of a plurality of actors in events such as sports, plays and operas. A video signal (image signal) from a camera or the like is sent by each wireless transmission means to a nearby wireless reception means corresponding thereto. In this case, by changing the radio frequency at the time of radio transmission / reception between the radio transmission means and the radio reception means by the frequency conversion means, it is possible to prevent the radio frequencies from overlapping.

The radio system between each radio transmission means and each radio reception means uses a direct frequency division multiplexing system, and the frequency conversion means converts the radio frequency based on different carrier waves in the mixer. You may As described above, the frequency conversion means is provided with the mixer, and the transmission data of the image information and the sound information can be transferred to another different frequency band by changing the carrier wave through the mixer and sending the data to the wireless transmission unit. As a result, it is possible to avoid frequency overlap when sending multiple data separately.

The event is an event performed in a predetermined space by one or more members belonging to a plurality of groups, and the frequency conversion means changes the radio frequency for each group. May be. This makes it possible to prevent overlapping of frequencies even when wirelessly transmitting and receiving image information and the like for each team in baseball.

The frequency conversion means may change the radio frequency for each wearer of the image pickup means. As a result, it is possible to prevent overlapping of frequencies even when wirelessly transmitting and receiving image information and the like for each sports player.

[0014]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the entire information processing system according to this embodiment.

The information processing system according to the present invention is roughly divided into a wearer side to be attached to an actor for capturing and wirelessly transmitting image information and sound information according to the motion of a person who performs an event such as sports. The image pickup means 100, the sound collecting means 110, the wireless means 120, and the information processing apparatus 300 on the side of the processing apparatus for wirelessly receiving the image information and the sound information and processing the image information and the sound information.

The information processing device 300 includes a wireless device 32 that receives wireless information from the wireless device 120 on the wearer side.
0, information processing means 350 for processing image information and sound information received by the wireless device 320, various databases 360 including storage means for storing processed images, CP
ROM and C that store programs executed by U and CPU
The control unit 370 is provided with a RAM or the like used when the PU performs various processes, and various memories 380 such as a frame memory for temporarily storing the received image information.

The wearer-side image pickup means 100 and the like are attached to the body of a person who has a great influence on the overall progress of an event performed in sports, theater, opera or the like, that is, the body of a key person. Therefore, for example, the image pickup unit 100 is a small CCD camera, the sound collection unit 110 is a small microphone, and the wireless device is a small wireless device so that the image pickup unit 100 and the like do not interfere with an event such as a sport. Those are preferable.

Examples of the above key persons include catchers and pitchers for baseball. In addition, in the case of soccer, the command tower, in the case of tennis and badminton, both players, in the case of football, the quarterback, etc. If the images viewed from these persons can be delivered to the viewer, the sense of presence in a sports game or the like will increase, and more dramatic images can be provided.

2 to 5 show examples of mounting the image pickup means 100 and the like. For example, as shown in FIG. 2, it is attached to a headgear 102 such as a helmet or a hat. In this case, for example, the image pickup means 10 is provided in the space at the top of the helmet or the like.
0, the sound collecting means 110 and the wireless device 120 are housed. In addition,
The antenna of the wireless means 120 is attached to the periphery of a helmet or the like so that it is not conspicuous.

As another example of attachment, the image pickup means 100 described above is used.
Etc. may be mounted on the temple 106 of the spectacles 104 as shown in FIG. A person who does not normally wear the eyeglasses 104 may wear the eyeglasses 104 that are not comfortable. In this case, the antenna of the wireless means 120 may use the frame of the glasses 104 or the like.

As another example of attachment, the image pickup means 100 is used.
Etc. may be attached to the upper side of the ear as shown in FIG. In this case, for example, the hearing aid may be attached or the headphones may be attached.

In the case of attaching to a baseball catcher, the image pickup means 100 and the sound collecting means 110 may be attached to the catcher mask 108 and the radio means 120 may be attached to the helmet 102 as shown in FIG. In this case, since the image pickup means 100 and the wireless means 120 are very close to each other, it is preferable to use transmission means 122, 122 for proximity using infrared rays, for example. As a result, it is possible to reduce the interference with the wireless means 120 using the radio waves of the main body.

Further, it is desirable that the optical system of the camera has a wide angle as the image pickup means 100. For example, by using a fisheye lens or the like, it becomes possible to capture an image in a wide range with an angle of view of about 180 degrees. If a video (image) taken with a wide-angle lens is used, it will be easy to process it so as to capture the target object by image processing.

Next, a wireless device according to this embodiment will be described. The wireless device 120 on the side of the wearer wirelessly transmits an encoder 130 for encoding image information from the image pickup unit 100 and sound information from the sound collecting unit 110 and the encoded information as wireless information. Transmission means 200,
A security means 140 for converting wireless information with a security code is provided as necessary.

The wireless device 320 on the processing device side receives the information from the wireless device 120 on the wearer side wirelessly, and the image information and sound information received by the wireless receiving device 400. A decoder or the like 330 for
A security unit 340 is provided to restore the wireless information to the original with a security code if necessary.

Here, an example using a digital FPU system as these wireless devices will be described with reference to FIGS.

In this case, the digital transmitter (Tx) serving as the wireless transmission means 200 has a transmission controller 210 as shown in FIG.
And a transmitting high frequency section 220, an antenna 230 and the like. The transmission control unit 210 inputs a TS (Transport Stream) format signal suitable for transmission from an encoder or the like, performs transmission path coding, and outputs an IF signal. Transmission high frequency section 2
20 receives the IF signal, performs frequency conversion and power amplification, and outputs an RF signal.

A digital F serving as the wireless receiving means 400
The PU receiver (Rx) includes a reception high frequency section 420 and a reception control section 4
10, an antenna 430 and the like. The receiving high-frequency unit 420 receives the RF signal, performs frequency conversion, and performs I conversion.
Output F signal. The reception control unit 410 receives the IF signal, performs transmission path decoding, and outputs a TS format signal to the decoder 330.

Here, as the wireless transmission / reception modulation method, for example, a transmission method using OFDM (Orthogonal Frequency Division Multiplexing Method), which is strong against multiple reflection, is applied. As an example of the basic parameters of OFDM, FIG. 7 shows the transmission parameters of the OFDM system and FIG. 8 shows the transmission capacity. As the transmission parameter, an FFT sampling clock that can secure the transmission parameter of the TS rate of 59.648 (Mbps) and 44.736 (Mbps) in the combination of the modulation method and the coding rate of the inner code is selected.

The calculation method of the FFT sampling clock will be described below. The target TS rate is TSR (Mbps), the number of data carriers is D (line), the band utilization efficiency determined by the modulation method is M (bit / Hz), the coding rate of the inner code is R, and the effective symbol length is Te ( ms) and the guard interval ratio is Gr, the following relationship holds.

TSR = D × M × R / (Te × (1 + Gr)) (1) Here, when Te is obtained, Te = D × M × R / (TSR × (1 + Gr)) (2) Here Then, assuming that the number of FFT points is P and the FFT sampling clock is Fs (MHz), Fs = 1 / (Te / P), so by substituting equation (2) and rearranging, Fs = TSR × (1 + Gr) × P / (D × M × R) (3) Here, TS rate TSR = 59.648 (Mbps), guard interval ratio Gr = 1/16, number of data carriers D = 672, 64
Bandwidth utilization efficiency M = 6 for QAM, number of FFT points P
When Fs is calculated by substituting = 1024 into the equation (3), it becomes Fs = 19.3145904761905 (MHz).

The encoder such as the encoder 130 (Encode
r), remultiplexer (ReMUX), or other digital FPU receiver (Rx) and the like and the digital FPU transmitter (Tx) are connected as a connection form, for example, outer interleave and error correction inner code In the case of connection between the two, the ARIB standard "data broadcasting clock data transmission interface for television broadcast program material transmission" (ARIB STD-B18) is used. The connection form between the digital FPU receiver (Rx) and the decoder (Decoder) is also a connection form corresponding to this.

As another connection form, for example, when connecting between information source coding / multiplexing and data frame synchronization, DVB-ASI (ETSI EN 50083-9 "Cabled distri
bution systems for television, sound and interacti
ve multimedia signals Part9: Interfaces for CATV / S
MATV headends and similar professional equipment f
or DVB / MPEG-2 transport streams ") 204byte / packet
You may use (with 16 dummy bytes).

In the above case, the clock signal and the data flow are basically in the same direction. The digital FPU transmitter can be operated with an external clock. Also,
The digital FPU transmitter has a clock supply function so that a clock can be supplied to an encoder, a remultiplexer, or the like as needed.

Next, the configuration of the transmission control unit 210 is shown in FIG.
Will be described with reference to. The transmission control unit performs data frame synchronization (option), simple scramble (option), energy spreading, error correction outer coding, outer interleaving, error correction inner coding, mapping, OFDM frame configuration, OFDM modulation, etc. Is output.

The above data frame synchronization is an option in the case of using the simple scramble, and is assumed to be used in combination with the simple scramble. Therefore, when simple scrambling is not used, data frame synchronization is also not used. In the data frame synchronization, the TS input from the codec or the remultiplexer to the transmission control unit is framed in 8TS packet units. The first sync byte of the data frame is 0xB8 by inverting the normal TS sync byte 0x47.

The above simple scramble is optional. The wireless transmission system handled in this standard is communication performed with a specific other party, and when scrambling is applied to reduce the equipment scale and power consumption, a 16-bit pseudo-random sequence (generation polynomial X ¹⁶ + X ¹² +
X ³ + X + 1) is added to a simple scramble system. The FPU does not need to be highly confidential, such as requiring direction adjustment to receive signals, so a simple scramble function can be added as needed (optional)
And Even when an arbitrary scrambling method other than the concealment method specified here is used together, the scrambling range must comply with the above rules.

The scrambling range is the payload of the transport stream excluding the sync byte of the transport packet and the transport packet transmitting the PSI. The scramble key is an initial value to be loaded into an LFSR (linear feedback shift register) that generates the above pseudo random sequence. Note that the key is not transmitted. The initial value is loaded to the LFSR immediately after the frame synchronization has passed, and the operation of the LFSR is continued until the next frame synchronization. In the portion where scrambling is prohibited, the addition of the pseudo-random sequence shall be simply stopped.

In the above energy diffusion, ISO / I
A pseudo random sequence for energy diffusion is added to the TS packet multiplexed by EC13818-1. The generation polynomial of the pseudo-random sequence is X ¹⁵ + X ¹⁴ +1. The initial value of the pseudo-random sequence generation circuit is set to "1001 0101 0000 000" from the low order. The range of addition is 2
16 bytes of dummy from each packet of 04 bytes, data frame synchronization byte (0xB8), TS synchronization byte (0
x47) is excluded. Initial value loading is OF
Immediately after the first synchronization byte (0xB8 or 0x47) of the TS packet accommodated in the DM transmission frame is completed. Note that the operation of the shift register continues even in the synchronization byte portion, and in this portion, addition of the pseudo random sequence is simply stopped.

The above error-correcting outer code is a shortened Reed-Solomon (204,188). The shortened Reed-Solomon code is a Reed-Solomon (255,239) encoder, in which 51 bytes of "0" are added before 188 bytes excluding the dummy 16 bytes of the 204 bytes of the input data, and encoded. It is generated by removing the first 51 bytes later. The Reed-Solomon (255,239) code polynomial is as follows.

Code generation polynomial: g (x) = (x + λ ⁰ ) (x +
λ ¹ ) (x + λ ² ) ... (x + λ ¹⁵ ) where λ = 02h.

Field generation polynomial: p (x) = x ⁸ + x ⁴ + x ³ + x ² +1 The byte correction capability of Reed Solomon (204,188) is 10 ⁻¹¹ or less at 10 ⁻³ input, It is 10 ^-19 or less at 10 ^-4 input.

The outer interleave is 17 in one block.
A block having a delay amount of bytes is arranged such that the n-th path has a delay amount of (n-1) blocks.
Convolutional interleaving is performed in which the Reed-Solomon-encoded 204-byte bit stream is sequentially supplied to the path one byte at a time. Here, it is assumed that the sync byte and the frame sync byte of the transport packet always pass through the path with no delay. Note that the deinterleaving circuit configuration is such that the insertion amount of the delay block of the leading path is 11, and the deinterleaving circuit decreases by 1 block in order of the path number.

As the error correction inner code, a constraint length of 7,
A punctured convolutional code whose coding rate is 1/2 is used. The generator polynomial of the original code is G ₁ = 171 _OCT , G ₂ = 13
3 _OCT , punctured R = 1/2, 2/3, 3/4, 5
It supports 5 types of coding rates of / 6 and 7/8. FIG. 10 shows a convolutional coding circuit with a constraint length of 7 and a coding rate of 1/2. In addition,
The puncturing pattern shall be reset with OFDM frame synchronization.

If the inner code is not required due to the characteristics of the transmission line, the inner code can be omitted. In this case, the input signal is used as it is as the output signal.

The frequency interleaving uses the following frequency interleaving function. Mseq10 () described in the frequency interleave function above
Shall return the value of the shift register of the circuit for generating the pseudo-random sequence shown in FIG. The initial value of the shift register is “1111 1111 11”. Here, the function F (i) is a function for converting the carrier position (i) before interleaving to the carrier position after interleaving. The generator polynomial is g (x) = x ¹⁰ + x ³ + 1.

In the time interleaving, carriers are dispersed on the time axis in order to improve fading characteristics. The time interleaving uses convolutional interleaving. As the time interleave length, five types of parameters can be selected by changing the value of the cell length (I) as shown in FIG. However, the parameter of cell length (I) = 168 is optional. FIG. 13 shows a configuration example of time interleaving.

Here, the OFDM frame is composed of the following five types of carriers. − CP (Continual Pilot) − TMCC (Transmission and Multiplexing Configu
ratio control) -AC (Auxiliary Channel) -Null (Null Carrier) -Data (Data Carrier) Of these, the arrangement of carriers other than the data carrier is shown in FIG.
4 shows.

In addition of the guard interval, FIG.
As shown in, the part corresponding to the guard interval length of the rear end part of the effective symbol is copied and added in front of the effective symbol.

Here, the signal format in the RF band in the above RF signal format is defined. Definition k: carrier number n: symbol number K: total number of carriers (K = 793) T _s: symbol period length T _g: guard period length T _u: effective symbol period length f _c: center frequency K _c of the RF signal: RF signal Carrier number corresponding to the center frequency of (K _c = 3
96) c (n, k): A complex signal point vector s (t) corresponding to a symbol number n and a carrier number k is an RF signal.

[0051]

[Equation 1] Next, a case where there are a plurality of wireless devices 120 described above will be described. When the image pickup means 100, the wireless device 120, etc. are attached to a plurality of key persons, it is necessary to prevent the frequencies of wireless transmission and reception from overlapping. For example, in the case of baseball, in order to attach the wireless devices 120 to the catchers of both teams, it is necessary that the frequencies from the wireless devices 120 do not overlap. The same applies when the wireless device 120 is attached to both the catcher and the pitcher.

In this case, the radio device 120 is provided with a mixer such as a mixer for frequency conversion, and the transmission data of image information and sound information is sent to the radio transmission section by changing the carrier wave via the mixer, thereby making another difference. Move to frequency band. As a result, it is possible to avoid frequency overlap when sending multiple data separately.

The frequency may be converted by using a different carrier depending on the wearer such as the pitcher or the catcher described above. Further, in an event such as a sports game held in a predetermined space performed by one or more members belonging to a plurality of groups, the frequency may be converted by using different carrier waves for each team. For example, in baseball, different carriers may be used for different teams.

Next, a security means for transmitting and receiving image information and the like in this embodiment will be described.

In the present embodiment, the actor of the event himself,
For example, the image capturing means 100 can be applied to a player who is playing sports.
Since the sound collecting means 110 is attached, it is necessary to prevent the image information and the sound information of the wearer from being used and abused by an unspecified number of persons. For example, in baseball, if a voice of a catcher is included in the sound information, the sign to the pitcher may be known in advance, or the next action may be predicted. In particular, when wirelessly transmitting and receiving image information and sound information data, the data may be received by an unspecified person, so it is necessary to prevent this.

Therefore, in the present embodiment, as shown in FIG.
As shown in FIG. 3, the security means 140 is provided in the wireless device 120 on the wearer side that transmits wireless data, and the security means 340 is provided in the wireless device 320 on the processing device side that receives wireless data.

The security means 140 and 340 prevent wireless communication from being received and abused by an unspecified person, for example, by converting wireless data with a security code.

The security means 140 in the wireless device 120 on the wearer side processes digital data to be transmitted using a predetermined security code and sends it to the wireless transmission means 200.

The security means 340 in the radio device 120 on the processing device side processes the digital data converted from the received data with the same security code as in the case of transmission to obtain the original data, and performs the next process. The information is sent to the information processing means 350.

The security means 140, 34
For 0, data transmitted by radio using a security code may be modulated by changing the energy spread data of OFDM. For example, a plurality of OFDM energy spread data may be associated with a security code and different data may be selected each time.

According to such a distribution method of the security code, for example, a security card is distributed corresponding to a game or an event. Further, the security code may be transmitted after the user is specified by using a mobile phone or the like. The security code may be embedded in the IC card.

In addition, in an event such as a sports game held in a predetermined space by one or more members belonging to a plurality of groups, different security codes may be used for each team. For example, in baseball, different security codes may be used for each team.

The security codes used in the security means 140 and 340 can be updated. For example, the wireless device 120 on the wearer side is provided with wireless receiving means, and the wireless device 320 on the processing device side is provided with wireless transmitting means so that wireless transmission and reception can be performed respectively, and security is provided in the rewritable memory of the information processing device 300. The code may be stored, and when it is updated by an input means such as a keyboard or a card reading device, the security code may be transmitted from the wireless device 320 on the processing device side to the wireless device 120 on the wearer side. . This allows the security code on the wearer side to be automatically updated.

In this case, the security code may be updated, that is, the input from the input means may not be possible without the ID and password. This makes it possible for only the third party who is different from the actor of the event to update the security code.

Next, in the information processing apparatus 300,
The information processing means 350 for processing the image information and sound information data received by the wireless device 320 will be described.

As shown in FIG. 1, the information processing means 350 includes an image information processing means 600 for performing data processing of image information in the data received by the wireless device 320 and a sound information processing means 700 for performing data processing of sound information. Prepare

The image information processing means 600 is a means for performing image processing according to the movement of the wearer, as shown in FIG. 16, a movement detecting means 610 for detecting the movement of the wearer of the image pickup means 100, and this movement detecting means. A blur correction unit 620 that performs blur correction of an image according to the motion of the wearer detected in 610 when the motion of the wearer is small, and the motion detection unit 6 described above.
An angle changing unit 630 that changes the angle of the image according to the movement of the wearer detected in 10 when the movement of the wearer is large is provided.

The motion detecting means 610, the image blur correcting means 620, and the angle changing means 630 are based on the software processing performed by the control section 370 based on the program recorded in a recording medium such as a computer-readable hard disk. It may be processed by a circuit of hardware, or may be processed by a combination of both.

Here, FIG. 17 shows an example of the circuit configuration of the motion detecting means 610. This circuit is composed of a reference frame memory 612 for storing a reference frame in the digitized image information data, a search memory 614 for storing a frame to be compared with the reference frame, a reference frame memory 612 and a search frame memory 614. Motion detection circuit 616 that performs motion detection based on transferred data
Equipped with.

Next, the motion detecting means 610, the image blur correcting means 620, and the angle changing means 630 perform image processing on the wearer (key person) based on the image information data received by the wireless receiving means 400, for example. An operation of detecting a motion and correcting a blur of an image or converting an angle according to the motion of the wearer will be described.

Here, a camera as the image pickup device 100 is attached to a key person such as a baseball catcher,
A case will be described in which a wide-angle lens such as a fish-eye lens is attached to this camera, the image information is such that an image of 180 degrees in front of the camera is projected, and the following processing is performed using the image information in this wide range.

It is considered that the key person wearing the camera does not look around vaguely, but mainly looks at the object of greatest interest to him. As a key person, a person who operates in response to the movement of another player or the ball, such as an athlete, should pay attention to the moving image. For example, when a player wears a camera on the head, it is assumed that there is a moving target image that the player is paying attention to in front of the head.

Then, when the movement of the key person is large, the target image with the movement that the key person focuses on changes, and the content of the image information from the camera is such that the original target image is out of the image range. It should move greatly as if it were about to stick out. Also, when the movement of the key person is small, the target image of interest of the key person has not changed, and as the content of the image information from the camera, the original target image does not go out of the image range and moves. Should be small.

Therefore, in order to detect the movement of the key person based on the image information from the camera attached to the key person, first, the moving image extracted from the image information is detected as the target image, and the target object is detected. It is considered that the motion of the wearer may be indirectly detected based on the motion of the image.

Therefore, first, the operation for detecting a moving image (target image) will be described. First, the digitalized image information data from the camera is stored in the frame memory. In this case, the image information may be digitized by a camera addition device provided in the information processing device.

Next, a field serving as a reference for comparison is determined and stored in the reference frame memory 612. The fields before and after that are stored in the search frame memory 614, and the motion detection circuit 616 compares the signal levels of the corresponding pixels to detect the difference. Then, the position of the pixel whose detected difference is larger than a predetermined value is detected. Then, a portion in which the detected pixels appear in a predetermined number of continuous scanning lines, that is, a group of moving pixels is detected as a target image.

For example, in MPEG2, the amount of calculation required for motion detection can be reduced by predicting a group of moving pixels (pixel block) and predetermining the search range. It should be noted that this motion detection is performed by detecting the level difference between corresponding pixels in frame units.

Next, image processing is performed according to the motion of the target image thus detected. First, when the blur of the image of the target image of interest is detected and it is determined that the image blur is small and the movement of the target image of interest on the screen is due to the movement on the camera side,
That is, when it is determined that the image of interest is within the imaging range of the camera and is unlikely to deviate from the range, the image blur is corrected (image blur correction means). Specifically, so-called image blur correction is performed to correct the image so that the image of interest is in the center.

For example, when a camera is attached to the catcher in baseball, the catcher pays attention to the ball when the pitcher enters the posture and throws the ball, so it is considered that the ball does not fall out of the imaging range of the camera. . In this case, when the image blur is corrected, the image becomes very easy for the viewer to see. In this way, it is possible to perform image processing so that the target image of interest looks moving only on the screen by the movement of the target target image, and it is possible to make the image easier for the viewer to see.

On the other hand, the blurring of the image of the target image is detected, and since the movement of the body of the key person is large, the blurring of the image is large and the target image is likely to deviate from the imaging range of the imaging means or deviates from it. When it is determined that the image has changed, another image of interest is switched to and image processing of the scene change is performed (angle changing means). Specifically, another target image of interest is taken out and image processing is performed so that the target image of interest comes to the center of the screen. That is, the conventional operation of changing the angle of the camera is performed by image processing.

For example, when the camera is attached to the catcher in baseball, the ball thrown by the pitcher has been noted, but when the batter hits the ball and becomes a fly, the catcher turns upward in an attempt to catch the ball. It will be. Then, the image of the ball is once out of the imaging range, but the image of the ball is processed so that the image of the ball comes to the center again when the catcher faces upward. This is the same as changing the camera angle. In this way, the viewer can know that the target image has been switched when the movement of the body of the key person is large, so that a more powerful and powerful image can be provided.

When searching the target image of interest from the new image information, a predetermined target may be searched for. When a predetermined object is found, image processing is performed so that the object is in the center of the screen, and when there is no predetermined object, the object with a high frequency of coming to the approximate center of the image from the camera is selected. As a virtual object, image processing is performed so that it is in the center of the screen (angle changing means). When the camera does not move significantly, the image blur correction unit 620 corrects the image so that the object is in the center, so-called image blur correction (image blur correction unit).

Considering how the batter runs after hitting in baseball, as shown in FIG. 18, there is no movement on the background scoreboard. If the camera attached to the key person does not move, the position of the pixels forming the scoreboard on the screen does not change. However, since the batter runs after hitting, the positions of the pixels forming the batter change with time. Therefore, it is preferable that the movement of the camera attached to the key person is performed by detecting the movement of the image of a stationary object such as a scoreboard as shown in FIG.

The motion detecting means 610 has
When a mass that vibrates in the direction (primary vibration) has an angular velocity, it vibrates in the direction orthogonal to it due to the "Coriolis" effect (secondary vibration).
The movement of the wearer may be detected based on the output from a so-called vibrating gyro, which utilizes the occurrence of. For example, this vibrating gyro is attached to the wearer of the image pickup means 100, the output from the vibrating gyro is transmitted as radio data from the wireless transmission means 200 together with image information, and the movement of the wearer is calculated from the data received by the wireless reception means 400. May be detected.

As the vibrating gyro, for example, a sound or beam shape may be used. However, like the vibrating gyro of Silicon Sensing Systems, the vibration mode can be changed vertically (horizontal or beam shape) with a rod (tuning fork or beam shape). Instead of shaking, it is preferable to use one that suspends the vibrator ring and vibrates the ring to an elliptic strain. Such a vibrating gyro has no fulcrum that is sensitive to external stress, elliptical vibration is unlikely to occur under external vibration or shock, and it can faithfully detect high-accuracy angular velocity even in a bad environment. This is because it is suitable for wearing a vibration gyro to athletes in sports such as.

As shown in FIG. 16, the image information processing means 100 is a means for performing image processing in accordance with the movement of an object, a movement detecting means 640 for detecting the movement of the object, and a movement detecting means 640 for detecting the movement of the object. And an image deforming unit 650 that deforms the image.

The movement detecting means 640 and the deforming means 650 of these objects may be software processing performed by the control unit 370 based on a program recorded in a recording medium such as a computer-readable hard disk. The processing may be performed by a circuit as hardware, or may be performed by a combination of both.

As the circuit configuration example of the motion detecting means 640, the one shown in FIG. 17 may be used similarly to the motion detecting means 610.

Next, the motion detecting means 640 and the deforming means 650 deform the image according to the movement of the object captured by the image capturing means such as a baseball, and transform the image into a deformed image or a dramatic image. The operation of converting to a different image will be described. Here, as in the above case, the camera is attached to a key person such as a baseball catcher as the image pickup device 100, a wide-angle lens such as a fisheye lens is attached to the camera, and the image information is such that a picture of 180 degrees in front of the camera is taken. Now, a case where the following processing is performed using the image information in this wide range will be described.

First, based on the image information data received from the camera and digitized, the key person recognizes the pixel block of the person or object of interest, that is, the image of interest.
The additional information such as the position and speed of the target image is detected. For example, in the case of a ball game, the position and speed of the ball are detected, and in the case of a play, changes in the facial expression of the target person, which is the target image captured by the camera, are recognized. In this case, a target object and a target person who are the triggers for deforming the image are determined in advance and stored in, for example, the target image database of the various databases 360 shown in FIG. It should be noted that from the image information received from the camera, the one that makes a special movement is captured, and at that point, the target object,
It may be stored as a target person in the target image database.

In the case of a ball game, the position, velocity, etc. of the ball are parameters for deforming. For example, in the case of baseball, the image of the ball captured by the camera attached to the catcher is recorded in advance in the search frame memory 614 shown in FIG. 17, for example, as a feature of the image signal of the ball compared with the background, and image information from the camera is recorded. The frame is stored in, for example, the reference frame memory 612, the data is compared by the motion detection circuit 616 to recognize the ball, and the size and position of the ball on the screen are measured. This can be recognized and measured by the change of the luminance signal on the scanning line of the video signal.

Here, an example of a method for detecting the velocity of the ball by the motion detection circuit 616 based on the image information (image signal) from the camera will be described. Calculation of the velocity of the ball, for example, as shown in FIG. 19, if the lens of the camera is fixed, the angle of view becomes constant, and the ratio of the size of the ball approaching from a distance and the screen size is Db / Da1 to Db / Da2 and Db.
/ It changes so that it may increase with Da1. Since the image information (image signal) is output from the camera along with the time change, the speed of the ball is detected by calculating the time change of the above ratio.

Next, the deformed image stored in advance at the position of the target image of interest such as the actual target object or target person recognized by the motion detection circuit 616 is fitted. For example, as shown in FIG. 20, when the image of the pitcher is the target image of interest, the deformed image of the pitcher is fitted in the position of the image of the actual pitcher, or when the image of the ball is the target image of interest, the ball is As it gets closer to the camera, the deformed image in which the size of the ball is made larger than the actual size is fitted into the position of the ball.

The speed of the ball in the ball game is, for example, 130 km / h for a pitcher ball in baseball.
Although it is not a big change of about 160 km, the difference is great for the batter. Therefore, as shown in FIG. 21, the color of the ball in the actual image may be changed according to the speed of the ball. Further, as shown in FIG. 22, a flame may be applied around the ball, a ring may be applied, or the like depending on the speed, and images may be combined by CG. This allows the viewer to feel the speed of the ball in real time. In FIG. 21 and FIG. 22 described above, FIG.
The speed of the ball increases in the order of (b) and (c).

Note that, for example, a deformed image created by deforming an actual image with CG (Computer Graphics) or the like in consideration of the characteristics of the target object and the target person is, for example, the deformed image database of the various databases 360 shown in FIG. It is also possible to store it in advance and retrieve necessary images from this deformed image database.

Further, as the target image, for example, in the case of baseball, in addition to the above-mentioned pitcher and ball, there are stealers and the like.

Next, the sound collecting means 11 attached to the wearer
The sound information processing means 700 for converting the sound information received from 0 into sound information that is more powerful than the actual sound will be described.

The sound information processing means 700 is the sound collecting means 110.
The sound information from is converted into a sound, a sound of a ball, a sound of a ball hitting a bat, a sound identification means 710 for identifying what the sound is, a sound identified by the sound identification means 710, and a more powerful sound. The sound conversion unit 720 and the specific sound masking unit 730 as a sound information security unit for masking only a specific sound among the sounds identified by the sound identifying unit 710 are provided.

The sound identifying means 710, the sound information converting means 720, and the specific voice masking means 730 are processed by the software executed by the control unit 370 based on the program recorded in the recording medium such as a computer-readable hard disk. It may be processed by a circuit of hardware, or may be processed by a combination of both.

In the case of baseball, for example, when the sound identifying means 710 recognizes the sound of cutting the wind of the ball, the sound information processing means 700 uses the motion detecting means 640 to detect the ball by the method shown in FIG. The sound information converting means 720 converts the wind blowing sound of the ball into a larger sound according to the speed of the ball by using the result of detecting the speed of the ball. For example, the higher the speed of the ball, the more the sound of cutting the wind of the ball is converted into a louder sound.

The sound information converting means 720 may increase the sound received by the catcher mitt according to the speed of the ball. Further, when the ball hits the bat, the sound of the ball hitting the bat may be increased according to the speed of the ball.

By converting the sound into a sound louder than the actual sound in accordance with the speed of the ball, the power of the ball is emphasized not only by the image but also by the sound, and more interesting and powerful image information and sound information are obtained. It can be delivered to the viewer.

Next, security means for sound information will be described. As described above, in the present embodiment, since the image capturing means 100 and the sound collecting means 110 are attached to the actor of the event, for example, the athlete who is engaged in sports, the image information and the sound information of the wearer are unspecified. It is necessary to prevent it from being used and abused by other people. Therefore, the security means 140 and 340 are provided so that only a specific person can wirelessly receive the data of such image information and sound information and restore the original data, but here, the specific voice information itself is masked. This is so that no one can acquire specific voice information. Therefore, even a specific person having the security code by the security means 140, 340 cannot make the specific voice information itself the original data.

Specifically, when a specific voice is identified by the sound identifying unit 710 in the voice information from the sound collecting unit 110, the specific voice masking unit 730 masks only the voice to convert the data. . Therefore, a voice such as a cheer of a spectator other than a specific voice can be acquired.

Specific voices include, for example, a meaningful voice as a human voice such as a folk ball, especially a voice of a person who performs an event such as sports. Also, a voice including a predetermined word may be masked as a human voice.

As a result, regarding specific voice information,
Since it can be prevented from being obtained by anyone, certain voices such as the voice of the catcher are masked in, for example, baseball, so the sign to the pitcher can be known in advance or the next action can be predicted. It is possible to prevent the possibility of being created. As a result, sufficient security can be ensured.

The image information and sound information data processed by the information processing means 350 as described above are stored in, for example, the image information database and sound information database of the various databases 360 shown in FIG.

Further, the information processing apparatus 300 is provided with a transmitting means for transmitting television radio waves, and the image information and sound information data processed by the information processing means 350 are transmitted in real time as analog or digital television broadcasting. May be. In this case, since the data processed by the information processing means 350 is digital data, it can be transmitted as it is as a digital broadcast.

When transmitting as a digital broadcast, the transmitting means is the wireless transmitting means 20 of the wireless device 120.
0, a means similar to the encoder 130 may be provided.

For example, when transmitting by CS digital broadcasting service, digital data of image information and sound information processed by the information processing means 350 is compressed and encoded and transmitted. PSI (Program Specific Informatio)
n) control information for channel selection and SI (Service Informa
electronic program guide (EPG: Electronic Program Guide)
de) create information to display. Information for conditional access is created from the contract information of the subscriber. These pieces of information are multiplexed into the transport stream (Tran
It becomes a single packet sequence such as sport stream) and after adding error correction code, QPSK (Quadrature Phase Shift K
eying) modulated and transmitted over satellite lines. The receiving side, which is the viewer, performs the reverse process of the transmitting side, and outputs data such as the image information and sound information of the program to a general television receiver.

The image information and sound information data processed by the information processing means 350 may be provided to the viewer terminal device via a network such as the Internet.

In this case, the content distribution service system can be constructed by providing the information processing apparatus 300 shown in FIG. 1 with a function as a business operator server apparatus, for example. FIG. 24 shows a configuration example of such a content distribution service system.

The business entity server 1200 uses image information and sound information data obtained by processing the data from the image pickup means 100 attached to the actor of the above-mentioned event with the information processing apparatus 300, and other image pickup means 102. Receiving means for receiving the image information picked up by 104 and the sound information data by the other sound collecting means 112 is provided. Therefore, in the content here, in addition to the image information and sound information data obtained by processing the data from the image pickup means 100 or the like by the information processing device 300, the image information picked up by the other image pickup means 102 and 104, other Data including sound information data obtained by the sound collection unit 112 and image processing performed by the spatial image processing unit 1500 and the line-of-sight angle image processing unit 1600 described later are also included.

The content distribution system 1000 described above
As shown in FIG. 24, the viewer terminal devices 1100-1 ... 11
00-n, the enterprise server device 1200, the billing processing server 1300, etc. are the network 140 such as the Internet.
Connected via 0. The viewer terminal device 1100
Are connected via the communication carrier 1420 of the provider 1410.

The viewer terminal device 1100 selects an image which outputs a selection signal in accordance with the selection of the image information desired to be distributed from the image display means such as a display and the image information which can be distributed from the enterprise server device 1200. Means, the selection signal from the image selection means
00 is provided.

The business entity server device 1200 includes a spatial image processing means 1500 and a line-of-sight angle image processing means 1600 in addition to the configuration of the information processing device 300 described above.

The spatial image processing means 1500 divides the play space in which the event takes place into a matrix into a matrix and divides it based on the images from the image pickup means 100 and 102 received by the receiving means. Processing for generating an image of the played space is performed.

In the case of baseball, for example, the field of the stadium corresponds to the play space here. For example, the entire play space is coordinate-converted into virtual three-dimensional coordinates on the basis of images from a plurality of image pickup means such as the image pickup means 100 attached to the catcher and the image pickup means 102 placed in the press seat of the stadium. Then, the play space of this virtual three-dimensional space is formed into a matrix and divided to generate an image showing each divided space such as the pitcher's mound space and the third baseman's space.

By distributing the images of these divided spaces via the network 1400 such as the Internet, the viewer can select an image of the desired play space and view it on the display of the terminal device 1100.

Also, the line-of-sight angle image processing means 1600
Is the image pickup means 100, 1 received by the receiving means.
Based on the image from 02 or the like, processing is performed to generate an image with a different line-of-sight angle, such as an image of the angle from the spectator seat or an image of the line-of-sight angle of the person wearing the imaging means in the venue space of the event.

For example, the image pickup means 1 attached to the catcher
00, the entire play space is coordinate-converted into virtual three-dimensional coordinates on the basis of images from a plurality of image pickup means such as the image pickup means 102 installed in the press seat of the stadium. Of the virtual three-dimensional space, for example, the pitcher's mound space has different angle images depending on the position of the spectator seats viewing this space. Therefore,
The viewing position in the virtual three-dimensional space is matrixed, the viewing angle for each viewing position is calculated, and a viewing angle image is generated.

By distributing these angle images via the network 1400 such as the Internet,
The viewer can select a desired angle image such as an angle image viewed from the arena seat or an angle image viewed from the outfield seat and view it from the display of the terminal device 1100 while staying at home.

Also, the angle image processing means 1600
Alternatively, the calculated line-of-sight angle may be switched according to the movement of the person who performs the event to generate an image for each person who performs the event. For example, in soccer, a player moves around a wide ground, so when viewing a favorite player from an angle viewed from a certain position, if the player moves farther from that position, the image of that player becomes smaller. Therefore, an image in which the line-of-sight angle is switched so that a specific player can be seen in a large image from a close position is generated. This will allow you to always see your favorite player at the angle from the best view.

The billing processing server 1300 determines the billing amount according to the selected image such as the image of the play space or the angle image based on the selection signal from the viewer terminal device 1100 and bills. .

For example, the charge amount may be determined according to the needs of the consumer who desires image distribution. When an advertisement is delivered as additional information together with an image, the amount to be charged may be determined according to the presence or absence of the advertisement.

Also, the amount to be charged may be determined according to the degree of processing of the image to be delivered together with the image. For example, as described above, an image on which image processing (processing) for changing the color of the ball as shown in FIG. 21 is performed according to the speed of the baseball, and a flame on the ball as shown in FIG. The amount to be charged may be changed depending on the image to which the added processing is applied. In this case, a higher amount of money may be charged for an image processed into a more interesting image.

Further, it is also possible to determine and charge a fee for each image from the person who inserts the image pickup means 100 or a specific person who holds an event. For example, every baseball player,
Charge for each soccer player according to the image of the player you want to see.

The preferred embodiments of the present invention have been described above with reference to the accompanying drawings, but it goes without saying that the present invention is not limited to such examples. It is obvious for those skilled in the art that various changes or modifications can be conceived within the scope described in the claims, and those modifications are naturally included in the technical scope of the present invention. Understood.

For example, although a case has been described with the present embodiment where the provider server apparatus 1200 includes all the configurations of the information processing apparatus 300, the configuration is not necessarily limited to this, and a configuration separate from the information processing apparatus is provided. And then
Data such as image information and sound information from the information processing apparatus 300 may be received by the business entity server apparatus 1200 and distributed. In addition, a server for images etc. is provided separately,
Data such as image information and sound information from the information processing device 300 may be stored therein.

[0130]

As described above in detail, according to the present invention, the image information and the sound information from the viewpoint of the player or the like can be changed to information that is easy to see and powerful according to the movement of the player, or can be made more interesting and fun information. It is possible to provide an information processing system that can be delivered to viewers.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image processing system according to an embodiment of the present invention.

FIG. 2 is a diagram showing an example of attachment of an image pickup means and the like according to the present embodiment.

FIG. 3 is a diagram showing another example of attachment of the image pickup means and the like according to the present embodiment.

FIG. 4 is a diagram showing another example of attachment of the image pickup means and the like according to the present embodiment.

FIG. 5 is a diagram showing another example of attachment of the image pickup means and the like according to the present embodiment.

FIG. 6 is a block diagram showing a configuration example of a wireless device according to the present embodiment.

FIG. 7 is a diagram showing OFDM system transmission parameters according to the present embodiment.

FIG. 8 is a diagram showing a transmission capacity of the OFDM system according to the present embodiment.

FIG. 9 is a block diagram showing a configuration of a transmission control unit according to the present embodiment.

FIG. 10 is a block diagram showing an example of a convolutional coding circuit according to the present embodiment.

FIG. 11 is a block diagram showing an example of a shift register of a circuit that generates a pseudo random sequence according to the present embodiment.

FIG. 12 is a block diagram showing an example of a time interleave length according to the present embodiment.

FIG. 13 is a block diagram showing a configuration example of time interleaving according to the present embodiment.

FIG. 14 is a block diagram showing an arrangement example of carriers other than the data carrier according to the present embodiment.

FIG. 15 is a diagram for explaining addition of guard intervals according to the present embodiment.

FIG. 16 is a block diagram showing a configuration of image information processing means according to the present embodiment.

FIG. 17 is a block diagram showing a configuration example of motion detecting means according to the present embodiment.

FIG. 18 is a diagram illustrating an image to be detected by the motion detection unit according to the present embodiment.

FIG. 19 is a diagram illustrating a ball velocity detection method according to the present embodiment.

FIG. 20 is a diagram showing an example in which a deformed image is fitted to the position of an actual image according to the present embodiment.

FIG. 21 is a diagram showing an example of deforming an actual image according to the present embodiment.

FIG. 22 is a diagram showing another example in which an actual image according to the present embodiment is deformed.

FIG. 23 is a diagram showing a configuration of sound information processing means according to the present embodiment.

FIG. 24 is a block diagram showing the configuration of a content distribution system according to this embodiment.

[Explanation of symbols]

100 ... Imaging means 110 ... Sound collecting means 120 ... Wireless device 200 ... Wireless transmission means 140 ... Security measures 300 ... Information processing device 320 ... Wireless device 340 ... Security measures 350 ... Information processing means 360 ... Various databases 400 ... Wireless receiving means 600 ... Image information processing means 610 ... Motion detection means for the imager wearer 620 ... Image blur detecting means 630 ... Angle conversion means 640 ... Motion detection means for focused image 650 ... Image deforming means 700 ... Sound information processing means 710 ... Sound identification means 720 ... Sound information conversion means 730 ... Specific voice masking means

   ─────────────────────────────────────────────────── ─── Continued front page    (72) Inventor Kazumasa Takahashi             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Nobuyuki Ishikawa             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation (72) Inventor Sojiro Kizu             6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Soni             -Inside the corporation F-term (reference) 5C064 BA07 BB01 BB02 BB05 BC01                       BC10 BC17 BC18 BC22 BC23                       BD02 BD08                 5K067 AA21 AA34 BB04 BB21 CC02                       DD51 DD52 EE02 EE12 FF02                       FF23 KK17

Claims (4)

[Claims] 1. An image pickup device mounted on each of the bodies of a plurality of persons performing an event, a wireless transmission device for wirelessly transmitting image information of the image pickup device, and image information from the image pickup device. Each wireless receiving means for receiving wirelessly, and frequency conversion means for converting a wireless frequency at the time of wirelessly transmitting and receiving between each wireless transmitting means and each wireless receiving means to a different wireless frequency are provided. And wireless system. 2. A radio system between each radio transmission unit and each radio reception unit uses a direct frequency division multiplexing system, and the frequency conversion unit converts a radio frequency based on different carrier waves in a mixer. The wireless system according to claim 1, wherein: 3. The event is an event performed in a predetermined space by one or more members belonging to a plurality of groups, and the frequency conversion means changes the radio frequency for each group. The wireless system according to claim 1, wherein the wireless system is provided. 4. The radio system according to claim 1, wherein the frequency conversion unit changes the radio frequency for each wearer of the imaging unit.