KR102602069B1 - Device for generating audio and video data corresponding to blocks inserted into the home plate - Google Patents

Abstract

According to an embodiment, an apparatus for generating audio data and video data corresponding to a block inserted into a groove plate includes a groove plate including a groove, a block inserted into the groove of the groove plate, a depth measuring camera for determining the position of the block, and the block. It may include a color discrimination camera for determining the color of and a processor for generating at least one of audio data and video data using at least one of the location information of the block and the color information of the block.

Description

Device for generating audio and video data corresponding to blocks inserted into the home plate}

The present invention relates to an apparatus for generating audio data and video data corresponding to a block inserted into a home board. More specifically, the present invention relates to an apparatus for generating audio data and video data using at least one of location information and color information generated by detecting a block inserted into a home board. It relates to a device for generating audio data and video data corresponding to blocks inserted into a home board for generating data.

Recently, as interest in educational toys has increased, interest in the infant and children's education and educational product industry is increasing. However, the educational and educational animation platforms are being developed and produced by companies in advanced countries such as Germany and the United States, occupying the global market and supplying them.

In relation to this, the government is providing various supports to foster the education industry, which will become a market opportunity in the future, but there are not many products and services that provide educational content that fit the purpose.

In addition, looking at domestic and international technology trends, in the field of children's toys, especially block toys, traditional Lego-type block toys account for most. Although various new products such as 3D puzzles have been released recently, block toys and various types of toys are generally used. Block play is often a non-digital form of simple craft play, such as shape blocks or magnetic blocks, and has limited functions, such as no video function and no integration with educational content.

In particular, in the case of young learners such as infants and toddlers, they have low concentration and easily lose interest, so there is a need to propose a learning device that can continuously stimulate the interest of young learners.

The present invention was derived from the above-described need, and seeks to provide a device that can continuously stimulate interest during learning or play by providing a digitalized function that outputs video or audio data to traditional block toy play. There is a purpose to it.

According to an embodiment, an apparatus for generating audio data and video data corresponding to a block inserted into a groove plate includes a groove plate including a groove, a block inserted into the groove of the groove plate, a depth measuring camera for determining the position of the block, and the block. It may include a color discrimination camera for determining the color of and a processor for generating at least one of audio data and video data using at least one of the location information of the block and the color information of the block.

The voice data may include at least one of pitch information corresponding to the location of the block and melody information corresponding to the color of the block.

It may further include a beam projector that emits a beam to the groove plate.

The block is placed in a scan area created by scanning the home plate using the beam of the beam projector, and the processor reproduces at least one of the audio data and the video data related to the block placed in the scan area. can be controlled.

The scan area for the home plate may be configured in line units.

The block may be one or multiple.

It may further include a case member accommodating the groove plate and a frame member implemented in a form surrounding a portion of the case member.

The frame member is composed of an upper frame and side frames extending from both ends of the upper frame, and at least one of the depth measurement camera, the color discrimination camera, and the beam projector may be mounted on the upper frame.

It may further include an audio output unit that reproduces the audio data, and at least one of the processor and the audio output unit may be mounted on the case member.

According to the present invention, users can experience playing audio data and video data together while playing with blocks, resulting in a more dynamic and interesting learning effect.

Additionally, various types of audio and video data can be played depending on the combination of blocks, making it possible to play back numerous audio and video data even if a limited space and a limited number of blocks are given.

Figure 1 is a block diagram of an apparatus for generating audio data and video data corresponding to blocks introduced into a home board according to an embodiment.
FIG. 2 is a flowchart illustrating a method of generating audio data and video data corresponding to blocks inserted into the homepage according to an embodiment.
FIG. 3A is an example of an image obtained as a result of depth measurement by a depth measurement camera, and FIG. 3B is an example of an RGB image obtained as a result of receiving visible light by a color discrimination camera.
Figure 4 is a perspective view of an apparatus for generating audio data and video data corresponding to blocks inserted into the home plate according to an embodiment.
Figure 5 is a view of the case member accommodating the groove plate when viewed from the front.
FIG. 6 is a diagram referenced to explain a process of reproducing at least one of audio data and video data generated using at least one of position information and color information of a block inserted into the home plate according to an embodiment.

The detailed description of the present invention described below refers to the accompanying drawings, which show by way of example specific embodiments in which the present invention may be practiced. These embodiments are described in sufficient detail to enable those skilled in the art to practice the invention. It should be understood that the various embodiments of the present invention are different from one another but are not necessarily mutually exclusive. For example, specific shapes, structures and characteristics described herein with respect to one embodiment may be implemented in other embodiments without departing from the spirit and scope of the invention. Additionally, it should be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the invention. Accordingly, the detailed description that follows is not intended to be taken in a limiting sense, and the scope of the invention is limited only by the appended claims, together with all equivalents to what those claims assert, if properly described. Similar reference numbers in the drawings refer to identical or similar functions across various aspects.

FIG. 1 is a block diagram of an apparatus 1 for generating audio and video data corresponding to a block introduced into a home board according to an embodiment, and FIG. 2 is a block diagram of audio data and video data corresponding to a block introduced into a home board according to an embodiment. This is a flow chart to explain the creation method.

Referring to FIGS. 1 and 2 together, the audio data and video data generating device 1 corresponding to the block inserted into the home plate includes a depth measurement camera 110, a color discrimination camera 120, a processor 130, and a memory ( 140), a beam projector 150, and an audio output unit 160.

Referring to FIG. 5 , the depth measuring camera 110 can determine the positions of the blocks 20, 21a, 21b, 22a, and 22b inserted into the groove 11 of the groove plate 10.

According to an embodiment, the depth measurement camera 110 may be a time of flight (TOF) camera. Specifically, the depth measurement camera 110 may be implemented as including an emitter configured to emit infrared light and a receiver configured to receive reflected light of the infrared light.

The processor 130 operatively coupled to the depth measurement camera 110 acquires information about reflected light of infrared light emitted from the depth measurement camera 110 using the depth measurement camera 110, and obtains information about the reflected light of the infrared light emitted from the depth measurement camera 110. Based on the information, information about the distance between the depth measurement camera 110 and each of a plurality of subjects within the field of view (FOV) of the depth measurement camera 110 may be obtained.

And, the processor 130 can determine the position of the block 20 inserted into the groove 11 of the groove plate 10 based on information about the distance. Specifically, Figure 3a is an example of an image obtained as a result of depth measurement by the depth measurement camera 110. Referring to this, in the case of the area where the block 20 is inserted into the groove 11, an image is formed like area A1. , it can be seen that in the area where the block 20 is not inserted into the groove 11, an image is formed like area A2. The processor 130 may determine the location of the block 20 placed on the home plate 10 by determining that the block 20 has been inserted into the A1 area based on the image of the A1 area. In other words, the position of the block 20 can be determined by measuring the height (or depth) of the block 20 inserted into area A1. At this time, if a subject that is different from the height of the block 20 is detected, it can be excluded from the position measurement.

According to another embodiment, the depth measurement camera 110 projects a pattern with certain rules onto a subject to be restored in three dimensions through a projector, photographs the image with a camera, and then uses the image to obtain a correspondence using structured light. (structured light, SL) It could also be a camera.

The color discrimination camera 120 may be an RGB camera including a receiving unit configured to receive visible light.

When the processor 130 determines the position of the block 20 using the depth measurement camera 110, the processor 130 acquires RGB images including the groove 11 of the groove plate 10 using the color discrimination camera 120. The color of the block 20 at the corresponding location can be determined (s210, s220).

Specifically, Figure 3b is an example of an RGB image obtained as a result of receiving visible light by the color discrimination camera 120. Referring to Figure 3b, the processor 130 determines the color of the block 20 located in area A1 of Figure 3a. You can.

The processor 130 uses at least one of the position information of the block 20 generated through the depth measurement camera 110 and the color information of the block 20 generated through the color discrimination camera 120 to produce audio data and video. At least one of the data can be generated (s230).

And, according to the embodiment, the voice data may include at least one of pitch information corresponding to the position of the block 20 and melody information corresponding to the color of the block 20. In addition, voice data may be generated by variously combining the corresponding pitch information and/or melody information based on information such as the size of the block 20, the shape of the block 20, and the height of the block 20. You can.

For reference, in the present invention, the pitch information refers to 'do, re, mi, fa, sol... It refers to information that can express various pitches of notes such as 'series name information, music code information such as 'sharp, flat' applicable to the same system name, and octave information, and the embodiment of the present invention is not limited thereto.

Melody information refers to various sound information generated according to various sound sources such as bird sounds, water sounds, baby laughter, and musical instrument sounds, and embodiments of the present invention are not limited thereto.

Referring to FIG. 5, for example, the first block 21a and the second block 21b inserted into the groove 11 are the same red, and the processor 130 generates a bird sound corresponding to red, but the first block 21a For (21a), a bird song of a relatively low octave can be generated, and for the second block (21b), a bird song of a relatively high octave can be generated. The processor 130 has a memory ( It can be identified by reading the mapping information previously recorded in 140).

Meanwhile, the third block 22a and the fourth block 22b inserted into the groove 11 are of the same yellow color, and the processor 130 generates a musical instrument sound corresponding to yellow, but the system name for the third block 22a is 'Mi' can be generated among the names, and 'Sol' can be generated among the system names for the fourth block 22b. The processor 130 has a memory 140 that indicates that the yellow color corresponds to the sound of the instrument, the position of the third block 22a corresponds to 'mi', and the position of the fourth block 22b should correspond to 'sol'. ) can be identified by reading the mapping information previously recorded in .

According to another embodiment, the processor 130 may generate melody information in response to the position of the block 20 and generate pitch information in response to the color of the block 20.

The processor 130 may generate image data using at least one of the location information of the block 20 and the color information of the block 20.

For example, the first block 21a and the second block 21b inserted into the groove 11 are of the same red color, and the processor 130 generates an image of a bird corresponding to red, but the first block 21a For the second block 21b, an image of a relatively small bird can be generated, and for the second block 21b, an image of a relatively large bird can be generated. The processor 130 determines that the color corresponding to red is an image of a bird, the position of the first block 21a corresponds to an image of a relatively small bird, and the position of the second block 21b corresponds to an image of a relatively large bird. What needs to be created can be determined by reading mapping information previously recorded in the memory 140.

Meanwhile, the third block 22a and the fourth block 22b inserted into the groove 11 are of the same yellow color, and the processor 130 generates a water droplet image corresponding to yellow, but the third block 22a has a relative yellow color. Small-sized water droplets can be generated, and relatively large-sized water droplets can be generated for the fourth block 22b. The processor 130 generates a water droplet image corresponding to yellow, the position of the third block 22a corresponds to a relatively small water droplet, and the position of the fourth block 22b corresponds to a relatively large water droplet. What needs to be done can be determined by reading mapping information previously recorded in the memory 140.

In addition, the processor 130 may generate image data corresponding to various combinations of information such as the size of the block 20, the shape of the block 20, and the height of the block 20.

Meanwhile, the processor 130 may output audio information and/or video information related to the block 20 corresponding to the scan position of the beam projector 150 on the home plate 10 (s240).

Specifically, referring again to FIG. 5 , at least one of audio data and video data may be generated and/or reproduced for each block 20 located in the scan area by the beam emitted from the beam projector 150. And, this scan area may be composed of line units in which the grooves 11 of the groove plate 10 are arranged in the column (col) direction.

For example, the position information and/or color information corresponding to each of the first block 21a and/or the third block 22a located in the 8th column are combined to generate first audio data and/or first video data. and/or are reproduced, and the position information and/or color information corresponding to each of the second block 21b and/or the fourth block 22b located in the 9th column are combined to produce second audio data and/or second video data. may be created and/or reproduced.

That is, one audio data and/or one video data can be generated and/or reproduced by a combination of location information and/or color information corresponding to one or a plurality of blocks 20 arranged in one scan area. .

For example, referring to FIG. 5 together, the first scan area where the first block 21a and the third block 22a are scanned using the beam emitted from the beam projector 150 has a relatively low field name. Generate and/or reproduce the sound of the instrument and/or the image of splashing water droplets, and for the second scan area where the second block 21b and the fourth block 22b are scanned, the sound of the instrument and/or the sound of the instrument with a relatively high name. You can create and/or play an image of water droplets splashing.

That is, when a plurality of blocks are located in each scan area, at least one of the position information and color information corresponding to each of the plurality of blocks is combined to generate different audio data or different image data for each scan area. It can be done.

Meanwhile, according to another embodiment, the processor 130 may generate voice data based on at least one of melody information corresponding to the position of the block 20 and pitch information corresponding to the color of the block 20. It may be possible.

The memory 140 can store data generated as a result of controlling each component of the processor 130, and the processor 130 can operate by reading data previously stored and/or generated in the memory 140.

The memory 140 may store and/or generate in advance the mapping relationship of audio data and/or video data corresponding to the position of the block 20 and/or the color of the block 20, and the processor 130 The information can be read and played back.

The beam projector 150 can perform a point scan on a subject using a beam. For example, a point scan can be performed in the X- and Y-axis directions by performing a line-by-line point scan on the subject surface. Point scanning creates a scan area by projecting light points onto the surface of the subject along a scan line.

Referring to FIG. 6B, the beam projector 150 according to the embodiment emits a beam to the groove plate 10 and when the block 20 is placed in the scan area SA2 created by scanning the groove plate 10 , the processor 130 may control the reproduction of audio data and/or video data E related to the block 20 arranged in the scan area SA2. In Figure 6b, it can be seen that the water droplet splashing effect related to the block 20 is added to the image and reproduced.

The audio output unit 160 may include a speaker for outputting audio data.

Figure 4 is a perspective view of the audio data and video data generating device 1 corresponding to the block inserted into the home plate according to an embodiment.

As shown in FIG. 4, the audio data and video data generating device 1 corresponding to the block inserted into the home plate includes a home plate 10, a block 20, a case member 30, and a frame member 40. It can be included.

For reference, Figure 5 is a view of the case member 30 accommodating the groove plate 10 when viewed from the front, and will be described below with reference to Figures 4 and 5 together.

The groove plate 10 recognizes the inserted block 20 and includes a first member 10a including a groove 11 for the processor 130 to operate, and a first member 10a for accommodating blocks that are not inserted into the groove 11. It may be configured to include a second member 10b.

The first member 10 may be configured to include a plurality of grooves 11 arranged in a grid shape.

Each of the plurality of grooves 11 may be arranged to be spaced apart from each other at equal intervals in the row and/or column direction. Accordingly, each of the rows (1st to nth rows) may be arranged to include the same number of grooves 11.

Depending on the embodiment, the size, height, shape, color, etc. of the block 20 can be manufactured in various ways, and correspondingly, the size, height, shape, etc. of the groove 11 can also be manufactured in various ways.

For example, the block 20 may be manufactured in various shapes such as a cylinder, triangular pillar, square pillar, or star pillar, and the lower surface of the groove 11 may also have a corresponding shape such as a circle, triangle, square, or star. It can be produced in various ways.

The heights of the grooves 11 may all be manufactured to be the same, but some grooves 11 may have different heights.

The case member 30 can accommodate and support the groove plate 10.

At least one of the processor 130 and the audio output unit 160 disposed below the groove plate 10 may be mounted on the case member 30.

The frame member 40 may be implemented in a form that surrounds a portion of the case member 30.

The frame member 40 may be implemented in a form that surrounds the central portion of both sides of the case member 30.

The frame member 40 may include an upper frame 41 and side frames 42 extending from both ends of the upper frame 41.

The upper frame 41 may be spaced a predetermined distance away from the groove plate 10 and placed on top of the groove plate 10.

One end of the side frame 42 may be in contact with the upper frame 41, and the other end may be in contact with portions of both sides of the case member 30.

At least one of the depth measurement camera 110, the color discrimination camera 120, and the beam projector 150 may be mounted on the upper frame 41.

In a form in which the frame member 40 surrounds a portion of the case member 30 and the upper frame 41 is disposed at the upper part of the groove plate 10 at a predetermined distance from the groove plate 10, the depth measurement camera 110, color At least one of the discrimination camera 120 and the beam projector 150 is implemented to be mounted on the upper frame 41 to more accurately detect the position and/or color of the groove plate 10 and the block 20 and accurately project the beam. By emitting light, scanning of the subject can be performed more easily.

FIG. 6 is referenced to explain a process of reproducing at least one of audio data and video data generated using at least one of the position information and color information of the block 20 inserted into the home plate 10 according to the embodiment. It is a drawing.

As shown in FIG. 6, it can be seen that the beam scan is performed in column units but in the row direction.

The scan area SA1 in FIG. 6A represents the scan area before detecting the block 20, and the scan area SA2 in FIG. 6B represents the scan area created by detecting the block 20.

At the same time as the scan area SA2 of FIG. 6B is created, an effect is applied to the image generated using at least one of the position information and color information of the block 20 located in the scan area SA2, resulting in image data showing water droplets ( You can see that E) has been played.

At this time, the beam projector 150 does not project the beam onto the block 20 so that the color of the block 20 can be accurately recognized.

The features, structures, effects, etc. described in the embodiments above are included in one embodiment of the present invention and are not necessarily limited to only one embodiment. Furthermore, the features, structures, effects, etc. illustrated in each embodiment can be combined or modified and implemented in other embodiments by a person with ordinary knowledge in the field to which the embodiments belong. Therefore, contents related to such combinations and modifications should be construed as being included in the scope of the present invention.

In addition, although the above description has been made focusing on the examples, this is only an example and does not limit the present invention, and those skilled in the art will understand the above examples without departing from the essential characteristics of the present embodiment. You will be able to see that various modifications and applications are possible. For example, each component specifically shown in the examples can be modified and implemented. And these variations and differences in application should be construed as being included in the scope of the present invention as defined in the appended claims.

Claims (8)

a groove plate including grooves;
a block inserted into the groove of the groove plate;
a depth measuring camera to determine the location of the block;
a color discrimination camera for determining the color of the block; and
A processor that generates at least one of audio data and video data using the location information of the block and the color information of the block,
The voice data includes pitch information corresponding to the position of the block and melody information corresponding to the color of the block,
The melody information represents various sound information generated according to various sound sources,
A device that generates audio and video data corresponding to blocks inserted into the home plate.
delete According to clause 1,
It further includes a beam projector that emits a beam to the groove plate,
The block is placed in a scan area created by scanning the home plate using the beam of the beam projector,
The processor controls reproduction of at least one of the audio data and the video data associated with the block arranged in the scan area,
A device that generates audio and video data corresponding to blocks inserted into the home plate.
According to clause 3,
The scan area for the home plate is configured in line units,
A device that generates audio and video data corresponding to blocks inserted into the home plate.
According to claim 3 or 4,
The block may be one or more individuals,
A device that generates audio and video data corresponding to blocks inserted into the home plate.
According to clause 3,
a case member accommodating the groove plate; and
Further comprising: a frame member implemented in a form surrounding a portion of the case member,
A device that generates audio and video data corresponding to blocks inserted into the home plate.
According to clause 6,
The frame member consists of an upper frame and side frames extending from both ends of the upper frame,
At least one of the depth measurement camera, the color discrimination camera, and the beam projector is mounted on the upper frame,
A device that generates audio and video data corresponding to blocks inserted into the home plate.
According to clause 7,
It further includes a voice output unit that reproduces the voice data,
At least one of the processor and the audio output unit is mounted on the case member,
A device that generates audio and video data corresponding to blocks inserted into the home plate.

Images