WO2018032629A1 - Teaching toy kit and strike action detection method thereof - Google Patents

Teaching toy kit and strike action detection method thereof Download PDF

Info

Publication number
WO2018032629A1
WO2018032629A1 PCT/CN2016/105736 CN2016105736W WO2018032629A1 WO 2018032629 A1 WO2018032629 A1 WO 2018032629A1 CN 2016105736 W CN2016105736 W CN 2016105736W WO 2018032629 A1 WO2018032629 A1 WO 2018032629A1
Authority
WO
WIPO (PCT)
Prior art keywords
tambourine
image
area
value
color
Prior art date
Application number
PCT/CN2016/105736
Other languages
French (fr)
Chinese (zh)
Inventor
杜鹏
范旭
暴满粟
孙贤军
宿方琪
Original Assignee
上海葡萄纬度科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 上海葡萄纬度科技有限公司 filed Critical 上海葡萄纬度科技有限公司
Publication of WO2018032629A1 publication Critical patent/WO2018032629A1/en

Links

Images

Classifications

    • G—PHYSICS
    • G09—EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09B—EDUCATIONAL OR DEMONSTRATION APPLIANCES; APPLIANCES FOR TEACHING, OR COMMUNICATING WITH, THE BLIND, DEAF OR MUTE; MODELS; PLANETARIA; GLOBES; MAPS; DIAGRAMS
    • G09B5/00—Electrically-operated educational appliances
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/40—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment
    • A63F13/42—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle
    • A63F13/428—Processing input control signals of video game devices, e.g. signals generated by the player or derived from the environment by mapping the input signals into game commands, e.g. mapping the displacement of a stylus on a touch screen to the steering angle of a virtual vehicle involving motion or position input signals, e.g. signals representing the rotation of an input controller or a player's arm motions sensed by accelerometers or gyroscopes
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/80—Special adaptations for executing a specific game genre or game mode
    • A63F13/814—Musical performances, e.g. by evaluating the player's ability to follow a notation
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63H—TOYS, e.g. TOPS, DOLLS, HOOPS OR BUILDING BLOCKS
    • A63H33/00—Other toys
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1062—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals being specially adapted to a type of game, e.g. steering wheel
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/10—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals
    • A63F2300/1087—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game characterized by input arrangements for converting player-generated signals into game device control signals comprising photodetecting means, e.g. a camera
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
    • A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
    • A63F2300/8047—Music games
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10024—Color image

Definitions

  • the invention relates to the technical field of computer vision detection and processing, in particular to an educational toy kit and a method for detecting the knocking action thereof.
  • an educational toy kit has been successfully developed in the field of computer vision and image processing technology, including: a bracket, a helmet detector, a bottom plate, a game program installed in a tablet computer, and being placed on a plane through a camera of a tablet computer. The image of the bottom plate.
  • the educational toy kit described above solves the problem of lack of interaction of games in the tablet computer, the hands-on ability is poor, and the game mode is single.
  • a tambourine game has been developed to enhance hands-on ability, and it is possible to identify the area information of the tambourine based on the tambourine color difference and determine whether the tambourine is tapped.
  • the state of the game enhances the fun and interactivity of the game.
  • the present invention provides an educational toy kit and a knocking action detecting method thereof, and the technical solution is as follows:
  • An educational toy kit comprising: a bracket, a helmet detector, a bottom plate and a tambourine, and the bracket is mounted on the bottom plate, the tambourine is placed on the bottom plate, the helmet detector is mounted on the bracket, and the first groove is arranged above the bottom plate; the bottom of the bracket
  • the utility model has a protrusion, the protrusion is installed in the first groove, the top part has a second groove and a third groove, the second groove is used for placing the tablet computer; the helmet detector is installed in the third groove;
  • the helmet detector further includes: a body, a fourth groove, two segments and a convex mirror, and the fourth groove is located in the body for holding different types of tablets, and holding the tablet in the fourth groove
  • the end of the screen is convex
  • the mirror the other end of the convex mirror is mounted on the edge of the helmet detector, the convex mirror is at an acute angle with the horizontal plane, the fourth groove holds the end of the tablet screen higher than the camera position of the tablet, and the two sectors are located on the convex surface. On the 2 edges of the mirror, it is used to hold the convex mirror and hold the tablet.
  • the method further includes: a positioning hole disposed on a longitudinal central axis of the third groove side wall.
  • the colors of the two tambourines are purple and cyan, respectively.
  • a method for detecting a tapping motion in an educational toy kit comprising the following steps:
  • Step 1 Install the game program on the tablet, and then place the bottom plate on the plane. Two tambourines of different colors are respectively placed on the left and right sides of the surface of the bottom plate, and the bottom end of the tablet is installed in the first groove. Mounting the helmet detector to the top of the tablet through the second recess;
  • Step 2 After the fixed installation, the color image of the tambourine is collected in real time through the front camera of the tablet computer, and when the hand is placed on the tambourine, when the tap is not tapped, the front camera collects an image that the tambourine is not tapped; When the hand is placed on the tambourine for tapping, the front camera collects an image of the tambourine in a tapping state;
  • f(x, y) (R xy , G xy , B xy );
  • R xy represents the color value of the image pixel in the red channel
  • G xy represents the color value of the image pixel in the green channel
  • B xy represents the color value of the image pixel in the blue channel
  • Step 3 detecting the color image of the tambourine collected in real time by the front camera in step 2, and obtaining the tapping state of the left and right hand drums on the bottom plate.
  • step three are:
  • the tambourine color image collected by the front camera in step 2 is divided into two images of the left hand drum area and the right hand drum area, and the tambourine contour information is extracted;
  • step 1) is:
  • a) First define the tambourine playable area in the game program installed on the tablet, and then divide the tambourine color image collected by the front camera in step 2 into the left hand drum area and the right hand drum area by image segmentation method.
  • the area image, the left tambourine area and the right tambourine area are all within a predefined tambourine playable area;
  • V max ⁇ C(R'), C(G'), C(B') ⁇ ;
  • H is the tone value
  • S is the saturation value
  • V is the brightness value
  • max ⁇ C(R'), C(G'), C(B') ⁇ means that one pixel is in red and green in the original image.
  • the maximum pixel value of the three channels of blue, min ⁇ C(R'), C(G'), C(B') ⁇ indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue.
  • Value, and the value range of H is between 0-360;
  • the color image is binarized according to the a priori threshold of the purple and cyan color of the tambourine in the HSV space.
  • the specific formula is as follows:
  • B(x, y) B_H(x,y)&B_S(x,y)&B_V(x,y) in a binary image, generating a binary image;
  • B(x, y) represents the binary pixel value of the image pixel point (x, y)
  • H(x, y) S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space
  • B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, and V regions, if yes, the value is 1, otherwise, the value is 0
  • H min and H min respectively indicate the a priori minimum and maximum of the color of the tambourine purple or cyan in the HSV color space.
  • S min and S max respectively represent the a priori minimum and maximum values of the saturation of the tambourine purple or cyan in the HSV color space
  • V min and V max respectively represent the prior minimum sums of the brightness of the tambourine purple or cyan in the HSV color space.
  • the binarized image is a grayscale image
  • the edge of the image refers to the part of the grayscale image where the grayscale changes relatively sharply.
  • the degree of change of the grayscale value is quantitatively represented by the gradient change between adjacent pixels, and the gradient is the first-order two-dimensional
  • the two-dimensional equivalent of the derivative is:
  • G x represents the difference of adjacent pixels in the x direction
  • G y represents the difference of adjacent pixels in the y direction
  • f[i, j+1] represents the pixel value of the image in the i th row and j+1th column.
  • f[i,j] represents the pixel value of the image in the i-th row and the j-th column
  • f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column
  • G(x, y) represents the gradient value at the (x, y) point of the image
  • the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;
  • the width of the rectangle is w
  • the height is h
  • the clockwise rotation angle of the rectangle and the horizontal direction is Q
  • the center point is A
  • the tambourine profile is filtered out, preferably, the aspect ratio threshold range is set to (0.05, 20);
  • step 2) are:
  • the contour area is the number of pixels in the contour, and the center point is the centroid of the contour
  • centroid coordinate of the outline is calculated as:
  • X represents the x coordinate of the center point (x, y)
  • m i represents the mass of the i mass point
  • x i represents the x coordinate of the i mass point
  • m i 1
  • the y coordinate of the center point (x, y) can be obtained.
  • Y represents the y coordinate of the center point (x, y)
  • y i represents the y coordinate of the i mass point
  • the actual area of the tambourine is estimated from the center point (x, y) value of the largest area profile according to the following formula:
  • A represents the actual area of the tambourine
  • a max represents the tampon area when the tambourine is placed in the playable area closest to the tablet
  • a min represents the tampon area when the tambourine is placed in the playable area from the farthest position of the tablet.
  • Y max and Y min respectively represent the y coordinate when the tambourine is placed at the farthest and closest point of the tablet in the playable area, Y min ⁇ y ⁇ Y max ;
  • Abig ⁇ A max ⁇ coff the tambourine is considered to have been tapped, otherwise the tambou is considered not to be tapped.
  • the tambourine indicated by Abig is covered and then processed by binarization graphics, the largest area that is not covered.
  • Area, coff represents an adjustable parameter, the value range is between 0 and 1, according to the experiment, the optimal value is 0.5.
  • the invention intelligently combines the application of computer vision graphic recognition technology with the HSV color space and binarization processing, can determine the tapping state of the tambourine, has a fast calculation speed, accurate positioning, hardware and software technology.
  • the technique is well unified, the game interaction design is ingenious; the appearance is simple, the judgment is faster, and the fun and the intuitiveness are enhanced.
  • the detection algorithm of the invention is more scientific and mature, and the combination of image color space conversion, contour detection and morphological processing algorithms can quickly determine the tapping state of the tambourine.
  • the calculation speed of the invention is fast; each positioning detection takes about 10ms, which provides a smooth experience for the player.
  • the performance of the invention is stable. In the case of different illumination and different tablet computers installed in the educational toy kit, the collection and test of 3,000 pictures are performed, and the false recognition rate and the missed detection rate are below 0.2%.
  • Figure 1 is a schematic view showing the structure of an educational toy kit of the present invention.
  • FIG. 2 is a rear elevational view of the bracket of an educational toy kit of the present invention.
  • FIG 3 is a perspective view of a stand of an educational toy kit of the present invention.
  • FIG. 4 is a schematic structural view of a bottom plate of an educational toy kit of the present invention.
  • Figure 5 is a schematic view showing the structure of a helmet probe of the grape exploration number educational toy kit of the present invention.
  • Figure 6 is a schematic view showing the application of the grape exploration number educational toy kit of the present invention.
  • FIG. 7 is a flow chart of a method for detecting a tapping motion in an educational toy kit of the present invention.
  • Bracket 1 protrusion 101, second groove 102, third groove 103, dish-shaped chassis 104, circular top frame 105, open handle 106, helmet detector 2, body 201, fourth groove 202 2 sectors 203, convex mirror 204, fixed fulcrum 205, bottom plate 3, first groove 301, tambourine 4.
  • a method for detecting a tapping motion in an educational toy kit includes the following steps:
  • Step 1 Install the game program on the tablet, and then place the bottom plate on the plane. Two tambourines of different colors are respectively placed on the left and right sides of the surface of the bottom plate, and the bottom end of the tablet is installed in the first groove. Mounting the helmet detector to the top of the tablet through the second recess;
  • Step 2 After the fixed installation, the color image of the tambourine is collected in real time through the front camera of the tablet computer, and when the hand is placed on the tambourine, when the tap is not tapped, the front camera collects an image that the tambourine is not tapped; When the hand is placed on the tambourine for tapping, the front camera collects an image of the tambourine in a tapping state;
  • f(x, y) (R xy , G xy , B xy );
  • R xy represents the color value of the image pixel in the red channel
  • G xy represents the color value of the image pixel in the green channel
  • B xy represents the color value of the image pixel in the blue channel
  • Step 3 detecting the color image of the tambourine collected in real time by the front camera in step 2, and obtaining the tapping state of the left and right hand drums on the bottom plate, the specific steps are:
  • the tambourine color image collected by the front camera in step 2 is divided into two images of the left hand drum area and the right hand drum area, and the tambourine contour information is extracted.
  • the specific steps are as follows:
  • a) First define the tambourine playable area in the game program installed on the tablet, and then divide the tambourine color image collected by the front camera in step 2 into the left hand drum area and the right hand drum area by image segmentation method.
  • the area image, the left tambourine area and the right tambourine area are all within a predefined tambourine playable area;
  • V max ⁇ C(R'), C(G'), C(B') ⁇ ;
  • H is the tone value
  • S is the saturation value
  • V is the brightness value
  • max ⁇ C(R'), C(G'), C(B') ⁇ means that one pixel is in red and green in the original image.
  • the maximum pixel value of the three channels of blue, min ⁇ C(R'), C(G'), C(B') ⁇ indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue.
  • Value, and the value range of H is between 0-360;
  • the color image is binarized according to the a priori threshold of the purple and cyan color of the tambourine in the HSV space.
  • the specific formula is as follows:
  • B(x, y) B_H(x,y)&B_S(x,y)&B_V(x,y) in a binary image, generating a binary image;
  • B(x, y) represents the binary pixel value of the image pixel point (x, y)
  • H(x, y) S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space
  • B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, and V regions, if yes, the value is 1, otherwise, the value is 0
  • H min and H min respectively indicate the a priori minimum and maximum of the color of the tambourine purple or cyan in the HSV color space.
  • S min and S max respectively represent the a priori minimum and maximum values of the saturation of the tambourine purple or cyan in the HSV color space
  • V min and V max respectively represent the prior minimum sums of the brightness of the tambourine purple or cyan in the HSV color space.
  • the binarized image is a grayscale image
  • the edge of the image refers to the part of the grayscale image where the grayscale changes relatively sharply.
  • the degree of change of the grayscale value is quantitatively represented by the gradient change between adjacent pixels, and the gradient is the first-order two-dimensional
  • the two-dimensional equivalent of the derivative is:
  • G x represents the difference of adjacent pixels in the x direction
  • G y represents the difference of adjacent pixels in the y direction
  • f[i, j+1] represents the pixel value of the image in the i th row and j+1th column.
  • f[i,j] represents the pixel value of the image in the i-th row and the j-th column
  • f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column
  • G(x, y) represents the gradient value at the (x, y) point of the image
  • the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;
  • the width of the rectangle is w
  • the height is h
  • the clockwise rotation angle of the rectangle and the horizontal direction is Q
  • the center point is A
  • the tambourine profile is filtered out, preferably, the aspect ratio threshold range is set to (0.05, 20);
  • the contour area is the number of pixels in the contour, and the center point is the centroid of the contour
  • centroid coordinate of the outline is calculated as:
  • X represents the x coordinate of the center point (x, y)
  • m i represents the mass of the i mass point
  • x i represents the x coordinate of the i mass point
  • m i 1
  • the y coordinate of the center point (x, y) can be obtained.
  • Y represents the y coordinate of the center point (x, y)
  • y i represents the y coordinate of the i mass point
  • the actual area of the tambourine is estimated from the center point (x, y) value of the largest area profile according to the following formula:
  • A represents the actual area of the tambourine
  • a max represents the tampon area when the tambourine is placed in the playable area closest to the tablet
  • a min represents the tampon area when the tambourine is placed in the playable area from the farthest position of the tablet.
  • Y max and Y min respectively represent the y coordinate when the tambourine is placed at the farthest and closest point of the tablet in the playable area, Y min ⁇ y ⁇ Y max ;
  • Abig ⁇ A max ⁇ coff the tambourine is considered to have been tapped, otherwise the tambou is considered not to be tapped.
  • the tambourine indicated by Abig is covered and then processed by binarization graphics, the largest area that is not covered.
  • Area, coff represents an adjustable parameter, the value range is between 0 and 1, according to the experiment, the optimal value is 0.5.
  • Figure 1 is a schematic view showing the structure of an educational toy kit of the present invention.
  • FIG. 2 is a rear elevational view of the bracket of an educational toy kit of the present invention.
  • FIG 3 is a perspective view of a stand of an educational toy kit of the present invention.
  • FIG. 4 is a schematic structural view of a bottom plate of an educational toy kit of the present invention.
  • Figure 5 is a schematic view showing the structure of a helmet probe of the grape exploration number educational toy kit of the present invention.
  • Figure 6 is a schematic view showing the application of the grape exploration number educational toy kit of the present invention.
  • a method for detecting a tapping motion in an educational toy kit includes: a bracket 1, a helmet detector 2, a bottom plate 3, and a tambourine 4, and the bracket 1 is mounted on the bottom plate 3, and the tambourine 4 is placed on the bottom plate.
  • the helmet detector 2 is mounted on the bracket 1; a first recess 301 is disposed above the bottom plate 3; the bracket 1 has a protrusion 101 at the bottom, the protrusion 101 is installed in the first recess 301, and the top has a second recess a groove 102 and a third groove 103, the second groove 102 is used for placing a tablet computer; the helmet detector 2 is mounted in the third groove 103; the helmet detector 2 further includes: a body 201, a fourth groove 202, 2 sectors 203 and convex mirror 204, and a fourth groove 202 is located in the body 201 for clamping different types of tablets, and a convex mirror 204 is disposed at the end of the fourth groove 202 holding the tablet screen.
  • the other end of the convex mirror 204 is mounted on the edge of the helmet detector 2, the convex mirror 204 is at an acute angle with the horizontal plane, and the fourth recess 202 holds the end of the tablet screen higher than the camera position of the tablet, and two sectors 203 is located on the two edges of the convex mirror 204 for fixing the convex surface 204 and gripping the tablet.
  • the positioning hole 104 is disposed on the longitudinal central axis of the sidewall of the third groove 103.
  • the number of tambourines 4 is two; preferably, the colors of the two tambourines 4 are purple and cyan, respectively.
  • FIG. 7 is a flow chart of a method for detecting a tapping motion in an educational toy kit of the present invention.
  • a method for detecting a tapping motion in an educational toy kit includes the following steps:
  • Step 1 Install the game program on the tablet, and then place the bottom plate on the plane. Two tambourines of different colors are respectively placed on the left and right sides of the surface of the bottom plate, and the bottom end of the tablet is installed in the first groove. Mounting the helmet detector to the top of the tablet through the second recess;
  • Step 2 After the fixed installation, the color image of the tambourine is collected in real time through the front camera of the tablet computer, and when the hand is placed on the tambourine, when the tap is not tapped, the front camera collects an image that the tambourine is not tapped; When the hand is placed on the tambourine for tapping, the front camera collects an image of the tambourine in a tapping state;
  • f(x, y) (R xy , G xy , B xy );
  • R xy represents the color value of the image pixel in the red channel
  • G xy represents the color value of the image pixel in the green channel
  • B xy represents the color value of the image pixel in the blue channel
  • Step 3 detecting the color image of the tambourine collected in real time by the front camera in step 2, and obtaining the tapping state of the left and right hand drums on the bottom plate, the specific steps are:
  • the tambourine color image collected by the front camera in step 2 is divided into two images of the left hand drum area and the right hand drum area, and the tambourine contour information is extracted.
  • the specific steps are as follows:
  • a) First define the tambourine playable area in the game program installed on the tablet, and then divide the tambourine color image collected by the front camera in step 2 into the left hand drum area and the right hand drum area by image segmentation method.
  • the area image, the left tambourine area and the right tambourine area are all within a predefined tambourine playable area;
  • V max ⁇ C(R'), C(G'), C(B') ⁇ ;
  • H is the tone value
  • S is the saturation value
  • V is the brightness value
  • max ⁇ C(R'), C(G'), C(B') ⁇ means that one pixel is in red and green in the original image.
  • the maximum pixel value of the three channels of blue, min ⁇ C(R'), C(G'), C(B') ⁇ indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue.
  • Value, and the value range of H is between 0-360;
  • the color image is binarized according to the a priori threshold of the purple and cyan color of the tambourine in the HSV space.
  • the specific formula is as follows:
  • B(x, y) B_H(x,y)&B_S(x,y)&B_V(x,y) in a binary image, generating a binary image;
  • B(x, y) represents the binary pixel value of the image pixel point (x, y)
  • H(x, y) S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space
  • B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, and V regions, if yes, the value is 1, otherwise, the value is 0
  • H min and H min respectively indicate the a priori minimum and maximum of the color of the tambourine purple or cyan in the HSV color space.
  • S min and S max respectively represent the a priori minimum and maximum values of the saturation of the tambourine purple or cyan in the HSV color space
  • V min and V max respectively represent the prior minimum sums of the brightness of the tambourine purple or cyan in the HSV color space.
  • the binarized image is a grayscale image
  • the edge of the image refers to the part of the grayscale image where the grayscale changes relatively sharply.
  • the degree of change of the grayscale value is quantitatively represented by the gradient change between adjacent pixels, and the gradient is the first-order two-dimensional
  • the two-dimensional equivalent of the derivative is:
  • G x represents the difference of the adjacent pixels in the x direction
  • Gy represents the difference of the adjacent pixels in the y direction
  • f[i, j+1] represents the pixel value of the image in the j+1th row of the i th row
  • f[i,j] represents the pixel value of the image in the i-th row and the j-th column
  • f[i+1,j] represents the pixel value of the image in the i-th row and the jth column
  • G(x, y) represents the gradient value at the (x, y) point of the image
  • the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;
  • the width of the rectangle is w
  • the height is h
  • the clockwise rotation angle of the rectangle and the horizontal direction is Q
  • the center point is A
  • the tambourine profile is filtered out, preferably, the aspect ratio threshold range is set to (0.05, 20);
  • the contour area is the number of pixels in the contour, and the center point is the centroid of the contour
  • centroid coordinate of the outline is calculated as:
  • X represents the x coordinate of the center point (x, y)
  • m i represents the mass of the i mass point
  • x i represents the x coordinate of the i mass point
  • m i 1
  • the y coordinate of the center point (x, y) can be obtained.
  • Y represents the y coordinate of the center point (x, y)
  • y i represents the y coordinate of the i mass point
  • the actual area of the tambourine is estimated from the center point (x, y) value of the largest area profile according to the following formula:
  • A represents the actual area of the tambourine
  • a max represents the tampon area when the tambourine is placed in the playable area closest to the tablet
  • a min represents the tampon area when the tambourine is placed in the playable area from the farthest position of the tablet.
  • Y max and Y min respectively represent the y coordinate when the tambourine is placed at the farthest and closest point of the tablet in the playable area, Y min ⁇ y ⁇ Y max ;
  • Abig ⁇ A max ⁇ coff the tambourine is considered to have been tapped, otherwise the tambou is considered not to be tapped.
  • the tambourine indicated by Abig is covered and then processed by binarization graphics, the largest area that is not covered.
  • Area, coff represents an adjustable parameter, the value range is between 0 and 1, according to the experiment, the optimal value is 0.5.
  • the invention intelligently combines the application of computer vision graphic recognition technology with the HSV color space and binarization processing, can judge the tapping state of the tambourine, has the operation speed fast, the positioning is accurate, and the hardware and the software technology are well unified. Up, the game interaction design is clever; the appearance is simple, the judgment is faster, and the fun and intuitiveness is enhanced.
  • the detection algorithm of the invention is more scientific and mature, and combines the algorithms of color space conversion, contour detection and morphological processing of images, and can quickly judge the tapping state of the tambourine.
  • the calculation speed of the invention is fast; each positioning detection takes about 10ms, which provides a smooth experience for the player.
  • the performance of the invention is stable, and in the case of different illumination and different tablet computers installed in the educational toy kit, the collection and test are performed on 3,000 pictures, and the false recognition rate and the missed detection rate are below 0.2%.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Business, Economics & Management (AREA)
  • Educational Administration (AREA)
  • Educational Technology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Toys (AREA)
  • Image Analysis (AREA)

Abstract

A teaching toy kit and a strike action detection method thereof. The teaching toy kit comprises a support (1), a helmet detector (2), a bottom plate (3), and handheld drums (4). The support (1) is mounted on the bottom plate (3). The handheld drums (4) are placed on the bottom plate (3). The helmet detector (2) is mounted on the support (1). A first recess (301) is provided at the upper part of the bottom plate (3). A protrusion (101) is provided at the bottom of the support (1) and mounted in the first recess (101). A second recess (102) and a third recess (103) are provided on the top, and the second recess (102) is used for holding a tablet computer. The helmet detector (2) is mounted in the third recess (103) and further comprises a body (201), a fourth recess (202), two fan-shaped blocks (203), and a convex mirror (204). The fourth recess (202) is positioned in the body (201). The convex mirror (204) is provided at an end point, where the screen of the tablet computer is held, of the fourth recess (202). By developing a handheld drum game, the manipulative ability is strengthened. Area information of a handheld drum can be identified on the basis of a color difference of handheld drums, and thereby determining whether the handheld drum is struck. The present invention makes the game funnier and more interactive.

Description

一种教育玩具套件及其敲击动作检测方法Educational toy kit and method for detecting percussion action 技术领域Technical field
本发明涉及计算机视觉检测处理技术领域，特别涉及一种教育玩具套件及其敲击动作检测方法。The invention relates to the technical field of computer vision detection and processing, in particular to an educational toy kit and a method for detecting the knocking action thereof.
背景技术Background technique
现在平板电脑上有许多有趣的幼教游戏应用程序或者儿童游戏，但往往只是让小朋友在屏幕上指指画画，互动性欠缺，长时间看着屏幕容易对眼睛造成伤害；而当下一些互动性强的传统性游戏玩具已经脱离了时代的发展，形式上无法满足孩子学习、玩耍的需求，也不便于孩子和家长的互动沟通。There are many interesting preschool game apps or children's games on the tablet, but often just let the children point finger on the screen, the interaction is lacking, and watching the screen for a long time is easy to cause damage to the eyes; and now some interactive Traditional game toys have been separated from the development of the times, and the form cannot meet the needs of children to learn and play, and it is not convenient for children and parents to interact.
为了解决上述问题，计算机视觉与图像处理技术领域成功地开发了一种教育玩具套件，包括：支架、头盔探测器，底板，在平板电脑内安装游戏程序，通过平板电脑的摄像头采集放置于平面上的底板的图像。In order to solve the above problems, an educational toy kit has been successfully developed in the field of computer vision and image processing technology, including: a bracket, a helmet detector, a bottom plate, a game program installed in a tablet computer, and being placed on a plane through a camera of a tablet computer. The image of the bottom plate.
技术问题technical problem
上述的教育玩具套件虽然解决了平板电脑中游戏的互动性欠缺的问题，动手能力差，游戏模式单一。Although the educational toy kit described above solves the problem of lack of interaction of games in the tablet computer, the hands-on ability is poor, and the game mode is single.
因此，计算机视觉与图像处理技术领域急需一种教育玩具套件及其数字的识别方法，开发了手鼓游戏，增强动手能力，能够基于手鼓颜色差异，识别手鼓的面积信息，判断出手鼓是否被敲击的状态，增强游戏的趣味性以及互动性。Therefore, there is an urgent need in the field of computer vision and image processing technology for an educational toy kit and its digital recognition method. A tambourine game has been developed to enhance hands-on ability, and it is possible to identify the area information of the tambourine based on the tambourine color difference and determine whether the tambourine is tapped. The state of the game enhances the fun and interactivity of the game.
技术解决方案Technical solution
本发明为了解决上述问题，提供了一种教育玩具套件及其敲击动作检测方法，技术方案如下：In order to solve the above problems, the present invention provides an educational toy kit and a knocking action detecting method thereof, and the technical solution is as follows:
一种教育玩具套件，包括：支架、头盔探测器、底板和手鼓，并且支架安装于底板上，手鼓放置于底板上，头盔探测器安装于支架上，底板上方设置有第一凹槽；支架底部具有凸起，凸起安装在第一凹槽内，顶部具有第二凹槽和第三凹槽，第二凹槽用于放置平板电脑；头盔探测器安装于第三凹槽内；An educational toy kit comprising: a bracket, a helmet detector, a bottom plate and a tambourine, and the bracket is mounted on the bottom plate, the tambourine is placed on the bottom plate, the helmet detector is mounted on the bracket, and the first groove is arranged above the bottom plate; the bottom of the bracket The utility model has a protrusion, the protrusion is installed in the first groove, the top part has a second groove and a third groove, the second groove is used for placing the tablet computer; the helmet detector is installed in the third groove;
头盔探测器，还包括：本体、第四凹槽、2个扇形块和凸面镜，并且第四凹槽位于本体内，用于夹持不同型号的平板电脑，在第四凹槽夹持平板电脑屏幕的端点处设置有凸 面镜，凸面镜的另一端安装于头盔探测器边缘上，凸面镜与水平面夹角成锐角，第四凹槽夹持平板电脑屏幕的端点高于平板电脑的摄像头位置，2个扇形块位于凸面镜的2个边缘上，用于固定凸面镜和夹持平板电脑。The helmet detector further includes: a body, a fourth groove, two segments and a convex mirror, and the fourth groove is located in the body for holding different types of tablets, and holding the tablet in the fourth groove The end of the screen is convex The mirror, the other end of the convex mirror is mounted on the edge of the helmet detector, the convex mirror is at an acute angle with the horizontal plane, the fourth groove holds the end of the tablet screen higher than the camera position of the tablet, and the two sectors are located on the convex surface. On the 2 edges of the mirror, it is used to hold the convex mirror and hold the tablet.
优选的，在上述的一种教育玩具套件中，还包括：定位孔，设置于第三凹槽侧壁的纵向中心轴上。Preferably, in the above educational toy kit, the method further includes: a positioning hole disposed on a longitudinal central axis of the third groove side wall.
优选的，在上述的一种教育玩具套件中，手鼓为2个。Preferably, in the above educational toy kit, there are two tambourines.
优选的，在上述的一种教育玩具套件中，2个手鼓的颜色分别为紫色与青色。Preferably, in the above educational toy kit, the colors of the two tambourines are purple and cyan, respectively.
一种教育玩具套件中敲击动作检测方法，包括如下步骤：A method for detecting a tapping motion in an educational toy kit, comprising the following steps:
步骤一，在平板电脑中安装游戏程序，再将底板放置于平面上，2个不同颜色的手鼓分别放置于底板表面的左、右两侧，平板电脑的底端安装于第一凹槽内，通过第二凹槽将头盔探测器安装于平板电脑的顶端；Step 1: Install the game program on the tablet, and then place the bottom plate on the plane. Two tambourines of different colors are respectively placed on the left and right sides of the surface of the bottom plate, and the bottom end of the tablet is installed in the first groove. Mounting the helmet detector to the top of the tablet through the second recess;
步骤二，固定安装好后，通过平板电脑的前置摄像头实时采集手鼓的彩色图像，当手放置于手鼓上，未敲击时，则前置摄像头采集到一幅手鼓未被敲击的图像；当手放置于手鼓上进行敲击时，则前置摄像头采集到一幅手鼓处于敲击状态下的图像；Step 2: After the fixed installation, the color image of the tambourine is collected in real time through the front camera of the tablet computer, and when the hand is placed on the tambourine, when the tap is not tapped, the front camera collects an image that the tambourine is not tapped; When the hand is placed on the tambourine for tapping, the front camera collects an image of the tambourine in a tapping state;
将平板电脑前置摄像头所获取的彩色图像定义为Ixy，Ixy＝f(x，y)；The color image obtained by the tablet front camera is defined as I xy , I xy = f(x, y);
其中，(x，y)表示图像像素点的位置坐标，f(x，y)表示图像的在(x，y)上的像素值；Where (x, y) represents the position coordinate of the image pixel point, and f(x, y) represents the pixel value of the image on (x, y);
由于摄像头采集的图像为彩色图片，因此f(x，y)＝(Rxy，Gxy，Bxy)；Since the image captured by the camera is a color picture, f(x, y) = (R xy , G xy , B xy );
其中，Rxy表示图像像素点在红色通道的色彩值，Gxy表示图像像素点在绿色通道的色彩值，Bxy表示图像像素点在蓝色通道的色彩值；Where R xy represents the color value of the image pixel in the red channel, G xy represents the color value of the image pixel in the green channel, and B xy represents the color value of the image pixel in the blue channel;
步骤三，对步骤二中前置摄像头实时采集到的手鼓彩色图像进行检测，得出位于底板上左、右手鼓的敲击状态。Step 3: detecting the color image of the tambourine collected in real time by the front camera in step 2, and obtaining the tapping state of the left and right hand drums on the bottom plate.
优选的，在上述的一种教育玩具套件中敲击动作检测方法中，步骤三的具体步骤为：Preferably, in the tapping motion detecting method in the educational toy kit described above, the specific steps of step three are:
1)由于2个手鼓颜色不同，因此将步骤二中前置摄像头实时采集到的手鼓彩色图像分为左手鼓区域和右手鼓区域2个图像，并且提取出手鼓轮廓信息；1) Since the color of the two tambourines is different, the tambourine color image collected by the front camera in step 2 is divided into two images of the left hand drum area and the right hand drum area, and the tambourine contour information is extracted;
2)通过提取出的手鼓轮廓信息，计算轮廓面积和中心点，并将轮廓面积与该中心点的面积阈值进行比较，得出左、右手鼓是否处于敲击状态。2) Calculate the contour area and the center point by extracting the contour information of the tambourine, and compare the contour area with the area threshold of the center point to determine whether the left and right hand drums are in a tapping state.
优选的，在上述的一种教育玩具套件中敲击动作检测方法中，步骤1)的具体步骤为： Preferably, in the tapping motion detecting method in the educational toy kit described above, the specific steps of step 1) are:
a)首先在平板电脑安装的游戏程序中预先定义手鼓可玩区域，然后通过图像分割方法将步骤二中前置摄像头实时采集到的手鼓彩色图像分为左手鼓区域和右手鼓区域2个感兴趣区域图像，左手鼓区域和右手鼓区域都处于预先定义的手鼓可玩区域内；a) First define the tambourine playable area in the game program installed on the tablet, and then divide the tambourine color image collected by the front camera in step 2 into the left hand drum area and the right hand drum area by image segmentation method. The area image, the left tambourine area and the right tambourine area are all within a predefined tambourine playable area;
b)因为手鼓的颜色在RGB颜色空间内不利于分割开来，对光照变化也比较敏感，所以，将提取出来的感兴趣区域图像由RGB颜色空间转换到侧重于色彩表示的HSV颜色空间，具体转换公式为：b) Because the color of the tambourine is not conducive to segmentation in the RGB color space, it is also sensitive to illumination changes. Therefore, the extracted image of the region of interest is converted from the RGB color space to the HSV color space focusing on the color representation. The conversion formula is:
V＝max{C(R′)、C(G′)、C(B′)}；V=max{C(R'), C(G'), C(B')};
Figure PCTCN2016105736-appb-000001
Figure PCTCN2016105736-appb-000001
Figure PCTCN2016105736-appb-000002
Figure PCTCN2016105736-appb-000002
其中，H表示色调值，S表示饱和度值，V表示亮度值，max{C(R′)、C(G′)、C(B′)}表示在原始图像中一个像素点在红、绿、蓝三个通道的像素最大值，min{C(R′)、C(G′)、C(B′)}表示在原始图像中一个像素点在红、绿、蓝三个通道的像素最小值，并且H的取值范围位于0-360之间；Where H is the tone value, S is the saturation value, V is the brightness value, and max{C(R'), C(G'), C(B')} means that one pixel is in red and green in the original image. The maximum pixel value of the three channels of blue, min{C(R'), C(G'), C(B')} indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue. Value, and the value range of H is between 0-360;
c)在HSV颜色空间内，根据手鼓紫色、青色颜色在HSV空间内的先验阈值，将彩色图像进行二值化处理，具体公式如下：c) In the HSV color space, the color image is binarized according to the a priori threshold of the purple and cyan color of the tambourine in the HSV space. The specific formula is as follows:
Figure PCTCN2016105736-appb-000003
Figure PCTCN2016105736-appb-000003
Figure PCTCN2016105736-appb-000004
Figure PCTCN2016105736-appb-000004
Figure PCTCN2016105736-appb-000005
Figure PCTCN2016105736-appb-000005
在二进制图像中B(x，y)＝B_H(x，y)&B_S(x，y)&B_V(x，y)，生成二进制图像；B(x, y)=B_H(x,y)&B_S(x,y)&B_V(x,y) in a binary image, generating a binary image;
其中，B(x，y)表示图像像素点(x，y)的二进制像素值，H(x，y)、S(x，y)、V(x，y)分别表示图像像素点(x，y)在HSV颜色空间内的色调值、饱和度值、亮度 值；B_H(x，y)、B_S(x，y)、B_V(x，y)分别表示图像像素点(x，y)是否分别在指定的H、S、V区域内，如果是，则取值为1，否则，取值为0；Hmin、Hmin分别表示手鼓紫色或者青色在HSV颜色空间内色调的先验最小和最大值；Smin、Smax分别表示手鼓紫色或者青色在HSV颜色空间内饱和度的先验最小和最大值；Vmin、Vmax分别表示手鼓紫色或者青色在HSV颜色空间内亮度的先验最小和最大值；Where B(x, y) represents the binary pixel value of the image pixel point (x, y), and H(x, y), S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space; B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, and V regions, if yes, the value is 1, otherwise, the value is 0; H min and H min respectively indicate the a priori minimum and maximum of the color of the tambourine purple or cyan in the HSV color space. Values; S min and S max respectively represent the a priori minimum and maximum values of the saturation of the tambourine purple or cyan in the HSV color space; V min and V max respectively represent the prior minimum sums of the brightness of the tambourine purple or cyan in the HSV color space. Maximum value
d)扫描二值化图像，找出所有边缘轮廓；d) scanning the binarized image to find all edge contours;
二值化图像为灰度图像，图像的边缘是指灰度图像中灰度变化比较剧烈的部分，灰度值的变化程度采用相邻像素间的梯度变化来定量表示，梯度是一阶二维导数的二维等效式，具体计算过程为：The binarized image is a grayscale image, and the edge of the image refers to the part of the grayscale image where the grayscale changes relatively sharply. The degree of change of the grayscale value is quantitatively represented by the gradient change between adjacent pixels, and the gradient is the first-order two-dimensional The two-dimensional equivalent of the derivative, the specific calculation process is:
首先，计算相邻像素的差分，具体公式为：First, calculate the difference between adjacent pixels. The specific formula is:
Gx＝f[i，j+1]-f[i，j]G x =f[i,j+1]-f[i,j]
Gy＝f[i，j]-f[i+1，j]G y =f[i,j]-f[i+1,j]
其中，Gx表示相邻像素在x方向上的差分，Gy表示相邻像素在y方向上的差分，f[i，j+1]表示图像在第i行第j+1列的像素值，f[i，j]表示图像在第i行第j列的像素值；f[i+1，j]表示图像在第i+1行第j列的像素值，Where G x represents the difference of adjacent pixels in the x direction, G y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column,
进一步地，计算相邻像素间的梯度，具体公式为：Further, the gradient between adjacent pixels is calculated, and the specific formula is:
Figure PCTCN2016105736-appb-000006
Figure PCTCN2016105736-appb-000006
其中，G(x，y)表示表示图像的在(x，y)点上梯度值，
Figure PCTCN2016105736-appb-000007
表示像素值在x方向上求导，
Figure PCTCN2016105736-appb-000008
表示像素值在y方向上求导；Where G(x, y) represents the gradient value at the (x, y) point of the image,
Figure PCTCN2016105736-appb-000007
Indicates that the pixel value is derived in the x direction.
Figure PCTCN2016105736-appb-000008
Indicates that the pixel value is derived in the y direction;
进一步地，计算边缘点的梯度幅值，所有边缘点的梯度幅值集合即为提取的边缘轮廓；Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;
进一步地，利用手鼓形状、边长比例、大小、角度和位置的先验知识过滤掉不合理的轮廓，具体滤除条件如下：Further, the prior knowledge of the tambourine shape, side length ratio, size, angle and position is used to filter out unreasonable contours, and the specific filtering conditions are as follows:
假设矩形的宽为w，高为h，矩形与水平方向的顺时针旋转角为Q，中心点为A，则计算出边缘轮廓的最小外包矩形，再根据如下规则判断是否需要滤除手鼓轮廓；Assuming that the width of the rectangle is w, the height is h, the clockwise rotation angle of the rectangle and the horizontal direction is Q, and the center point is A, the minimum outer envelope of the edge contour is calculated, and then it is judged whether the tambourine contour needs to be filtered according to the following rules;
①如果宽高比超过宽高比阈值范围，则滤除该手鼓轮廓，优选的，宽高比阈值范围设置为(0.05，20)； 1 If the aspect ratio exceeds the aspect ratio threshold range, the tambourine profile is filtered out, preferably, the aspect ratio threshold range is set to (0.05, 20);
②如果中心点A与矩形的4个角和4条边非常接近，则滤除该手鼓轮廓；2 If the center point A is very close to the 4 corners and 4 sides of the rectangle, the tambourine profile is filtered out;
③如果w与h的乘积超过乘积阈值范围，则滤除该手鼓轮廓。3 If the product of w and h exceeds the product threshold range, the tambourine profile is filtered out.
优选的，在上述的一种教育玩具套件中敲击动作检测方法中，步骤2)的具体步骤为：Preferably, in the tapping motion detecting method in the educational toy kit described above, the specific steps of step 2) are:
e)通过步骤d)中提取出的手鼓轮廓信息，计算轮廓面积和中心点；e) calculating the contour area and the center point by using the tambourine contour information extracted in step d);
轮廓面积就是轮廓里面像素点的个数，中心点就是轮廓的质心；The contour area is the number of pixels in the contour, and the center point is the centroid of the contour;
假设一个面积为N的轮廓，则轮廓的质心坐标计算公式为：Assuming an outline of area N, the centroid coordinate of the outline is calculated as:
Figure PCTCN2016105736-appb-000009
Figure PCTCN2016105736-appb-000009
其中，X表示中心点(x，y)的x坐标，mi表示i质点的质量，xi表示i质点的x坐标，如果i质点的质量与中心点的质量相等，则mi＝1；Where X represents the x coordinate of the center point (x, y), m i represents the mass of the i mass point, x i represents the x coordinate of the i mass point, and if the mass of the i mass point is equal to the mass of the center point, m i =1;
上述公式则可写为
Figure PCTCN2016105736-appb-000010
The above formula can be written as
Figure PCTCN2016105736-appb-000010
同理，可得出中心点(x，y)的y坐标
Figure PCTCN2016105736-appb-000011
Similarly, the y coordinate of the center point (x, y) can be obtained.
Figure PCTCN2016105736-appb-000011
Y表示中心点(x，y)的y坐标，yi表示i质点的y坐标，因此中心点(x，y)可以表示为
Figure PCTCN2016105736-appb-000012
Y represents the y coordinate of the center point (x, y), and y i represents the y coordinate of the i mass point, so the center point (x, y) can be expressed as
Figure PCTCN2016105736-appb-000012
由最大面积轮廓的中心点(x，y)值根据以下公式估算出该处手鼓的实际面积：The actual area of the tambourine is estimated from the center point (x, y) value of the largest area profile according to the following formula:
Figure PCTCN2016105736-appb-000013
Figure PCTCN2016105736-appb-000013
其中，A表示手鼓的实际面积，Amax表示当手鼓放置在可玩区域距离平板电脑最近位置时的手鼓面积，Amin表示当手鼓放置在可玩区域距离平板电脑最远位置时的手鼓面积。Ymax和Ymin分别表示当手鼓放置在可玩区域距离平板电脑最远和最近处点的y坐标，Ymin≤y≤Ymax；Where A represents the actual area of the tambourine, A max represents the tampon area when the tambourine is placed in the playable area closest to the tablet, and A min represents the tampon area when the tambourine is placed in the playable area from the farthest position of the tablet. Y max and Y min respectively represent the y coordinate when the tambourine is placed at the farthest and closest point of the tablet in the playable area, Y min ≤ y ≤ Y max ;
如果Abig<Amax×coff，则认为手鼓被敲击了，否则认为手鼓没有被敲击，其中，Abig表示的手鼓被覆盖后，经过二值化图形处理，没有被覆盖掉的最大的区域的面积，coff表示一个可调节的参数，取值范围为0～1之间，根据实验，得出最优值为0.5。If Abig<A max ×coff, the tambourine is considered to have been tapped, otherwise the tambou is considered not to be tapped. The tambourine indicated by Abig is covered and then processed by binarization graphics, the largest area that is not covered. Area, coff represents an adjustable parameter, the value range is between 0 and 1, according to the experiment, the optimal value is 0.5.
有益效果Beneficial effect
1、本发明巧妙的将应用计算机视觉图形识别技术与HSV颜色空间、二值化处理相结合使用，能够判断出手鼓的敲击状态，具有运算速度快，定位准确，将硬件与软件技 术很好地统一起来，游戏交互设计巧妙；美观简单，判断更加快速，同时增强了趣味性和直观性。1. The invention intelligently combines the application of computer vision graphic recognition technology with the HSV color space and binarization processing, can determine the tapping state of the tambourine, has a fast calculation speed, accurate positioning, hardware and software technology. The technique is well unified, the game interaction design is ingenious; the appearance is simple, the judgment is faster, and the fun and the intuitiveness are enhanced.
2、本发明检测算法更加科学、成熟，将图像的色彩空间转换、轮廓检测、形态学处理等算法相结合使用，能够快速的判断出手鼓的敲击状态。2. The detection algorithm of the invention is more scientific and mature, and the combination of image color space conversion, contour detection and morphological processing algorithms can quickly determine the tapping state of the tambourine.
3、本发明计算速度快；每次定位检测耗时在10ms左右，为玩家提供流畅的使用体验。3. The calculation speed of the invention is fast; each positioning detection takes about 10ms, which provides a smooth experience for the player.
4、本发明性能稳定，在不同光照、对不同平板电脑安装于教育玩具套件内的情况下，针对3千幅图片进行了采集测试，误识别率和漏检率在0.2％以下。4. The performance of the invention is stable. In the case of different illumination and different tablet computers installed in the educational toy kit, the collection and test of 3,000 pictures are performed, and the false recognition rate and the missed detection rate are below 0.2%.
附图说明DRAWINGS
下面结合附图和具体实施方式来详细说明本发明：The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.
图1是本发明一种教育玩具套件的结构示意图。BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a schematic view showing the structure of an educational toy kit of the present invention.
图2是本发明一种教育玩具套件的支架的后视图。2 is a rear elevational view of the bracket of an educational toy kit of the present invention.
图3是本发明一种教育玩具套件的支架的立体图。3 is a perspective view of a stand of an educational toy kit of the present invention.
图4是本发明一种教育玩具套件的底板的结构示意图。4 is a schematic structural view of a bottom plate of an educational toy kit of the present invention.
图5是本发明葡萄探索号教育玩具套件的头盔探测器的结构示意图。Figure 5 is a schematic view showing the structure of a helmet probe of the grape exploration number educational toy kit of the present invention.
图6是本发明葡萄探索号教育玩具套件的应用示意图。Figure 6 is a schematic view showing the application of the grape exploration number educational toy kit of the present invention.
图7是本发明一种教育玩具套件中敲击动作检测方法的流程图。7 is a flow chart of a method for detecting a tapping motion in an educational toy kit of the present invention.
其中，图1-7中的附图标记与部件名称之间的对应关系为：The correspondence between the reference numerals in Figure 1-7 and the part names is:
支架1，凸起101，第二凹槽102，第三凹槽103，碟状底架104，圆形顶架105，露空提手106，头盔探测器2，本体201，第四凹槽202，2个扇形块203，凸面镜204，固定支点205，底板3，第一凹槽301，手鼓4。 Bracket 1, protrusion 101, second groove 102, third groove 103, dish-shaped chassis 104, circular top frame 105, open handle 106, helmet detector 2, body 201, fourth groove 202 2 sectors 203, convex mirror 204, fixed fulcrum 205, bottom plate 3, first groove 301, tambourine 4.
本发明的最佳实施方式BEST MODE FOR CARRYING OUT THE INVENTION
如图7所示，一种教育玩具套件中敲击动作检测方法，包括如下步骤：As shown in FIG. 7, a method for detecting a tapping motion in an educational toy kit includes the following steps:
步骤一，在平板电脑中安装游戏程序，再将底板放置于平面上，2个不同颜色的手鼓分别放置于底板表面的左、右两侧，平板电脑的底端安装于第一凹槽内，通过第二凹槽将头盔探测器安装于平板电脑的顶端；Step 1: Install the game program on the tablet, and then place the bottom plate on the plane. Two tambourines of different colors are respectively placed on the left and right sides of the surface of the bottom plate, and the bottom end of the tablet is installed in the first groove. Mounting the helmet detector to the top of the tablet through the second recess;
步骤二，固定安装好后，通过平板电脑的前置摄像头实时采集手鼓的彩色图像，当手放置于手鼓上，未敲击时，则前置摄像头采集到一幅手鼓未被敲击的图像；当手放置于手鼓上进行敲击时，则前置摄像头采集到一幅手鼓处于敲击状态下的图像；Step 2: After the fixed installation, the color image of the tambourine is collected in real time through the front camera of the tablet computer, and when the hand is placed on the tambourine, when the tap is not tapped, the front camera collects an image that the tambourine is not tapped; When the hand is placed on the tambourine for tapping, the front camera collects an image of the tambourine in a tapping state;
将平板电脑前置摄像头所获取的彩色图像定义为Ixy，Ixy＝f(x，y)； The color image obtained by the tablet front camera is defined as I xy , I xy = f(x, y);
其中，(x，y)表示图像像素点的位置坐标，f(x，y)表示图像的在(x，y)上的像素值；Where (x, y) represents the position coordinate of the image pixel point, and f(x, y) represents the pixel value of the image on (x, y);
由于摄像头采集的图像为彩色图片，因此f(x，y)＝(Rxy，Gxy，Bxy)；Since the image captured by the camera is a color picture, f(x, y) = (R xy , G xy , B xy );
其中，Rxy表示图像像素点在红色通道的色彩值，Gxy表示图像像素点在绿色通道的色彩值，Bxy表示图像像素点在蓝色通道的色彩值；Where R xy represents the color value of the image pixel in the red channel, G xy represents the color value of the image pixel in the green channel, and B xy represents the color value of the image pixel in the blue channel;
步骤三，对步骤二中前置摄像头实时采集到的手鼓彩色图像进行检测，得出位于底板上左、右手鼓的敲击状态，具体步骤为：Step 3: detecting the color image of the tambourine collected in real time by the front camera in step 2, and obtaining the tapping state of the left and right hand drums on the bottom plate, the specific steps are:
1)由于2个手鼓颜色不同，因此将步骤二中前置摄像头实时采集到的手鼓彩色图像分为左手鼓区域和右手鼓区域2个图像，并且提取出手鼓轮廓信息，具体步骤为：1) Since the two tambourine colors are different, the tambourine color image collected by the front camera in step 2 is divided into two images of the left hand drum area and the right hand drum area, and the tambourine contour information is extracted. The specific steps are as follows:
a)首先在平板电脑安装的游戏程序中预先定义手鼓可玩区域，然后通过图像分割方法将步骤二中前置摄像头实时采集到的手鼓彩色图像分为左手鼓区域和右手鼓区域2个感兴趣区域图像，左手鼓区域和右手鼓区域都处于预先定义的手鼓可玩区域内；a) First define the tambourine playable area in the game program installed on the tablet, and then divide the tambourine color image collected by the front camera in step 2 into the left hand drum area and the right hand drum area by image segmentation method. The area image, the left tambourine area and the right tambourine area are all within a predefined tambourine playable area;
b)因为手鼓的颜色在RGB颜色空间内不利于分割开来，对光照变化也比较敏感，所以，将提取出来的感兴趣区域图像由RGB颜色空间转换到侧重于色彩表示的HSV颜色空间，具体转换公式为：b) Because the color of the tambourine is not conducive to segmentation in the RGB color space, it is also sensitive to illumination changes. Therefore, the extracted image of the region of interest is converted from the RGB color space to the HSV color space focusing on the color representation. The conversion formula is:
V＝max{C(R′)、C(G′)、C(B′)}；V=max{C(R'), C(G'), C(B')};
Figure PCTCN2016105736-appb-000014
Figure PCTCN2016105736-appb-000014
Figure PCTCN2016105736-appb-000015
Figure PCTCN2016105736-appb-000015
其中，H表示色调值，S表示饱和度值，V表示亮度值，max{C(R′)、C(G′)、C(B′)}表示在原始图像中一个像素点在红、绿、蓝三个通道的像素最大值，min{C(R′)、C(G′)、C(B′)}表示在原始图像中一个像素点在红、绿、蓝三个通道的像素最小值，并且H的取值范围位于0-360之间；Where H is the tone value, S is the saturation value, V is the brightness value, and max{C(R'), C(G'), C(B')} means that one pixel is in red and green in the original image. The maximum pixel value of the three channels of blue, min{C(R'), C(G'), C(B')} indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue. Value, and the value range of H is between 0-360;
c)在HSV颜色空间内，根据手鼓紫色、青色颜色在HSV空间内的先验阈值，将彩色图像进行二值化处理，具体公式如下： c) In the HSV color space, the color image is binarized according to the a priori threshold of the purple and cyan color of the tambourine in the HSV space. The specific formula is as follows:
Figure PCTCN2016105736-appb-000016
Figure PCTCN2016105736-appb-000016
Figure PCTCN2016105736-appb-000017
Figure PCTCN2016105736-appb-000017
Figure PCTCN2016105736-appb-000018
Figure PCTCN2016105736-appb-000018
在二进制图像中B(x，y)＝B_H(x，y)&B_S(x，y)&B_V(x，y)，生成二进制图像；B(x, y)=B_H(x,y)&B_S(x,y)&B_V(x,y) in a binary image, generating a binary image;
其中，B(x，y)表示图像像素点(x，y)的二进制像素值，H(x，y)、S(x，y)、V(x，y)分别表示图像像素点(x，y)在HSV颜色空间内的色调值、饱和度值、亮度值；B_H(x，y)、B_S(x，y)、B_V(x，y)分别表示图像像素点(x，y)是否分别在指定的H、S、V区域内，如果是，则取值为1，否则，取值为0；Hmin、Hmin分别表示手鼓紫色或者青色在HSV颜色空间内色调的先验最小和最大值；Smin、Smax分别表示手鼓紫色或者青色在HSV颜色空间内饱和度的先验最小和最大值；Vmin、Vmax分别表示手鼓紫色或者青色在HSV颜色空间内亮度的先验最小和最大值；Where B(x, y) represents the binary pixel value of the image pixel point (x, y), and H(x, y), S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space; B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, and V regions, if yes, the value is 1, otherwise, the value is 0; H min and H min respectively indicate the a priori minimum and maximum of the color of the tambourine purple or cyan in the HSV color space. Values; S min and S max respectively represent the a priori minimum and maximum values of the saturation of the tambourine purple or cyan in the HSV color space; V min and V max respectively represent the prior minimum sums of the brightness of the tambourine purple or cyan in the HSV color space. Maximum value
d)扫描二值化图像，找出所有边缘轮廓；d) scanning the binarized image to find all edge contours;
二值化图像为灰度图像，图像的边缘是指灰度图像中灰度变化比较剧烈的部分，灰度值的变化程度采用相邻像素间的梯度变化来定量表示，梯度是一阶二维导数的二维等效式，具体计算过程为：The binarized image is a grayscale image, and the edge of the image refers to the part of the grayscale image where the grayscale changes relatively sharply. The degree of change of the grayscale value is quantitatively represented by the gradient change between adjacent pixels, and the gradient is the first-order two-dimensional The two-dimensional equivalent of the derivative, the specific calculation process is:
首先，计算相邻像素的差分，具体公式为：First, calculate the difference between adjacent pixels. The specific formula is:
Gx＝f[i，j+1]-f[i，j]G x =f[i,j+1]-f[i,j]
Gy＝f[i，j]-f[i+1，j]G y =f[i,j]-f[i+1,j]
其中，Gx表示相邻像素在x方向上的差分，Gy表示相邻像素在y方向上的差分，f[i，j+1]表示图像在第i行第j+1列的像素值，f[i，j]表示图像在第i行第j列的像素值；f[i+1，j]表示图像在第i+1行第j列的像素值，Where G x represents the difference of adjacent pixels in the x direction, G y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column,
进一步地，计算相邻像素间的梯度，具体公式为：Further, the gradient between adjacent pixels is calculated, and the specific formula is:
Figure PCTCN2016105736-appb-000019
Figure PCTCN2016105736-appb-000019
其中，G(x，y)表示表示图像的在(x，y)点上梯度值，表示像素值在x方向上求导，
Figure PCTCN2016105736-appb-000021
表示像素值在y方向上求导；Where G(x, y) represents the gradient value at the (x, y) point of the image, Indicates that the pixel value is derived in the x direction.
Figure PCTCN2016105736-appb-000021
Indicates that the pixel value is derived in the y direction;
进一步地，计算边缘点的梯度幅值，所有边缘点的梯度幅值集合即为提取的边缘轮廓；Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;
进一步地，利用手鼓形状、边长比例、大小、角度和位置的先验知识过滤掉不合理的轮廓；Further, the prior knowledge of the shape of the tambourine, the ratio of the length of the side, the size, the angle and the position are used to filter out the unreasonable contour;
假设矩形的宽为w，高为h，矩形与水平方向的顺时针旋转角为Q，中心点为A，则计算出边缘轮廓的最小外包矩形，再根据如下规则判断是否需要滤除手鼓轮廓；Assuming that the width of the rectangle is w, the height is h, the clockwise rotation angle of the rectangle and the horizontal direction is Q, and the center point is A, the minimum outer envelope of the edge contour is calculated, and then it is judged whether the tambourine contour needs to be filtered according to the following rules;
①如果宽高比超过宽高比阈值范围，则滤除该手鼓轮廓，优选的，宽高比阈值范围设置为(0.05，20)；1 If the aspect ratio exceeds the aspect ratio threshold range, the tambourine profile is filtered out, preferably, the aspect ratio threshold range is set to (0.05, 20);
②如果中心点A与矩形的4个角和4条边非常接近，则滤除该手鼓轮廓；2 If the center point A is very close to the 4 corners and 4 sides of the rectangle, the tambourine profile is filtered out;
③如果w与h的乘积超过乘积阈值范围，则滤除该手鼓轮廓；3 if the product of w and h exceeds the product threshold range, the tambourine profile is filtered out;
2)通过提取出的手鼓轮廓信息，计算轮廓面积和中心点，并将轮廓面积与该中心点的面积阈值进行比较，得出左、右手鼓是否处于敲击状态，具体步骤为：2) Calculate the contour area and the center point by extracting the contour information of the tambourine, and compare the contour area with the area threshold of the center point to determine whether the left and right hand drums are in a tapping state. The specific steps are as follows:
e)通过步骤d)中提取出的手鼓轮廓信息，计算轮廓面积和中心点；e) calculating the contour area and the center point by using the tambourine contour information extracted in step d);
轮廓面积就是轮廓里面像素点的个数，中心点就是轮廓的质心；The contour area is the number of pixels in the contour, and the center point is the centroid of the contour;
假设一个面积为N的轮廓，则轮廓的质心坐标计算公式为：Assuming an outline of area N, the centroid coordinate of the outline is calculated as:
Figure PCTCN2016105736-appb-000022
Figure PCTCN2016105736-appb-000022
其中，X表示中心点(x，y)的x坐标，mi表示i质点的质量，xi表示i质点的x坐标，如果i质点的质量与中心点的质量相等，则mi＝1；Where X represents the x coordinate of the center point (x, y), m i represents the mass of the i mass point, x i represents the x coordinate of the i mass point, and if the mass of the i mass point is equal to the mass of the center point, m i =1;
上述公式则可写为
Figure PCTCN2016105736-appb-000023
The above formula can be written as
Figure PCTCN2016105736-appb-000023
同理，可得出中心点(x，y)的y坐标
Figure PCTCN2016105736-appb-000024
Similarly, the y coordinate of the center point (x, y) can be obtained.
Figure PCTCN2016105736-appb-000024
Y表示中心点(x，y)的y坐标，yi表示i质点的y坐标，因此中心点(x，y)可以表示为
Figure PCTCN2016105736-appb-000025
Y represents the y coordinate of the center point (x, y), and y i represents the y coordinate of the i mass point, so the center point (x, y) can be expressed as
Figure PCTCN2016105736-appb-000025
由最大面积轮廓的中心点(x，y)值根据以下公式估算出该处手鼓的实际面积：The actual area of the tambourine is estimated from the center point (x, y) value of the largest area profile according to the following formula:
Figure PCTCN2016105736-appb-000026
Figure PCTCN2016105736-appb-000026
其中，A表示手鼓的实际面积，Amax表示当手鼓放置在可玩区域距离平板电脑最近位置时的手鼓面积，Amin表示当手鼓放置在可玩区域距离平板电脑最远位置时的手鼓面积。Ymax和Ymin分别表示当手鼓放置在可玩区域距离平板电脑最远和最近处点的y坐标，Ymin≤y≤Ymax；Where A represents the actual area of the tambourine, A max represents the tampon area when the tambourine is placed in the playable area closest to the tablet, and A min represents the tampon area when the tambourine is placed in the playable area from the farthest position of the tablet. Y max and Y min respectively represent the y coordinate when the tambourine is placed at the farthest and closest point of the tablet in the playable area, Y min ≤ y ≤ Y max ;
如果Abig<Amax×coff，则认为手鼓被敲击了，否则认为手鼓没有被敲击，其中，Abig表示的手鼓被覆盖后，经过二值化图形处理，没有被覆盖掉的最大的区域的面积，coff表示一个可调节的参数，取值范围为0～1之间，根据实验，得出最优值为0.5。If Abig<A max ×coff, the tambourine is considered to have been tapped, otherwise the tambou is considered not to be tapped. The tambourine indicated by Abig is covered and then processed by binarization graphics, the largest area that is not covered. Area, coff represents an adjustable parameter, the value range is between 0 and 1, according to the experiment, the optimal value is 0.5.
本发明的实施方式Embodiments of the invention
为了使本发明技术实现的措施、创作特征、达成目的与功效易于明白了解，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而不是全部的实施例。基于本发明中的实施例，本领域普通技术人员在没有作出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the invention, and not all of the embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative efforts are within the scope of the present invention.
实施例1：Example 1:
图1是本发明一种教育玩具套件的结构示意图。BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 is a schematic view showing the structure of an educational toy kit of the present invention.
图2是本发明一种教育玩具套件的支架的后视图。2 is a rear elevational view of the bracket of an educational toy kit of the present invention.
图3是本发明一种教育玩具套件的支架的立体图。3 is a perspective view of a stand of an educational toy kit of the present invention.
图4是本发明一种教育玩具套件的底板的结构示意图。4 is a schematic structural view of a bottom plate of an educational toy kit of the present invention.
图5是本发明葡萄探索号教育玩具套件的头盔探测器的结构示意图。Figure 5 is a schematic view showing the structure of a helmet probe of the grape exploration number educational toy kit of the present invention.
图6是本发明葡萄探索号教育玩具套件的应用示意图。Figure 6 is a schematic view showing the application of the grape exploration number educational toy kit of the present invention.
如图1-6所示，一种教育玩具套件中敲击动作检测方法，包括：支架1、头盔探测器2、底板3和手鼓4，并且支架1安装于底板3上，手鼓4放置于底板3上，头盔探测器2安装于支架1上；底板3上方设置有第一凹槽301；支架1，底部具有凸起101，凸起101安装在第一凹槽301内，顶部具有第二凹槽102和第三凹槽103，第二凹槽102用于放置平板电脑；头盔探测器2，安装于第三凹槽103内；头盔探测器2还包括：本体201、第四凹槽202、2个扇形块203和凸面镜204，并且第四凹槽202位于本体201内，用于夹持不同型号的平板电脑，在第四凹槽202夹持平板电脑屏幕的端点处设置有凸面镜204，凸面镜204的另一端安装于头盔探测器2边缘上，凸面镜204与水平面夹角成锐角，第四凹槽202夹持平板电脑屏幕的端点高于平板电脑的摄像头位置，2个扇形块203位于凸面镜204的2个边缘上，用于固定凸面镜204和夹持平板电脑。 As shown in FIG. 1-6, a method for detecting a tapping motion in an educational toy kit includes: a bracket 1, a helmet detector 2, a bottom plate 3, and a tambourine 4, and the bracket 1 is mounted on the bottom plate 3, and the tambourine 4 is placed on the bottom plate. 3, the helmet detector 2 is mounted on the bracket 1; a first recess 301 is disposed above the bottom plate 3; the bracket 1 has a protrusion 101 at the bottom, the protrusion 101 is installed in the first recess 301, and the top has a second recess a groove 102 and a third groove 103, the second groove 102 is used for placing a tablet computer; the helmet detector 2 is mounted in the third groove 103; the helmet detector 2 further includes: a body 201, a fourth groove 202, 2 sectors 203 and convex mirror 204, and a fourth groove 202 is located in the body 201 for clamping different types of tablets, and a convex mirror 204 is disposed at the end of the fourth groove 202 holding the tablet screen. The other end of the convex mirror 204 is mounted on the edge of the helmet detector 2, the convex mirror 204 is at an acute angle with the horizontal plane, and the fourth recess 202 holds the end of the tablet screen higher than the camera position of the tablet, and two sectors 203 is located on the two edges of the convex mirror 204 for fixing the convex surface 204 and gripping the tablet.
本实施例中，还包括：定位孔104，设置于第三凹槽103侧壁的纵向中心轴上。In this embodiment, the positioning hole 104 is disposed on the longitudinal central axis of the sidewall of the third groove 103.
本实施例中，手鼓4为2个；优选的，2个手鼓4的颜色分别为紫色与青色。In this embodiment, the number of tambourines 4 is two; preferably, the colors of the two tambourines 4 are purple and cyan, respectively.
图7是本发明一种教育玩具套件中敲击动作检测方法的流程图。7 is a flow chart of a method for detecting a tapping motion in an educational toy kit of the present invention.
如图7所示，一种教育玩具套件中敲击动作检测方法，包括如下步骤：As shown in FIG. 7, a method for detecting a tapping motion in an educational toy kit includes the following steps:
步骤一，在平板电脑中安装游戏程序，再将底板放置于平面上，2个不同颜色的手鼓分别放置于底板表面的左、右两侧，平板电脑的底端安装于第一凹槽内，通过第二凹槽将头盔探测器安装于平板电脑的顶端；Step 1: Install the game program on the tablet, and then place the bottom plate on the plane. Two tambourines of different colors are respectively placed on the left and right sides of the surface of the bottom plate, and the bottom end of the tablet is installed in the first groove. Mounting the helmet detector to the top of the tablet through the second recess;
步骤二，固定安装好后，通过平板电脑的前置摄像头实时采集手鼓的彩色图像，当手放置于手鼓上，未敲击时，则前置摄像头采集到一幅手鼓未被敲击的图像；当手放置于手鼓上进行敲击时，则前置摄像头采集到一幅手鼓处于敲击状态下的图像；Step 2: After the fixed installation, the color image of the tambourine is collected in real time through the front camera of the tablet computer, and when the hand is placed on the tambourine, when the tap is not tapped, the front camera collects an image that the tambourine is not tapped; When the hand is placed on the tambourine for tapping, the front camera collects an image of the tambourine in a tapping state;
将平板电脑前置摄像头所获取的彩色图像定义为Ixy，Ixy＝f(x，y)；The color image obtained by the tablet front camera is defined as I xy , I xy = f(x, y);
其中，(x，y)表示图像像素点的位置坐标，f(x，y)表示图像的在(x，y)上的像素值；Where (x, y) represents the position coordinate of the image pixel point, and f(x, y) represents the pixel value of the image on (x, y);
由于摄像头采集的图像为彩色图片，因此f(x，y)＝(Rxy，Gxy，Bxy)；Since the image captured by the camera is a color picture, f(x, y) = (R xy , G xy , B xy );
其中，Rxy表示图像像素点在红色通道的色彩值，Gxy表示图像像素点在绿色通道的色彩值，Bxy表示图像像素点在蓝色通道的色彩值；Where R xy represents the color value of the image pixel in the red channel, G xy represents the color value of the image pixel in the green channel, and B xy represents the color value of the image pixel in the blue channel;
步骤三，对步骤二中前置摄像头实时采集到的手鼓彩色图像进行检测，得出位于底板上左、右手鼓的敲击状态，具体步骤为：Step 3: detecting the color image of the tambourine collected in real time by the front camera in step 2, and obtaining the tapping state of the left and right hand drums on the bottom plate, the specific steps are:
1)由于2个手鼓颜色不同，因此将步骤二中前置摄像头实时采集到的手鼓彩色图像分为左手鼓区域和右手鼓区域2个图像，并且提取出手鼓轮廓信息，具体步骤为：1) Since the two tambourine colors are different, the tambourine color image collected by the front camera in step 2 is divided into two images of the left hand drum area and the right hand drum area, and the tambourine contour information is extracted. The specific steps are as follows:
a)首先在平板电脑安装的游戏程序中预先定义手鼓可玩区域，然后通过图像分割方法将步骤二中前置摄像头实时采集到的手鼓彩色图像分为左手鼓区域和右手鼓区域2个感兴趣区域图像，左手鼓区域和右手鼓区域都处于预先定义的手鼓可玩区域内；a) First define the tambourine playable area in the game program installed on the tablet, and then divide the tambourine color image collected by the front camera in step 2 into the left hand drum area and the right hand drum area by image segmentation method. The area image, the left tambourine area and the right tambourine area are all within a predefined tambourine playable area;
b)因为手鼓的颜色在RGB颜色空间内不利于分割开来，对光照变化也比较敏感，所以，将提取出来的感兴趣区域图像由RGB颜色空间转换到侧重于色彩表示的HSV颜色空间，具体转换公式为：b) Because the color of the tambourine is not conducive to segmentation in the RGB color space, it is also sensitive to illumination changes. Therefore, the extracted image of the region of interest is converted from the RGB color space to the HSV color space focusing on the color representation. The conversion formula is:
V＝max{C(R′)、C(G′)、C(B′)}；V=max{C(R'), C(G'), C(B')};
Figure PCTCN2016105736-appb-000027
Figure PCTCN2016105736-appb-000027
Figure PCTCN2016105736-appb-000028
Figure PCTCN2016105736-appb-000028
其中，H表示色调值，S表示饱和度值，V表示亮度值，max{C(R′)、C(G′)、C(B′)}表示在原始图像中一个像素点在红、绿、蓝三个通道的像素最大值，min{C(R′)、C(G′)、C(B′)}表示在原始图像中一个像素点在红、绿、蓝三个通道的像素最小值，并且H的取值范围位于0-360之间；Where H is the tone value, S is the saturation value, V is the brightness value, and max{C(R'), C(G'), C(B')} means that one pixel is in red and green in the original image. The maximum pixel value of the three channels of blue, min{C(R'), C(G'), C(B')} indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue. Value, and the value range of H is between 0-360;
c)在HSV颜色空间内，根据手鼓紫色、青色颜色在HSV空间内的先验阈值，将彩色图像进行二值化处理，具体公式如下：c) In the HSV color space, the color image is binarized according to the a priori threshold of the purple and cyan color of the tambourine in the HSV space. The specific formula is as follows:
Figure PCTCN2016105736-appb-000029
Figure PCTCN2016105736-appb-000029
Figure PCTCN2016105736-appb-000030
Figure PCTCN2016105736-appb-000030
Figure PCTCN2016105736-appb-000031
Figure PCTCN2016105736-appb-000031
在二进制图像中B(x，y)＝B_H(x，y)&B_S(x，y)&B_V(x，y)，生成二进制图像；B(x, y)=B_H(x,y)&B_S(x,y)&B_V(x,y) in a binary image, generating a binary image;
其中，B(x，y)表示图像像素点(x，y)的二进制像素值，H(x，y)、S(x，y)、V(x，y)分别表示图像像素点(x，y)在HSV颜色空间内的色调值、饱和度值、亮度值；B_H(x，y)、B_S(x，y)、B_V(x，y)分别表示图像像素点(x，y)是否分别在指定的H、S、V区域内，如果是，则取值为1，否则，取值为0；Hmin、Hmin分别表示手鼓紫色或者青色在HSV颜色空间内色调的先验最小和最大值；Smin、Smax分别表示手鼓紫色或者青色在HSV颜色空间内饱和度的先验最小和最大值；Vmin、Vmax分别表示手鼓紫色或者青色在HSV颜色空间内亮度的先验最小和最大值；Where B(x, y) represents the binary pixel value of the image pixel point (x, y), and H(x, y), S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space; B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, and V regions, if yes, the value is 1, otherwise, the value is 0; H min and H min respectively indicate the a priori minimum and maximum of the color of the tambourine purple or cyan in the HSV color space. Values; S min and S max respectively represent the a priori minimum and maximum values of the saturation of the tambourine purple or cyan in the HSV color space; V min and V max respectively represent the prior minimum sums of the brightness of the tambourine purple or cyan in the HSV color space. Maximum value
d)扫描二值化图像，找出所有边缘轮廓；d) scanning the binarized image to find all edge contours;
二值化图像为灰度图像，图像的边缘是指灰度图像中灰度变化比较剧烈的部分，灰度值的变化程度采用相邻像素间的梯度变化来定量表示，梯度是一阶二维导数的二维等效式，具体计算过程为： The binarized image is a grayscale image, and the edge of the image refers to the part of the grayscale image where the grayscale changes relatively sharply. The degree of change of the grayscale value is quantitatively represented by the gradient change between adjacent pixels, and the gradient is the first-order two-dimensional The two-dimensional equivalent of the derivative, the specific calculation process is:
首先，计算相邻像素的差分，具体公式为：First, calculate the difference between adjacent pixels. The specific formula is:
Gx＝f[i，j+1]-f[i，j]G x =f[i,j+1]-f[i,j]
Gy＝f[i，j]-f[i+1，j]G y =f[i,j]-f[i+1,j]
其中，Gx表示相邻像素在x方向上的差分，Gy表示相邻像素在y方向上的差分，f[i，j+1]表示图像在第i行第j+1列的像素值，f[i，j]表示图像在第i行第j列的像素值；f[i+1，j]表示图像在第i+1行第j列的像素值，Where G x represents the difference of the adjacent pixels in the x direction, Gy represents the difference of the adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the j+1th row of the i th row, f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the jth column,
进一步地，计算相邻像素间的梯度，具体公式为：Further, the gradient between adjacent pixels is calculated, and the specific formula is:
Figure PCTCN2016105736-appb-000032
Figure PCTCN2016105736-appb-000032
其中，G(x，y)表示表示图像的在(x，y)点上梯度值，
Figure PCTCN2016105736-appb-000033
表示像素值在x方向上求导，
Figure PCTCN2016105736-appb-000034
表示像素值在y方向上求导；Where G(x, y) represents the gradient value at the (x, y) point of the image,
Figure PCTCN2016105736-appb-000033
Indicates that the pixel value is derived in the x direction.
Figure PCTCN2016105736-appb-000034
Indicates that the pixel value is derived in the y direction;
进一步地，计算边缘点的梯度幅值，所有边缘点的梯度幅值集合即为提取的边缘轮廓；Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;
进一步地，利用手鼓形状、边长比例、大小、角度和位置的先验知识过滤掉不合理的轮廓；Further, the prior knowledge of the shape of the tambourine, the ratio of the length of the side, the size, the angle and the position are used to filter out the unreasonable contour;
假设矩形的宽为w，高为h，矩形与水平方向的顺时针旋转角为Q，中心点为A，则计算出边缘轮廓的最小外包矩形，再根据如下规则判断是否需要滤除手鼓轮廓；Assuming that the width of the rectangle is w, the height is h, the clockwise rotation angle of the rectangle and the horizontal direction is Q, and the center point is A, the minimum outer envelope of the edge contour is calculated, and then it is judged whether the tambourine contour needs to be filtered according to the following rules;
①如果宽高比超过宽高比阈值范围，则滤除该手鼓轮廓，优选的，宽高比阈值范围设置为(0.05，20)；1 If the aspect ratio exceeds the aspect ratio threshold range, the tambourine profile is filtered out, preferably, the aspect ratio threshold range is set to (0.05, 20);
②如果中心点A与矩形的4个角和4条边非常接近，则滤除该手鼓轮廓；2 If the center point A is very close to the 4 corners and 4 sides of the rectangle, the tambourine profile is filtered out;
③如果w与h的乘积超过乘积阈值范围，则滤除该手鼓轮廓；3 if the product of w and h exceeds the product threshold range, the tambourine profile is filtered out;
2)通过提取出的手鼓轮廓信息，计算轮廓面积和中心点，并将轮廓面积与该中心点的面积阈值进行比较，得出左、右手鼓是否处于敲击状态，具体步骤为：2) Calculate the contour area and the center point by extracting the contour information of the tambourine, and compare the contour area with the area threshold of the center point to determine whether the left and right hand drums are in a tapping state. The specific steps are as follows:
e)通过步骤d)中提取出的手鼓轮廓信息，计算轮廓面积和中心点；e) calculating the contour area and the center point by using the tambourine contour information extracted in step d);
轮廓面积就是轮廓里面像素点的个数，中心点就是轮廓的质心；The contour area is the number of pixels in the contour, and the center point is the centroid of the contour;
假设一个面积为N的轮廓，则轮廓的质心坐标计算公式为：Assuming an outline of area N, the centroid coordinate of the outline is calculated as:
Figure PCTCN2016105736-appb-000035
Figure PCTCN2016105736-appb-000035
其中，X表示中心点(x，y)的x坐标，mi表示i质点的质量，xi表示i质点的x坐标，如果i质点的质量与中心点的质量相等，则mi＝1；Where X represents the x coordinate of the center point (x, y), m i represents the mass of the i mass point, x i represents the x coordinate of the i mass point, and if the mass of the i mass point is equal to the mass of the center point, m i =1;
上述公式则可写为
Figure PCTCN2016105736-appb-000036
The above formula can be written as
Figure PCTCN2016105736-appb-000036
同理，可得出中心点(x，y)的y坐标
Figure PCTCN2016105736-appb-000037
Similarly, the y coordinate of the center point (x, y) can be obtained.
Figure PCTCN2016105736-appb-000037
Y表示中心点(x，y)的y坐标，yi表示i质点的y坐标，因此中心点(x，y)可以表示为
Figure PCTCN2016105736-appb-000038
Y represents the y coordinate of the center point (x, y), and y i represents the y coordinate of the i mass point, so the center point (x, y) can be expressed as
Figure PCTCN2016105736-appb-000038
由最大面积轮廓的中心点(x，y)值根据以下公式估算出该处手鼓的实际面积：The actual area of the tambourine is estimated from the center point (x, y) value of the largest area profile according to the following formula:
Figure PCTCN2016105736-appb-000039
Figure PCTCN2016105736-appb-000039
其中，A表示手鼓的实际面积，Amax表示当手鼓放置在可玩区域距离平板电脑最近位置时的手鼓面积，Amin表示当手鼓放置在可玩区域距离平板电脑最远位置时的手鼓面积。Ymax和Ymin分别表示当手鼓放置在可玩区域距离平板电脑最远和最近处点的y坐标，Ymin≤y≤Ymax；Where A represents the actual area of the tambourine, A max represents the tampon area when the tambourine is placed in the playable area closest to the tablet, and A min represents the tampon area when the tambourine is placed in the playable area from the farthest position of the tablet. Y max and Y min respectively represent the y coordinate when the tambourine is placed at the farthest and closest point of the tablet in the playable area, Y min ≤ y ≤ Y max ;
如果Abig<Amax×coff，则认为手鼓被敲击了，否则认为手鼓没有被敲击，其中，Abig表示的手鼓被覆盖后，经过二值化图形处理，没有被覆盖掉的最大的区域的面积，coff表示一个可调节的参数，取值范围为0～1之间，根据实验，得出最优值为0.5。If Abig<A max ×coff, the tambourine is considered to have been tapped, otherwise the tambou is considered not to be tapped. The tambourine indicated by Abig is covered and then processed by binarization graphics, the largest area that is not covered. Area, coff represents an adjustable parameter, the value range is between 0 and 1, according to the experiment, the optimal value is 0.5.
本发明巧妙的将应用计算机视觉图形识别技术与HSV颜色空间、二值化处理相结合使用，能够判断出手鼓的敲击状态，具有运算速度快，定位准确，将硬件与软件技术很好地统一起来，游戏交互设计巧妙；美观简单，判断更加快速，同时增强了趣味性和直观性。The invention intelligently combines the application of computer vision graphic recognition technology with the HSV color space and binarization processing, can judge the tapping state of the tambourine, has the operation speed fast, the positioning is accurate, and the hardware and the software technology are well unified. Up, the game interaction design is clever; the appearance is simple, the judgment is faster, and the fun and intuitiveness is enhanced.
本发明检测算法更加科学、成熟，将图像的色彩空间转换、轮廓检测、形态学处理等算法相结合使用，能够快速的判断出手鼓的敲击状态。The detection algorithm of the invention is more scientific and mature, and combines the algorithms of color space conversion, contour detection and morphological processing of images, and can quickly judge the tapping state of the tambourine.
本发明计算速度快；每次定位检测耗时在10ms左右，为玩家提供流畅的使用体验。The calculation speed of the invention is fast; each positioning detection takes about 10ms, which provides a smooth experience for the player.
本发明性能稳定，在不同光照、对不同平板电脑安装于教育玩具套件内的情况下，针对3千幅图片进行了采集测试，误识别率和漏检率在0.2％以下。The performance of the invention is stable, and in the case of different illumination and different tablet computers installed in the educational toy kit, the collection and test are performed on 3,000 pictures, and the false recognition rate and the missed detection rate are below 0.2%.
以上显示和描述了本发明的基本原理、主要特征和本发明的优点。本行业的技术人员应该了解，本发明不受上述实施例的限制，上述实施例和说明书中描述的只是说明本发明的原理，在不脱离本发明精神和范围的前提下本发明还会有各种变化和改进，这些 变化和改进都落入要求保护的本发明范围内。本发明要求保护范围由所附的权利要求书及其等同物界定。The basic principles, main features, and advantages of the present invention are shown and described above. It should be understood by those skilled in the art that the present invention is not limited by the foregoing embodiments, and that the present invention is described in the foregoing description and the description of the present invention. Changes and improvements, these Variations and modifications are intended to fall within the scope of the invention as claimed. The scope of the invention is defined by the appended claims and their equivalents.
工业实用性Industrial applicability
所属领域技术人员根据上文的记载容易得知，本发明技术方案适合在工业中制造并在生产、生活中使用，因此本发明具备工业实用性。 It is easily known to those skilled in the art from the above description that the technical solution of the present invention is suitable for industrial production and is used in production and life, and therefore the present invention has industrial applicability.

Claims (8)

  1. 一种教育玩具套件，其特征在于，包括：支架、头盔探测器、底板和手鼓，并且所述支架安装于底板上，所述手鼓放置于底板上，所述头盔探测器安装于支架上，所述底板上方设置有第一凹槽；所述支架底部具有凸起，所述凸起安装在第一凹槽内，顶部具有第二凹槽和第三凹槽，所述第二凹槽用于放置平板电脑；所述头盔探测器安装于第三凹槽内；An educational toy kit, comprising: a bracket, a helmet detector, a bottom plate and a tambourine, and the bracket is mounted on the bottom plate, the tambourine is placed on the bottom plate, and the helmet detector is mounted on the bracket, a first groove is disposed above the bottom plate; the bottom of the bracket has a protrusion, the protrusion is installed in the first groove, and the top has a second groove and a third groove, and the second groove is used for Positioning a tablet; the helmet detector is mounted in the third recess;
    所述头盔探测器，还包括：本体、第四凹槽、2个扇形块和凸面镜，并且所述第四凹槽位于本体内，用于夹持不同型号的平板电脑，在所述第四凹槽夹持平板电脑屏幕的端点处设置有凸面镜，所述凸面镜的另一端安装于头盔探测器边缘上，所述凸面镜与水平面夹角成锐角，所述第四凹槽夹持平板电脑屏幕的端点高于平板电脑的摄像头位置，2个所述扇形块位于凸面镜的2个边缘上，用于固定所述凸面镜和夹持平板电脑。The helmet detector further includes: a body, a fourth groove, two segments and a convex mirror, and the fourth groove is located in the body for clamping different types of tablets, in the fourth A convex mirror is disposed at an end of the groove holding tablet screen, and the other end of the convex mirror is mounted on an edge of the helmet detector, the convex mirror is at an acute angle with the horizontal plane, and the fourth groove holds the flat plate The end of the computer screen is higher than the camera position of the tablet, and the two sectors are located on the two edges of the convex mirror for fixing the convex mirror and holding the tablet.
  2. 根据权利要求1所述的一种教育玩具套件，其特征在于，还包括：定位孔，设置于所述第三凹槽侧壁的纵向中心轴上。An educational toy kit according to claim 1, further comprising: a positioning hole provided on a longitudinal central axis of the third groove side wall.
  3. 根据权利要求1所述的一种教育玩具套件，其特征在于，所述手鼓为2个。The educational toy kit according to claim 1, wherein the tambourine has two.
  4. 根据权利要求3所述的一种教育玩具套件，其特征在于，2个所述手鼓的颜色分别为紫色与青色。An educational toy kit according to claim 3, wherein the color of the two tambourines is purple and cyan, respectively.
  5. 一种教育玩具套件中敲击动作检测方法，其特征在于，包括如下步骤：A method for detecting a tapping motion in an educational toy kit, comprising the steps of:
    步骤一，在平板电脑中安装游戏程序，再将底板放置于平面上，2个不同颜色的手鼓分别放置于底板表面的左、右两侧，平板电脑的底端安装于第一凹槽内，通过第二凹槽将头盔探测器安装于平板电脑的顶端；Step 1: Install the game program on the tablet, and then place the bottom plate on the plane. Two tambourines of different colors are respectively placed on the left and right sides of the surface of the bottom plate, and the bottom end of the tablet is installed in the first groove. Mounting the helmet detector to the top of the tablet through the second recess;
    步骤二，固定安装好后，通过平板电脑的前置摄像头实时采集手鼓的彩色图像，当手放置于手鼓上，未敲击时，则前置摄像头采集到一幅手鼓未被敲击的图像；当手放置于手鼓上进行敲击时，则前置摄像头采集到一幅手鼓处于敲击状态下的图像；Step 2: After the fixed installation, the color image of the tambourine is collected in real time through the front camera of the tablet computer, and when the hand is placed on the tambourine, when the tap is not tapped, the front camera collects an image that the tambourine is not tapped; When the hand is placed on the tambourine for tapping, the front camera collects an image of the tambourine in a tapping state;
    将平板电脑前置摄像头所获取的彩色图像定义为Ixy，Ixy＝f(x，y)；The color image obtained by the tablet front camera is defined as I xy , I xy = f(x, y);
    其中，(x，y)表示图像像素点的位置坐标，f(x，y)表示图像的在(x，y)上的像素值；Where (x, y) represents the position coordinate of the image pixel point, and f(x, y) represents the pixel value of the image on (x, y);
    由于摄像头采集的图像为彩色图片，因此f(x，y)＝(Rxy，Gxy，Bxy)；Since the image captured by the camera is a color picture, f(x, y) = (R xy , G xy , B xy );
    其中，Rxy表示图像像素点在红色通道的色彩值，Gxy表示图像像素点在绿色通道的色彩值，Bxy表示图像像素点在蓝色通道的色彩值； Where R xy represents the color value of the image pixel in the red channel, G xy represents the color value of the image pixel in the green channel, and B xy represents the color value of the image pixel in the blue channel;
    步骤三，对步骤二中前置摄像头实时采集到的手鼓彩色图像进行检测，得出位于底板上左、右手鼓的敲击状态。Step 3: detecting the color image of the tambourine collected in real time by the front camera in step 2, and obtaining the tapping state of the left and right hand drums on the bottom plate.
  6. 根据权利要求5所述的一种教育玩具套件中敲击动作检测方法，其特征在于，所述步骤三的具体步骤为：The method for detecting a tapping motion in an educational toy kit according to claim 5, wherein the specific steps of the third step are:
    1)由于2个手鼓颜色不同，因此将步骤二中前置摄像头实时采集到的手鼓彩色图像分为左手鼓区域和右手鼓区域2个图像，并且提取出手鼓轮廓信息；1) Since the color of the two tambourines is different, the tambourine color image collected by the front camera in step 2 is divided into two images of the left hand drum area and the right hand drum area, and the tambourine contour information is extracted;
    2)通过提取出的手鼓轮廓信息，计算轮廓面积和中心点，并将轮廓面积与该中心点的面积阈值进行比较，得出左、右手鼓是否处于敲击状态。2) Calculate the contour area and the center point by extracting the contour information of the tambourine, and compare the contour area with the area threshold of the center point to determine whether the left and right hand drums are in a tapping state.
  7. 根据权利要求6所述的一种教育玩具套件中敲击动作检测方法，其特征在于，所述步骤1)的具体步骤为：The method for detecting a tapping motion in an educational toy kit according to claim 6, wherein the specific steps of the step 1) are:
    a)首先在平板电脑安装的游戏程序中预先定义手鼓可玩区域，然后通过图像分割方法将步骤二中前置摄像头实时采集到的手鼓彩色图像分为左手鼓区域和右手鼓区域2个感兴趣区域图像，左手鼓区域和右手鼓区域都处于预先定义的手鼓可玩区域内；a) First define the tambourine playable area in the game program installed on the tablet, and then divide the tambourine color image collected by the front camera in step 2 into the left hand drum area and the right hand drum area by image segmentation method. The area image, the left tambourine area and the right tambourine area are all within a predefined tambourine playable area;
    b)因为手鼓的颜色在RGB颜色空间内不利于分割开来，对光照变化也比较敏感，所以，将提取出来的感兴趣区域图像由RGB颜色空间转换到侧重于色彩表示的HSV颜色空间，具体转换公式为：b) Because the color of the tambourine is not conducive to segmentation in the RGB color space, it is also sensitive to illumination changes. Therefore, the extracted image of the region of interest is converted from the RGB color space to the HSV color space focusing on the color representation. The conversion formula is:
    V＝max{C(R′)、C(G′)、C(B′)}；V=max{C(R'), C(G'), C(B')};
    Figure PCTCN2016105736-appb-100001
    Figure PCTCN2016105736-appb-100001
    Figure PCTCN2016105736-appb-100002
    Figure PCTCN2016105736-appb-100002
    其中，H表示色调值，S表示饱和度值，V表示亮度值，max{C(R′)、C(G′)、C(B′)}表示在原始图像中一个像素点在红、绿、蓝三个通道的像素最大值，min{C(R′)、C(G′)、C(B′)}表示在原始图像中一个像素点在红、绿、蓝三个通道的像素最小值，并且H的取值范围位于0-360(0～360度)之间；Where H is the tone value, S is the saturation value, V is the brightness value, and max{C(R'), C(G'), C(B')} means that one pixel is in red and green in the original image. The maximum pixel value of the three channels of blue, min{C(R'), C(G'), C(B')} indicates that the pixel of one pixel in the original image is the smallest in the three channels of red, green and blue. Value, and the value range of H is between 0-360 (0-360 degrees);
    c)在HSV颜色空间内，根据手鼓紫色、青色颜色在HSV空间内的先验阈值，将彩色图像进行二值化处理，具体公式如下： c) In the HSV color space, the color image is binarized according to the a priori threshold of the purple and cyan color of the tambourine in the HSV space. The specific formula is as follows:
    Figure PCTCN2016105736-appb-100003
    Figure PCTCN2016105736-appb-100003
    Figure PCTCN2016105736-appb-100004
    Figure PCTCN2016105736-appb-100004
    Figure PCTCN2016105736-appb-100005
    Figure PCTCN2016105736-appb-100005
    在二进制图像中B(x，y)＝B_H(x，y)&B_S(x，y)&B_V(x，y)，生成二进制图像；B(x, y)=B_H(x,y)&B_S(x,y)&B_V(x,y) in a binary image, generating a binary image;
    其中，B(x，y)表示图像像素点(x，y)的二进制像素值，H(x，y)、S(x，y)、V(x，y)分别表示图像像素点(x，y)在HSV颜色空间内的色调值、饱和度值、亮度值；B_H(x，y)、B_S(x，y)、B_V(x，y)分别表示图像像素点(x，y)是否分别在指定的H、S、V区域内，如果是，则取值为1，否则，取值为0；Hmin、Hmin分别表示手鼓紫色或者青色在HSV颜色空间内色调的先验最小和最大值；Smin、Smax分别表示手鼓紫色或者青色在HSV颜色空间内饱和度的先验最小和最大值；Vmin、Vmax分别表示手鼓紫色或者青色在HSV颜色空间内亮度的先验最小和最大值；Where B(x, y) represents the binary pixel value of the image pixel point (x, y), and H(x, y), S(x, y), V(x, y) respectively represent the image pixel point (x, y) the hue value, saturation value, and brightness value in the HSV color space; B_H(x, y), B_S(x, y), B_V(x, y) respectively indicate whether the image pixel points (x, y) are respectively In the specified H, S, and V regions, if yes, the value is 1, otherwise, the value is 0; H min and H min respectively indicate the a priori minimum and maximum of the color of the tambourine purple or cyan in the HSV color space. Values; S min and S max respectively represent the a priori minimum and maximum values of the saturation of the tambourine purple or cyan in the HSV color space; V min and V max respectively represent the prior minimum sums of the brightness of the tambourine purple or cyan in the HSV color space. Maximum value
    d)扫描二值化图像，找出所有边缘轮廓；d) scanning the binarized image to find all edge contours;
    二值化图像为灰度图像，图像的边缘是指灰度图像中灰度变化比较剧烈的部分，灰度值的变化程度采用相邻像素间的梯度变化来定量表示，梯度是一阶二维导数的二维等效式，具体计算过程为：The binarized image is a grayscale image, and the edge of the image refers to the part of the grayscale image where the grayscale changes relatively sharply. The degree of change of the grayscale value is quantitatively represented by the gradient change between adjacent pixels, and the gradient is the first-order two-dimensional The two-dimensional equivalent of the derivative, the specific calculation process is:
    首先，计算相邻像素的差分，具体公式为：First, calculate the difference between adjacent pixels. The specific formula is:
    Gx＝f[i，j+1]-f[i，j]G x =f[i,j+1]-f[i,j]
    Gy＝f[i，j]-f[i+1，j]G y =f[i,j]-f[i+1,j]
    其中，Gx表示相邻像素在x方向上的差分，Gy表示相邻像素在y方向上的差分，f[i，j+1]表示图像在第i行第j+1列的像素值，f[i，j]表示图像在第i行第j列的像素值；f[i+1，j]表示图像在第i+1行第j列的像素值；Where G x represents the difference of adjacent pixels in the x direction, G y represents the difference of adjacent pixels in the y direction, and f[i, j+1] represents the pixel value of the image in the i th row and j+1th column. , f[i,j] represents the pixel value of the image in the i-th row and the j-th column; f[i+1,j] represents the pixel value of the image in the i-th row and the j-th column;
    进一步地，计算相邻像素间的梯度，具体公式为：Further, the gradient between adjacent pixels is calculated, and the specific formula is:
    Figure PCTCN2016105736-appb-100006
    Figure PCTCN2016105736-appb-100006
    其中，G(x，y)表示表示图像的在(x，y)点上梯度值，
    Figure PCTCN2016105736-appb-100007
    表示像素值在x方向上求导，
    Figure PCTCN2016105736-appb-100008
    表示像素值在y方向上求导；    Where G(x, y) represents the gradient value at the (x, y) point of the image,
    Figure PCTCN2016105736-appb-100007
    Indicates that the pixel value is derived in the x direction.
    Figure PCTCN2016105736-appb-100008
    Indicates that the pixel value is derived in the y direction;
    进一步地，计算边缘点的梯度幅值，所有边缘点的梯度幅值集合即为提取的边缘轮廓；Further, the gradient magnitude of the edge point is calculated, and the gradient magnitude set of all the edge points is the extracted edge contour;
    进一步地，利用手鼓形状、边长比例、大小、角度和位置的先验知识过滤掉不合理的轮廓，具体滤除条件如下：Further, the prior knowledge of the tambourine shape, side length ratio, size, angle and position is used to filter out unreasonable contours, and the specific filtering conditions are as follows:
    假设矩形的宽为w，高为h，矩形与水平方向的顺时针旋转角为Q，中心点为A，则计算出边缘轮廓的最小外包矩形，再根据如下规则判断是否需要滤除手鼓轮廓；Assuming that the width of the rectangle is w, the height is h, the clockwise rotation angle of the rectangle and the horizontal direction is Q, and the center point is A, the minimum outer envelope of the edge contour is calculated, and then it is judged whether the tambourine contour needs to be filtered according to the following rules;
    ①如果宽高比超过宽高比阈值范围，则滤除该手鼓轮廓，优选的，宽高比阈值范围设置为(0.05，20)；1 If the aspect ratio exceeds the aspect ratio threshold range, the tambourine profile is filtered out, preferably, the aspect ratio threshold range is set to (0.05, 20);
    ②如果中心点A与矩形的4个角和4条边非常接近，则滤除该手鼓轮廓；2 If the center point A is very close to the 4 corners and 4 sides of the rectangle, the tambourine profile is filtered out;
    ③如果w与h的乘积超过乘积阈值范围，则滤除该手鼓轮廓。3 If the product of w and h exceeds the product threshold range, the tambourine profile is filtered out.
  8. 根据权利要求7所述的一种教育玩具套件中敲击动作检测方法，其特征在于，所述步骤2)的具体步骤为：The method for detecting a tapping motion in an educational toy kit according to claim 7, wherein the specific steps of the step 2) are:
    e)通过所述步骤d)中提取出的手鼓轮廓信息，计算轮廓面积和中心点；e) calculating the contour area and the center point by using the tambourine contour information extracted in the step d);
    轮廓面积就是轮廓里面像素点的个数，中心点就是轮廓的质心；The contour area is the number of pixels in the contour, and the center point is the centroid of the contour;
    假设一个面积为N的轮廓，则轮廓的质心坐标计算公式为：Assuming an outline of area N, the centroid coordinate of the outline is calculated as:
    Figure PCTCN2016105736-appb-100009
    Figure PCTCN2016105736-appb-100009
    其中，X表示中心点(x，y)的x坐标，mi表示i质点的质量，xi表示i质点的x坐标，如果i质点的质量与中心点的质量相等，则mi＝1；Where X represents the x coordinate of the center point (x, y), m i represents the mass of the i mass point, x i represents the x coordinate of the i mass point, and if the mass of the i mass point is equal to the mass of the center point, m i =1;
    上述公式则可写为
    Figure PCTCN2016105736-appb-100010
    The above formula can be written as
    Figure PCTCN2016105736-appb-100010
    同理，可得出中心点(x，y)的y坐标
    Figure PCTCN2016105736-appb-100011
    Similarly, the y coordinate of the center point (x, y) can be obtained.
    Figure PCTCN2016105736-appb-100011
    Y表示中心点(x，y)的y坐标，yi表示i质点的y坐标，因此中心点(x，y)可以表示为
    Figure PCTCN2016105736-appb-100012
    Y represents the y coordinate of the center point (x, y), and y i represents the y coordinate of the i mass point, so the center point (x, y) can be expressed as
    Figure PCTCN2016105736-appb-100012
    由最大面积轮廓的中心点(x，y)值根据以下公式估算出该处手鼓的实际面积：The actual area of the tambourine is estimated from the center point (x, y) value of the largest area profile according to the following formula:
    Figure PCTCN2016105736-appb-100013
    Figure PCTCN2016105736-appb-100013
    其中，A表示该处手鼓的实际面积，Amax表示当手鼓放置在可玩区域距离平板电脑最近位置时的手鼓面积，Amin表示当手鼓放置在可玩区域距离平板电脑最远位置时的手鼓面积；Ymax和Ymin分别表示当手鼓放置在可玩区域距离平板电脑最远和最近处点的y坐标，Ymin≤y≤Ymax；Where A represents the actual area of the tambourine, A max represents the tambourine area when the tambourine is placed in the playable area closest to the tablet, and A min represents the tambourine when the tambourine is placed in the playable area from the farthest position of the tablet. Area; Y max and Y min respectively represent the y coordinate when the tambourine is placed at the farthest and closest point of the tablet in the playable area, Y min ≤ y ≤ Y max ;
    如果Sbig<Amax×coff，则认为手鼓被敲击了，否则认为手鼓没有被敲击，其中，Abig表示手鼓被覆盖后，经过二值化图形处理，没有被覆盖掉的最大的区域面积，coff表示一个可调节的参数，取值范围为0～1之间，根据实验，得出最优值为0.5。 If Sbig<A max ×coff, the tambourine is considered to have been tapped, otherwise the tambou is considered not to be tapped. Among them, Abig indicates that the tambourine is covered and the binarized graphics are processed, and the largest area of the area is not covered. Coff represents an adjustable parameter, ranging from 0 to 1. According to the experiment, the optimal value is 0.5.
PCT/CN2016/105736 2016-08-19 2016-11-14 Teaching toy kit and strike action detection method thereof WO2018032629A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201610697875.0A CN106296710A (en) 2016-08-19 2016-08-19 A kind of Educational toy external member and hammer action detection method thereof
CN201610697875.0 2016-08-19

Publications (1)

Publication Number Publication Date
WO2018032629A1 true WO2018032629A1 (en) 2018-02-22

Family

ID=57661764

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2016/105736 WO2018032629A1 (en) 2016-08-19 2016-11-14 Teaching toy kit and strike action detection method thereof

Country Status (2)

Country Link
CN (1) CN106296710A (en)
WO (1) WO2018032629A1 (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114494060A (en) * 2022-01-20 2022-05-13 齐鲁工业大学 Fresnel lens center positioning method based on Opencv

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9158389B1 (en) * 2012-10-15 2015-10-13 Tangible Play, Inc. Virtualization of tangible interface objects
CN105194884A (en) * 2015-10-27 2015-12-30 上海葡萄纬度科技有限公司 Educational toy suite
CN303606294S (en) * 2015-10-14 2016-03-02
CN105498200A (en) * 2016-01-26 2016-04-20 上海葡萄纬度科技有限公司 Educational toy set and tangram color recognition method thereof
CN205164140U (en) * 2015-10-27 2016-04-20 上海葡萄纬度科技有限公司 Education toy external member
CN105709434A (en) * 2016-01-26 2016-06-29 上海葡萄纬度科技有限公司 Educational toy suite and reflector position detecting method thereof
CN106023723A (en) * 2016-06-15 2016-10-12 上海葡萄纬度科技有限公司 Educational toy set and convex mirror imaging correction method

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9158389B1 (en) * 2012-10-15 2015-10-13 Tangible Play, Inc. Virtualization of tangible interface objects
CN303606294S (en) * 2015-10-14 2016-03-02
CN105194884A (en) * 2015-10-27 2015-12-30 上海葡萄纬度科技有限公司 Educational toy suite
CN205164140U (en) * 2015-10-27 2016-04-20 上海葡萄纬度科技有限公司 Education toy external member
CN105498200A (en) * 2016-01-26 2016-04-20 上海葡萄纬度科技有限公司 Educational toy set and tangram color recognition method thereof
CN105709434A (en) * 2016-01-26 2016-06-29 上海葡萄纬度科技有限公司 Educational toy suite and reflector position detecting method thereof
CN106023723A (en) * 2016-06-15 2016-10-12 上海葡萄纬度科技有限公司 Educational toy set and convex mirror imaging correction method

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114494060A (en) * 2022-01-20 2022-05-13 齐鲁工业大学 Fresnel lens center positioning method based on Opencv
CN114494060B (en) * 2022-01-20 2024-02-23 齐鲁工业大学 Opencv-based Fresnel lens center positioning method

Also Published As

Publication number Publication date
CN106296710A (en) 2017-01-04

Similar Documents

Publication Publication Date Title
CN105719318B (en) Magic square color identification method based on HSV in a kind of Educational toy external member
WO2017128606A1 (en) Educational toy kit and tangram color recognition method therefor
WO2018032630A1 (en) Teaching toy kit and method for identifying programming module by using color and counter
WO2018032626A1 (en) Teaching toy kit and number identification method thereof
CN105718878B (en) The aerial hand-written and aerial exchange method in the first visual angle based on concatenated convolutional neural network
CN103218605B (en) A kind of fast human-eye positioning method based on integral projection and rim detection
JP6079832B2 (en) Human computer interaction system, hand-to-hand pointing point positioning method, and finger gesture determination method
WO2017128604A1 (en) Educational toy kit and shape matching-based magic cube location detection method therefor
WO2017128607A1 (en) Educational toy kit and method for detecting placement shape and position of tangram
CN110210360B (en) Rope skipping counting method based on video image target recognition
CN108734743A (en) Method, apparatus, medium and electronic equipment for demarcating photographic device
WO2018032631A1 (en) Teaching toy kit and circuit element and electric wire identification method thereof
WO2017128602A1 (en) Educational toy kit, and method of detecting and positioning alignment hole thereof
WO2017128603A1 (en) Educational toy kit and mirror position detection method thereof
CN102096471B (en) Human-computer interaction method based on machine vision
WO2018032627A1 (en) Teaching toy kit and marine mold identification method thereof
WO2022052941A1 (en) Intelligent identification method and system for giving assistance with piano teaching, and intelligent piano training method and system
CN113723264A (en) Method and system for intelligently identifying playing errors for assisting piano teaching
CN108520260A (en) The recognition methods of visible foreign matters in bottled oral solution
CN109448131A (en) A kind of virtual piano based on Kinect plays the construction method of system
WO2018032629A1 (en) Teaching toy kit and strike action detection method thereof
CN108274476A (en) A kind of method of anthropomorphic robot crawl sphere
WO2018032628A1 (en) Teaching toy kit and colored card identification method thereof
CN108960094A (en) A kind of driver&#39;s smoking motion detection algorithm based on histograms of oriented gradients
CN103839032B (en) A kind of recognition methods and electronic equipment

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16913384

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 16913384

Country of ref document: EP

Kind code of ref document: A1