KR102185320B1 - Apparatus for generating images for analyzing user's posture - Google Patents

Abstract

An image generating device for analyzing a user's exercise posture according to the present invention may be connected to an image sensing device including at least one image sensor. Here, the image generating device for analyzing the exercise posture of the user receives image data photographed by a user performing a specific exercise every time unit from the image sensing device, and the received image according to a predefined image processing process. A 3D image generator that processes the data to generate depth image data; And a posture image generation unit that generates user posture information related to the user’s exercise posture based on the generated depth image data, and generates a posture image visually expressing the generated user posture information.

Description

Image generating device for analyzing user's exercise posture {APPARATUS FOR GENERATING IMAGES FOR ANALYZING USER'S POSTURE}

The present invention relates to an image processing technology capable of generating an image that intuitively expresses the user's exercise posture so that the user's exercise posture can be corrected.

Recently, with the improvement of living conditions, interest in leisure activities of modern people is increasing, and in particular, interest in sports activities for health promotion is increasing rapidly. In particular, in the case of sports using hitting devices such as golf and baseball, experience-based service technology applying virtual reality or augmented reality technology is provided through screen golf courses, screen baseball fields, etc., so that you can enjoy it regardless of time and space. It is gaining great popularity.

In general, in the case of sports using hitting devices such as golf and baseball, it is very difficult to stably and repeatedly use the correct posture, and the practice of the batting posture is performed in a manner of correcting the posture while receiving instruction from an expert. However, in the case of taking lessons from experts, it is inevitably burdensome in terms of cost and time, and various techniques for solving this problem have been provided.

Korean Patent No. 1504538 relates to a technology for generating exercise posture analysis information for correcting a golf swing posture, and to a technology for providing various information that can be referred to correct a user's posture using a pressure sensor or the like. The prior art is a method of sensing the pressure distribution of the user's foot, and this method has a disadvantage that it is difficult to accurately analyze the movement of the entire user's body.

As a technique for overcoming this disadvantage, a technique for generating reference information for posture correction by analyzing an image photographed of a user's body has been provided. A motion tracking technique using depth image analysis such as Korean Patent No. 1683194 can be applied to this technique. A general process of generating a depth image is a method of storing image data received every time unit from a stereo camera in a memory device and then generating a depth image using a predefined algorithm based on the stored image data. However, in such a conventional technique, it is impossible to generate a depth image in real time as the central processing unit (CPU) performs excessive computation, and even if real-time processing is possible, there is a limit to the resolution.

Currently, a technology capable of providing information for correcting a user's exercise posture by analyzing a user's movement at high speed has not been proposed, and if such a technology is provided, it is expected to generate great profits in the sports-related industry.

Korean Patent Registration No. 1504538 Korean Patent Registration No. 1683194

Accordingly, the present invention has been devised to solve the above-described problem, and generates an image for analyzing the user's exercise posture, which can create an image that intuitively expresses the user's exercise posture to correct the user's exercise posture I want to provide a device.

Other objects of the present invention will become more apparent through preferred embodiments described below.

According to an aspect of the present invention, an image generating device for analyzing a user's exercise posture may be connected to an image sensing device including at least one image sensor. Here, the image generating device for analyzing the exercise posture of the user receives image data photographed by a user performing a specific exercise every time unit from the image sensing device, and the received image according to a predefined image processing process. A 3D image generator that processes the data to generate depth image data; And a posture image generation unit that generates user posture information related to the user’s exercise posture based on the generated depth image data, and generates a posture image visually expressing the generated user posture information.

In one embodiment, the 3D image generating unit may include a data processing unit that processes image data received from the image sensing device according to a plurality of predefined image processing processes independently performed; And a memory unit for independently storing image data received from the image sensing device and a plurality of image data processed according to the plurality of image processing processes in each allocated data storage area.

In one embodiment, the data processing unit may include a direct memory access unit that directly stores the received image data and each of the processed image data in the allocated data storage area without passing through a central processing unit. .

In one embodiment, the data processing unit compresses image data received from the image sensing device to a predefined size to generate downscale image data, and based on the image data received from the image sensing device. It may include an edge processing unit that generates edge image data.

In one embodiment, the direct memory access unit stores image data received from the image sensing device in an original image storage area of the memory unit, and the generated downscale image data in a downscale image storage area of the memory unit, the generated The edge image data may be independently stored in the edge image storage area of the memory unit.

In one embodiment, the image sensing device includes a stereo camera including first and second image sensors disposed at positions spaced apart to have parallax, and the data processing unit according to a predefined image processing process Depth image data may be generated by processing the first and second image data received from the first and second image sensors.

In one embodiment, the direct memory access unit stores the first and second image data received from the first and second image sensing devices into the first and second original image storage areas of the memory unit, respectively, and the first and second image data. 2 The first and second downscaled image data generated based on the first and second image data received from the image sensing device are respectively stored in the first and second downscaled image storage areas of the memory unit, and the first and second 2 The first and second edge image data generated based on the first and second image data received from the image sensing device may be independently stored in the first and second edge image storage areas of the memory unit, respectively.

In one embodiment, the data processing unit receives first and second edge image data stored in the first and second edge image storage areas from the direct memory access unit, and receives the received first and second edge image data. It may include a depth image generator that generates depth image data based on.

In an embodiment, the posture image generator may include a feature point extracting unit for extracting a motion feature point associated with the user's motion posture from the generated depth image data.

In an embodiment, the posture image generator may include a motion analysis unit that generates user posture information related to the user’s exercise posture based on a position change of the extracted exercise feature point over time.

In one embodiment, the posture image generator may include an image processing unit that generates a posture image visually expressing the generated user posture information.

In one embodiment, the image sensing device includes a 2D camera that photographs a user performing a specific exercise at a fixed position, and the image processing unit is selected from among 2D image data received from the 2D camera and the generated depth image data. The posture image may be generated by matching at least one of the generated user posture information.

The present invention creates and provides an image that intuitively expresses the user's exercise posture, so that it can be used to correct the user's exercise posture.

In addition, according to the present invention, a depth image may be generated in real time by processing image data received from a stereo camera in parallel at high speed, thereby generating and providing an image for analyzing an exercise posture at a high speed.

1 is a block diagram illustrating an image generation system for analyzing a user's exercise posture according to the present invention.
2 is a block diagram illustrating an image generation system for analyzing a user's exercise posture according to an embodiment of the present invention.
3 is a block diagram illustrating a 3D image generator for generating a depth image according to the present invention.
4 is a block diagram illustrating a 3D image generator according to an embodiment of the present invention.
5 is a block diagram illustrating an operation of a 3D image generator according to an embodiment of the present invention.
6 is a block diagram illustrating an operation of a 3D image generator according to another embodiment of the present invention.
7 is a block diagram illustrating a posture image generator for generating a posture image according to an embodiment of the present invention.
8 to 10 are reference diagrams for explaining a process of generating a posture image according to an embodiment of the present invention.

Since the present invention can apply various transformations and have various embodiments, specific embodiments are illustrated in the drawings and will be described in detail in the detailed description. However, this is not intended to limit the present invention to a specific embodiment, it is to be understood to include all conversions, equivalents, and substitutes included in the spirit and scope of the present invention. In describing the present invention, when it is determined that a detailed description of a related known technology may obscure the subject matter of the present invention, a detailed description thereof will be omitted.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. These terms are used only for the purpose of distinguishing one component from another component.

The terms used in the present application are only used to describe specific embodiments, and are not intended to limit the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In the present application, terms such as "comprise" or "have" are intended to designate the presence of features, numbers, steps, actions, components, parts, or combinations thereof described in the specification, but one or more other features. It is to be understood that the presence or addition of elements or numbers, steps, actions, components, parts, or combinations thereof, does not preclude in advance. Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram illustrating an image generation system (hereinafter, an image generation system) for analyzing a user's exercise posture according to the present invention, and FIG. 2 is a block diagram illustrating an image generation system according to an embodiment of the present invention. It is a block diagram.

In an embodiment, the exercise performed by the user corresponds to at least one of golf and baseball, and the exercise posture of the user may correspond to at least one of a golf swing posture and a baseball batting posture. However, this is not intended to limit the scope of the present invention, and sports other than golf and baseball are irrelevant if it is an exercise to which the technical idea described below can be applied.

1 and 2, the image processing system 10 includes an image sensing device 100 and an image generating device 200 (hereinafter, referred to as an image generating device) for analyzing a user's exercise posture.

The image sensing device 100 is a device that senses an image to generate image data, and may correspond to, for example, a camera device. Here, the image sensing device 100 may be configured to include a plurality of camera devices.

In one embodiment, it may be configured to include at least one of the stereo camera 110 and the 2D camera 120.

In one embodiment, the stereo camera 110 may be configured to include two image sensors (left and right) disposed at spaced positions to have parallax.

In one embodiment, the 2D camera 120 may be configured to include an image sensor capable of generating an image of higher resolution than a stereo camera. Here, the 2D camera 120 may include a color image sensor.

The image generating device 200 is connected to the image sensing device 100 and corresponds to a device that processes image data received from the image sensing device 100 to generate an image (hereinafter, a posture image) for analyzing a user's exercise posture. do.

The image generating device 200 receives image data captured by a user performing a specific exercise every time unit from the image sensing device 100, and processes the received image data according to a predefined image processing process to perform a depth image ( depth image) and a 3D image generator 210 that generates data.

The image generating device 200 generates user posture information related to the user's exercise posture based on depth image data generated by the 3D image generator 210, and a posture image visually expressing the generated user posture information It includes a posture image generator 220 that generates.

In an embodiment, the 3D image generator 210 may generate depth image data based on image data (left, right) received from the stereo camera 110.

In one embodiment, the posture image generator 220 generates user posture information (eg, swing plane) related to the user’s exercise posture based on the depth image data, but the 2D image data received from the 2D camera 120 and At least one of the depth image data received from the 3D image generator 210 and the generated user posture information may be matched to generate a posture image.

Hereinafter, each component of the image generating apparatus according to the present invention and various embodiments of the present invention and operations of each component will be described in detail with reference to FIGS. 3 to 10.

First, a technology for generating a depth image according to an embodiment of the present invention will be described in detail with reference to FIGS. 3 to 6. The depth image generation technology according to the present invention can be processed at a high speed (real time) compared to the prior art, and a remarkable effect in terms of this speed will be clearly understood through the following description.

3 is a block diagram illustrating a 3D image generator 210 for generating a depth image according to the present invention.

Referring to FIG. 3, the 3D image generator 210 is connected to the stereo camera 110 and may receive image data captured by a user performing a specific exercise from the stereo camera 110 every time unit.

The 3D image generator 210 is connected to the image sensing device 100 and corresponds to a device capable of processing and storing image data received from the image sensing device 100. Here, the 3D image generator 210 includes a data processing unit 310 that processes image data received from the stereo camera 110 according to a predefined image processing process, and image data received from the stereo camera 110. It may include a (original image) and a memory unit 320 that stores image data processed by the data processing unit 310.

In one embodiment, at least some components of the 3D image generator 210 may be formed by being included in a field programmable gate array (FPGA).

Hereinafter, each component of the 3D image generator 210 according to the present invention and various embodiments of the present invention and the operation of each component will be described in detail with reference to FIGS. 4 to 6.

4 is a block diagram illustrating a 3D image generator 210 according to an embodiment of the present invention.

Referring to FIG. 4, the 3D image generation unit 210 processes image data received from the stereo camera 110 according to a predefined image processing process, and receives the image data from the stereo camera 110. A memory unit 320 for storing image data (original image) and image data processed by the data processing unit 310 may be included.

The data processing unit 310 may process image data (original image) received from the stereo camera 110 according to a plurality of predefined image processing processes each independently performed.

In one embodiment, the data processing unit 310 centrally processes image data (original image) received from the stereo camera 110 and a plurality of processed image data (eg, downscale image data, edge image data). It may include a direct memory access unit 311 that directly stores in the allocated data storage area without passing through the device (CPU). For example, the direct memory access unit 311 may be implemented with direct memory access (DMA).

In one embodiment, the data processing unit 310 may include a down scale unit 312 for generating downscale image data by compressing image data (original image) received from the stereo camera 110 into a predefined size. have. For example, the down scale unit 312 reduces the image data (original image) received from the image sensing device 100 to 1/16 (for example, an image having a resolution of 1280 * 1024 is converted to an image having a resolution of 320 * 256). Downscaled image data can be generated.

In an embodiment, the data processing unit 310 may include an edge processing unit 313 that generates edge image data based on image data received from the stereo camera 110. Here, the edge processing unit 313 uses an edge extraction algorithm (eg, sobel filter) on the image data (original image) received from the stereo camera 110 or the downscale image data generated by the downscale unit 312. ) Can be applied to create edge image data.

On the other hand, in the present invention, the edge image data is basic image data (image data with clearly processed boundary lines) that can be used in the process of generating a depth image, and is not limited to the type and name of the edge extraction algorithm. If it is basic image data that can be used to generate a depth image, it should be interpreted as edge image data according to the present invention.

In an embodiment, the data processing unit 310 may include a depth image generation unit 314 that generates depth image data based on edge image data. A process of generating the depth image data by the depth image generator 314 will be described later.

The memory unit 320 may independently store image data (original image) received from the stereo camera 110 and a plurality of image data processed by the data processing unit 310 in each allocated data storage area.

In one embodiment, the memory unit 320 may correspond to a volatile memory, and preferably may correspond to a dynamic random access memory (DRAM).

In one embodiment, the memory unit 320 includes an original image storage area 321 for storing image data (original image) received from the stereo camera 110 and a downscale image generated by the downscale unit 312. A down-scale image storage area 322 for storing data and an edge image storage area 323 for storing edge image data generated by the edge processing unit 313 may be included.

In the above, each configuration of the 3D image generation unit 210 according to an embodiment of the present invention has been described with reference to FIG. 4, and hereinafter, a 3D image generation unit according to an embodiment of the present invention with reference to FIG. 5 The operation of each component included in 210 will be described.

5 is a block diagram for explaining the operation of the 3D image generator 210 according to an embodiment of the present invention.

Referring to FIG. 5, the data processing unit 310 receives image data from the stereo camera 110.

In one embodiment, the direct memory access unit 311 directly transfers the image data (original image) received from the stereo camera 110 to the original image storage area 321 of the memory unit 320 without passing through the central processing unit. Can be saved.

In an embodiment, the down-scale unit 312 may generate down-scale image data by compressing image data (original image) received from the stereo camera 110 into a predetermined size. Here, the direct memory access unit 311 directly stores the downscale image data generated by the downscale unit 312 in the downscale image storage area 322 of the memory unit 320 without passing through the central processing unit. I can.

In an embodiment, the edge processing unit 313 may generate edge image data based on image data (original image) received from the stereo camera 110. In another embodiment, the edge processing unit 313 directly receives the downscale image data stored in the downscale image storage area 223 from the memory access unit 311, and based on the received downscale image data, the edge image data Can be created. Here, the edge processing unit 313 applies an edge extraction algorithm (eg, sobel filter) to image data (original image) or downscale image data received from the stereo camera 110 to generate edge image data. I can. Thereafter, the direct memory access unit 311 may directly store the edge image data generated by the edge processing unit 313 in the edge image storage area 323 of the memory unit 320 without passing through the central processing unit.

Here, the downscale unit 312 and the edge processing unit 313 may operate independently, and the direct memory access unit 311 simultaneously stores original image data, downscale image data, and edge image data to the memory unit 320 Can be stored in each allocated storage area.

The depth image generator 314 may directly receive edge image data stored in the edge image storage area 323 from the memory access unit 311 and generate depth image data based on the received edge image data. A process of generating the depth image data by the depth image generator 314 will be described later.

In the above, the operation of each component included in the 3D image generating unit 210 according to an embodiment of the present invention has been described with reference to FIG. 5. Hereinafter, referring to FIG. 6, the operation of each component according to a preferred embodiment of the present invention is described. The operation of each component included in the dimensional image generator 210 will be described.

6 is a block diagram illustrating the operation of the 3D image generator 210 according to another embodiment of the present invention.

Referring to FIG. 6, the stereo camera 110 may be configured to include first and second image sensors 111 and 112 disposed at spaced positions to have parallax. Herein, the stereo camera 110 may correspond to a camera device including first and second image sensors 111 and 112 corresponding to left and right image sensors, and the first and second image sensors ( Each of 111 and 112 may generate first and second image data and provide them to the data processing unit 310.

In an embodiment, the 3D image generator 210 may include a timing generator (not shown) for synchronizing the first image sensor 111 and the second image sensor 112. Here, the timing generator is configured to include a time identifier in the first and second image data generated by the first image sensor 111 and the second image sensor 112, so that the first and second image data are It can be generated in correspondence with time (e.g., time identifiers corresponding to n1, n2, …, nx per unit time dn from n0 are assigned to each of the first image data L and the second image data R) To be. Meanwhile, a known configuration may be applied to the timing generator, and a configuration for synchronizing the first image sensor 111 and the second image sensor 112 should be interpreted as corresponding to the timing generator according to the present invention.

In one embodiment, the memory unit 320 may be formed of DRAM.

The data processing unit 310 receives first and second image data from the first and second image sensors 111 and 112, respectively.

In one embodiment, the direct memory access unit 311 is configured to transmit the first and second image data (original images) received from the first and second image sensors 111 and 112 to the DRAM 320 without passing through the central processing unit. ) Sequentially stored in each of the first and second original image storage areas 321-1 and 321-2 ([(L, n0), (L, n1), ……(L, nx)], [( R, n0), (R, n1),… (R, nx)]).

In one embodiment, the first and second down-scale units 312-1 and 312-2 convert first and second image data (original images) received from the first and second image sensors 111 and 112. First and second downscale image data may be generated by compression to a predefined size, respectively. Here, the direct memory access unit 311 is a DRAM device without passing through the central processing unit for the first and second downscale image data generated by the first and second downscale units 312-1 and 312-2. Sequentially stored in each of the first and second downscaled image storage areas 322-1 and 322-2 of 320 ([(L, n0), (L, n1), ……(L, nx)]) , [(R, n0), (R, n1), ……(R, nx)]).

In one embodiment, the first and second edge processing units 213-1 and 213-2 are based on first and second image data (original images) received from the first and second image sensors 111 and 112. The first and second edge image data may be generated respectively. In another embodiment, the first and second edge processing units 313-1 and 313-2 are directly connected to the first and second downscale image storage areas 323-1 and 323-2 from the memory access unit 311. The first and second downscale image data stored in may be provided, and first and second edge image data may be generated based on the provided first and second downscale image data, respectively. Thereafter, the direct memory access unit 311 transmits the first and second edge image data generated by the first and second edge processing units 313-1 and 313-2 to the DRAM 320 without passing through the central processing unit. Sequentially stored in each of the first and second edge image storage areas 323-1 and 323-2 of ([(L, n0), (L, n1), ……(L, nx)], [(R , n0), (R, n1),… (R, nx)]).

Here, the down scale units 312-1 and 312-2 and the edge processing units 313-1 and 313-2 may operate independently, respectively, and the direct memory access unit 311 is the first and second original Image data, first and second downscale image data, and first and second edge image data may be simultaneously stored in each allocated storage area of DRAM 320.

The depth image generator 314 receives and receives the first and second edge image data stored in the first and second edge image storage areas 323-1 and 323-2 from the direct memory access unit 311, Depth image data may be generated based on the first and second edge image data.

Hereinafter, a process of generating depth image data by the 3D image generator 210 according to the present invention will be described in more detail.

The direct memory access unit 311 of the data processing unit 310 includes first and second image data (original image) generated from the first image sensor 111 and the second image sensor 112 included in the stereo camera 110. ) May be stored in each of the first and second original image storage areas 321-1 and 321-2 of the DRAM 320 without passing through the central processing unit.

At the same time, the first and second down-scale units 312-1 and 312-2 compress the first and second image data (original image) to a predefined size, respectively, to obtain first and second down-scale image data. And the direct memory access unit 311 transmits the first and second downscale image data to the first and second downscale image storage areas 322-1 and 322 of the DRAM 320 without passing through the central processing unit. -2) Can be stored in each. Here, the first and second down-scale units 312-1 and 312-2 apply a horizontal correction algorithm to the first and second down-scale image data, so that the first and second image sensors are S/W. The horizontal of (111, 112) can be corrected.

In one embodiment, the first and second edge processing units 313-1 and 313-2 are directly from the memory access unit 311 to the first and second downscale image storage areas 323-1 and 323-2. The stored first and second downscale image data may be provided, and first and second edge image data (eg, gradient images) may be generated by applying a Sobel filter to the provided first and second downscale image data. . Thereafter, the direct memory access unit 311 transmits the first and second edge image data generated by the first and second edge processing units 313-1 and 313-2 to the DRAM 220 without passing through the central processing unit. It may be stored in the first and second edge image storage areas 223-1 and 223-2 of. In another embodiment, the first and second edge processing units 313-1 and 313-2 apply a Sobel filter to the first and second original image data to provide first and second edge image data (eg, gradient image). ) Can be generated, and according to the present embodiment, high-quality depth image data can be generated compared to the case of using the first and second downscale images.

The depth image generator 314 receives and receives the first and second edge image data stored in the first and second edge image storage areas 323-1 and 323-2 from the direct memory access unit 311, Depth image data may be generated based on the first and second edge image data. More specifically, the depth image generator 314 may generate a disparity map by detecting a block matching left and right for each block from the first and second edge image data (eg, gradient image). Depth image data may be generated by using values such as baseline and focal length of the camera 110. Meanwhile, the algorithm for generating the depth image is not a core configuration of the present invention, and various known algorithms can be applied. That is, the core technical idea of the present invention is of an image processing apparatus that independently processes image data received from the stereo camera 110 in parallel, stores it in the memory unit 320, and allows the stored image to be used for depth image generation. Since it is a configuration, it should be interpreted as not being limited to a specific algorithm for generating a depth image.

According to the 3D image generation unit 210 according to the present invention described above, data parallel processing is possible in units of 128 bits, and high-speed image processing is possible at a level of about 860 fps. It can be created and provided.

In the above, a technique for generating a depth image according to various embodiments of the present invention has been described in detail with reference to FIGS. 3 to 6. Hereinafter, a technique for generating a posture image according to an embodiment of the present invention will be described in detail with reference to FIGS. 7 to 10.

7 is a block diagram illustrating a posture image generator 220 for generating a posture image according to an embodiment of the present invention, and FIGS. 8 to 10 are a process of generating a posture image according to an embodiment of the present invention. Is a reference diagram (an example of a depth image generated by a 3D image generator, and FIG. 8A is an example of a depth image at a viewpoint n1 and a viewpoint 8B is an n2).

Referring to FIG. 7, the posture image generating unit 220 is a device that generates a posture image based on depth image data generated by the 3D image generating unit 210. In one embodiment, the feature point extracting unit 410 ), a motion analysis unit 420, and an image processing unit 430.

In an embodiment, the feature point extracting unit 410 may extract a motion feature point related to a user's motion posture from depth image data generated by the 3D image generator 210. Here, the motion feature point may be predefined by the manager of the image generating device.

In an embodiment, the exercise feature point may be defined in association with at least one of a user's head, shoulders, waist, hand, knee, foot, and striking device.

For example, referring to FIG. 8(A), the feature point extraction unit 410 is a motion feature point (e.g., head, shoulder, waist, etc.) defined in depth image data generated by the 3D image generator 210. Both hands, knees, feet, and the end of the striking device) can be extracted.

Here, a specific marker is attached to the end of the clothing (hat, top, bottom, shoes, etc.) worn by the user and the striking device, so that the feature point extraction unit 410 determines the point corresponding to the corresponding marker in the depth image data. Even if there is no such marker, a specific point is defined as a motion feature point in the first depth image data (T=n0) in time, and the feature point extraction unit 410 determines the depth image data (T=n1, n2, In a method of tracking a corresponding point in …… nx), motion feature points may be extracted from depth image data (T=nx) of a specific viewpoint. Meanwhile, as an algorithm for extracting feature points from image data, various known algorithms may be applied, and those skilled in the image processing technology field will be able to clearly understand and implement the technical idea of the present invention through the above description.

In an embodiment, the motion analysis unit 420 may generate user posture information related to the user’s exercise posture based on a position change of the motion feature point extracted by the feature point extractor 410 over time. Here, the user's posture information may correspond to data that can visually represent at least one of a swing trajectory according to a user's movement, a position of a rotation axis, a movement of a rotation axis, a position of a center of gravity, and a movement of the center of gravity.

An example of a process of generating user posture information will be described in more detail with reference to FIG. 9.

The motion analysis unit 420 may track a position change of the motion feature point over time in the depth image data. For example, as shown in FIG. 9(A), the motion analysis unit 420 tracks the three-dimensional position coordinates of the motion feature points defined as wrist and shoulder in depth image data, and the x, y, z axes based on the center point Star position coordinate function can be derived. Here, the motion analysis unit 420 may apply a noise removal filter to remove noise of the three-dimensional position coordinate function of the derived motion feature points, and, for example, apply a noise removal filter based on the least squares method or RANSAC. Thus, a corrected position coordinate function as shown in FIG. 9(B) can be derived. Thereafter, the motion analysis unit 420 derives a plane in which the straight lines connected to the movement feature points for the wrist and shoulder move over time based on the corrected position coordinate function, and generates user posture information (swing plane representing the swing trajectory). can do. In the same manner, user posture information (position or position change of the swing axis) may be generated by tracking the 3D position change of the movement feature points defined as the head center and waist center positions. On the other hand, the method of tracking the three-dimensional position change of the feature point in the depth image is not a core technical idea of the present invention, and it goes without saying that various known algorithms other than the least squares method or RANSAC may be applied.

In an embodiment, the image processing unit 430 may generate a posture image based on the user posture information generated by the motion analysis unit 420. Here, the posture image may refer to an image that visually and intuitively expresses an exercise posture according to a user's movement.

In one embodiment, the image processing unit 430 may generate a posture image by matching depth image data generated by the 3D image generating unit 210 with user posture information generated by the motion analysis unit 420. have. In another embodiment, the image processing unit 430 matches the 2D image data received from the 2D camera 120 of the image sensing device 100 with the user's posture information generated by the motion analysis unit 420 to obtain a posture image. Can be created.

For example, the image processing unit 430 may generate a posture image that intuitively represents the user's golf swing posture, as shown in FIG. 10, wherein the image processing unit 430 is a swing plane on the depth image at a specific point in time. The posture image may be generated by matching user posture information including 910 and the rotation axis position 920. (In other words, the position of the swing plane and rotation axis is expressed using the depth image as the background)

Alternatively, the image processing unit 430 may generate a posture image by matching user posture information including the swing plane 910 and the rotation axis position 920 with the 2D image at a specific viewpoint. In an embodiment, the 2D image may correspond to a high-quality color image compared to a depth image. (In other words, the position of the swing plane and rotation axis is expressed using a high-quality color image as the background)

Preferred embodiments of the present invention described above are disclosed for the purpose of illustration, and those skilled in the art who have ordinary knowledge of the present invention will be able to make various modifications, changes, additions within the spirit and scope of the present invention, such modifications, changes and The addition should be seen as falling within the scope of the following claims.

10: image processing system
100: image sensing device
110: stereo camera, 120: 2D camera
111: first image sensor, 112: second image sensor
200: image generating device
210: 3D image generator
220: posture image generator
310: data processing unit
311: Direct memory access unit
312: down scale unit
313: edge processing unit
314: depth image generator
320: memory unit
321: Original image storage area
322: downscale image storage area
323: Edge image storage area
410: feature point extraction unit
420: motion analysis unit
430: image processing unit

Claims (12)

In the image generating device for analyzing a user's exercise posture, which can be connected to an image sensing device including at least one image sensor,
3, in which a user performing a specific exercise receives photographed image data from the image sensing device every time unit, and processes the received image data according to a predefined image processing process to generate depth image data. A dimensional image generator; And
And a posture image generator for generating user posture information related to the user’s exercise posture based on the generated depth image data, and generating a posture image visually expressing the generated user posture information, and
The 3D image generating unit,
A data processing unit that processes image data received from the image sensing device according to a plurality of predefined image processing processes each independently performed; And
And a memory unit for independently storing image data received from the image sensing device and a plurality of image data processed according to the plurality of image processing processes in each allocated data storage area; and
The data processing unit,
A down scale unit for generating down scale image data by compressing the image data received from the image sensing device into a predefined size;
An edge processing unit generating edge image data based on the image data received from the image sensing device; And
A direct memory access unit for directly storing the received image data and each of the processed image data in the allocated data storage area without passing through a central processing unit;
The direct memory access unit,
The image data received from the image sensing device is stored in an original image storage area of the memory unit,
The generated downscale image data in a downscale image storage area of the memory unit,
The generated edge image data is stored in an edge image storage area of the memory unit,
Each is stored independently,
The 3D image generator is characterized in that by performing image processing at a speed of 860 fps by parallel processing data in units of 128 bits, a posture image for analyzing a user's exercise posture can be generated and provided to the user at high speed. An image generating device for analyzing the exercise posture of a person
delete delete delete delete The method of claim 1,
The image sensing device includes a stereo camera including first and second image sensors disposed at positions spaced apart to have parallax,
The data processing unit processes the first and second image data received from the first and second image sensors according to a predefined image processing process to generate depth image data. An image generating device for posture analysis.
The method of claim 6, wherein the direct memory access unit
The first and second image data received from the first and second image sensing devices are respectively stored in first and second original image storage areas of the memory unit,
First and second downscale image data generated based on the first and second image data received from the first and second image sensing devices are respectively stored in first and second downscale image storage areas of the memory unit,
First and second edge image data generated based on the first and second image data received from the first and second image sensing devices are respectively stored in the first and second edge image storage areas of the memory unit,
An image generating device for analyzing a user's exercise posture, characterized in that each independently stored.
The method of claim 7, wherein the data processing unit
Receive first and second edge image data stored in the first and second edge image storage areas from the direct memory access unit, and provide a depth image based on the received first and second edge image data An image generating apparatus for analyzing a user's exercise posture, comprising: a depth image generator that generates data.
The method of claim 1, wherein the posture image generator
And a feature point extracting unit for extracting motion feature points associated with the user's motion posture from the generated depth image data.
The method of claim 9, wherein the posture image generating unit
And a motion analysis unit that generates user posture information related to the user’s exercise posture based on a position change of the extracted exercise feature point over time.
The method of claim 10, wherein the posture image generator
And an image processing unit for generating a posture image visually expressing the generated user posture information.
The method of claim 11,
The image sensing device includes a 2D camera for photographing a user performing a specific exercise at a fixed position,
The image processing unit generates the posture image by matching at least one of the 2D image data received from the 2D camera and the generated depth image data with the generated user posture information to analyze the user’s exercise posture. Device for generating images.

Images