WO2024212554A1 - 用于智能机器人的书法临摹方法、装置、设备及存储介质 - Google Patents

用于智能机器人的书法临摹方法、装置、设备及存储介质 Download PDF

Info

Publication number
WO2024212554A1
WO2024212554A1 PCT/CN2023/137196 CN2023137196W WO2024212554A1 WO 2024212554 A1 WO2024212554 A1 WO 2024212554A1 CN 2023137196 W CN2023137196 W CN 2023137196W WO 2024212554 A1 WO2024212554 A1 WO 2024212554A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
calligraphy
writing
robot
target feature
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2023/137196
Other languages
English (en)
French (fr)
Inventor
冯伟
许睿烁
王卫军
周凯臣
安鲸
车其姝
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen Institute of Advanced Technology of CAS
Original Assignee
Shenzhen Institute of Advanced Technology of CAS
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen Institute of Advanced Technology of CAS filed Critical Shenzhen Institute of Advanced Technology of CAS
Publication of WO2024212554A1 publication Critical patent/WO2024212554A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Program-controlled manipulators
    • B25J9/16—Program controls
    • B25J9/1656—Program controls characterised by programming, planning systems for manipulators
    • B25J9/1664—Program controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/26—Segmentation of patterns in the image field; Cutting or merging of image elements to establish the pattern region, e.g. clustering-based techniques; Detection of occlusion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/146—Aligning or centring of the image pick-up or image-field
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/146—Aligning or centring of the image pick-up or image-field
    • G06V30/147—Determination of region of interest
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/14—Image acquisition
    • G06V30/148—Segmentation of character regions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/16—Image preprocessing
    • G06V30/162—Quantising the image signal
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/16—Image preprocessing
    • G06V30/164—Noise filtering
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/18—Extraction of features or characteristics of the image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/19—Recognition using electronic means
    • G06V30/191—Design or setup of recognition systems or techniques; Extraction of features in feature space; Clustering techniques; Blind source separation
    • G06V30/19147—Obtaining sets of training patterns; Bootstrap methods, e.g. bagging or boosting
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10—Character recognition
    • G06V30/22—Character recognition characterised by the type of writing
    • G06V30/226—Character recognition characterised by the type of writing of cursive writing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20092—Interactive image processing based on input by user
    • G06T2207/20104—Interactive definition of region of interest [ROI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Definitions

  • the present application relates to the technical field of intelligent robots, and in particular to a calligraphy copying method, device, equipment and storage medium for intelligent robots.
  • Chinese calligraphy has a history of thousands of years, and countless famous calligraphers have left behind a large number of masterpieces.
  • Calligraphy is not only a culture of the Chinese nation, but also a way of carrying the history of the Chinese nation.
  • the excellent culture of the Chinese nation has received more and more attention, and the Ministry of Education plans to include calligraphy in primary and secondary school education and the high school entrance examination.
  • Chinese calligraphy is ever-changing, and ordinary people need to spend a long time practicing to reach a good level of writing.
  • the present application provides a calligraphy copying method, device, equipment and storage medium for an intelligent robot to solve the problem that the existing calligraphy robot cannot copy calligraphy in real time and has poor flexibility.
  • a technical solution adopted in the present application is: to provide a calligraphy copying method for an intelligent robot, which includes: using a pre-set camera to shoot the calligraphy writing process in real time to obtain a writing image; using a pre-trained image recognition model to sequentially identify and segment the area of the calligraphy in each frame of the writing image in the order of acquisition to obtain a target feature image; extracting calligraphy style features from each frame of the target feature image in turn, and obtaining the three-dimensional coordinates of the motion trajectory of the robot end effector based on the calligraphy style feature analysis; based on the inverse kinematics solution, determining the motion control parameters of the robot manipulator arm according to each three-dimensional coordinate in turn, and controlling the robot to copy the calligraphy according to the motion control parameters.
  • a pre-trained image recognition model is used to sequentially identify and segment the areas of calligraphy in each frame of a writing image in the order of acquisition to obtain a target feature image, including: inputting the writing image into a pre-trained image recognition model to identify a first area including the calligraphy and a second area including a hand and a pen in the writing image; performing image segmentation on the first area and the second area in the writing image, and then performing mask superposition processing on the second area to obtain a first feature image; based on an image binarization method, eliminating the second area in the first feature image and retaining the first area to obtain a target feature image.
  • the writing image into a pre-trained image recognition model before inputting the writing image into a pre-trained image recognition model, it also includes: filtering shadow noise data in the writing image based on a first preset method.
  • the target feature image after obtaining the target feature image, it also includes: when the text in the target feature image is tilted, adjusting the direction of the tilted text in the target feature image to a preset direction based on an affine transformation.
  • calligraphy style features are extracted from each frame of the target feature image in turn, and the three-dimensional coordinates of the motion trajectory of the robot end effector are obtained based on the calligraphy style feature analysis, including: performing frame difference method on each two adjacent frames of the target feature image in turn to obtain a writing trajectory image between each two adjacent frames of the target feature image; obtaining the coordinates of the center point of the writing trajectory image to obtain the coordinate values in the X-axis and Y-axis directions; calculating the area of the writing trajectory in the writing trajectory image; based on a preset mapping method, obtaining the coordinate value of the robot end effector in the Z-axis direction according to the area mapping; and obtaining the three-dimensional coordinates based on the coordinate values in the X-axis and Y-axis directions and the coordinate value in the Z-axis direction.
  • a calligraphy copying device for an intelligent robot which includes: a shooting module, which is used to use a pre-set camera to shoot the calligraphy writing process in real time to obtain a writing image; a preprocessing module, which is used to use a pre-trained image recognition model to sequentially identify and segment the area of the calligraphy in each frame of the writing image in the order of acquisition to obtain a target feature image; an extraction module, which is used to extract the calligraphy style features from each frame of the target feature image in turn, and obtain the three-dimensional coordinates of the motion trajectory of the robot end effector according to the calligraphy style feature analysis; a control module, which is used to determine the motion control parameters of the robot manipulator arm according to each three-dimensional coordinate based on the inverse kinematic solution, and control the robot to copy calligraphy according to the motion control parameters.
  • a shooting module which is used to use a pre-set camera to shoot the calligraphy writing process in real time to obtain a writing image
  • a preprocessing module
  • a computer device which includes a processor and a memory coupled to the processor, wherein program instructions are stored in the memory, and when the program instructions are executed by the processor, the processor executes the steps of any one of the above-mentioned calligraphy copying methods for an intelligent robot.
  • another technical solution adopted in the present application is: to provide a storage medium storing program instructions of the calligraphy copying method for an intelligent robot that can implement any of the above items.
  • the beneficial effect of the present application is that the calligraphy copying method for an intelligent robot of the present application captures the writing image of the calligraphy writing process in real time, extracts the calligraphy style features from the continuous multi-frame writing images, and then extracts the three-dimensional coordinates of the motion trajectory of the robot end effector from the calligraphy style features, and then uses the inverse kinematic solution method to control the robot in real time to copy the photographed calligraphy according to the three-dimensional coordinates. It can flexibly display different calligraphy styles, and the copying method is not limited to the content taught in advance by the robot, so it has higher practical value. Moreover, the real-time copying method copies in the order in which the writing images are acquired, thereby accurately copying the stroke order of the calligraphy and avoiding displaying the wrong calligraphy writing method.
  • FIG1 is a schematic diagram of a flow chart of a calligraphy copying method for an intelligent robot according to an embodiment of the present invention
  • FIG2 is a schematic diagram of filtering shadow noise data according to an embodiment of the present invention.
  • FIG. 3 is a schematic diagram of extracting target image features using an image recognition model according to an embodiment of the present invention.
  • FIG4 is a schematic diagram of adjusting a tilted font by using an affine transformation method according to an embodiment of the present invention
  • FIG5 is a schematic diagram of filtering salt and pepper noise data according to an embodiment of the present invention.
  • FIG6 is a schematic diagram of functional modules of a calligraphy copying device for an intelligent robot according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention.
  • FIG. 8 is a schematic diagram of the structure of a storage medium according to an embodiment of the present invention.
  • first”, “second” and “third” in this application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features.
  • the features defined as “first”, “second” and “third” can explicitly or implicitly include at least one of the features.
  • the meaning of “multiple” is at least two, such as two, three, etc., unless otherwise clearly and specifically defined. All directional indications (such as up, down, left, right, front, back%) in the embodiments of this application are only used to explain the relative position relationship, movement, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication also changes accordingly.
  • FIG1 is a flow chart of a calligraphy copying method for an intelligent robot according to an embodiment of the present invention. It should be noted that the method of the present invention is not limited to the flow sequence shown in FIG1 if substantially the same results are achieved. As shown in FIG1 , the calligraphy copying method for an intelligent robot includes the following steps:
  • Step S101 Use a pre-set camera to shoot the calligraphy writing process in real time to obtain a writing image.
  • the purpose of this embodiment is to copy calligraphy in real time, so it is necessary to use a pre-set camera to shoot the calligraphy writing process.
  • the camera can be installed on a handheld terminal or a fixed terminal, such as a mobile phone, a tablet computer, a laptop computer, etc.
  • the handheld terminal or the fixed terminal can be connected to the robot through a wired network or a wireless network, so that the real-time calligraphy copying process can be realized remotely.
  • the calligrapher can shoot the calligraphy writing process through a mobile phone and send the shooting process to the robot in real time, and the robot will copy the writing process in real time.
  • Step S102 using a pre-trained image recognition model, sequentially identify and segment the calligraphy writing area in each frame of the writing image in the order of acquisition to obtain a target feature image.
  • the camera captures the calligraphy writing process, it may capture the hand and the pen, resulting in the inability to accurately obtain the calligraphy features from the writing image. Therefore, in this embodiment, after obtaining the writing image, the writing image is input into a pre-trained image recognition model to identify the areas in the image belonging to the hand and the pen, as well as the areas belonging to the calligraphy writing, and then the calligraphy writing areas are segmented to obtain a target feature image containing the calligraphy.
  • the image recognition model is implemented based on a deep neural network, such as a convolutional neural network, and the image recognition model is iteratively trained using pre-labeled sample data until the image recognition model reaches a preset accuracy.
  • the continuous frame writing images captured in real time are sequentially input into the image recognition model in the order of acquisition for processing, so as to obtain continuous frame target feature images.
  • step S102 specifically includes:
  • the acquired writing image is input into the image recognition model to recognize the second area corresponding to the hand and the pen in the writing image.
  • the method further includes: filtering shadow noise data in the writing image based on a first preset method.
  • the shadow noise data in the writing image is filtered before the writing image is input into the image recognition model for recognition.
  • the first preset method is preferably an image binarization method, and the shadow noise in the writing image is filtered out by the image binarization method.
  • the shadow noise in the writing image is filtered out by the image binarization method.
  • the second area in the first feature image is eliminated, the first area is retained, and the target feature image is obtained.
  • this embodiment performs instance segmentation on the first region and the second region, performs mask overlay processing on the second region to distinguish the first region from the second region, and then performs image binarization processing on the first feature image according to a preset threshold value, thereby eliminating the hand and pen in the second region, and obtaining a target feature image including the first region.
  • the method further includes: when the text in the target feature image is tilted, adjusting the direction of the tilted text in the target feature image to a preset direction based on an affine transformation.
  • Step S103 extracting calligraphy style features from each frame of the target feature image in turn, and obtaining the three-dimensional coordinates of the motion trajectory of the robot end effector based on the calligraphy style feature analysis.
  • the calligraphy style features include the order of calligraphy strokes, the starting point of the strokes, etc. After obtaining the target feature image, the calligraphy style features are extracted from the target feature image, and then the three-dimensional coordinates of the motion trajectory of the robot end effector are obtained according to the calligraphy style feature analysis.
  • step S103 specifically includes:
  • this embodiment uses a frame difference method to perform a differential operation on two adjacent frames of target feature images to obtain the outline of the writing track between the two frames of target feature images, and uses the outline as the writing track image.
  • the method further includes: filtering salt and pepper noise data in the writing track image based on a second preset method.
  • this embodiment uses the second preset method to filter the salt and pepper noise data in the writing track image.
  • the second preset method is image binarization, as shown in FIG5
  • FIG5 (A) is a writing track image including salt and pepper noise data
  • FIG5 (B) is a writing track image after filtering out the salt and pepper noise data.
  • the writing track image After obtaining the writing track image, confirm the center point of the writing track in the writing track image, and then obtain the coordinates corresponding to the center point. It should be understood that the shooting interval between two adjacent frames of images is extremely short, so the change between two adjacent frames of images is small.
  • This embodiment constructs the smallest circle around the writing track, and then uses the center of the circle as the center point of the writing track, and then obtains the coordinate value of the center point in the pre-constructed rectangular coordinate system.
  • this embodiment calculates the area of the writing track in the writing track image.
  • the coordinate value of the robot end effector in the Z-axis direction is obtained according to the area mapping.
  • the thickness of the strokes of the robot writing font is related to the displacement of the robot mechanical arm in the Z-axis direction.
  • this embodiment pre-constructs a mapping relationship between the area of the writing trajectory and the coordinate value of the robot end effector in the Z-axis direction, so that after obtaining the area of the writing trajectory, the coordinate value of the robot end effector in the Z-axis direction is obtained according to the area mapping.
  • the coordinate values in the X-axis, Y-axis, and Z-axis directions are used as the three-dimensional coordinates of the robot end effector.
  • Step S104 Based on the inverse kinematics solution, the motion control parameters of the robot arm are determined according to each three-dimensional coordinate in turn, and the robot is controlled to copy calligraphy according to the motion control parameters.
  • the kinematic inverse solution is performed based on the three-dimensional coordinates to obtain the motion control parameters of the robot's mechanical arm, and then the robot's motion is controlled based on the motion control parameters to complete the copying of the calligraphy.
  • the calligraphy copying method for an intelligent robot in an embodiment of the present invention captures a writing image of the calligraphy writing process in real time, extracts calligraphy style features from multiple consecutive writing images, and then extracts the three-dimensional coordinates of the motion trajectory of the robot's end effector from the calligraphy style features. Then, the robot is controlled in real time to copy the photographed calligraphy according to the three-dimensional coordinates by using an inverse kinematic solution.
  • Different calligraphy styles can be flexibly displayed, and the copying method is not limited to the content taught in advance by the robot, so it has higher practical value.
  • the real-time copying method copies in the order in which the writing images are acquired, thereby accurately copying the stroke order of the calligraphy and avoiding displaying the wrong calligraphy writing method.
  • Fig. 6 is a functional module diagram of a calligraphy copying device for an intelligent robot according to an embodiment of the present invention.
  • the calligraphy copying device 20 for an intelligent robot includes a shooting module 21 , a preprocessing module 22 , an extraction module 23 and a control module 24 .
  • the shooting module 21 is used to shoot the calligraphy writing process in real time using a preset camera to obtain a writing image
  • a preprocessing module 22 is used to use a pre-trained image recognition model to sequentially identify and segment the calligraphy writing area in each frame of the writing image in the order of acquisition to obtain a target feature image;
  • An extraction module 23 is used to extract calligraphy style features from each frame of the target feature image in turn, and obtain the three-dimensional coordinates of the motion trajectory of the robot end effector according to the calligraphy style feature analysis;
  • the control module 24 is used to determine the motion control parameters of the robot arm according to each three-dimensional coordinate in turn based on the inverse kinematics solution, and control the robot to copy calligraphy according to the motion control parameters.
  • the preprocessing module 22 uses a pre-trained image recognition model to sequentially identify and segment the areas of calligraphy in each frame of the writing image in the order of acquisition to obtain the target feature image, specifically including: inputting the writing image into a pre-trained image recognition model to identify a first area including the calligraphy and a second area including the hand and the pen in the writing image; performing image segmentation on the first area and the second area in the writing image, and then performing mask superposition processing on the second area to obtain a first feature image; based on the image binarization method, eliminating the second area in the first feature image, retaining the first area, and obtaining the target feature image.
  • a pre-trained image recognition model to sequentially identify and segment the areas of calligraphy in each frame of the writing image in the order of acquisition to obtain the target feature image, specifically including: inputting the writing image into a pre-trained image recognition model to identify a first area including the calligraphy and a second area including the hand and the pen in the writing image; performing image segmentation on
  • the preprocessing module 22 before the preprocessing module 22 performs the operation of inputting the writing image into a pre-trained image recognition model, it is further used to: filter shadow noise data in the writing image based on a first preset method.
  • the preprocessing module 22 uses a pre-trained image recognition model to sequentially identify and segment the areas of calligraphy in each frame of writing image in the order of acquisition to obtain a target feature image, or performs an image binarization-based method to eliminate the second area in the first feature image and retain the first area to obtain the target feature image. It is also used for: when the text in the target feature image is tilted, adjusting the direction of the tilted text in the target feature image to a preset direction based on an affine transformation.
  • the extraction module 23 performs operations of extracting calligraphy style features from each frame of the target feature image in turn, and obtaining the three-dimensional coordinates of the motion trajectory of the robot end effector based on the calligraphy style feature analysis, specifically including: performing frame difference method on each two adjacent frames of the target feature image in turn to obtain the writing trajectory image between each two adjacent frames of the target feature image; obtaining the coordinates of the center point of the writing trajectory image to obtain the coordinate values in the X-axis and Y-axis directions; calculating the area of the writing trajectory in the writing trajectory image; based on a preset mapping method, obtaining the coordinate value of the robot end effector in the Z-axis direction according to the area mapping; and obtaining the three-dimensional coordinates according to the coordinate values in the X-axis and Y-axis directions and the coordinate value in the Z-axis direction.
  • the extraction module 23 After the extraction module 23 performs the operation of obtaining the writing track image between each two adjacent frames of target feature images, it is further used to: filter the salt and pepper noise data in the writing track image based on a second preset method.
  • each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other.
  • the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.
  • Figure 7 is a schematic diagram of the structure of a computer device according to an embodiment of the present invention.
  • the computer device 30 includes a processor 31 and a memory 32 coupled to the processor 31, wherein the memory 32 stores program instructions, and when the program instructions are executed by the processor 31, the processor 31 executes the steps of the calligraphy copying method for an intelligent robot described in any of the above embodiments.
  • the processor 31 may also be referred to as a CPU (Central Processing Unit).
  • the processor 31 may be an integrated circuit chip having the ability to process signals.
  • the processor 31 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.
  • DSP digital signal processor
  • ASIC application-specific integrated circuit
  • FPGA field programmable gate array
  • the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.
  • the storage medium of the embodiment of the present invention stores program instructions 41 that can implement the above-mentioned calligraphy copying method for an intelligent robot, wherein the program instructions 41 can be stored in the above-mentioned storage medium in the form of a software product, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application.
  • a computer device which can be a personal computer, a server, or a network device, etc.
  • processor processor
  • the aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer device such as a computer, a server, a mobile phone, and a tablet.
  • program codes such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, or a computer device such as a computer, a server, a mobile phone, and a tablet.
  • the disclosed computer equipment, devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.
  • the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
  • each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.
  • the above integrated unit may be implemented in the form of hardware or in the form of software functional units. The above is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the specification and drawings of this application, or directly or indirectly used in other related technical fields, is also included in the patent protection scope of the present application.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Robotics (AREA)
  • Mechanical Engineering (AREA)
  • Geometry (AREA)
  • Biomedical Technology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Data Mining & Analysis (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Image Analysis (AREA)

Abstract

本发明公开了一种用于智能机器人的书法临摹方法、装置、设备及存储介质，其中方法包括：利用预先设置的摄像头实时拍摄书法的书写过程，得到书写图像；利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像；依次从每帧目标特征图像中提取得到书法风格特征，并根据书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标；基于运动学逆解，依次根据每个三维坐标确定机器人机械臂的运动控制参数，并根据运动控制参数控制机器人临摹书法。本发明通过实时拍摄书法的书写图像，再从书写图像中提取书法特征，并根据书法特征控制机器人实时临摹书法。

Description

用于智能机器人的书法临摹方法、装置、设备及存储介质 技术领域
本申请涉及智能机器人技术领域，特别是涉及一种用于智能机器人的书法临摹方法、装置、设备及存储介质。
 背景技术
中国书法已有几千年的历史，无数书法名家留下了大量墨宝。书法不光是中华民族的一种文化更是中华民族历史的一种承载方式。如今中华民族的优秀文化受到越来越受的关注，教育部计划将书法纳入到中小学教育以及中考中。然而中国书法千变万化，普通人需要花费很长时间进行练习才能达到较好的是书写水平。
随着科学技术的发展机器人开始逐渐走进人们的生活中，在机器人学中很多工作都是关于机器人的模仿学，并且随着机器人的小型化，机器人已经被尝试进行书法的书写和示教领域。如果拥有一台小型书法机器人对书法进行模仿演示教学，可以更加的直观并且很大程度上提高小朋友的书写兴趣。
 技术问题
然而目前书法机器人，大多数是进行的提前示教，手动录入书法的轨迹，然后由机器人将提前录入的书法临摹出来，其临摹的模式固定，书法的风格完全依赖于提前录入的书法，不能实时临摹书法的书写过程，灵活性差、实用价值低。
 技术解决方案
有鉴于此，本申请提供一种用于智能机器人的书法临摹方法、装置、设备及存储介质，以解决现有书法机器人不能实时临摹书法、灵活性差的问题。
为解决上述技术问题，本申请采用的一个技术方案是：提供一种用于智能机器人的书法临摹方法，其包括：利用预先设置的摄像头实时拍摄书法的书写过程，得到书写图像；利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像；依次从每帧目标特征图像中提取得到书法风格特征，并根据书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标；基于运动学逆解，依次根据每个三维坐标确定机器人机械臂的运动控制参数，并根据运动控制参数控制机器人临摹书法。
作为本申请的进一步改进，利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像，包括：将书写图像输入至预先训练好的图像识别模型，以识别书写图像中包括书写书法的第一区域、以及包括手部和笔的第二区域；对书写图像中的第一区域和第二区域进行图像分割，再对第二区域进行掩膜叠加处理，得到第一特征图像；基于图像二值化的方式，剔除第一特征图像中的第二区域，保留第一区域，得到目标特征图像。
作为本申请的进一步改进，将书写图像输入至预先训练好的图像识别模型之前，还包括：基于第一预设方式过滤书写图像中的阴影噪声数据。
作为本申请的进一步改进，得到目标特征图像之后，还包括：当目标特征图像中的文字倾斜时，基于仿射变换的方式将目标特征图像中倾斜文字的方向调整为预设方向。
作为本申请的进一步改进，依次从每帧目标特征图像中提取得到书法风格特征，并根据书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标，包括：依次对每相邻两帧目标特征图像进行帧差法，得到每相邻两帧目标特征图像之间的书写轨迹图像；获取书写轨迹图像的中心点的坐标，得到X轴和Y轴方向上的坐标值；计算书写轨迹图像中书写轨迹的面积；基于预设映射方式，根据面积映射得到机器人末端执行器在Z轴方向上的坐标值；根据X轴和Y轴方向上的坐标值、Z轴方向上的坐标值得到三维坐标。
作为本申请的进一步改进，得到每相邻两帧目标特征图像之间的书写轨迹图像之后，还包括：基于第二预设方式过滤书写轨迹图像中的椒盐噪声数据。
作为本申请的进一步改进，预设映射方式表示为：S=A×z，其中，S表示面积，z表示Z轴方向上的坐标值，A表示预设参数。
为解决上述技术问题，本申请采用的又一个技术方案是：提供一种用于智能机器人的书法临摹装置，其包括：拍摄模块，用于利用预先设置的摄像头实时拍摄书法的书写过程，得到书写图像；预处理模块，用于利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像；提取模块，用于依次从每帧目标特征图像中提取得到书法风格特征，并根据书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标；控制模块，用于基于运动学逆解，依次根据每个三维坐标确定机器人机械臂的运动控制参数，并根据运动控制参数控制机器人临摹书法。
为解决上述技术问题，本申请采用的再一个技术方案是：提供一种计算机设备，所述计算机设备包括处理器、与所述处理器耦接的存储器，所述存储器中存储有程序指令，所述程序指令被所述处理器执行时，使得所述处理器执行如上述任一项的用于智能机器人的书法临摹方法的步骤。
为解决上述技术问题，本申请采用的再一个技术方案是：提供一种存储介质，存储有能够实现上述任一项的用于智能机器人的书法临摹方法的程序指令。
 有益效果
本申请的有益效果是：本申请的用于智能机器人的书法临摹方法通过实时拍摄书法书写过程的书写图像，再从连续的多帧书写图像中提取得到书法风格特征，再从书法风格特征中提取得到机器人末端执行器的运动轨迹的三维坐标，然后利用运动学逆解的方式，根据三维坐标实时控制机器人临摹出拍摄的书法，其能够对不同书法风格进行灵活展示，临摹方式不局限于机器人提前示教的内容，实用价值更高，并且，该实时临摹的方式按照书写图像的获取顺序依次进行临摹，从而准确临摹书法的笔画顺序，避免展示错误的书法书写方式。
 附图说明
图1是本发明实施例的用于智能机器人的书法临摹方法的一流程示意图；
图2是本发明实施例过滤阴影噪声数据的示意图；
图3是本发明实施例利用图像识别模型提取目标图像特征的示意图；
图4是本发明实施例利用仿射变换方式调整倾斜字体的示意图；
图5是本发明实施例过滤椒盐噪声数据的示意图；
图6是本发明实施例的用于智能机器人的书法临摹装置的功能模块示意图；
图7是本发明实施例的计算机设备的结构示意图；
图8是本发明实施例的存储介质的结构示意图。
 本发明的实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅是本申请的一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
本申请中的术语“第一”、“第二”、“第三”仅用于描述目的，而不能理解为指示或暗示相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”、“第三”的特征可以明示或者隐含地包括至少一个该特征。本申请的描述中，“多个”的含义是至少两个，例如两个，三个等，除非另有明确具体的限定。本申请实施例中所有方向性指示（诸如上、下、左、右、前、后……）仅用于解释在某一特定姿态（如附图所示）下各部件之间的相对位置关系、运动情况等，如果该特定姿态发生改变时，则该方向性指示也相应地随之改变。此外，术语“包括”和“具有”以及它们任何变形，意图在于覆盖不排他的包含。例如包含了一系列步骤或单元的过程、方法、系统、产品或设备没有限定于已列出的步骤或单元，而是可选地还包括没有列出的步骤或单元，或可选地还包括对于这些过程、方法、产品或设备固有的其它步骤或单元。
在本文中提及“实施例”意味着，结合实施例描述的特定特征、结构或特性可以包含在本申请的至少一个实施例中。在说明书中的各个位置出现该短语并不一定均是指相同的实施例，也不是与其它实施例互斥的独立的或备选的实施例。本领域技术人员显式地和隐式地理解的是，本文所描述的实施例可以与其它实施例相结合。
图1是本发明实施例的用于智能机器人的书法临摹方法的流程示意图。需注意的是，若有实质上相同的结果，本发明的方法并不以图1所示的流程顺序为限。如图1所示，该用于智能机器人的书法临摹方法包括步骤：
步骤S101：利用预先设置的摄像头实时拍摄书法的书写过程，得到书写图像。
需要说明的是，本实施例的目的在于实时临摹书法，因此，需要通过预先设置好的摄像头来拍摄书法的书写过程。进一步的，本实施例中，该摄像头可安装在手持终端或固定终端上，如手机、平板电脑、笔记本电脑等，该手持终端或固定终端可通过有线网络或无线网络与机器人通信连接，从而使得该实时临摹书法的过程能够远程实现，例如，书法家可通过手机拍摄书法的书写过程，并实时将拍摄过程发送至机器人，机器人根据书写过程进行实时临摹。
步骤S102：利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像。
需要说明的是，摄像头在拍摄书法的书写过程时，其可能会拍摄到手部和笔，导致从书写图像中无法准确获取书法的特征，因此，本实施例中，在获取到书写图像后，将书写图像输入至预先训练好的图像识别模型，以识别图像中属于手部和笔的区域，以及属于书法书写的区域，再将书法书写的区域分割出来，得到包含书法的目标特征图像。其中，该图像识别模型基于深度神经网络实现，如卷积神经网络，通过利用预先标注的样本数据对图像识别模型进行迭代训练直至图像识别模型达到预设精度即可。
具体地，本实施例将实时拍摄的连续帧书写图像按照获取顺序依次输入至图像识别模型中进行处理，得到连续帧目标特征图像。
进一步的，步骤S102具体包括：
1、将书写图像输入至预先训练好的图像识别模型，以识别书写图像中包括书写书法的第一区域、以及包括手部和笔的第二区域。
具体地，将获取到的书写图像输入至图像识别模型，从而识别书写图像中的手部和笔对应的第二区域。
进一步的，将书写图像输入至预先训练好的图像识别模型的步骤之前，还包括：基于第一预设方式过滤书写图像中的阴影噪声数据。
具体地，在使用摄像机对书法进行图像采集的过程通常会产生一些噪声干扰，其会影响到后续对书法临摹的准确性，为了避免书写图像中的阴影噪声影响图像识别模型的识别精度，本实施例中，在将书写图像输入至图像识别模型进行识别之前，对书写图像中的阴影噪声数据进行过滤。其中，该第一预设方式优选为图像二值化的方式，通过图像二值化的方式将书写图像中的阴影噪声过滤掉，具体可参阅图2中过滤阴影噪声数据的示例。
2、对书写图像中的第一区域和第二区域进行图像分割，再对第二区域进行掩膜叠加处理，得到第一特征图像。
3、基于图像二值化的方式，剔除第一特征图像中的第二区域，保留第一区域，得到目标特征图像。
具体地，如图3所示，本实施例对第一区域和和第二区域进行实例分割，再对第二区域进行掩膜叠加处理以对第一区域和第二区域进行区分，再根据预先设定的阈值对第一特征图像进行图像二值化处理，从而将第二区域中的手部和笔剔除，得到包括第一区域的目标特征图像。
进一步的，在得到目标特征图像之后，还包括：当目标特征图像中的文字倾斜时，基于仿射变换的方式将目标特征图像中倾斜文字的方向调整为预设方向。
具体地，摄像头在拍摄书法的书写过程时，其拍摄角度并不垂直，导致拍摄的文字呈现倾斜状，为了使得后续能够准确获取到书法的风格特征，如图4所示，本实施例在目标特征图像中的文字倾斜时，基于仿射变换的方式将目标特征图像中文字的方向调整为与摄像机拍摄角度垂直的角度。
步骤S103：依次从每帧目标特征图像中提取得到书法风格特征，并根据书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标。
具体地，该书法风格特征包括书法的笔画顺序、笔画的起始点等。在得到目标特征图像后，从该目标特征图像中提取得到书法风格特征，再根据书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标。
进一步的，步骤S103具体包括：
1、依次对每相邻两帧目标特征图像进行帧差法，得到每相邻两帧目标特征图像之间的书写轨迹图像。
具体地，本实施例采用帧差法对相邻两帧目标特征图像作差分运算，得到两帧目标特征图像之间书写轨迹的轮廓，并以该轮廓作为书写轨迹图像。
进一步的，得到每相邻两帧目标特征图像之间的书写轨迹图像的步骤之后，还包括：基于第二预设方式过滤书写轨迹图像中的椒盐噪声数据。
具体地，帧差法过程中会导致当前帧轨迹附近产生椒盐噪声，因此，本实施例利用第二预设方式过滤书写轨迹图像中的椒盐噪声数据。本实施例中，该第二预设方式为图像二值化，如图5所示，图5（A）为包括椒盐噪声数据的书写轨迹图像，图5（B）为过滤掉椒盐噪声数据的书写轨迹图像。
2、获取书写轨迹图像的中心点的坐标，得到X轴和Y轴方向上的坐标值。
具体地，在得到书写轨迹图像后，确认书写轨迹图像中书写轨迹的中心点，再获取该中心点对应的坐标。需要理解的是，相邻两帧图像之间的拍摄间隔极短，因此，相邻两帧图像之间的变化较小，本实施例构建环绕书写轨迹的最小圆环，再以该圆环的圆心作为该书写轨迹的中心点，再获取该中心点在预先构建的直角坐标系的坐标值。
3、计算书写轨迹图像中书写轨迹的面积。
具体地，本实施例在获取到书写轨迹图像后，计算该书写轨迹图像中书写轨迹的面积。
4、基于预设映射方式，根据面积映射得到机器人末端执行器在Z轴方向上的坐标值。
需要说明的是，控制机器人末端执行器进行运动的方式有多种，其中一种为根据用户输入的三维坐标进行运动学逆解，得到机器人机械臂的运动控制参数，再根据运动控制参数控制机器人运动。通常地，机器人书写字体的笔画粗细跟机器人机械臂在Z轴方向上的位移量相关，当机械臂带动毛笔向纸面方向移动的距离较大时，毛笔前端压迫在纸面的程度大，书写的笔画较粗；当机械臂带动毛笔向纸面方向移动的距离较小时，毛笔前端压迫在纸面的程度小，书写的笔画较细。基于此，本实施例预先构建书写轨迹的面积与机器人末端执行器在Z轴方向上的坐标值的映射关系，从而在获取到书写轨迹的面积后，根据面积映射得到机器人末端执行器在Z轴方向上的坐标值。
进一步的，预设映射方式表示为：S=A×z，其中，S表示面积，z表示Z轴方向上的坐标值，A表示预设参数，该预设参数可通过实验得到。
5、根据X轴和Y轴方向上的坐标值、Z轴方向上的坐标值得到三维坐标。
具体地，在得到X轴、Y轴、Z轴方向上的坐标值后，以X轴、Y轴、Z轴方向上的坐标值作为机器人末端执行器的三维坐标。
步骤S104：基于运动学逆解，依次根据每个三维坐标确定机器人机械臂的运动控制参数，并根据运动控制参数控制机器人临摹书法。
具体地，在得到机器人末端执行器后续运动轨迹的三维坐标后，根据三维坐标进行运动学逆解，得到机器人机械臂的运动控制参数，再根据运动控制参数控制机器人运动，以完成对书法的临摹。
本发明实施例的用于智能机器人的书法临摹方法通过实时拍摄书法书写过程的书写图像，再从连续的多帧书写图像中提取得到书法风格特征，再从书法风格特征中提取得到机器人末端执行器的运动轨迹的三维坐标，然后利用运动学逆解的方式，根据三维坐标实时控制机器人临摹出拍摄的书法，其能够对不同书法风格进行灵活展示，临摹方式不局限于机器人提前示教的内容，实用价值更高，并且，该实时临摹的方式按照书写图像的获取顺序依次进行临摹，从而准确临摹书法的笔画顺序，避免展示错误的书法书写方式。
图6是本发明实施例的用于智能机器人的书法临摹装置的功能模块示意图。如图6所示，该用于智能机器人的书法临摹装置20包括拍摄模块21、预处理模块22、提取模块23和控制模块24。
拍摄模块21，用于利用预先设置的摄像头实时拍摄书法的书写过程，得到书写图像；
预处理模块22，用于利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像；
提取模块23，用于依次从每帧目标特征图像中提取得到书法风格特征，并根据书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标；
控制模块24，用于基于运动学逆解，依次根据每个三维坐标确定机器人机械臂的运动控制参数，并根据运动控制参数控制机器人临摹书法。
可选地，预处理模块22执行利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像的操作，具体包括：将书写图像输入至预先训练好的图像识别模型，以识别书写图像中包括书写书法的第一区域、以及包括手部和笔的第二区域；对书写图像中的第一区域和第二区域进行图像分割，再对第二区域进行掩膜叠加处理，得到第一特征图像；基于图像二值化的方式，剔除第一特征图像中的第二区域，保留第一区域，得到目标特征图像。
可选地，预处理模块22执行将书写图像输入至预先训练好的图像识别模型的操作之前，还用于：基于第一预设方式过滤书写图像中的阴影噪声数据。
可选地，预处理模块22执行利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像的操作，或执行基于图像二值化的方式，剔除第一特征图像中的第二区域，保留第一区域，得到目标特征图像的操作之后，还用于：当目标特征图像中的文字倾斜时，基于仿射变换的方式将目标特征图像中倾斜文字的方向调整为预设方向。
可选地，提取模块23执行依次从每帧目标特征图像中提取得到书法风格特征，并根据书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标的操作，具体包括：依次对每相邻两帧目标特征图像进行帧差法，得到每相邻两帧目标特征图像之间的书写轨迹图像；获取书写轨迹图像的中心点的坐标，得到X轴和Y轴方向上的坐标值；计算书写轨迹图像中书写轨迹的面积；基于预设映射方式，根据面积映射得到机器人末端执行器在Z轴方向上的坐标值；根据X轴和Y轴方向上的坐标值、Z轴方向上的坐标值得到三维坐标。
可选地，提取模块23执行得到每相邻两帧目标特征图像之间的书写轨迹图像的操作之后，还用于：基于第二预设方式过滤书写轨迹图像中的椒盐噪声数据。
可选地，预设映射方式表示为：S=A×z，其中，S表示面积，z表示Z轴方向上的坐标值，A表示预设参数。
关于上述实施例用于智能机器人的书法临摹装置中各模块实现技术方案的其他细节，可参见上述实施例中的用于智能机器人的书法临摹方法中的描述，此处不再赘述。
需要说明的是，本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。对于装置类实施例而言，由于其与方法实施例基本相似，所以描述的比较简单，相关之处参见方法实施例的部分说明即可。
请参阅图7，图7为本发明实施例的计算机设备的结构示意图。如图7所示，该计算机设备30包括处理器31及和处理器31耦接的存储器32，存储器32中存储有程序指令，程序指令被处理器31执行时，使得处理器31执行上述任一实施例所述的用于智能机器人的书法临摹方法步骤。
其中，处理器31还可以称为CPU（Central Processing Unit，中央处理单元）。处理器31可能是一种集成电路芯片，具有信号的处理能力。处理器31还可以是通用处理器、数字信号处理器（DSP）、专用集成电路（ASIC）、现场可编程门阵列（FPGA）或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
参阅图8，图8为本发明实施例的存储介质的结构示意图。本发明实施例的存储介质存储有能够实现上述用于智能机器人的书法临摹方法的程序指令41，其中，该程序指令41可以以软件产品的形式存储在上述存储介质中，包括若干指令用以使得一台计算机设备（可以是个人计算机，服务器，或者网络设备等）或处理器（processor）执行本申请各个实施方式所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器（ROM，Read-Only Memory）、随机存取存储器（RAM，Random Access Memory）、磁碟或者光盘等各种可以存储程序代码的介质，或者是计算机、服务器、手机、平板等计算机设备。
在本申请所提供的几个实施例中，应该理解到，所揭露的计算机设备，装置和方法，可以通过其它的方式实现。例如，以上所描述的装置实施例仅仅是示意性的，例如，单元的划分，仅仅为一种逻辑功能划分，实际实现时可以有另外的划分方式，例如多个单元或组件可以结合或者可以集成到另一个系统，或一些特征可以忽略，或不执行。另一点，所显示或讨论的相互之间的耦合或直接耦合或通信连接可以是通过一些接口，装置或单元的间接耦合或通信连接，可以是电性，机械或其它的形式。
另外，在本发明各个实施例中的各功能单元可以集成在一个处理单元中，也可以是各个单元单独物理存在，也可以两个或两个以上单元集成在一个单元中。上述集成的单元既可以采用硬件的形式实现，也可以采用软件功能单元的形式实现。以上仅为本申请的实施方式，并非因此限制本申请的专利范围，凡是利用本申请说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本申请的专利保护范围内。

Claims (10)

  1. 一种用于智能机器人的书法临摹方法，其特征在于，其包括：
    利用预先设置的摄像头实时拍摄书法的书写过程，得到书写图像；
    利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像；
    依次从每帧目标特征图像中提取得到书法风格特征，并根据所述书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标；
    基于运动学逆解，依次根据每个三维坐标确定所述机器人机械臂的运动控制参数，并根据所述运动控制参数控制所述机器人临摹书法。
  2. 根据权利要求1所述的用于智能机器人的书法临摹方法，其特征在于，所述利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像，包括：
    将所述书写图像输入至所述预先训练好的图像识别模型，以识别所述书写图像中包括书写书法的第一区域、以及包括手部和笔的第二区域；
    对所述书写图像中的所述第一区域和所述第二区域进行图像分割，再对所述第二区域进行掩膜叠加处理，得到第一特征图像；
    基于图像二值化的方式，剔除所述第一特征图像中的所述第二区域，保留所述第一区域，得到目标特征图像。
  3. 根据权利要求2所述的用于智能机器人的书法临摹方法，其特征在于，所述将所述书写图像输入至所述预先训练好的图像识别模型之前，还包括：
    基于第一预设方式过滤所述书写图像中的阴影噪声数据。
  4. 根据权利要求1或2所述的用于智能机器人的书法临摹方法，其特征在于，所述得到目标特征图像之后，还包括：
    当所述目标特征图像中的文字倾斜时，基于仿射变换的方式将所述目标特征图像中倾斜文字的方向调整为预设方向。
  5. 根据权利要求1所述的用于智能机器人的书法临摹方法，其特征在于，所述依次从每帧目标特征图像中提取得到书法风格特征，并根据所述书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标，包括：
    依次对每相邻两帧目标特征图像进行帧差法，得到每相邻两帧目标特征图像之间的书写轨迹图像；
    获取所述书写轨迹图像的中心点的坐标，得到X轴和Y轴方向上的坐标值；
    计算所述书写轨迹图像中书写轨迹的面积；
    基于预设映射方式，根据所述面积映射得到所述机器人末端执行器在Z轴方向上的坐标值；
    根据所述X轴和Y轴方向上的坐标值、所述Z轴方向上的坐标值得到所述三维坐标。
  6. 根据权利要求5所述的用于智能机器人的书法临摹方法，其特征在于，所述得到每相邻两帧目标特征图像之间的书写轨迹图像之后，还包括：
    基于第二预设方式过滤所述书写轨迹图像中的椒盐噪声数据。
  7. 根据权利要求5所述的用于智能机器人的书法临摹方法，其特征在于，所述预设映射方式表示为：S=A×z，其中，S表示所述面积，z表示所述Z轴方向上的坐标值，A表示预设参数。
  8. 一种用于智能机器人的书法临摹装置，其特征在于，其包括：
    拍摄模块，用于利用预先设置的摄像头实时拍摄书法的书写过程，得到书写图像；
    预处理模块，用于利用预先训练好的图像识别模型，按获取顺序依次识别并分割每帧书写图像中书写书法的区域，得到目标特征图像；
    提取模块，用于依次从每帧目标特征图像中提取得到书法风格特征，并根据所述书法风格特征分析得到机器人末端执行器的运动轨迹的三维坐标；
    控制模块，用于基于运动学逆解，依次根据每个三维坐标确定所述机器人机械臂的运动控制参数，并根据所述运动控制参数控制所述机器人临摹书法。
  9. 一种计算机设备，其特征在于，所述计算机设备包括处理器、与所述处理器耦接的存储器，所述存储器中存储有程序指令，所述程序指令被所述处理器执行时，使得所述处理器执行如权利要求1-7中任一项权利要求所述的用于智能机器人的书法临摹方法的步骤。
  10. 一种存储介质，其特征在于，存储有能够实现如权利要求1-7中任一项所述的用于智能机器人的书法临摹方法的程序指令。
PCT/CN2023/137196 2023-04-12 2023-12-07 用于智能机器人的书法临摹方法、装置、设备及存储介质 Ceased WO2024212554A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310439907.7A CN118799768A (zh) 2023-04-12 2023-04-12 用于智能机器人的书法临摹方法、装置、设备及存储介质
CN202310439907.7 2023-04-12

Publications (1)

Publication Number Publication Date
WO2024212554A1 true WO2024212554A1 (zh) 2024-10-17

Family

ID=93034524

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/137196 Ceased WO2024212554A1 (zh) 2023-04-12 2023-12-07 用于智能机器人的书法临摹方法、装置、设备及存储介质

Country Status (2)

Country Link
CN (1) CN118799768A (zh)
WO (1) WO2024212554A1 (zh)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108764070A (zh) * 2018-05-11 2018-11-06 西北大学 一种基于书写视频的笔画分割方法及书法临摹指导方法
CN109664300A (zh) * 2019-01-09 2019-04-23 湘潭大学 一种基于力觉学习的机器人多风格书法临摹方法
CN111152234A (zh) * 2019-12-27 2020-05-15 深圳市越疆科技有限公司 用于机器人的书法临摹方法、装置及机器人
CN111950514A (zh) * 2020-08-26 2020-11-17 重庆邮电大学 一种基于深度摄像头的空中手写识别系统及方法
US20210012540A1 (en) * 2017-05-05 2021-01-14 Boe Technology Group Co., Ltd. Calligraphy-painting device, calligraphy-painting apparatus, and auxiliary method for calligraphy-painting
CN113408418A (zh) * 2021-06-18 2021-09-17 西安电子科技大学 一种书法字体与文字内容同步识别方法及系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20210012540A1 (en) * 2017-05-05 2021-01-14 Boe Technology Group Co., Ltd. Calligraphy-painting device, calligraphy-painting apparatus, and auxiliary method for calligraphy-painting
CN108764070A (zh) * 2018-05-11 2018-11-06 西北大学 一种基于书写视频的笔画分割方法及书法临摹指导方法
CN109664300A (zh) * 2019-01-09 2019-04-23 湘潭大学 一种基于力觉学习的机器人多风格书法临摹方法
CN111152234A (zh) * 2019-12-27 2020-05-15 深圳市越疆科技有限公司 用于机器人的书法临摹方法、装置及机器人
CN111950514A (zh) * 2020-08-26 2020-11-17 重庆邮电大学 一种基于深度摄像头的空中手写识别系统及方法
CN113408418A (zh) * 2021-06-18 2021-09-17 西安电子科技大学 一种书法字体与文字内容同步识别方法及系统

Also Published As

Publication number Publication date
CN118799768A (zh) 2024-10-18

Similar Documents

Publication Publication Date Title
CN110532984B (zh) 关键点检测方法、手势识别方法、装置及系统
CN109255813B (zh) 一种面向人机协作的手持物体位姿实时检测方法
US11379996B2 (en) Deformable object tracking
Hackenberg et al. Lightweight palm and finger tracking for real-time 3D gesture control
CN109993073B (zh) 一种基于Leap Motion的复杂动态手势识别方法
CN100407798C (zh) 三维几何建模系统和方法
CN108776773B (zh) 一种基于深度图像的三维手势识别方法及交互系统
CN101794384B (zh) 基于人体轮廓图提取与分组运动图查询的投篮动作识别
CN115816460A (zh) 一种基于深度学习目标检测与图像分割的机械手抓取方法
CN106708270B (zh) 一种虚拟现实设备的显示方法、装置以及虚拟现实设备
CN112506340B (zh) 设备控制方法、装置、电子设备及存储介质
CN114012722A (zh) 一种基于深度学习和边缘检测的机械臂抓取目标方法
TW201322058A (zh) 手勢辨識系統及方法
CN115471561A (zh) 对象关键点定位方法、清洁机器人控制方法及相关设备
CN105930773A (zh) 动作识别方法及装置
CN109087337B (zh) 基于分层卷积特征的长时间目标跟踪方法及系统
CN105844258A (zh) 动作识别方法及装置
CN114029952A (zh) 机器人操作控制方法、装置和系统
CN108537156B (zh) 一种抗遮挡的手部关键节点追踪方法
CN110390281B (zh) 一种基于感知设备的手语识别系统及其工作方法
CN112965602A (zh) 一种基于手势的人机交互方法及设备
CN113504063B (zh) 一种基于多轴机械臂的立体空间触屏设备可视化测试方法
Kiyokawa et al. Efficient collection and automatic annotation of real-world object images by taking advantage of post-diminished multiple visual markers
WO2024212554A1 (zh) 用于智能机器人的书法临摹方法、装置、设备及存储介质
CN113807280A (zh) 一种基于Kinect的虚拟船舶机舱系统与方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23932825

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE