CN111666940A - Chat screenshot content processing method and device, electronic equipment and readable storage medium - Google Patents

Chat screenshot content processing method and device, electronic equipment and readable storage medium Download PDF

Info

Publication number
CN111666940A
CN111666940A CN202010506616.1A CN202010506616A CN111666940A CN 111666940 A CN111666940 A CN 111666940A CN 202010506616 A CN202010506616 A CN 202010506616A CN 111666940 A CN111666940 A CN 111666940A
Authority
CN
China
Prior art keywords
chat
information
content
screenshot
category
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010506616.1A
Other languages
Chinese (zh)
Other versions
CN111666940B (en
Inventor
叶唐陟
陈星�
李骈臻
田兴业
戴晓雪
吴棨贤
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Xiamen Meitu Technology Co Ltd
Original Assignee
Xiamen Meitu Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Xiamen Meitu Technology Co Ltd filed Critical Xiamen Meitu Technology Co Ltd
Priority to CN202010506616.1A priority Critical patent/CN111666940B/en
Publication of CN111666940A publication Critical patent/CN111666940A/en
Application granted granted Critical
Publication of CN111666940B publication Critical patent/CN111666940B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/60Type of objects
    • G06V20/62Text, e.g. of license plates, overlay texts or captions on TV images
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/24Classification techniques
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/451Execution arrangements for user interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q50/00Information and communication technology [ICT] specially adapted for implementation of business processes of specific business sectors, e.g. utilities or tourism
    • G06Q50/01Social networking
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04LTRANSMISSION OF DIGITAL INFORMATION, e.g. TELEGRAPHIC COMMUNICATION
    • H04L51/00User-to-user messaging in packet-switching networks, transmitted according to store-and-forward or real-time protocols, e.g. e-mail
    • H04L51/04Real-time or near real-time messaging, e.g. instant messaging [IM]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V30/00Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
    • G06V30/10Character recognition

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Business, Economics & Management (AREA)
  • General Engineering & Computer Science (AREA)
  • Computer Networks & Wireless Communication (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Signal Processing (AREA)
  • Human Computer Interaction (AREA)
  • Artificial Intelligence (AREA)
  • Computing Systems (AREA)
  • Evolutionary Computation (AREA)
  • Health & Medical Sciences (AREA)
  • Economics (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Resources & Organizations (AREA)
  • Marketing (AREA)
  • Primary Health Care (AREA)
  • Strategic Management (AREA)
  • Tourism & Hospitality (AREA)
  • General Business, Economics & Management (AREA)
  • Information Transfer Between Computers (AREA)

Abstract

The embodiment of the application provides a chat screenshot content processing method and device, electronic equipment and a readable storage medium, and relates to the technical field of image processing. The method comprises the steps of firstly obtaining a chat screenshot to be identified. And then, inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information in the chat screenshot to be identified and the position information of each piece of chat content information. And finally, arranging at least one chat content message according to the position information of each chat content message to obtain the restored chat content of the chat screenshot to be identified. Therefore, the context information can be associated according to the coordinate information of the chat content information, the sequence of the conversation can be obtained, and the accuracy of processing the chat screenshot content can be further improved.

Description

Chat screenshot content processing method and device, electronic equipment and readable storage medium
Technical Field
The application relates to the technical field of image processing, in particular to a method and a device for processing chat screenshot content, electronic equipment and a readable storage medium.
Background
Social Instant Messaging (Instant Messaging) is currently the most popular way of communicating on a network. The use time of people of instant messaging software represented by WeChat and QQ in China is far longer than that of other types of software. Taking QQ and WeChat as examples, most chat interfaces of chat software conform to the same style, the left side is a guest view angle, the right side is a main view angle, each message is associated with a head portrait, and the top is a conversation name.
Chat conversations that occur by users on instant messaging software are often shared in screenshots to social media. But the form of the screenshot is different from the text format and is difficult to retrieve.
The current general Optical Character Recognition (OCR) algorithm can recognize the characters in the screenshot, but cannot associate the context information to obtain the sequence of the dialog.
Disclosure of Invention
In view of the above, the present application provides a method and an apparatus for processing a chat screenshot content, an electronic device, and a readable storage medium to solve the above problem.
The embodiment of the application can be realized as follows:
in a first aspect, an embodiment of the present application provides a method for processing a chat screenshot content, where the method includes:
obtaining a chat screenshot to be identified;
inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information in the chat screenshot to be identified and position information of each piece of chat content information;
and arranging at least one chat content message according to the position information of each chat content message to obtain the restored chat content of the chat screenshot to be identified.
In an optional embodiment, the chat screenshots to be identified are multiple;
after the step of arranging at least one chat content information according to the location information of each chat content information, the method further includes:
and deleting repeated chat content information in the plurality of chat screenshots to be identified.
In an optional embodiment, each chat screenshot to be identified has a label for representing a chat order;
the step of deleting repeated chat content information in the plurality of chat screenshots to be identified comprises the following steps:
the method comprises the steps of obtaining last chat content information in a target chat screenshot to be identified to obtain first content information, wherein the target chat screenshot to be identified is any one of a plurality of chat screenshots;
determining a next chat screenshot to be identified of the target chat screenshot to be identified according to the label of the target chat screenshot to be identified;
all chat content information of the next chat screenshot to be identified is obtained, and a plurality of second content information is obtained;
and matching the first content information with each piece of second content information, and deleting the target second content information and all second content information before the target second content information if the target second content information identical to the first content information exists.
In an alternative embodiment, the content recognition model includes a target detection model and a character recognition model;
the step of inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information and position information of each piece of chat content information in the chat screenshot to be identified comprises the following steps:
performing target detection on the chat screenshot to be identified by using the target detection model to obtain at least one chat content image block, the category attribute of each chat content image block and the coordinate information of each chat content image block in a preset coordinate system;
based on the category attribute of each chat content image block, identifying each chat content image block by using the character identification model to obtain chat content information in each chat content image block;
taking the coordinate information and the category attribute of each chat content image block as the position information of each chat content image block;
and associating the position information of each chat content image block with the chat content information of each chat content image block in a one-to-one correspondence manner.
In an alternative embodiment, the content recognition model includes a target detection model and a character recognition model;
the step of inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information and position information of each piece of chat content information in the chat screenshot to be identified comprises the following steps:
performing target detection on the chat screenshot to be identified by using the target detection model to obtain at least one chat content image block, the category attribute of each chat content image block and the coordinate information of each chat content image block in a preset coordinate system;
based on the category attribute of each chat content image block, identifying each chat content image block by using the character identification model to obtain chat content information in each chat content image block;
according to the category attribute of each chat content image block, carrying out hierarchy numbering on all chat content image blocks;
taking the level number, the coordinate information and the category attribute of each chat content image block as the position information of each chat content image block;
and associating the position information of each chat content image block with the chat content information of each chat content image block in a one-to-one correspondence manner.
In an optional implementation manner, the step of arranging at least one piece of chat content information according to the location information of each piece of chat content information includes:
acquiring a hierarchy number, coordinate information and category attributes corresponding to each chat content information;
and arranging each chat content information according to the hierarchy number, the coordinate information and the category attribute corresponding to each chat content information.
In an optional implementation manner, the category attribute includes a message category, a nickname category, and a head portrait category, the message category includes a message character sub-category and a message expression sub-category, the nickname category includes a nickname character sub-category and a nickname expression sub-category, the character recognition model includes an expression recognition sub-model and a character recognition sub-model, and the chat content information includes Unicode encoding information and character information;
the step of identifying each chat content image block by using the character identification model based on the category attribute of each chat content image block to obtain the chat content information in each chat content image block comprises the following steps:
recognizing the chat content image blocks with the category attributes of the message expression sub-category by using the expression recognition submodels to obtain Unicode coding information of the chat content image blocks with the category attributes of the message expression sub-category;
recognizing the chat content image blocks with the category attributes of the nickname expression sub-categories by using the expression recognition sub-models to obtain Unicode coded information of the chat content image blocks with the category attributes of the nickname expression sub-categories;
recognizing the chat content image blocks with each category attribute being the message character sub-category by using the character recognition submodel to obtain character information of the chat content image blocks with each category attribute being the message character sub-category;
and identifying the chat content graphic blocks with each category attribute being the nickname character sub-category by utilizing the character identification submodel to obtain the character information of the chat content graphic blocks with each category attribute being the nickname character sub-category.
In an alternative embodiment, the location information comprises coordinate information;
the step of arranging at least one chat content information according to the position information of each chat content information comprises:
acquiring coordinate information of each chat content information;
and arranging each chatting content information according to the coordinate information of each chatting content information.
In an optional embodiment, the location information further includes category attributes, where the category attributes include a nickname category and an avatar category;
after the step of arranging at least one chat content information according to the location information of each chat content information, the method further includes:
responding to a sharing operation, and acquiring a chat screenshot to be shared, wherein the chat screenshot to be shared is obtained after the chat content in the chat screenshot to be identified is restored;
screening the chat content information with the category attributes of head portrait category and nickname category in the chat screenshot to be shared;
and sending the chat screenshot to be shared after the shielding processing to a target object.
In a second aspect, an embodiment of the present application provides a device for processing a chat screenshot content, where the device includes:
the obtaining module is used for obtaining the chat screenshot to be identified;
the content identification module is used for inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information in the chat screenshot to be identified and position information of each piece of chat content information;
and the arrangement module is used for arranging at least one chat content message according to the position information of each chat content message to obtain the restored chat content of the chat screenshot to be identified.
In a third aspect, an embodiment of the present application provides an electronic device, where the electronic device includes a processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, and when the electronic device runs, the processor and the memory communicate with each other through the bus, and the processor executes the machine-readable instructions to perform the steps of the chat screenshot content processing method according to any one of the foregoing embodiments.
In a fourth aspect, an embodiment of the present application provides a readable storage medium, where a computer program is stored in the readable storage medium, and when the computer program is executed, the method for processing a chat screenshot content according to any one of the foregoing embodiments is implemented.
The embodiment of the application provides a chat screenshot content processing method and device, electronic equipment and a readable storage medium. The method comprises the steps of firstly obtaining a chat screenshot to be identified. And then, inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information in the chat screenshot to be identified and the position information of each piece of chat content information. And finally, arranging at least one chat content message according to the position information of each chat content message to obtain the restored chat content of the chat screenshot to be identified. Therefore, the context information can be associated according to the coordinate information of the chat content information, the sequence of the conversation can be obtained, and the accuracy of processing the chat screenshot content can be further improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are required to be used in the embodiments will be briefly described below, it should be understood that the following drawings only illustrate some embodiments of the present application and therefore should not be considered as limiting the scope, and for those skilled in the art, other related drawings can be obtained from the drawings without inventive effort.
Fig. 1 is a block diagram of an electronic device according to an embodiment of the present disclosure.
Fig. 2 is a flowchart of a chat screenshot content processing method according to an embodiment of the present application.
Fig. 3 is a second flowchart of a chat screenshot content processing method according to the embodiment of the present application.
Fig. 4 is a schematic diagram of a target to-be-recognized chat screenshot provided in an embodiment of the present application.
Fig. 5 is a schematic diagram of a next chat screenshot to be recognized of the target chat screenshot to be recognized according to the embodiment of the present application.
Fig. 6 is a schematic diagram of a to-be-recognized chat screenshot including a plurality of chat content tiles according to an embodiment of the present application.
Fig. 7 is a functional block diagram of a chat screenshot content processing apparatus according to an embodiment of the present application.
Icon: 100-an electronic device; 110-a memory; 120-a processor; 130-chat screenshot content processing means; 131-an acquisition module; 132-a content identification module; 133-arrangement module.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are some embodiments of the present application, but not all embodiments. The components of the embodiments of the present application, generally described and illustrated in the figures herein, can be arranged and designed in a wide variety of different configurations.
Thus, the following detailed description of the embodiments of the present application, presented in the accompanying drawings, is not intended to limit the scope of the claimed application, but is merely representative of selected embodiments of the application. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
It should be noted that: like reference numbers and letters refer to like items in the following figures, and thus, once an item is defined in one figure, it need not be further defined and explained in subsequent figures.
In the description of the present application, it should be noted that if the terms "upper", "lower", "inside", "outside", etc. are used for indicating the orientation or positional relationship based on the orientation or positional relationship shown in the drawings or the orientation or positional relationship which the present invention product is usually put into use, it is only for convenience of describing the present application and simplifying the description, but it is not intended to indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation and be operated, and thus, should not be construed as limiting the present application.
Furthermore, the appearances of the terms "first," "second," and the like, if any, are used solely to distinguish one from another and are not to be construed as indicating or implying relative importance.
It should be noted that the features of the embodiments of the present application may be combined with each other without conflict.
As introduced by the background, social Instant Messaging (Instant Messaging) is currently the most popular way of communicating on a network. The use time of people of instant messaging software represented by WeChat and QQ in China is far longer than that of other types of software. Taking QQ and WeChat as examples, most chat interfaces of chat software conform to the same style, the left side is a guest view angle, the right side is a main view angle, each message is associated with a head portrait, and the top is a conversation name.
Chat conversations that occur by users on instant messaging software are often shared in screenshots to social media. But the form of the screenshot is different from the text format and is difficult to retrieve.
At present, a common OCR algorithm can identify characters in a screenshot, but cannot associate context information to obtain the sequence of conversations.
In view of this, embodiments of the present application provide a method and an apparatus for processing a chat screenshot content, an electronic device, and a readable storage medium. The method identifies the characters in the screenshot and the position information of the characters through a pre-established content identification model, and sequences the characters according to the position information to solve the problems.
Referring to fig. 1, fig. 1 is a block diagram of an electronic device 100 according to an embodiment of the present disclosure. The device may include a processor 120, a memory 110, a chat screenshot content processing apparatus 130 and a bus, where the memory 110 stores machine-readable instructions executable by the processor 120, when the electronic device 100 operates, the processor 120 and the memory 110 communicate with each other through the bus, and the processor 120 executes the machine-readable instructions and performs the steps of the chat screenshot content processing method.
The memory 110, the processor 120, and other components are electrically connected to each other, directly or indirectly, to enable transmission or interaction of signals.
For example, the components may be electrically connected to each other via one or more communication buses or signal lines. The chat screenshot content processing apparatus 130 includes at least one software function module that can be stored in the memory 110 in the form of software or firmware (firmware). Processor 120 is configured to execute executable modules stored in memory 110, such as software functional modules or computer programs included in chat screenshot content processing apparatus 130.
The Memory 110 may be, but is not limited to, a Random ACCess Memory (RAM), a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Read-Only Memory (EPROM), an electrically Erasable Read-Only Memory (EEPROM), and the like.
The processor 120 may be an integrated circuit chip having signal processing capabilities. The processor 120 may be a general-purpose processor, and includes a Central Processing Unit (CPU), a Network Processor (NP), and so on.
But may also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware components. The various methods, steps, and logic blocks disclosed in the embodiments of the present application may be implemented or performed. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.
In the embodiment of the present application, the memory 110 is used for storing a program, and the processor 120 is used for executing the program after receiving the execution instruction. The method defined by the process disclosed in any of the embodiments of the present application can be applied to the processor 120, or implemented by the processor 120.
In the embodiment of the present application, the electronic device 100 may be, but is not limited to, a smart phone, a personal computer, a tablet computer, or the like having a processing function.
It will be appreciated that the configuration shown in figure 1 is merely illustrative. Electronic device 100 may also have more or fewer components than shown in FIG. 1, or a different configuration than shown in FIG. 1. The components shown in fig. 1 may be implemented in hardware, software, or a combination thereof.
As a possible implementation manner, an embodiment of the present application provides a method for processing a chat screenshot content, please refer to fig. 2 in combination, and fig. 2 is a flowchart of the method for processing the chat screenshot content according to the embodiment of the present application. The following is described in detail with reference to the specific flow shown in fig. 2.
And step S1, obtaining the chat screenshot to be identified.
Step S2, inputting the chat screenshot to be identified into a pre-established content identification model, and obtaining at least one piece of chat content information in the chat screenshot to be identified and the position information of each piece of chat content information.
And step S3, arranging at least one chat content message according to the position information of each chat content message to obtain the restored chat content of the chat screenshot to be identified.
The pre-established content recognition model can be obtained by training with a neural network after training samples are marked manually. For example, the neural network may be a combination of any one of a yolo object detection model, a fast-RCNN network, an ssd (single shot multi box detector) object detection model, a centrnet object detection model, an FCOS object detection model, and an OCR model. The specific training process and method can refer to the prior art, and are not described herein.
The embodiment of the application provides a chat screenshot content processing method, which identifies characters and position information of the characters in a screenshot through a pre-established content identification model, sorts the characters according to the position information, and solves the problem that context information cannot be associated to obtain the sequence of conversations in the chat screenshot.
Further, as another possible implementation manner, please refer to fig. 3 in combination, when there are a plurality of chat screenshots to be identified, since the chat screenshots may include repeated chat contents, if each chat screenshot is identified according to the method shown in fig. 2, the problem of repeated redundancy of the obtained restored chat contents is often caused.
Therefore, after the step of arranging at least one chat content information according to the location information of each chat content information, the method further comprises the steps of:
and step S4, deleting repeated chat content information in the plurality of chat screenshots to be identified.
The repeated chat content information in the plurality of chat screenshots to be identified can be deleted in the following modes:
the method comprises the steps of firstly, obtaining the last chat content information in a target to-be-identified chat screenshot to obtain first content information, wherein the target to-be-identified chat screenshot is any one of a plurality of chat screenshots, and each to-be-identified chat screenshot is provided with a label for representing a chat sequence.
And then, determining the next chat screenshot to be identified of the target chat screenshot to be identified according to the label of the target chat screenshot to be identified.
And then, obtaining all the chat content information of the next chat screenshot to be identified to obtain a plurality of second content information.
And finally, matching the first content information with each piece of second content information, and deleting the target second content information and all second content information before the target second content information if the target second content information identical to the first content information exists.
For example, please refer to fig. 4 and 5 in combination, fig. 4 is a schematic diagram of a target to-be-identified chat screenshot, and fig. 5 is a schematic diagram of a chat screenshot to be identified next to the target to-be-identified chat screenshot. Fig. 4 a is the last chat content information, i.e. the first content information, in the target to-be-recognized chat screenshot. B1, B2, B3, B4, B5, B6 and B7 in fig. 5 are a plurality of second content information in the next chat screenshot to be recognized.
By the matching, it is known that the target second content information B2 identical to the first content information exists in fig. 5, the target second content information B2, and the second content information B1 before B2 are deleted.
Therefore, the problem of repeated redundancy of the restored chat content can be solved.
Further, in order to make the detection more accurate and avoid the situation of mistaken deletion when deleting the repeated content, the repeated chat content information in the chat screenshots to be identified can be deleted in the following manner.
Similarly, on the basis, the first content information is matched with each second content information, and if the target second content information identical to the first content information exists, the previous chat content information of the first content information and the previous chat content information of the target second content information are acquired.
Next, the previous chat content information of the first content information is used as new first content information, and the previous chat content information of the target second content information is used as new second content information.
And then, repeating the step of matching the first content information with each second content information until all target second content information which is repeated with the target chat screenshot to be identified in the next chat screenshot to be identified is found out, and deleting all the target second content information.
Therefore, under the condition of solving the problem of repeated redundancy of the restored chat content, the accuracy of deleting the repeated content is further improved, and the accuracy of processing the chat screenshot content is improved.
Further, the content recognition model includes a target detection model and a character recognition model.
As a possible implementation manner, at least one chat content information in the chat screenshot to be identified and the location information of each chat content information can be obtained through the following steps.
Firstly, target detection is carried out on the chat screenshot to be recognized by using a target detection model, and at least one chat content image block, the category attribute of each chat content image block and the coordinate information of each chat content image block in a preset coordinate system are obtained.
And then, based on the category attribute of each chat content image block, identifying each chat content image block by using a character identification model to obtain chat content information in each chat content image block.
Then, the coordinate information and the category attribute of each chat content image block are used as the position information of each chat content image block.
And finally, associating the position information of each chat content image block with the chat content information of each chat content image block in a one-to-one correspondence manner.
The target detection model can be obtained by training any one of a yolo target detection model, a master-RCNN network, an SSD (Single shot Multi Box Detector) target detection model, a CenterNet target detection model and an FCOS target detection model, and the character recognition model can be obtained by training an OCR model.
As one possible implementation scenario, as shown in fig. 6, the image blocks with dashed boxes in fig. 6 are a plurality of chat content image blocks obtained by performing target detection by using a target detection model. The category attribute of each image block of the chat content may be any one of a message category, a nickname category, and an avatar category. For example, the category attribute of chat content image block 1 in fig. 6 is a nickname category (hereinafter, chat content image block is simply referred to as image block). The category attribute of the image block 2 is the head portrait category, and the category attribute of the image block 3 is the message category.
The coordinate information is obtained based on a preset coordinate system, for each chat screenshot to be recognized, the original point of the preset coordinate system is located at the upper left end of the chat screenshot to be recognized, the upper right end of the chat screenshot to be recognized is an x-axis, and the y-axis is perpendicular to the x-axis based on the original point.
Meanwhile, the coordinate information of each chat content image block may be a coordinate of the upper left end of each chat content image block in a preset coordinate system, or a coordinate of the geometric center of each chat content image block in the preset coordinate system.
Based on the above embodiment, at least one piece of chat content information in the to-be-identified chat screenshot, and the coordinate information and the category attribute of each piece of chat content information can be obtained.
As another possible implementation manner, in order to make the obtained restored chat content of the chat screenshot to be recognized more accurate, the dialog and the avatar are associated, when there is a line feed in the chat content information, at least one piece of chat content information and the position information of each piece of chat content information in the chat screenshot to be recognized can be obtained by the following method:
firstly, target detection is carried out on the chat screenshot to be recognized by using a target detection model, and at least one chat content image block, the category attribute of each chat content image block and the coordinate information of each chat content image block in a preset coordinate system are obtained.
And then, based on the category attribute of each chat content image block, identifying each chat content image block by using a character identification model to obtain chat content information in each chat content image block.
And then, carrying out hierarchy numbering on all chat content image blocks according to the category attribute of each chat content image block.
Then, the level number, the coordinate information, and the category attribute of each chat content tile are used as the position information of each chat content tile.
And finally, associating the position information of each chat content image block with the chat content information of each chat content image block in a one-to-one correspondence manner.
The principle of the coordinate information of the target detection model and the character recognition model can refer to the related contents mentioned in the previous embodiment, and is not described herein again.
Referring to fig. 6, as a possible implementation scenario, after all the chat content tiles are hierarchically numbered according to the category attribute of each chat content tile, as shown in fig. 6.
Further, the message category comprises a message character sub-category and a message expression sub-category, the nickname category comprises a nickname character sub-category and a nickname expression sub-category, the character recognition model comprises an expression recognition sub-model and a character recognition sub-model, and the chat content information comprises uniform code (Unicode) coding information and character information.
As a possible implementation manner, the chat content information in each chat content image block can be obtained by:
firstly, recognizing the chat content image blocks with the category attributes of the message expression sub-category by using the expression recognition sub-model to obtain Unicode coding information of the chat content image blocks with the category attributes of the message expression sub-category.
And then, identifying the chat content image blocks with the category attributes of the nickname expression sub-categories by using the expression identification sub-models to obtain Unicode coded information of the chat content image blocks with the category attributes of the nickname expression sub-categories.
And then, recognizing the chat content image block with each category attribute being the message character sub-category by using the character recognition submodel to obtain character information of the chat content image block with each category attribute being the message character sub-category.
And finally, recognizing the chat content graphic blocks with each category attribute being the nickname character sub-category by utilizing the character recognition sub-model to obtain the character information of the chat content graphic blocks with each category attribute being the nickname character sub-category.
For example, referring to fig. 6, the category attribute of the chat content image block 1 in fig. 6 is a nickname category, which includes characters and emoji emoticons, that is, the chat content image block 1-2 is a nickname emoticon sub-category, and the chat content image block 1-1 is a nickname character sub-category. The category attribute of the chat content image block 5 is a message category, and the chat content image block also includes characters and emoji emoticons, that is, the chat content image block 5-1 is a message character subcategory, and the chat content image block 5-2 is a message emotion subcategory.
The emotion recognition submodel is used for respectively recognizing the chat content image blocks 1-1 and 5-2 to respectively obtain Unicode coding information of the two chat content image blocks so as to classify and recognize the emotions in the chat content image blocks.
And respectively identifying the chat content image blocks 1-2 and 5-1 by using the character identification submodel to obtain character information of the chat content image blocks so as to identify the character content of the chat content image blocks.
Further, as a possible implementation, the at least one chat content information may be arranged by:
first, coordinate information of each chat content information is acquired.
Next, each of the pieces of chat content information is arranged based on the coordinate information of each of the pieces of chat content information.
For example, as one possible implementation scenario, chat content information having a category attribute of a message category is screened out, and the chat content information is arranged in order from top to bottom according to the magnitude order of y coordinates in the coordinate information of each chat content information, thereby generating a conversation list. And for the chat content information in the same line, sorting the chat content information from left to right according to the size sequence of the x coordinate.
Screening out the chat content information with the category attribute as the head portrait category, and sequentially arranging the chat content information from top to bottom according to the magnitude sequence of the y coordinate in the coordinate information of each chat content information to generate a head portrait list.
Screening out the chat content information with the category attribute as the nickname category, and sequentially arranging the chat content information from top to bottom according to the magnitude sequence of the y coordinate in the coordinate information of each chat content information to generate a nickname list.
And performing associated sequencing on the conversation list, the head portrait list and the nickname list to restore the reading sequence of each chat content message to obtain the restored chat content of the chat screenshot to be identified.
It should be understood that, in other implementation scenarios, the order of some steps in the method for arranging each chat content information according to the coordinate information of each chat content information set forth in the embodiments of the present application may be interchanged according to actual needs, or some steps may be omitted or deleted.
For example, the steps of generating the avatar list, generating the nickname list, and generating the dialog list may be interchanged as necessary, and the order of generation is not limited herein.
In this way, the restored chat content of the chat screenshot to be identified can be obtained, but based on the above manner, each piece of chat content information is arranged according to the coordinate information of each piece of chat content information, and when the chat content information has a line feed or one category of the chat content information includes a plurality of subcategories, the obtained restored chat content of the chat screenshot to be identified may have a disordered arrangement order.
Therefore, in order to solve the above problem, as another possible implementation, at least one piece of chat content information may be further arranged by:
first, a hierarchy number, coordinate information, and category attributes corresponding to each chat content information are obtained.
And then, arranging each piece of chat content information according to the corresponding hierarchy number, coordinate information and category attribute of each piece of chat content information.
For example, as a possible implementation scenario, corresponding chat content information is sequentially obtained according to the hierarchy number, and all the chat content information is arranged according to the above-mentioned ways of generating the dialog list, the avatar list and the nickname list according to the coordinate information of the chat content, so as to obtain the restored chat content of the chat screenshot to be identified.
Further, on the basis of the chat screenshot content processing method provided by the embodiment of the application, the embodiment of the application also provides a method, and the method can help a user to share privacy such as nicknames and head portraits hidden by one key when the user shares the screenshot, and does not need to manually repair the screenshot.
Firstly, responding to a sharing operation, and acquiring a chat screenshot to be shared, wherein the chat screenshot to be shared is obtained after the chat content in the chat screenshot to be identified is restored.
And then, shielding the chat content information with the category attribute of the avatar category and the nickname category in the chat screenshot to be shared.
And finally, sending the shielded chat screenshot to be shared to a target object.
Therefore, the chat content in the chat screenshot to be identified is restored, the chat screenshot to be shared is obtained in response to the sharing operation, and the chat content information with the category attributes of the head portrait category and the nickname category in the chat screenshot to be shared is shielded, so that the privacy of the nickname, the head portrait and the like can be hidden when the user shares the chat screenshot, the user does not need to manually repair the image, and the user experience is improved.
Further, on the basis of the method for Processing the chat screenshot content provided in the embodiment of the present application, the restored chat content may be classified and labeled based on a Natural Language Processing (NLP) technology, so as to recommend the chat content to an interested user.
Further, on the basis of the method for processing the chat screenshot content provided by the embodiment of the application, the restored chat content obtained after processing can be searched out through the text keywords by using various search engines, so that the efficiency and accuracy of searching the chat content can be improved.
Based on the same inventive concept, please refer to fig. 7 in combination, an embodiment of the present application further provides a device 130 for processing a chat screenshot content corresponding to the method for processing a chat screenshot content, where the device includes:
the obtaining module 131 is configured to obtain a chat screenshot to be identified.
The content identification module 132 is configured to input the chat screenshot to be identified into a pre-established content identification model, so as to obtain at least one piece of chat content information in the chat screenshot to be identified and location information of each piece of chat content information.
The arranging module 133 is configured to arrange at least one piece of chat content information according to the location information of each piece of chat content information, so as to obtain a restored chat content of the chat screenshot to be identified.
Because the principle of the device in the embodiment of the present application for solving the problem is similar to the chat screenshot content processing method described above in the embodiment of the present application, the implementation principle of the device can be referred to the implementation principle of the method, and repeated details are not described again.
The embodiment also provides a readable storage medium, in which a computer program is stored, and when the computer program is executed, the method for processing the chat screenshot content is implemented.
In summary, the embodiment of the present application provides a method and an apparatus for processing a chat screenshot content, an electronic device, and a readable storage medium. The method comprises the steps of firstly obtaining a chat screenshot to be identified. And then, inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information in the chat screenshot to be identified and the position information of each piece of chat content information. And finally, arranging at least one chat content message according to the position information of each chat content message to obtain the restored chat content of the chat screenshot to be identified. Therefore, the context information can be associated according to the coordinate information of the chat content information, the sequence of the conversation can be obtained, and the accuracy of processing the chat screenshot content can be further improved.
In the embodiments provided in the present application, it should be understood that the disclosed apparatus and method may be implemented in other ways. The apparatus embodiments described above are merely illustrative, and for example, the flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods and computer program products according to embodiments of the present application. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of code, which comprises one or more executable instructions for implementing the specified logical function(s). It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems which perform the specified functions or acts, or combinations of special purpose hardware and computer instructions.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The above description is only for the specific embodiments of the present application, but the scope of the present application is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present application should be covered within the scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims (12)

1. A chat screenshot content processing method is characterized by comprising the following steps:
obtaining a chat screenshot to be identified;
inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information in the chat screenshot to be identified and position information of each piece of chat content information;
and arranging at least one chat content message according to the position information of each chat content message to obtain the restored chat content of the chat screenshot to be identified.
2. The method for processing the content of the chat screenshot according to claim 1, wherein the number of the chat screenshot to be recognized is plural;
after the step of arranging at least one chat content information according to the location information of each chat content information, the method further includes:
and deleting repeated chat content information in the plurality of chat screenshots to be identified.
3. The method for processing the contents of the chat screenshots, according to claim 2, each chat screenshot to be identified has a label for representing the chat sequence;
the step of deleting repeated chat content information in the plurality of chat screenshots to be identified comprises the following steps:
the method comprises the steps of obtaining last chat content information in a target chat screenshot to be identified to obtain first content information, wherein the target chat screenshot to be identified is any one of a plurality of chat screenshots;
determining a next chat screenshot to be identified of the target chat screenshot to be identified according to the label of the target chat screenshot to be identified;
all chat content information of the next chat screenshot to be identified is obtained, and a plurality of second content information is obtained;
and matching the first content information with each piece of second content information, and deleting the target second content information and all second content information before the target second content information if the target second content information identical to the first content information exists.
4. The method of processing chat screenshot content according to claim 1, wherein the content recognition model comprises a target detection model and a character recognition model;
the step of inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information and position information of each piece of chat content information in the chat screenshot to be identified comprises the following steps:
performing target detection on the chat screenshot to be identified by using the target detection model to obtain at least one chat content image block, the category attribute of each chat content image block and the coordinate information of each chat content image block in a preset coordinate system;
based on the category attribute of each chat content image block, identifying each chat content image block by using the character identification model to obtain chat content information in each chat content image block;
taking the coordinate information and the category attribute of each chat content image block as the position information of each chat content image block;
and associating the position information of each chat content image block with the chat content information of each chat content image block in a one-to-one correspondence manner.
5. The method of processing chat screenshot content according to claim 1, wherein the content recognition model comprises a target detection model and a character recognition model;
the step of inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information and position information of each piece of chat content information in the chat screenshot to be identified comprises the following steps:
performing target detection on the chat screenshot to be identified by using the target detection model to obtain at least one chat content image block, the category attribute of each chat content image block and the coordinate information of each chat content image block in a preset coordinate system;
based on the category attribute of each chat content image block, identifying each chat content image block by using the character identification model to obtain chat content information in each chat content image block;
according to the category attribute of each chat content image block, carrying out hierarchy numbering on all chat content image blocks;
taking the level number, the coordinate information and the category attribute of each chat content image block as the position information of each chat content image block;
and associating the position information of each chat content image block with the chat content information of each chat content image block in a one-to-one correspondence manner.
6. The method for processing the chat screenshot content according to claim 5, wherein the step of arranging at least one piece of chat content information according to the location information of each piece of chat content information comprises:
acquiring a hierarchy number, coordinate information and category attributes corresponding to each chat content information;
and arranging each chat content information according to the hierarchy number, the coordinate information and the category attribute corresponding to each chat content information.
7. The method for processing the chat screenshot content according to claim 4 or 5, wherein the category attributes include a message category, a nickname category and a head portrait category, the message category includes a message character sub-category and a message emotion sub-category, the nickname category includes a nickname character sub-category and a nickname emotion sub-category, the character recognition model includes an emotion recognition sub-model and a character recognition sub-model, and the chat content information includes Unicode coding information and character information;
the step of identifying each chat content image block by using the character identification model based on the category attribute of each chat content image block to obtain the chat content information in each chat content image block comprises the following steps:
recognizing the chat content image blocks with the category attributes of the message expression sub-category by using the expression recognition submodels to obtain Unicode coding information of the chat content image blocks with the category attributes of the message expression sub-category;
recognizing the chat content image blocks with the category attributes of the nickname expression sub-categories by using the expression recognition sub-models to obtain Unicode coded information of the chat content image blocks with the category attributes of the nickname expression sub-categories;
recognizing the chat content image blocks with each category attribute being the message character sub-category by using the character recognition submodel to obtain character information of the chat content image blocks with each category attribute being the message character sub-category;
and identifying the chat content graphic blocks with each category attribute being the nickname character sub-category by utilizing the character identification submodel to obtain the character information of the chat content graphic blocks with each category attribute being the nickname character sub-category.
8. The method of processing chat screenshot content according to claim 1 or above, wherein the location information comprises coordinate information;
the step of arranging at least one chat content information according to the position information of each chat content information comprises:
acquiring coordinate information of each chat content information;
and arranging each chatting content information according to the coordinate information of each chatting content information.
9. The method of processing chat screenshot content according to claim 1, wherein the location information further includes category attributes, the category attributes including a nickname category and an avatar category;
after the step of arranging at least one chat content information according to the location information of each chat content information, the method further includes:
responding to a sharing operation, and acquiring a chat screenshot to be shared, wherein the chat screenshot to be shared is obtained after the chat content in the chat screenshot to be identified is restored;
screening the chat content information with the category attributes of head portrait category and nickname category in the chat screenshot to be shared;
and sending the chat screenshot to be shared after the shielding processing to a target object.
10. A chat screenshot content processing apparatus, the apparatus comprising:
the obtaining module is used for obtaining the chat screenshot to be identified;
the content identification module is used for inputting the chat screenshot to be identified into a pre-established content identification model to obtain at least one piece of chat content information in the chat screenshot to be identified and position information of each piece of chat content information;
and the arrangement module is used for arranging at least one chat content message according to the position information of each chat content message to obtain the restored chat content of the chat screenshot to be identified.
11. An electronic device, comprising a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, when the electronic device is running, the processor and the memory communicate with each other via the bus, and the processor executes the machine-readable instructions to perform the steps of the method for processing contents of a chat screenshot according to any one of claims 1 to 9.
12. A readable storage medium, wherein a computer program is stored in the readable storage medium, and when the computer program is executed, the method for processing the content of the chat screenshot in any one of claims 1-9 is implemented.
CN202010506616.1A 2020-06-05 2020-06-05 Chat screenshot content processing method and device, electronic equipment and readable storage medium Active CN111666940B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010506616.1A CN111666940B (en) 2020-06-05 2020-06-05 Chat screenshot content processing method and device, electronic equipment and readable storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010506616.1A CN111666940B (en) 2020-06-05 2020-06-05 Chat screenshot content processing method and device, electronic equipment and readable storage medium

Publications (2)

Publication Number Publication Date
CN111666940A true CN111666940A (en) 2020-09-15
CN111666940B CN111666940B (en) 2024-01-16

Family

ID=72386816

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010506616.1A Active CN111666940B (en) 2020-06-05 2020-06-05 Chat screenshot content processing method and device, electronic equipment and readable storage medium

Country Status (1)

Country Link
CN (1) CN111666940B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113343769A (en) * 2021-05-11 2021-09-03 广州市申迪计算机系统有限公司 Method and device for automatically receiving text messages
CN113438526A (en) * 2021-06-25 2021-09-24 维沃移动通信有限公司 Screen content sharing method, screen content display device, screen content equipment and storage medium
CN113905001A (en) * 2021-09-15 2022-01-07 深圳市六度人和科技有限公司 Automatic reply method and device, computer equipment and readable storage medium
CN115022268A (en) * 2022-06-24 2022-09-06 深圳市六度人和科技有限公司 Session identification method and device, readable storage medium and computer equipment
CN116522400A (en) * 2023-07-03 2023-08-01 荣耀终端有限公司 Image processing method and terminal equipment

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030038884A1 (en) * 2000-02-25 2003-02-27 Nobuyuki Matsushita Method and apparatus for producing communication data, method and apparatus for reproducing communication data, and program storage medium
CN104715233A (en) * 2014-12-30 2015-06-17 上海孩子国科教设备有限公司 Character conversion method and system
CN107948048A (en) * 2017-10-31 2018-04-20 阿里巴巴集团控股有限公司 Forward the method, apparatus and electronic equipment of chat message
CN109032726A (en) * 2018-06-29 2018-12-18 北京明朝万达科技股份有限公司 The acquisition methods and device of chat message
CN109726712A (en) * 2018-11-13 2019-05-07 平安科技(深圳)有限公司 Character recognition method, device and storage medium, server
CN109919014A (en) * 2019-01-28 2019-06-21 平安科技(深圳)有限公司 OCR recognition methods and its electronic equipment

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030038884A1 (en) * 2000-02-25 2003-02-27 Nobuyuki Matsushita Method and apparatus for producing communication data, method and apparatus for reproducing communication data, and program storage medium
CN104715233A (en) * 2014-12-30 2015-06-17 上海孩子国科教设备有限公司 Character conversion method and system
CN107948048A (en) * 2017-10-31 2018-04-20 阿里巴巴集团控股有限公司 Forward the method, apparatus and electronic equipment of chat message
CN109032726A (en) * 2018-06-29 2018-12-18 北京明朝万达科技股份有限公司 The acquisition methods and device of chat message
CN109726712A (en) * 2018-11-13 2019-05-07 平安科技(深圳)有限公司 Character recognition method, device and storage medium, server
CN109919014A (en) * 2019-01-28 2019-06-21 平安科技(深圳)有限公司 OCR recognition methods and its electronic equipment

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113343769A (en) * 2021-05-11 2021-09-03 广州市申迪计算机系统有限公司 Method and device for automatically receiving text messages
CN113438526A (en) * 2021-06-25 2021-09-24 维沃移动通信有限公司 Screen content sharing method, screen content display device, screen content equipment and storage medium
CN113905001A (en) * 2021-09-15 2022-01-07 深圳市六度人和科技有限公司 Automatic reply method and device, computer equipment and readable storage medium
CN113905001B (en) * 2021-09-15 2023-06-06 深圳市六度人和科技有限公司 Automatic reply method, device, computer equipment and readable storage medium
CN115022268A (en) * 2022-06-24 2022-09-06 深圳市六度人和科技有限公司 Session identification method and device, readable storage medium and computer equipment
CN115022268B (en) * 2022-06-24 2023-05-12 深圳市六度人和科技有限公司 Session identification method and device, readable storage medium and computer equipment
CN116522400A (en) * 2023-07-03 2023-08-01 荣耀终端有限公司 Image processing method and terminal equipment
CN116522400B (en) * 2023-07-03 2024-05-14 荣耀终端有限公司 Image processing method and terminal equipment

Also Published As

Publication number Publication date
CN111666940B (en) 2024-01-16

Similar Documents

Publication Publication Date Title
CN111666940B (en) Chat screenshot content processing method and device, electronic equipment and readable storage medium
US11195007B2 (en) Classification of piping and instrumental diagram information using machine-learning
CN106446816B (en) Face recognition method and device
CN109685052A (en) Method for processing text images, device, electronic equipment and computer-readable medium
CN111460131A (en) Method, device and equipment for extracting official document abstract and computer readable storage medium
CN109074547B (en) Text message ordering based on message content
WO2018075191A1 (en) Techniques for scheduling language models and character recognition models for handwriting inputs
CN111104516B (en) Text classification method and device and electronic equipment
CN110597965B (en) Emotion polarity analysis method and device for article, electronic equipment and storage medium
CN110490237B (en) Data processing method and device, storage medium and electronic equipment
CN112650858A (en) Method and device for acquiring emergency assistance information, computer equipment and medium
CN113627439A (en) Text structuring method, processing device, electronic device and storage medium
CN113963147A (en) Key information extraction method and system based on semantic segmentation
CN113326363B (en) Searching method and device, prediction model training method and device and electronic equipment
CN109190615B (en) Shape-near word recognition determination method, device, computer device and storage medium
CN114328816A (en) Data processing method and device, electronic equipment and computer readable storage medium
CN112579781A (en) Text classification method and device, electronic equipment and medium
CN110647537A (en) Data searching method, device and storage medium
CN116185812A (en) Automatic testing method, device and medium for software system functions
CN115270711A (en) Electronic signature method, electronic signature device, electronic apparatus, and storage medium
CN113742559A (en) Keyword detection method and device, electronic equipment and storage medium
CN114492306A (en) Corpus labeling method and device, electronic equipment and storage medium
WO2019231640A1 (en) System and method for automatic language detection for handwritten text
US12033411B2 (en) Stroke based control of handwriting input
US20210350122A1 (en) Stroke based control of handwriting input

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant