CN112633270A - Skeleton screen generation method, device and equipment based on picture recognition and storage medium - Google Patents
Skeleton screen generation method, device and equipment based on picture recognition and storage medium Download PDFInfo
- Publication number
- CN112633270A CN112633270A CN202011594807.4A CN202011594807A CN112633270A CN 112633270 A CN112633270 A CN 112633270A CN 202011594807 A CN202011594807 A CN 202011594807A CN 112633270 A CN112633270 A CN 112633270A
- Authority
- CN
- China
- Prior art keywords
- skeleton screen
- page
- coordinate information
- skeleton
- region
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 57
- 238000004590 computer program Methods 0.000 claims description 10
- 238000007781 pre-processing Methods 0.000 claims description 9
- 238000009877 rendering Methods 0.000 claims description 8
- 239000000126 substance Substances 0.000 claims 1
- 238000011161 development Methods 0.000 abstract description 27
- 230000008569 process Effects 0.000 abstract description 17
- 238000012423 maintenance Methods 0.000 abstract description 9
- 238000005516 engineering process Methods 0.000 abstract description 8
- 238000013473 artificial intelligence Methods 0.000 abstract description 2
- 238000012545 processing Methods 0.000 description 13
- 230000006870 function Effects 0.000 description 11
- 238000010586 diagram Methods 0.000 description 8
- 238000013461 design Methods 0.000 description 6
- 239000000243 solution Substances 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 238000012937 correction Methods 0.000 description 3
- 238000002347 injection Methods 0.000 description 3
- 239000007924 injection Substances 0.000 description 3
- 230000009467 reduction Effects 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 238000004891 communication Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 239000004973 liquid crystal related substance Substances 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- VYZAMTAEIAYCRO-UHFFFAOYSA-N Chromium Chemical compound [Cr] VYZAMTAEIAYCRO-UHFFFAOYSA-N 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 230000003044 adaptive effect Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000001035 drying Methods 0.000 description 1
- 230000008451 emotion Effects 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000003032 molecular docking Methods 0.000 description 1
- 230000007935 neutral effect Effects 0.000 description 1
- 230000008447 perception Effects 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/22—Image preprocessing by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/20—Image preprocessing
- G06V10/24—Aligning, centring, orientation detection or correction of the image
- G06V10/243—Aligning, centring, orientation detection or correction of the image by compensating for image skew or non-uniform image deformations
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V30/00—Character recognition; Recognising digital ink; Document-oriented image-based pattern recognition
- G06V30/10—Character recognition
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Theoretical Computer Science (AREA)
- User Interface Of Digital Computer (AREA)
Abstract
The invention relates to the technical field of artificial intelligence, and provides a skeleton screen generation method, a skeleton screen generation device, skeleton screen generation equipment and a storage medium based on picture identification. The method comprises the following steps: acquiring a screenshot corresponding to a page loaded with real data; performing OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element; and automatically generating a skeleton screen according to the region coordinate information. According to the method, the characteristics of simple layout and single style of the skeleton screen are combined, the picture recognition technology is adopted to recognize the page loaded with the real data, the area coordinate information corresponding to the page elements is obtained, then the skeleton screen is automatically generated, the consistency of the structure of the skeleton screen and the structure of the page is ensured, the development process of the skeleton screen is simplified, and the development and maintenance cost is reduced; the framework screen code does not need to be written manually, and the code structure does not need to be analyzed and configured; the framework screen code structure is clear and reasonable, and the size is small; the time of the white screen is greatly reduced, the page loading experience is optimized, and the user experience is improved.
Description
Technical Field
The invention relates to the technical field of artificial intelligence, in particular to a skeleton screen generation method, a skeleton screen generation device, skeleton screen generation equipment and a storage medium based on picture identification.
Background
At present, App market competition is intense, and the user experience of an App product is crucial to the success or failure of the product; optimizing the page loading experience is an important detail for improving the user experience. The Loading of App page data is gradually converted from Loading and a progress bar into a skeleton screen Loading mode, so that the waiting experience of a user is improved.
The skeleton screen is that before the real data of the page is loaded, the rough structure of the page is displayed for the user, and the user can have rough expectation on the loaded content by generally adopting a light gray occupation bitmap; the method has the advantages that the attention of a user to the loading process can be relieved, the attention is transferred to the loaded content, the user feels that the content is gradually loaded and is about to be presented, the scorching emotion is reduced, the loading process becomes smooth subjectively, the waiting time of the user perception is short, the page is rendered after the requested data is returned, and the data content needing to be displayed is supplemented; the effect is more obvious under the condition of poor network environment or large loading data quantity.
The skeleton screen is characterized in that: the layout is simple, the style is single, and a page rough frame is drawn by using blocks which are light gray or neutral filling colors; but the realization process is not simple due to the simple page layout; the related links are more, for example, in the implementation process, the design can be completed only by participation of a designer and a developer, the designer gives a skeleton screen design draft, and a development engineer writes skeleton screen codes manually; meanwhile, due to different page interaction requirements, the skeleton screen can be changed frequently, so that the skeleton screen is a great problem for maintenance of the skeleton screen and needs a part of manpower input.
At present, the main skeleton screen generation schemes include the following:
firstly, a designer provides a skeleton screen design draft, and a developer finishes writing of related codes of a skeleton screen and then manually puts the related codes into an html file. According to the scheme, designers need to participate, developers need to write skeleton screen codes manually, html files need to be injected manually, the flow is long, the flexibility is not high, and the change cost is high; meanwhile, due to the independent design, the situation that the actual content of the skeleton screen is inconsistent with the actual content of the page is easy to occur.
Firstly, a designer provides a skeleton screen design draft, developers finish writing skeleton screen components, and the skeleton screen components are analyzed and automatically injected into html files through an ssr render plug-in. Although the scheme realizes automatic injection of the skeleton screen code into the html file, the situations of long period, poor flexibility and inconsistency of the skeleton screen and the page actual content are still easy to occur.
Thirdly, a design draft is not needed, a webpage needing to generate a skeleton screen is opened by controlling a head less Chrome through Puppeneer at a server, elements in the webpage are deleted or added, and existing elements are subjected to pattern stacking, so that under the condition that the layout of the webpage is not changed, pictures and characters are hidden, and are displayed as gray blocks through pattern covering; the modified code is then extracted for use as a skeleton screen. Although the scheme can automatically generate the skeleton screen without the participation of a designer, certain requirements are made on the page structure, and if the expected effect is achieved, a plurality of configurations are required, such as: fine-grained configuration, designated node configuration and adaptive unit requirements of the framework screen are carried out through DOM attributes, and the use threshold is high; meanwhile, the generated skeleton screen code is complex and large in size; puppeteeer has about 80M, is easy to install and fail, can be used only in an intranet office environment, and has use limitation.
In summary, how to simplify the framework screen development process, ensure automatic generation, ensure consistency with a real page structure, make the framework screen code simple, simplify the usage, have no invasiveness to the page structure, and reduce development and maintenance costs is a main technical problem to be solved in the field at present.
Disclosure of Invention
The invention aims to provide a method, a device, equipment and a storage medium for generating a skeleton screen based on picture recognition, so as to solve the problems that the structure of the skeleton screen is inconsistent with the structure of a page, the generation flow of the skeleton screen is complex and the like.
The above purpose is realized by the following technical scheme:
according to an aspect of the present invention, the present invention provides a skeleton screen generation method based on picture recognition, where the skeleton screen generation method includes:
acquiring a screenshot corresponding to a page loaded with real data;
performing OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element;
and automatically generating a skeleton screen according to the region coordinate information.
Optionally, the step of performing OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element includes:
preprocessing the screenshot;
dividing the preprocessed screenshot into logic areas to obtain the logic areas; the logic area comprises a title bar, a left column, a right column, an upper column and a lower column;
finely dividing the logic area to obtain page elements;
and judging the shape outline of the page element according to the pixel characteristics to obtain the area coordinate information corresponding to the page element.
Optionally, the step of automatically generating the skeleton screen according to the region coordinate information includes: calculating the width and height of the area block corresponding to each page element according to the point coordinates in the area coordinate information of each page element; and automatically generating the skeleton screen according to the region coordinate information and the width and the height of the region block.
Optionally, the step of automatically generating the skeleton screen according to the region coordinate information and the width and height of the region block includes: positioning the left side and the top of the area block according to the coordinates of the upper left corner point; positioning the right side and the bottom of the area block according to the coordinates of the lower right corner point; and according to the width and the height of the region block, giving a region block style, and automatically generating a skeleton screen.
Optionally, the step of giving a region block style and automatically generating a skeleton screen includes: giving a region block style and converting the region block style into a skeleton screen code; automatically injecting the skeleton screen code into an html file; and rendering the page to obtain the skeleton screen.
Optionally, the skeleton screen code is automatically injected into the html file by externally referring to the Js file.
Optionally, the real data is real page data presented to the user; the page elements comprise text and pictures; the region coordinate information includes a page element type and a point coordinate of a page element.
According to another aspect of the present invention, the present invention provides a skeleton screen generating device based on picture recognition, including:
the acquisition unit is used for acquiring a screenshot corresponding to the page loaded with the real data;
the recognition unit is used for carrying out OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element;
and the generating unit is used for automatically generating the skeleton screen according to the region coordinate information.
According to still another aspect of the present invention, there is provided an electronic apparatus including: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the picture recognition based skeleton screen generation method.
According to still another aspect of the present invention, there is also provided a computer-readable storage medium, in which a computer program is stored, and the computer program, when executed by a processor, implements the steps of the method for generating a skeleton screen based on picture recognition.
Has the advantages that: according to the method, the device and the equipment for generating the skeleton screen based on the picture recognition and the storage medium, the picture of the page loaded with the real data is recognized by adopting an OCR picture recognition technology to obtain the region coordinate information of page elements such as the picture, the text and the like, and then the skeleton screen is automatically generated according to the region coordinate information, so that the consistency of the page structure and the skeleton screen structure is ensured, the development process of the skeleton screen is simplified, and the development and maintenance cost is reduced; the time of the white screen is greatly reduced, the page loading experience is optimized, and the user experience is improved.
Drawings
Other objects and results of the present invention will become more apparent and more readily appreciated as the same becomes better understood by reference to the following description taken in conjunction with the accompanying drawings. In the drawings:
FIG. 1 is a schematic flow chart diagram of a skeleton screen generation method based on picture recognition according to an embodiment of the present invention;
FIG. 2 is a schematic flow chart of obtaining region coordinate information according to an embodiment of the present invention;
FIG. 3 is a schematic flow chart diagram for automatically generating a skeleton screen in one embodiment of the present invention;
FIG. 4 is a schematic flow chart of automatically generating a skeleton screen in another embodiment of the present invention;
FIG. 5 is a diagram illustrating an integrated information list page loaded with real data according to an embodiment of the present invention;
FIG. 6 is a logic block diagram of a method for generating a skeleton screen based on picture recognition according to an embodiment of the present invention;
FIG. 7 is a schematic diagram of a skeleton screen of the integrated information obtained in the embodiment of the present invention;
FIG. 8 is a block diagram of a skeleton screen generating device based on picture recognition according to the present invention;
fig. 9 is a schematic structural diagram of the electronic device of the present invention.
Detailed Description
The technical solution in the embodiment of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiment of the present invention:
fig. 1 schematically shows a flowchart of a skeleton screen generation method based on picture recognition according to an embodiment of the present invention. As shown in fig. 1, the method for generating a skeleton screen based on picture recognition provided by the present invention includes the following steps:
step S100, acquiring a screenshot corresponding to a page loaded with real data; wherein the real data is real page data that is ultimately presented to the user for viewing.
S200, performing OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element; the page elements can include elements such as pictures, texts, videos and buttons; the area coordinate information may include contents of a page element type and a point coordinate of a page element.
And step S300, automatically generating a skeleton screen according to the region coordinate information.
According to the method, the characteristics of simple framework screen layout and single style are combined, the page loaded with real data is identified by adopting an OCR picture identification technology to obtain the region coordinate information of page elements such as texts, pictures and the like, and then the region block style is given according to the corresponding region coordinate information to automatically generate the framework screen, so that the consistency of the framework screen structure and the page structure is ensured, the development process of the framework screen is simplified, and the development and maintenance cost is reduced; the framework screen development efficiency is improved. The skeleton screen code structure generated by the skeleton screen generation method based on the picture recognition is very clear and reasonable and has small volume. By adopting the method for generating the skeleton screen based on the picture recognition, the white screen time is greatly reduced, the page loading experience is optimized, and the user experience is improved.
In step S100, in the step of obtaining the screenshot corresponding to the page loaded with the real data, the page loaded with the real data is obtained by rendering the real data through page loading after the developer completes the development of the page function, where the real data is the real page data finally presented to the user. The invention generates the skeleton screen by the real data page, namely, the page is obtained instantly, the consistency of the page structure and the skeleton screen structure can be ensured, and the skeleton screen can be updated with lower cost when the page is changed interactively.
Fig. 5 schematically shows a general information list page loaded with real data. Illustratively, at the front end, a developer completes function development of the comprehensive information list page, and then loads real data to render the page, so as to obtain the comprehensive information list page loaded with the real data as shown in the figure. Based on the page, the next screenshot, OCR processing, etc. may be performed.
The "function development of the list page" refers to complete page development, and includes, for example, page layout, interface docking, and the like. The page elements in a page may include: pictures, text, video, buttons, etc., the following embodiments of the present invention are merely exemplary of text and pictures as common page elements; taking the comprehensive information list page as an example, as shown in the figure, the upper part of the page is a title, each title comprises two rows of page elements, the left row can be a text element, the right row can be a picture element, and the text elements and the picture elements can be in one-to-one correspondence.
In step S200, in the step of performing OCR recognition on the screenshot to obtain the region coordinate information corresponding to the page element, the screenshot may be recognized by an OCR platform by using an OCR picture recognition technology. FIG. 6 schematically shows a logic block diagram of a skeleton screen generation method based on image recognition according to an embodiment of the present invention, as shown in FIG. 6, the skeleton screen generation method includes a front end and an OCR platform, the front end completes page function development, and renders a page and captures a picture by using real data; then uploading the screenshot to an OCR platform connected with a front-end interface, identifying the characters and the pictures in the screenshot by the OCR platform, generating coordinate information of the character and picture areas, and returning the coordinate information of each area to the front end; and the front end generates each region style according to the coordinate information of each region, such as the coordinates of the upper left corner and the lower right corner, namely, automatically generating the skeleton screen, and then performing subsequent operations such as rendering of the skeleton screen page. The OCR platform can comprise an image preprocessing module, a layout module, a character recognition module, a character editing module and the like. The whole process is short, and the generation process is flexible; the characteristics of simple layout and single style of the skeleton screen are perfectly matched with the region identification characteristics of the picture identification technology, and the skeleton screen is suitable for generating approximate outlines of the skeleton screen; and moreover, the skeleton screen finally obtained by OCR image recognition has concise skeleton screen codes, the skeleton screen codes are non-invasive to the page structure, and the generated skeleton screen codes are very clear and reasonable in structure and small in size.
The invention adopts OCR picture recognition technology to recognize the screenshot of the page loaded with real data, can quickly and accurately recognize the corresponding area coordinate information of page elements such as pictures, texts and the like from the screenshot, simplifies the whole development process, reduces the development and maintenance cost, greatly improves the development efficiency of the skeleton screen, and can be applied, popularized and implemented.
Fig. 2 schematically illustrates a flow of obtaining area coordinate information in an embodiment of the present invention, and as shown in fig. 2, the step of obtaining area coordinate information corresponding to a page element by performing OCR recognition on the screenshot may include:
and step 210, preprocessing the screenshot. The preprocessing may include graying processing, noise reduction processing, tilt correction, image cropping, and the like. The graying processing is to perform graying processing on the color screenshot to obtain a black-white picture, for example, only gray foreground information and white background information are displayed after processing. The noise reduction processing is to perform the drying processing according to the noise characteristics. The inclination correction is to correct the inclined screenshot. Image cropping is the cropping of the screenshot into the real required area to reduce unnecessary running consumption and improve the speed of OCR recognition.
Step 220, performing logic area division on the preprocessed screenshot to obtain a logic area, and performing fine division on the logic area to obtain page elements; the logic area comprises a title bar, a left column, a right column, an upper column and a lower column; the page elements may include: text, title, picture, table, line, etc.;
and step 230, judging the shape outline of the page element according to the pixel characteristics to obtain the area coordinate information corresponding to the page element. Wherein the region coordinate information may include: page element type, point coordinate data of the page element, etc. Wherein the page element type mainly indicates whether the page element is a picture or a text or other page element. The point coordinate data of the page element may be, for example, coordinates of each vertex of a rectangular frame, for example, the rectangular frame. It should be noted that, the area corresponding to the page element is taken as a rectangular frame for explanation, but of course, the area corresponding to the page element may be in other shapes, such as a circle.
Preferably, the area coordinate information may further include area position information of the page element, for example, whether the page element belongs to a header at the top end or not, such as "general information".
Illustratively, taking the outline of a page element as a rectangular frame as an example, the area coordinate information corresponding to the page element is obtained as follows: the text message text is "product notice", and the coordinate information of four vertexes of the rectangular box area:
x：498.525，y：1401.702；
x：492.745，y：1180.304；
x：562.105，y：1178.838；
x：567.885，y：1398.770。
FIG. 3 schematically illustrates a process for automatically generating a skeletal screen in one embodiment of the invention. As shown in fig. 3, the step of automatically generating the skeleton screen according to the region coordinate information may specifically include:
step S310, calculating the width and height of the area block corresponding to each page element according to the point coordinates in the area coordinate information of each page element. The point coordinates may be point coordinates of two end points on a certain diagonal line, for example, a rectangular frame. When the area of the page element is circular, the outline of the block area can be determined according to the circle center, the radius and the like.
And step S320, automatically generating a skeleton screen according to the region coordinate information and the width and the height of the region block. In consideration of page performance and rationality, the width and height of the area can be calculated according to the area coordinate information, and then the skeleton screen is generated.
FIG. 4 schematically illustrates a flow of automatically generating a skeletal screen, according to an embodiment. As shown in fig. 4, taking the outline of the page element as a rectangular frame as an example, the step S320 may include the following steps:
step S321, positioning the left side and the top of the area block according to the coordinates of the upper left corner point;
step S322, positioning the right and the bottom of the area block according to the coordinates of the lower right corner point;
and step S323, then, giving a region block style according to the width and the height of the region block, and automatically generating a skeleton screen. Specifically, according to the specification requirement of designing the H5 static page in the row, the skeleton screen is generated by using a rem unit and a layout scheme in the compatible row. The layout scheme refers to a scheme for multi-terminal adaptation of the H5 page, namely, the page layout is completed when the page function development completed by a developer is completed. The embodiment is explained by taking a rectangular frame as an example, the rectangular frame belongs to region positioning, and is not specific to positioning of a certain point, the page performance and the reasonability are considered, the region width and the height are obtained by calculation according to region coordinate information, then the region block pattern is finally given according to the region block width and the height and the region coordinate information, and further, the skeleton screen is automatically generated. The region block pattern may include color blocks, such as positions, sizes, background colors, and the like of gray color blocks, and it should be noted that the gray color blocks are taken as an example in the present invention, but not limited thereto, and may also be other color blocks.
In an optional embodiment, the step of giving the gray region block pattern and automatically generating the skeleton screen may include: giving a grey area block style and converting the grey area block style into a skeleton screen code; automatically injecting the skeleton screen code into an html file; and rendering the page, and obtaining the skeleton screen when analyzing the code. The skeleton screen code is also a color block code, which is a color block described in the present invention, for example, a gray area block code. Optionally, in the step of automatically injecting the skeleton screen code into the html file, the skeleton screen code may be automatically injected into the html file by specifically using a manner of externally referencing a Js file.
FIG. 7 schematically illustrates the structure of a skeleton screen for integrating information in the practice of the present invention; that is, the comprehensive information list page loaded with the real data shown in fig. 5 is adopted, screenshot is performed, OCR recognition is adopted, and finally the skeleton screen is obtained. As shown in fig. 7, the left side is a text gray area block corresponding to fig. 5, and the right side is a picture gray area block corresponding to fig. 5. According to the embodiment of the invention, the automatic generation of the skeleton screen is dynamically realized through the js code at the front end, so that the development process of the skeleton screen is simplified, the use mode of the skeleton screen is simplified, and the development and maintenance cost is reduced.
In the embodiment of the invention, the skeleton screen generation method based on the picture recognition combines the characteristics of simple skeleton screen layout and single style, adopts an OCR picture recognition technology to recognize the page loaded with the real data to obtain the region coordinate information of the text and the picture, then obtains the width and the height of the region block according to the region coordinate information of the text and the picture, and further automatically generates the skeleton screen, thereby ensuring the consistency of the structure of the skeleton screen and the structure of the page, simplifying the development process of the skeleton screen, and reducing the development and maintenance cost.
In the embodiment of the invention, the skeleton screen generation method based on the picture recognition greatly reduces the white screen time, optimizes the page loading experience and improves the user experience. The "white screen" time of the skeleton screen, which is the time until the real content (dynamically loaded) of the page reaches the browser, is the time of the web load. When a page is loaded in the same network environment, the white screen time of the prior art is about 990 milliseconds, but the white screen time of the invention can be reduced to 30 milliseconds, and the skeleton screen displays 970 milliseconds, so that the invention greatly reduces the white screen time and improves the user experience; in addition, under the environment with poor network, the technical effect of short white screen time of the invention is more obvious compared with the prior art. The invention is suitable for most single-page application projects at present, and the html file only needs to contain the simple skeleton screen code, so that the content is less and the loading is fast.
In the embodiment of the invention, the skeleton screen generation method based on the picture recognition does not need a designer to participate, does not need to develop a manual skeleton screen writing code, and shortens the development process and period. In the embodiment of the invention, the skeleton screen generation method based on the picture recognition does not involve the analysis of the page code structure, does not need to be configured, and has the advantages of simple and flexible use mode and wide application range.
Fig. 8 schematically shows a structure of a skeleton screen generating apparatus based on picture recognition according to an embodiment of the present invention. As shown in fig. 8, the present application provides a skeleton screen generating device based on picture recognition, where the skeleton screen generating device includes: an acquisition unit 100, a recognition unit 200, and a generation unit 300.
The obtaining unit 100 is configured to obtain a screenshot corresponding to a page loaded with real data.
The recognition unit 200 is configured to perform OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element.
The generating unit 300 is configured to automatically generate a skeleton screen according to the region coordinate information.
In an alternative embodiment, the identification unit 200 may include: the device comprises a preprocessing module, a dividing module and an area coordinate information acquisition module. The preprocessing module is used for executing the step of preprocessing the screenshot. Optionally, the preprocessing module may further include sub-modules such as graying processing, noise reduction processing, tilt correction, and image cropping. And the dividing module is used for executing the step of dividing the layout of the preprocessed screenshot to obtain the area of each page element. And the area coordinate information acquisition module is used for executing the step of acquiring the area coordinate information corresponding to the page element according to the page element area.
In an optional embodiment, the generating unit 300 may include: the device comprises an area block calculation module and a skeleton screen generation module. The area block calculation module is used for calculating the width and height of the area block according to the point coordinate in the area coordinate information of each page element. And the skeleton screen generating module is used for executing the step of automatically generating the skeleton screen according to the region coordinate information and the width and the height of the region block.
Optionally, the skeleton screen generating module may include: the transformation submodule comprises a transformation submodule, an injection submodule and a rendering submodule. The transformation module is used for executing the step of transforming the given region block pattern into the skeleton screen code. And the injection submodule is used for executing the step of automatically injecting the skeleton screen code into the html file. And the rendering submodule is used for executing page rendering to obtain a skeleton screen.
In an optional embodiment, the generating unit 300 may further include a positioning sub-module. The positioning sub-module may be configured to perform the step of positioning the left side and the top of the region block according to the coordinates of the upper left corner point; and the step for positioning the right and the bottom of the area block according to the coordinates of the lower right corner point.
In the embodiment of the application, the skeleton screen generation device based on image recognition adopts an OCR image recognition technology to recognize the page loaded with real data by combining the characteristics of simple skeleton screen layout and single style, so as to obtain page elements, and then automatically generates the skeleton screen according to corresponding region coordinate information, so that the consistency of the skeleton screen structure and the page structure is ensured, the development process of the skeleton screen is simplified, and the development and maintenance cost is reduced; the generated skeleton screen code structure is very clear and reasonable and has small volume; therefore, the time for white screen is greatly reduced, the page loading experience is optimized, and the user experience is improved; moreover, a designer does not need to participate, and a manually written skeleton screen code does not need to be developed, so that the development flow and period are shortened; the method has the advantages that the method does not involve analysis on the page code structure, does not need configuration, is simple and flexible in use mode and wide in application range, is suitable for most single-page application projects at present, only needs to contain the simple skeleton screen code in the html file, and is less in content and fast in loading.
Fig. 9 schematically shows the structure of the electronic device in the present invention. As shown in fig. 9, in the present invention, the electronic device 1 includes: memory 11, processor 10, a bus, and a computer program, such as skeleton screen generation program 12, stored in memory 11 and executable on the processor 10. When the processor 10 executes the computer program, the steps of the method for generating the skeleton screen based on picture recognition according to the present invention are implemented. For example, acquiring a screenshot corresponding to a page loaded with real data; performing OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element; and automatically generating a skeleton screen according to the region coordinate information.
The memory 11 includes at least one type of readable storage medium including flash memory, removable hard disks, multimedia cards, card-type memory (e.g., SD or DX memory, etc.), magnetic memory, magnetic disks, optical disks, etc. The memory may in some embodiments be an internal storage unit of the electronic device 1, such as a removable hard disk of the electronic device 1. The memory 11 may also be an external storage device of the electronic device 1 in other embodiments, such as a plug-in mobile hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like, which are provided on the electronic device 1. Further, the memory 11 may also include both an internal storage unit and an external storage device of the electronic device 1. The memory 11 may be used not only to store application software installed in the electronic device 1 and various types of data, such as codes of the skeleton screen generating program 12, but also to temporarily store data that has been output or is to be output.
The processor 10 may be composed of an integrated circuit in some embodiments, for example, a single packaged integrated circuit, or may be composed of a plurality of integrated circuits packaged with the same or different functions, including one or more Central Processing Units (CPUs), microprocessors, digital Processing chips, graphics processors, and combinations of various control chips. The processor 10 is a Control Unit (Control Unit) of the electronic device 1, connects various components of the electronic device 1 by using various interfaces and lines, and executes various functions and processes data of the electronic device 1 by running or executing programs or modules stored in the memory 11 and calling data stored in the memory 11.
The bus may be a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. The bus is arranged to enable connection communication between the memory 11 and at least one processor 10 or the like.
It will be appreciated by a person skilled in the art that the electronic device 1 with the above-described components does not constitute a limitation of the electronic device 1, and that fewer or more components may be included, or that certain components may be combined, or that different arrangements of components may be provided. For example, although not shown, the electronic device 1 may further include a power supply (such as a battery) for supplying power to each component, and preferably, the power supply may be logically connected to the at least one processor 10 through a power management device, so as to implement functions of charge management, discharge management, power consumption management, and the like through the power management device. The power supply may also include any component of one or more dc or ac power sources, recharging devices, power failure detection circuitry, power converters or inverters, power status indicators, and the like. The electronic device 1 may further include various sensors, a bluetooth module, a Wi-Fi module, and the like, which are not described herein again.
Further, the electronic device 1 may further include a network interface, and optionally, the network interface may include a wired interface and/or a wireless interface (such as a WI-FI interface, a bluetooth interface, etc.), which are generally used for establishing a communication connection between the electronic device 1 and other electronic devices.
Optionally, the electronic device 1 may further comprise a user interface, which may be a Display (Display), an input unit (such as a Keyboard), or optionally a standard wired interface, or a wireless interface. Alternatively, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, an OLED (Organic Light-Emitting Diode) touch device, or the like. The display, which may also be referred to as a display screen or display unit, is suitable for displaying information processed in the electronic device 1 and for displaying a visualized user interface, among other things.
It is to be understood that the described embodiments are for purposes of illustration only and that the scope of the appended claims is not limited to such structures.
The skeleton screen generating program 12 stored in the memory 11 of the electronic device 1 is a combination of a plurality of instructions, and when running in the processor 10, can implement the steps of the method, such as: acquiring a screenshot corresponding to a page loaded with real data; performing OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element; automatically generating a skeleton screen according to the region coordinate information; or may implement the functions of each unit and/or module, for example, implement the acquiring unit 100 to acquire the function of the screenshot corresponding to the page loaded with the real data; the recognition unit 200 is configured to perform OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element. The generating unit 300 is configured to automatically generate a skeleton screen according to the region coordinate information.
In the present invention, the integrated module/unit of the electronic device 1 may be stored in a computer readable storage medium if it is implemented in the form of a software functional unit and sold or used as a stand-alone product. The computer readable storage medium may be any tangible medium containing or storing a program or instructions, and may store a computer program that can be executed, where when the computer program is executed by the processor 10, the computer program implements, through hardware related to the stored program instructions, functions of each step in the method for generating a skeleton screen based on picture recognition or each unit/module in the system according to the present invention. For example, acquiring a screenshot corresponding to a page loaded with real data; performing OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element; and automatically generating a skeleton screen according to the region coordinate information. The computer-readable medium may include, for example: any entity or device capable of carrying said computer program code, recording medium, U-disk, removable hard disk, magnetic disk, optical disk, computer Memory, Read-Only Memory (ROM).
In the embodiments provided in the present invention, it should be understood that the disclosed apparatus and device may be implemented in other ways. The above-described system embodiments are merely illustrative, and for example, the division of the modules is only one logical functional division, and other divisions may be realized in practice.
The modules described as separate parts may or may not be physically separate, and parts displayed as modules may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment.
In addition, functional modules in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, or in a form of hardware plus a software functional module.
It will be evident to those skilled in the art that the invention is not limited to the details of the foregoing illustrative embodiments, and that the present invention may be embodied in other specific forms without departing from the spirit or essential attributes thereof.
The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Any reference signs in the claims shall not be construed as limiting the claim concerned.
Furthermore, it is obvious that the word "comprising" does not exclude other elements or steps, and the singular does not exclude the plural. A plurality of units or modules recited in the apparatus claims may also be implemented by one unit or module through software or hardware. The terms second, etc. are used to denote names, but not any particular order.
Finally, it should be noted that the above embodiments are only for illustrating the technical solutions of the present invention and not for limiting, and although the present invention is described in detail with reference to the preferred embodiments, it should be understood by those skilled in the art that modifications or equivalent substitutions may be made on the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention.
Claims (10)
1. A skeleton screen generation method based on picture recognition is characterized by comprising the following steps:
acquiring a screenshot corresponding to a page loaded with real data;
performing OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element;
and automatically generating a skeleton screen according to the region coordinate information.
2. The method for generating the skeleton screen based on the picture recognition according to claim 1, wherein the step of performing OCR recognition on the screenshot to obtain the region coordinate information corresponding to the page element comprises:
preprocessing the screenshot;
dividing the preprocessed screenshot into logic areas to obtain the logic areas; the logic area comprises a title bar, a left column, a right column, an upper column and a lower column;
finely dividing the logic area to obtain page elements;
and judging the shape outline of the page element according to the pixel characteristics to obtain the area coordinate information corresponding to the page element.
3. The method for generating a skeleton screen based on picture recognition according to claim 1, wherein the step of automatically generating the skeleton screen according to the region coordinate information comprises:
calculating the width and height of the area block corresponding to each page element according to the point coordinates in the area coordinate information of each page element;
and automatically generating the skeleton screen according to the region coordinate information and the width and the height of the region block.
4. The method for generating the skeleton screen based on the picture recognition according to claim 3, wherein the step of automatically generating the skeleton screen according to the region coordinate information and the width and the height of the region block comprises:
positioning the left side and the top of the area block according to the coordinates of the upper left corner point;
positioning the right side and the bottom of the area block according to the coordinates of the lower right corner point;
and giving a region block pattern according to the width and the height of the region block, and automatically generating a skeleton screen.
5. The method for generating a skeleton screen based on picture recognition according to claim 4, wherein the step of giving a region block style and automatically generating the skeleton screen comprises:
giving a region block style and converting the region block style into a skeleton screen code;
automatically injecting the skeleton screen code into an html file;
and rendering the page to obtain the skeleton screen.
6. The method for generating the skeleton screen based on the picture recognition according to claim 5, wherein the skeleton screen code is automatically injected into the html file by externally referring to the Js file.
7. The picture recognition-based skeleton screen generation method of claim 1, wherein the real data is real page data presented to a user; the page elements comprise text and pictures; the region coordinate information includes a page element type and a point coordinate of a page element.
8. The utility model provides a skeleton screen generation device based on picture discernment which characterized in that includes:
the acquisition unit is used for acquiring a screenshot corresponding to the page loaded with the real data;
the recognition unit is used for carrying out OCR recognition on the screenshot to obtain area coordinate information corresponding to the page element;
and the generating unit is used for automatically generating the skeleton screen according to the region coordinate information.
9. An electronic device, characterized in that the electronic device comprises:
at least one processor; and the number of the first and second groups,
a memory communicatively coupled to the at least one processor; wherein the content of the first and second substances,
the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the steps of the picture recognition based skeleton screen generation method of any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program, wherein the computer program, when executed by a processor, implements each step in the picture recognition-based skeleton screen generation method according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011594807.4A CN112633270A (en) | 2020-12-29 | 2020-12-29 | Skeleton screen generation method, device and equipment based on picture recognition and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202011594807.4A CN112633270A (en) | 2020-12-29 | 2020-12-29 | Skeleton screen generation method, device and equipment based on picture recognition and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN112633270A true CN112633270A (en) | 2021-04-09 |
Family
ID=75286100
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202011594807.4A Pending CN112633270A (en) | 2020-12-29 | 2020-12-29 | Skeleton screen generation method, device and equipment based on picture recognition and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN112633270A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113553523A (en) * | 2021-06-10 | 2021-10-26 | 北京达佳互联信息技术有限公司 | Method and device for generating skeleton screen page, electronic equipment, medium and product |
| CN113569177A (en) * | 2021-07-16 | 2021-10-29 | 上海淇玥信息技术有限公司 | Method and device for optimizing skipped web pages and electronic equipment |
| CN116781771A (en) * | 2023-08-21 | 2023-09-19 | 南京粒聚智能科技有限公司 | Automatic screen capturing picture analysis method of station machine by using OCR technology |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110489682A (en) * | 2019-07-08 | 2019-11-22 | 北京三快在线科技有限公司 | A kind of page loading method and device |
| CN112015634A (en) * | 2019-05-31 | 2020-12-01 | 百度在线网络技术（北京）有限公司 | Page structure information generation method and device and electronic equipment |
| CN112100543A (en) * | 2020-08-31 | 2020-12-18 | 东软集团股份有限公司 | Method and device for rendering webpage, storage medium, electronic equipment and server |
-
2020
- 2020-12-29 CN CN202011594807.4A patent/CN112633270A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112015634A (en) * | 2019-05-31 | 2020-12-01 | 百度在线网络技术（北京）有限公司 | Page structure information generation method and device and electronic equipment |
| CN110489682A (en) * | 2019-07-08 | 2019-11-22 | 北京三快在线科技有限公司 | A kind of page loading method and device |
| CN112100543A (en) * | 2020-08-31 | 2020-12-18 | 东软集团股份有限公司 | Method and device for rendering webpage, storage medium, electronic equipment and server |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113553523A (en) * | 2021-06-10 | 2021-10-26 | 北京达佳互联信息技术有限公司 | Method and device for generating skeleton screen page, electronic equipment, medium and product |
| CN113569177A (en) * | 2021-07-16 | 2021-10-29 | 上海淇玥信息技术有限公司 | Method and device for optimizing skipped web pages and electronic equipment |
| CN116781771A (en) * | 2023-08-21 | 2023-09-19 | 南京粒聚智能科技有限公司 | Automatic screen capturing picture analysis method of station machine by using OCR technology |
| CN116781771B (en) * | 2023-08-21 | 2023-11-17 | 南京粒聚智能科技有限公司 | Automatic screen capturing picture analysis method of station machine by using OCR technology |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112633270A (en) | Skeleton screen generation method, device and equipment based on picture recognition and storage medium | |
| CN110377285B (en) | Method and device for generating page skeleton screen and computer equipment | |
| CN108010112A (en) | Animation processing method, device and storage medium | |
| CN112991377A (en) | Keying algorithm based on opencv and shader fusion technology | |
| CN112233125A (en) | Image segmentation method and device, electronic equipment and computer readable storage medium | |
| CN110969004B (en) | Automatic typesetting method and system for graphics context, server and medium | |
| CN112419136A (en) | Picture watermark adding method and device, electronic equipment and storage medium | |
| CN112528909A (en) | Living body detection method, living body detection device, electronic apparatus, and computer-readable storage medium | |
| CN113222874B (en) | Data enhancement method, device, equipment and storage medium applied to target detection | |
| CN111476760A (en) | Medical image generation method and device, electronic equipment and medium | |
| CN114708461A (en) | Multi-modal learning model-based classification method, device, equipment and storage medium | |
| US9153193B2 (en) | Primitive rendering using a single primitive type | |
| CN112486807A (en) | Pressure testing method and device, electronic equipment and readable storage medium | |
| CN111932563A (en) | Picture region segmentation method and device, electronic equipment and storage medium | |
| CA1200025A (en) | Graphic and textual image generator for a raster scan display | |
| CN112233194B (en) | Medical picture optimization method, device, equipment and computer readable storage medium | |
| CN103488619B (en) | Method and device for processing document file | |
| US20230222716A1 (en) | Method and apparatus for automatically generating banner image, and computer-readable storage medium | |
| CN113010129A (en) | Virtual studio full-flow multi-terminal blackboard writing extraction method and device | |
| CN112633134A (en) | In-vehicle face recognition method, device and medium based on image recognition | |
| CN111667411A (en) | Image transmission method and device, electronic equipment and storage medium | |
| JPS63137378A (en) | Graphics processing system | |
| JPS60239261A (en) | Controlling system of printing of card information | |
| CN111539315B (en) | Model training method and device based on black box model, electronic equipment and medium | |
| CN115100081B (en) | LCD display screen gray scale image enhancement method, device, equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |