WO2024251060A1 - 图文排版方法、装置、介质和电子设备 - Google Patents

图文排版方法、装置、介质和电子设备 Download PDF

Info

Publication number
WO2024251060A1
WO2024251060A1 PCT/CN2024/096949 CN2024096949W WO2024251060A1 WO 2024251060 A1 WO2024251060 A1 WO 2024251060A1 CN 2024096949 W CN2024096949 W CN 2024096949W WO 2024251060 A1 WO2024251060 A1 WO 2024251060A1
Authority
WO
WIPO (PCT)
Prior art keywords
laid out
saliency
map
candidate
candidate frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2024/096949
Other languages
English (en)
French (fr)
Inventor
杨茂柯
薛昊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beijing Zitiao Network Technology Co Ltd
Original Assignee
Beijing Zitiao Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Zitiao Network Technology Co Ltd filed Critical Beijing Zitiao Network Technology Co Ltd
Publication of WO2024251060A1 publication Critical patent/WO2024251060A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/53—Querying
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/50—Information retrieval; Database structures therefor; File system structures therefor of still image data
    • G06F16/58—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually
    • G06F16/5866—Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using information manually generated, e.g. tags, keywords, comments, manually generated location and time information
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Definitions

  • the present disclosure relates to the field of computer technology, and in particular to a graphic typesetting method, device, medium and electronic device.
  • GAN Generative Adversarial Network
  • the present disclosure provides a method for typesetting text and graphics, comprising: obtaining a saliency map based on a background map; searching on the saliency map in combination with a material to be laid out to obtain a candidate frame of the material to be laid out; determining a saliency value of the candidate frame; and determining a placement position of the material to be laid out from the candidate frame based on the saliency value.
  • the present disclosure provides a graphic typesetting device, comprising: an acquisition module, configured to obtain a saliency map based on a background map; a search module, configured to search on the saliency map in combination with a material to be laid out, to obtain a candidate frame of the material to be laid out; a first determination module, configured to determine a saliency value of the candidate frame; and a second determination module, configured to determine a placement position of the material to be laid out from the candidate frame based on the saliency value.
  • the present disclosure provides a computer-readable medium having a computer program stored thereon, which, when executed by a processing device, implements the steps of any one of the methods described in the first aspect.
  • the present disclosure provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device configured to execute the computer program in the storage device to implement the steps of any one of the methods described in the first aspect.
  • the present disclosure provides a computer program product, comprising instructions, which, when executed by a processor, cause The processor implements the steps of any one of the methods described in the first aspect.
  • the present disclosure provides a computer program, comprising program code, which, when executed by a processor, causes the steps of any one of the methods described in the first aspect to be implemented.
  • FIG1 is a flow chart of a method for typesetting text and images according to an embodiment of the present disclosure.
  • FIG. 2 shows a schematic diagram of a background map and its corresponding saliency map.
  • FIG. 3 is another flow chart of a method for typesetting text and images according to an embodiment of the present disclosure.
  • FIG. 4 is another flow chart of a method for typesetting text and images according to an embodiment of the present disclosure.
  • FIG. 5 shows a schematic diagram of extracting candidate boxes of multiple scales using a sliding window pyramid.
  • FIG. 6 is a schematic diagram of placement positions of materials to be laid out according to an embodiment of the present disclosure.
  • FIG. 7 is a schematic block diagram of a graphic typesetting device according to an embodiment of the present disclosure.
  • FIG. 8 shows a schematic structural diagram of an electronic device suitable for implementing the embodiments of the present disclosure.
  • a prompt message is sent to the user to clearly prompt the user that the operation requested to be performed will require obtaining and using the user's personal information.
  • the user can autonomously choose whether to provide personal information to software or hardware such as an electronic device, application, server, or storage medium that performs the operation of the technical solution of the present disclosure according to the prompt message.
  • the prompt information in response to receiving an active request from the user, may be sent to the user in the form of a pop-up window, in which the prompt information may be presented in text form.
  • the pop-up window may also carry a selection control for the user to choose "agree” or “disagree” to provide personal information to the electronic device.
  • the resulting image after arrangement/layout usually has the problem that the text box position is not harmonious and beautiful, or the text box blocks the main body of the poster.
  • the present disclosure proposes an improved graphic layout scheme, which will be described below with reference to the accompanying drawings.
  • Fig. 1 is a flow chart of a method for typesetting text and images according to an embodiment of the present disclosure. As shown in Fig. 1, the method for typesetting text and images includes the following steps S11 to S14.
  • step S11 a saliency map is obtained based on the background map.
  • the background image is the image on which the layout material is placed.
  • the material to be laid out can be a text box or other materials.
  • the saliency map is an image used to show the uniqueness of each pixel in the background image.
  • the size of the saliency map is the same as that of the background image.
  • Figure 2 shows a schematic diagram of the background image and its corresponding saliency map.
  • the white part represents the salient area and the black part represents the non-salient area.
  • step S12 a search is performed on the saliency map in combination with the material to be laid out to obtain a candidate frame of the material to be laid out.
  • the search may be performed based on the shape parameters of the material to be laid out, such as size, shape, etc., for example, a frame that matches the shape of the material to be laid out is found as a candidate frame.
  • the search may be performed on the saliency map in combination with the material to be laid out in a sliding window manner. That is, a window of the shape of the material to be laid out is used to perform a sliding window operation on the entire saliency map to obtain all candidate frames that can place the material to be laid out.
  • the candidate frame may be one or more.
  • step S13 the significance value of the candidate box is determined.
  • the significance value of each candidate box among all the candidate boxes where the material to be laid out can be placed is determined.
  • an integral map of the saliency map may be first determined, and then the saliency values of each candidate box may be determined based on the integral map.
  • the integral map refers to a map of the integral values of each pixel in the image, and the integral value of a certain pixel is the sum of all elements contained in a rectangle with the pixel in the upper left corner of the image as the upper left vertex and the pixel as the lower right vertex.
  • step S14 based on the saliency value, the placement position of the material to be laid out is determined from the candidate frames.
  • a specific candidate frame is selected from all candidate frames where the material to be laid out can be placed to place the material to be laid out.
  • the candidate box with the smallest significance value can be selected as the placement position of the material to be laid out; or the N candidate boxes with the smallest significance values can be selected by sorting the significance values, so that the user can independently decide in which candidate box to place the material to be laid out from these N candidate boxes.
  • the placement position is determined based on the saliency value, it is possible to ensure that the material to be laid out is not placed in the salient area of the background map, but is placed in the non-salient area of the background map, thereby greatly reducing the occlusion between the materials during the automated layout process (for example, reducing the occlusion of the text box on the subject in the background map), and improving the aesthetics of the automated poster.
  • FIG. 3 is another flow chart of a method for typesetting text and images according to an embodiment of the present disclosure.
  • step S31 a saliency map is obtained based on the background map.
  • step S32 the material to be laid out to be searched on the saliency map is selected from the multiple materials to be laid out according to the size priority.
  • the multiple materials to be laid out can be sorted according to the area size, and the material to be laid out to be searched on the saliency map is selected in descending order.
  • step S33 a search is performed on the saliency map in combination with the selected material to be laid out to obtain a candidate frame of the selected material to be laid out.
  • step S34 the saliency value of the candidate box is determined. How to determine the saliency value has been described in detail above and will not be repeated here.
  • step S35 based on the saliency value, the placement position of the selected material to be laid out is determined from the candidate frame.
  • step S36 the determined placement position is marked as significant on the significance map. In this way, the placement position will be considered as a significant area in subsequent processes, and no other materials to be laid out will be placed there.
  • step S37 it is determined whether all the materials to be laid out have been placed. If they have been placed, the graphic layout process ends. If not, the process goes to step S32 to place the next material to be laid out.
  • Fig. 4 is another flow chart of a method for typesetting text and graphics according to an embodiment of the present disclosure.
  • the method for typesetting text and graphics can be applied to a scenario where an existing poster is resized to obtain a new poster, for example, an existing horizontal poster is resized to obtain a vertical poster.
  • step S41 the material to be laid out is cut out from the poster to be resized to obtain a background image, and the background image is resized to obtain a background base image.
  • step S41 you can first cut out the material to be laid out (such as a text box) from the existing horizontal poster to obtain a background image, and then resize the background image to obtain a vertical background base image.
  • material to be laid out such as a text box
  • step S42 a saliency map is obtained based on the background map.
  • step S43 the material to be laid out to be searched on the saliency map is selected from the multiple materials to be laid out according to the size priority.
  • the multiple materials to be laid out can be sorted according to the area size, and the material to be laid out to be searched on the saliency map is selected in descending order.
  • step S44 a search is performed on the saliency map based on the selected material to be laid out to obtain a candidate frame of the selected material to be laid out.
  • step S45 the saliency value of the candidate box is determined. The above has described in detail how to determine the saliency value. This will not be elaborated on.
  • step S46 the significance value is processed based on the position and size of the material to be laid out in the poster to be converted in size, and the placement position of the selected material to be laid out is determined from the candidate frame based on the processed significance value.
  • the significance value is processed based on the position and size of the material to be laid out in the poster to be converted in size, which can be achieved in the following way: first, the similarity between the position and size of the candidate box and the position and size of the material to be laid out in the poster to be converted in size is calculated.
  • the similarity between the two is relatively high; then, the similarity is normalized, for example, the similarity is normalized to a range of 0 to 1; then, the normalized similarity is multiplied by the significance value to obtain the processed significance value.
  • step S47 the determined placement position is marked as significant on the significance map. In this way, the placement position will be considered as a significant area in subsequent processes, and no other materials to be laid out will be placed there.
  • step S48 it is determined whether all the materials to be laid out have been placed. If they have been placed, the graphic layout process ends. If not, the process goes to step S43 to place the next material to be laid out.
  • the material to be laid out before searching on the saliency map in combination with the material to be laid out, the material to be laid out may be subjected to a multi-scale transformation, for example, the material to be laid out may be enlarged or reduced, and the ratio of enlargement or reduction may be, for example, in the range of 1 to 2 times.
  • the material to be laid out that retains the original size, the material to be laid out that is enlarged, the material to be laid out that is reduced, etc. may be obtained.
  • the material to be laid out that has been multi-scale transformed may be searched on the saliency map in sequence.
  • the first-scale material to be laid out, the second-scale material to be laid out, and the third-scale material to be laid out may be searched on the saliency map in sequence in combination, thereby obtaining the candidate box for the material to be laid out 1.
  • This processing method may also be referred to as using a sliding window.
  • Pyramid extraction of candidate boxes of multiple scales may also be referred to as using a sliding window.
  • Figure 5 shows a schematic diagram of extracting candidate boxes of multiple scales using a sliding window pyramid. Each rectangular box in Figure 5 is the extracted candidate boxes of multiple scales.
  • determining the placement position of the material to be laid out from the candidate frames may include: for candidate frames with similar positions and a difference in saliency values less than a preset threshold, using a non-maximum suppression method to determine the placement position of the material to be laid out from the candidate frames. In this way, it is possible to ensure that a more appropriate placement position is determined for the material to be laid out.
  • FIG6 is a schematic diagram of the placement position of the material to be laid out according to an embodiment of the present disclosure.
  • the rectangular box in the upper left corner is the final candidate box position, and the upper left corner of the right figure shows the material to be laid out placed in the candidate box. It can be seen that the final placement position of the material to be laid out does not block the subject in the background base map.
  • the graphic typesetting device provided by the embodiment of the present disclosure can execute the graphic typesetting method provided by any embodiment of the present disclosure.
  • the embodiment of the present disclosure can divide the device into functional units according to the above method example. For example, each functional module/unit can be divided corresponding to each function, or two or more functions can be integrated into one processing module. It is worth noting that the various modules/units included in the above device are only divided according to functional logic, but are not limited to the division described in the text.
  • each module/unit can be implemented in various appropriate ways, such as hardware, firmware, or any appropriate combination.
  • Fig. 7 is a schematic block diagram of a graphic typesetting device according to an embodiment of the present disclosure.
  • the graphic typesetting device includes: an acquisition module 71, configured to obtain a saliency map based on a background base map; a search module 72, configured to search on the saliency map in combination with the material to be laid out, and obtain a candidate frame of the material to be laid out; a first determination module 73, configured to determine the saliency value of the candidate frame; and a second determination module 74, configured to determine the placement position of the material to be laid out from the candidate frame based on the saliency value.
  • the number of the materials to be laid out is multiple, and the search module 72 is further configured to: According to the size priority, a material to be laid out to be searched on the saliency map is selected from the plurality of materials to be laid out; and the search is performed on the saliency map in combination with the selected material to be laid out.
  • the search module 72 is further configured to: perform multi-scale transformation on the material to be laid out; and sequentially search on the saliency map based on the material to be laid out after the multi-scale transformation.
  • the first determination module 73 is further configured to: determine an integral map of the saliency map; and determine a saliency value of the candidate box based on the integral map.
  • the graphic typesetting apparatus further comprises a marking module configured to: after determining the placement position of the to-be-layout material from the candidate frame, mark the determined placement position as significant on the significance map.
  • the graphic typesetting device further includes a cutting module configured to: cut out the material to be laid out from the poster to be converted in size to obtain a background image; perform size conversion on the background image to obtain the background base image;
  • the second determination module 74 is further configured to: process the significance value based on the position and size of the material to be laid out in the poster to be converted in size; and determine the placement position of the material to be laid out from the candidate frame based on the processed significance value.
  • the second determination module 74 may also be configured to: calculate the similarity between the position and size of the candidate box and the position and size of the material to be laid out in the poster to be converted in size; normalize the similarity; multiply the normalized similarity by the significance value to obtain the processed significance value.
  • the second determination module 74 may also be configured to: for candidate frames that are similar in position and whose difference in significance values is less than a preset threshold, use a non-maximum suppression method to determine the placement position of the material to be laid out from the candidate frames.
  • the embodiment of the present disclosure further provides a computer-readable medium having a computer program stored thereon, which implements the steps of any method in the present disclosure when the program is executed by a processing device.
  • An embodiment of the present disclosure further provides an electronic device, comprising: a storage device on which a computer program is stored; and a processing device configured to execute the computer program in the storage device to implement the steps of any method in the present disclosure.
  • the electronic device includes hardware structures and/or software modules corresponding to executing each function.
  • the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. The technicians can It is possible to use different methods to implement the described functionality, but such implementation should not be considered beyond the scope of the present disclosure.
  • the terminal device in the embodiment of the present disclosure may include, but is not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc.
  • PDAs personal digital assistants
  • PADs tablet computers
  • PMPs portable multimedia players
  • vehicle-mounted terminals such as vehicle-mounted navigation terminals
  • fixed terminals such as digital TVs, desktop computers, etc.
  • the electronic device shown in FIG8 is only an example and should not bring any limitation to the functions and scope of use of
  • the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603.
  • a processing device e.g., a central processing unit, a graphics processing unit, etc.
  • RAM random access memory
  • various programs and data required for the operation of the electronic device 600 are also stored.
  • the processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604.
  • An input/output (I/O) interface 605 is also connected to the bus 604.
  • the following devices may be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609.
  • the communication device 609 may allow the electronic device 600 to communicate wirelessly or wired with other devices to exchange data.
  • FIG. 8 shows an electronic device 600 with various devices, it should be understood that it is not required to implement or have all the devices shown. More or fewer devices may be implemented or have alternatively.
  • an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart.
  • the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602.
  • the processing device 601 the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.
  • the above-mentioned computer-readable medium of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two.
  • the computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above.
  • Computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.
  • a computer-readable storage medium may be any tangible medium containing or storing a program that can be used or communicated with an instruction execution system, device or device.
  • a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above.
  • a computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in combination with an instruction execution system, device, or device.
  • the program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.
  • the client and server may communicate using any currently known or future developed network protocol such as HTTP (HyperText Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network).
  • HTTP HyperText Transfer Protocol
  • Examples of communication networks include a local area network ("LAN”), a wide area network ("WAN”), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.
  • the computer-readable medium may be included in the electronic device, or may exist independently without being installed in the electronic device.
  • the computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device executes the graphic typesetting method provided in any embodiment of the present disclosure.
  • Computer program code for performing the operations of the present disclosure may be written in one or more programming languages or a combination thereof, including, but not limited to, object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages.
  • the program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
  • LAN local area network
  • WAN wide area network
  • Internet service provider e.g., via the Internet using an Internet service provider
  • each box in the flowchart or block diagram may represent a module, a program segment, or a portion of a code, which contains one or more executable instructions for implementing the specified logical functions.
  • the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession may actually be executed substantially in parallel, and they may sometimes be executed in the opposite order, depending on the functions involved.
  • each box in the block diagram and/or flowchart, and the combination of boxes in the block diagram and/or flowchart may be implemented using a dedicated hardware-based system that performs the specified functions or operations, or may be implemented using dedicated hardware and a computer. It is implemented by a combination of machine instructions.
  • modules involved in the embodiments described in the present disclosure may be implemented by software or hardware.
  • the name of a module does not limit the module itself in some cases.
  • an acquisition module may also be described as a "module for obtaining a saliency map based on a background map”.
  • exemplary types of hardware logic components include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.
  • FPGAs field programmable gate arrays
  • ASICs application specific integrated circuits
  • ASSPs application specific standard products
  • SOCs systems on chips
  • CPLDs complex programmable logic devices
  • a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment.
  • a machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium.
  • a machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing.
  • a more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or flash memory erasable programmable read-only memory
  • CD-ROM portable compact disk read-only memory
  • CD-ROM compact disk read-only memory
  • magnetic storage device or any suitable combination of the foregoing.
  • Example 1 provides a graphic typesetting method, including: obtaining a saliency map based on a background map; searching on the saliency map in combination with the material to be laid out to obtain a candidate frame of the material to be laid out; determining a saliency value of the candidate frame; and determining a placement position of the material to be laid out from the candidate frame based on the saliency value.
  • Example 2 provides the method of Example 1, wherein the number of the materials to be laid out is multiple, and the method further comprises: selecting the material to be laid out to be searched on the saliency map from the multiple materials to be laid out according to size priority;
  • the searching on the saliency map in combination with the material to be laid out includes: searching on the saliency map in combination with the selected material to be laid out.
  • Example 3 provides the method of Example 1, wherein the method further comprises: performing a multi-scale transformation on the material to be laid out;
  • the searching on the saliency map in combination with the material to be laid out includes: searching on the saliency map in sequence based on the material to be laid out after multi-scale transformation.
  • Example 4 provides the method of Example 1, wherein the determining The saliency value of the candidate box includes: determining an integral map of the saliency map; and determining the saliency value of the candidate box based on the integral map.
  • Example 5 provides the method of Example 1, wherein the method further includes: after determining the placement position of the material to be laid out from the candidate frame, marking the determined placement position as significant on the significance map.
  • Example 6 provides a method of any one of Examples 1 to 5, wherein the method further comprises: cutting out the material to be laid out from the poster to be resized to obtain a background image; resizing the background image to obtain the background base image;
  • the step of determining the placement position of the material to be laid out from the candidate frame based on the significance value comprises: processing the significance value based on the position and size of the material to be laid out in the poster to be resized; and determining the placement position of the material to be laid out from the candidate frame based on the processed significance value.
  • Example 7 provides the method of Example 6, wherein the significance value is processed based on the position and size of the material to be laid out in the poster to be resized, including: calculating the similarity between the position and size of the candidate box and the position and size of the material to be laid out in the poster to be resized; normalizing the similarity; multiplying the normalized similarity by the significance value to obtain the processed significance value.
  • Example 8 provides a method of any one of Examples 1 to 5 and 7, wherein the step of determining the placement position of the material to be laid out from the candidate frame based on the significance value comprises: for candidate frames that are similar in position and whose significance value difference is less than a preset threshold, using a non-maximum suppression method to determine the placement position of the material to be laid out from the candidate frame.
  • Example 9 provides a graphic typesetting device, comprising: an acquisition module, configured to obtain a saliency map based on a background map; a search module, configured to search on the saliency map in combination with the material to be laid out, to obtain a candidate frame of the material to be laid out; a first determination module, configured to determine a saliency value of the candidate frame; and a second determination module, configured to determine a placement position of the material to be laid out from the candidate frame based on the saliency value.
  • Example 10 provides a computer-readable medium having a computer program and/or instructions stored thereon, which implement the steps of any of the methods described in Examples 1-8 when executed by a processing device.
  • Example 11 provides an electronic device, comprising: a storage device on which a computer program and/or instructions are stored; and a processing device configured to execute the computer program and/or instructions in the storage device to implement the steps of any one of the methods described in Examples 1-8.
  • Example 12 provides a computer program product, comprising instructions, which, when executed by a processor, enable the processor to implement the steps of the method described in any one of Examples 1 to 8.
  • Example 13 provides a computer program, including program code, which, when executed by a processor, causes the steps of the method described in any one of Examples 1 to 8 to be implemented.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Library & Information Science (AREA)
  • Processing Or Creating Images (AREA)

Abstract

本公开涉及一种图文排版方法、装置、介质和电子设备。该图文排版方法包括：基于背景底图获得显著性图；结合待布局素材在所述显著性图上进行查找，得到所述待布局素材的候选框；确定所述候选框的显著性值；基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置。

Description

图文排版方法、装置、介质和电子设备
相关申请的交叉引用
本申请是以申请号为202310667291.9、申请日为2023年6月6日的中国申请为基础，并主张其优先权，该中国申请的公开内容在此作为整体引入本申请中。
技术领域
本公开涉及计算机技术领域，具体地，涉及一种图文排版方法、装置、介质和电子设备。
背景技术
目前的自动化图文海报生成技术中，通常使用生成对抗网络(Generative Adversarial Network，GAN)等算法生成文本框之间的排布。
发明内容
提供该发明内容部分以便以简要的形式介绍构思，这些构思将在后面的具体实施方式部分被详细描述。该发明内容部分并不旨在标识要求保护的技术方案的关键特征或必要特征，也不旨在用于限制所要求的保护的技术方案的范围。
第一方面，本公开提供一种图文排版方法，包括：基于背景底图获得显著性图；结合待布局素材在所述显著性图上进行查找，得到所述待布局素材的候选框；确定所述候选框的显著性值；基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置。
第二方面，本公开提供一种图文排版装置，包括：获取模块，被配置为基于背景底图获得显著性图；查找模块，被配置为结合待布局素材在所述显著性图上进行查找，得到所述待布局素材的候选框；第一确定模块，被配置为确定所述候选框的显著性值；第二确定模块，被配置为基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置。
第三方面，本公开提供一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现第一方面中任一项所述方法的步骤。
第四方面，本公开提供一种电子设备，包括：存储装置，其上存储有计算机程序；处理装置，被配置为执行所述存储装置中的所述计算机程序，以实现第一方面中任一项所述方法的步骤。
第五方面，本公开提供一种计算机程序产品，包含指令，该指令在由处理器执行时使 得处理器实现第一方面中任一项所述方法的步骤。
第六方面，本公开提供一种计算机程序，包括程序代码，该程序代码在由处理器执行时导致实现第一方面中任一项所述方法的步骤。
本公开的其他特征和优点将在随后的具体实施方式部分予以详细说明。
附图说明
结合附图并参考以下具体实施方式，本公开各实施例的上述和其他特征、优点及方面将变得更加明显。贯穿附图中，相同或相似的附图标记表示相同或相似的元素。应当理解附图是示意性的，原件和元素不一定按照比例绘制。在附图中：
图1是根据本公开一种实施例的图文排版方法的流程图。
图2示出了背景底图及其对应的显著性图的示意图。
图3是根据本公开一种实施例的图文排版方法的又一流程图。
图4是根据本公开一种实施例的图文排版方法的又一流程图。
图5示出了使用滑窗金字塔提取多个尺度的候选框的示意图。
图6是根据本公开实施例的待布局素材的放置位置示意图。
图7是根据本公开一种实施例的图文排版装置的示意框图。
图8示出了适于用来实现本公开实施例的电子设备的结构示意图。
具体实施方式
下面将参照附图更详细地描述本公开的实施例。虽然附图中显示了本公开的某些实施例，然而应当理解的是，本公开可以通过各种形式来实现，而且不应该被解释为限于这里阐述的实施例，相反提供这些实施例是为了更加透彻和完整地理解本公开。应当理解的是，本公开的附图及实施例仅用于示例性作用，并非用于限制本公开的保护范围。
应当理解，本公开的方法实施方式中记载的各个步骤可以按照不同的顺序执行，和/或并行执行。此外，方法实施方式可以包括附加的步骤和/或省略执行示出的步骤。本公开的范围在此方面不受限制。
本文使用的术语“包括”及其变形是开放性包括，即“包括但不限于”。术语“基于”是“至少部分地基于”。术语“一个实施例”表示“至少一个实施例”；术语“另一实施例”表示“至少一个另外的实施例”；术语“一些实施例”表示“至少一些实施例”。其他术语的相关定义将在下文描述中给出。
需要注意，本公开中提及的“第一”、“第二”等概念仅用于对不同的装置、模块或单元 进行区分，并非用于限定这些装置、模块或单元所执行的功能的顺序或者相互依存关系。
需要注意，本公开中提及的“一个”、“多个”的修饰是示意性而非限制性的，本领域技术人员应当理解，除非在上下文另有明确指出，否则应该理解为“一个或多个”。
本公开实施方式中的多个装置之间所交互的消息或者信息的名称仅用于说明性的目的，而并不是用于对这些消息或信息的范围进行限制。
可以理解的是，在使用本公开各实施例公开的技术方案之前，均应当依据相关法律法规通过恰当的方式对本公开所涉及个人信息的类型、使用范围、使用场景等告知用户并获得用户的授权。
例如，在响应于接收到用户的主动请求时，向用户发送提示信息，以明确地提示用户，其请求执行的操作将需要获取和使用到用户的个人信息。从而，使得用户可以根据提示信息来自主地选择是否向执行本公开技术方案的操作的电子设备、应用程序、服务器或存储介质等软件或硬件提供个人信息。
作为一种可选的但非限定性的实现方式，响应于接收到用户的主动请求，向用户发送提示信息的方式例如可以是弹窗的方式，弹窗中可以以文字的方式呈现提示信息。此外，弹窗中还可以承载供用户选择“同意”或者“不同意”向电子设备提供个人信息的选择控件。
可以理解的是，上述通知和获取用户授权过程仅是示意性的，不对本公开的实现方式构成限定，其它满足相关法律法规的方式也可应用于本公开的实现方式中。
同时，可以理解的是，本技术方案所涉及的数据(包括但不限于数据本身、数据的获取或使用)应当遵循相应法律法规及相关规定的要求。
在目前的自动化图文海报生成技术中，排布/布局后的成图通常存在着文本框位置不协调美观、或是文本框遮挡海报主体的问题。鉴于此，本公开提出了改进的图文排版方案，以下将参照附图进行描述。
图1是根据本公开一种实施例的图文排版方法的流程图。如图1所示，该图文排版方法包括以下步骤S11至S14。
在步骤S11中，基于背景底图获得显著性图。
背景底图指的是要在其上放置待布局素材的图像。
待布局素材可以是文本框或者其他素材。
显著性图是用于显示背景底图中的每个像素独特性的图像。显著性图的大小与背景底图的大小相同。图2示出了背景底图及其对应的显著性图的示意图，图2中右侧的显著性图中，白色部分表示显著性区域，黑色部分表示非显著性区域。
在步骤S12中，结合待布局素材在显著性图上进行查找，得到待布局素材的候选框。
在一些实施例中，可以基于待布局素材的外形参数，诸如尺寸/大小、形状等等来进行查找，例如找到可与待布局素材的外形匹配的框作为候选框。在一些实施例中，可以通过滑窗的方式来结合待布局素材在显著性图上进行查找。也即，使用待布局素材形状的窗口在整个显著性图上进行滑窗操作，得到可以放置待布局素材的所有候选框。特别地，所述候选框可以为一个或多个。
在步骤S13中，确定候选框的显著性值。特别地，确定可以放置待布局素材的所有候选框中各候选框的显著性值。
在一些实施例中，可以首先确定显著性图的积分图，然后基于积分图，确定各个候选框的显著性值。积分图指的是图像中每个像素点的积分值的图，某个像素点的积分值是以图像左上角像素点为左上角顶点、以该像素点为右下角顶点的矩形中包含的所有元素之和。通过采用积分图的方式，能够加速显著性值的计算速度。
在步骤S14中，基于显著性值，从候选框中确定待布局素材的放置位置。特别地，从可以放置待布局素材的所有候选框中选择特定候选框以放置待布局素材。
例如，可以选取显著性值最小的候选框作为待布局素材的放置位置；也可以按照显著性值排序，选取显著性值最小的N个候选框，这样用户就能够从这N个候选框中自主决定将待布局素材放置在哪个候选框中。
应指出，可选择地，作为一个示例，如果所查找到的候选框仅一个，则可以无需执行显著性值确定以及基于该显著性值进行的选取，而直接使用所查找到的一个候选框作来放置待布局素材。当然，即使仅一个候选框，也可进行显著性值确定。
通过采用上述技术方案，由于能够基于背景底图获得显著性图，基于待布局素材在显著性图上进行查找，得到待布局素材的候选框，然后确定候选框的显著性值，并基于显著性值，从候选框中确定待布局素材的放置位置，因此能够在图文海报生产过程中自动化地布局各种素材(例如，文本框及其他素材)。进一步地，由于是根据显著性值来确定放置位置，因此能够确保在背景底图的显著区域中不放置待布局素材，而是在背景底图的非显著性区域中放置待布局素材，从而能够在自动化布局的过程中大幅减少各素材之间的遮挡(例如，减少文本框对背景底图中的主体的遮挡)，提升自动化海报成图的美观度。
图3是根据本公开一种实施例的图文排版方法的又一流程图。
如图3所示，在步骤S31中，基于背景底图获得显著性图。
在步骤S32中，按照大小优先级，从多个待布局素材中选取要在显著性图上进行查找的待布局素材。例如，可以按照面积大小对多个待布局素材进行排序，并按照从大到小的顺序来选取要在显著性图上进行查找的待布局素材。
在步骤S33中，结合所选取的待布局素材在显著性图上进行查找，得到所选取的待布局素材的候选框。前文中已经详细描述如何进行查找，在此不再赘述。
在步骤S34中，确定候选框的显著性值。前文中已经详细描述如何确定显著性值，在此不再赘述。
在步骤S35中，基于显著性值，从候选框中确定所选取的待布局素材的放置位置。
在步骤S36中，在显著性图上将所确定的放置位置标记为显著。这样，该放置位置在后续的过程中就会被认为是显著区域，不会再被放置其他的待布局素材。
在步骤S37中，判断是否所有的待布局素材均已放置完毕。若均已放置完毕，则结束图文排版流程。若没有均放置完毕，则转至步骤S32，以放置下一个待布局素材。
通过采用上述技术方案，由于能够基于背景底图获得显著性图，结合待布局素材在显著性图上进行查找，得到待布局素材的候选框，然后确定候选框的显著性值，并基于显著性值，从候选框中确定待布局素材的放置位置，因此能够在图文海报生产过程中自动化地布局各种素材(例如，文本框及其他素材)，而且由于是根据显著性值来确定放置位置，因此能够确保在背景底图的显著区域中不放置待布局素材，而是在背景底图的非显著性区域中放置待布局素材，从而能够在自动化布局的过程中大幅减少各素材之间的遮挡(例如，减少文本框对背景底图中的主体的遮挡)，提升自动化海报成图的美观度。
图4是根据本公开一种实施例的图文排版方法的又一流程图。该图文排版方法能够应用于对已有海报进行尺寸变换得到新海报的场景中，例如对已有的横版海报进行尺寸变换得到竖版海报。
如图4所示，在步骤S41中，从待尺寸转换的海报中剪裁掉待布局素材，得到背景图，并对背景图进行尺寸转换，得到背景底图。
举例而言，如果希望将已有的横版海报进行尺寸变换得到竖版海报，则在步骤S41中可以先从已有的横版海报中剪裁掉待布局素材(例如文本框)，得到背景图，然后对背景图进行尺寸转换，得到竖版的背景底图。
在步骤S42中，基于背景底图获得显著性图。
在步骤S43中，按照大小优先级，从多个待布局素材中选取要在显著性图上进行查找的待布局素材。例如，可以按照面积大小对多个待布局素材进行排序，并按照从大到小的顺序来选取要在显著性图上进行查找的待布局素材。
在步骤S44中，基于所选取的待布局素材在显著性图上进行查找，得到所选取的待布局素材的候选框。前文中已经详细描述如何进行查找，在此不再赘述。
在步骤S45中，确定候选框的显著性值。前文中已经详细描述如何确定显著性值，在 此不再赘述。
在步骤S46中，基于待布局素材在待尺寸转换的海报中的位置和尺寸，对显著性值进行处理，基于处理后的显著性值，从候选框中确定所选取的待布局素材的放置位置。
在一些实施例中，基于待布局素材在待尺寸转换的海报中的位置和尺寸，对显著性值进行处理，可以通过如下方式来实现：首先，计算候选框的位置和尺寸与待布局素材在待尺寸转换的海报中的位置和尺寸之间的相似度，例如，如果候选框的位置比较接近于待布局素材在待尺寸转换的海报中的位置且候选框的尺寸比较接近于待布局素材在待尺寸转换的海报中的尺寸，则两者之间的相似度比较高；然后，对相似度进行归一化处理，例如将相似度归一化到0至1的范围内；然后，将归一化处理后的相似度与显著性值相乘，得到处理后的显著性值。
在步骤S47中，在显著性图上将所确定的放置位置标记为显著。这样，该放置位置在后续的过程中就会被认为是显著区域，不会再被放置其他的待布局素材。
在步骤S48中，判断是否所有的待布局素材均已放置完毕。若均已放置完毕，则结束图文排版流程。若没有均放置完毕，则转至步骤S43，以放置下一个待布局素材。
通过采用上述技术方案，由于能够基于背景底图获得显著性图，结合待布局素材在显著性图上进行查找，得到待布局素材的候选框，然后确定候选框的显著性值，并基于显著性值，从候选框中确定待布局素材的放置位置，因此能够在图文海报生产过程中自动化地布局各种素材(例如，文本框及其他素材)，而且由于是根据显著性值来确定放置位置，因此能够确保在背景底图的显著区域中不放置待布局素材，而是在背景底图的非显著性区域中放置待布局素材，从而能够在自动化布局的过程中大幅减少各个素材之间的遮挡(例如，减少文本框对背景底图中的主体的遮挡)，提升自动化海报成图的美观度。另外，由于在确定放置位置时考虑了待布局素材在待尺寸转换的海报中的位置和尺寸，因此能够确保最终确定的放置位置更为合适。
在一些实施例中，在结合待布局素材在显著性图上进行查找之前，可以先对待布局素材进行多尺度变换，例如，对待布局素材进行放大、缩小等，放大或缩小的比例可以位于例如1至2倍的范围内。通过多尺度变换，可以得到保留原尺寸的待布局素材、被放大的待布局素材、被缩小的待布局素材等。在多尺度变换之后，就可以结合多尺度变换后的待布局素材依次在显著性图上进行查找。举例而言，对于待布局素材1，假设经过多尺度变换后，得到了第一尺度待布局素材、第二尺度待布局素材和第三尺度待布局素材，则可以依次结合第一尺度待布局素材、第二尺度待布局素材和第三尺度待布局素材分别在显著性图上进行查找，从而得到针对待布局素材1的候选框。这种处理方式也可以称为使用滑窗 金字塔提取多个尺度的候选框。图5示出了使用滑窗金字塔提取多个尺度的候选框的示意图。图5中的各个长方形框即是被提取的多个尺度的候选框。
通过对待布局素材进行多尺度变换、然后再在显著性图上进行查找，能够确保将待布局素材放置到背景底图的合适位置上。
在一些实施例中，基于显著性值，从候选框中确定待布局素材的放置位置，可以包括：针对位置相近且显著性值的差值小于预设阈值的候选框，使用非极大值抑制方法从候选框中确定待布局素材的放置位置。这样就能够确保为待布局素材确定更为合适的放置位置。
图6是根据本公开实施例的待布局素材的放置位置示意图。图6左图中，左上角的长方形框是最终确定的候选框位置，右图中左上角示出了在该候选框中放置的待布局素材。可见，待布局素材的最终放置位置没有遮挡背景底图中的主体。
以下将描述根据本公开的实施例的图文排版装置。本公开实施例所提供的图文排版装置可执行本公开任意实施例所提供的图文排版方法，本公开实施例可以根据上述方法示例对装置进行功能单元的划分，例如，可以对应各个功能划分各个功能模块/单元，也可以将两个或两个以上的功能集成在一个处理模块中。值得注意的是，上述装置所包括的各个模块/单元只是按照功能逻辑进行划分的，但并不局限于文中所述的划分，实际实现时可以有另外的划分方式，只要能够实现相应的功能即可；另外，各模块/单元的具体名称也只是为了便于相互区分，并不用于限制本公开实施例的保护范围。此外，各模块/单元可采用各种适当方式来实现，例如硬件、固件、或任何适当组合来实现的。
图7是根据本公开一种实施例的图文排版装置的示意框图。如图7所示，该图文排版装置包括：获取模块71，被配置为基于背景底图获得显著性图；查找模块72，被配置为结合待布局素材在显著性图上进行查找，得到待布局素材的候选框；第一确定模块73，被配置为确定候选框的显著性值；第二确定模块74，被配置为基于显著性值，从候选框中确定待布局素材的放置位置。
通过采用上述技术方案，由于能够基于背景底图获得显著性图，结合待布局素材在显著性图上进行查找，得到待布局素材的候选框，然后确定候选框的显著性值，并基于显著性值，从候选框中确定待布局素材的放置位置，因此能够在图文海报生产过程中自动化地布局各种素材(例如，文本框及其他素材)，而且由于是根据显著性值来确定放置位置，因此能够确保在背景底图的显著区域中不放置待布局素材，而是在背景底图的非显著性区域中放置待布局素材，从而能够在自动化布局的过程中大幅减少各个素材之间的遮挡(例如，减少文本框对背景底图中的主体的遮挡)，提升自动化海报成图的美观度。
在一些示例性实施例中，所述待布局素材的数量为多个，所述查找模块72还配置为： 按照大小优先级，从多个所述待布局素材中选取要在所述显著性图上进行查找的待布局素材；结合所选取的待布局素材在所述显著性图上进行查找。
在一些示例性实施例中，所述查找模块72还配置为：对所述待布局素材进行多尺度变换；基于多尺度变换后的所述待布局素材依次在所述显著性图上进行查找。
在一些示例性实施例中，所述第一确定模块73还配置为：确定所述显著性图的积分图；基于所述积分图，确定所述候选框的显著性值。
在一些示例性实施例中，所述图文排版装置还包括标记模块，被配置为：在从所述候选框中确定所述待布局素材的放置位置之后，在所述显著性图上将所确定的放置位置标记为显著。
在一些示例性实施例中，所述图文排版装置还包括剪裁模块，被配置为：从待尺寸转换的海报中剪裁掉所述待布局素材，得到背景图；对所述背景图进行尺寸转换，得到所述背景底图；
所述第二确定模块74还配置为：基于所述待布局素材在所述待尺寸转换的海报中的位置和尺寸，对所述显著性值进行处理；基于处理后的显著性值，从所述候选框中确定所述待布局素材的放置位置。
在一些示例性实施例中，所述第二确定模块74还可配置为：计算所述候选框的位置和尺寸与所述待布局素材在所述待尺寸转换的海报中的位置和尺寸之间的相似度；对所述相似度进行归一化处理；将归一化处理后的相似度与所述显著性值相乘，得到所述处理后的显著性值。
在一些示例性实施例中，所述第二确定模块74还可配置为：针对位置相近且所述显著性值的差值小于预设阈值的候选框，使用非极大值抑制方法从所述候选框中确定所述待布局素材的放置位置。
本公开实施例还提供一种计算机可读介质，其上存储有计算机程序，该程序被处理装置执行时实现本公开中任一项方法的步骤。
本公开实施例还提供一种电子设备，包括：存储装置，其上存储有计算机程序；处理装置，被配置为执行存储装置中的计算机程序，以实现本公开中任一项方法的步骤。
上述主要对本公开实施例提供的方案进行了介绍。可以理解的是，为了实现上述功能，电子设备包含了执行各个功能相应的硬件结构和/或软件模块。本领域技术人员应该很容易意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，本公开能够以硬件或硬件和计算机软件的结合形式来实现。某个功能究竟以硬件还是计算机软件驱动硬件的方式来执行，取决于技术方案的特定应用和设计约束条件。技术人员可以对每个特定的应 用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本公开的范围。
下面参考图8，其示出了适于用来实现本公开实施例的电子设备600的结构示意图。本公开实施例中的终端设备可以包括但不限于诸如移动电话、笔记本电脑、数字广播接收器、PDA(个人数字助理)、PAD(平板电脑)、PMP(便携式多媒体播放器)、车载终端(例如车载导航终端)等等的移动终端以及诸如数字TV、台式计算机等等的固定终端。图8示出的电子设备仅仅是一个示例，不应对本公开实施例的功能和使用范围带来任何限制。
如图8所示，电子设备600可以包括处理装置(例如中央处理器、图形处理器等)601，其可以根据存储在只读存储器(ROM)602中的程序或者从存储装置608加载到随机访问存储器(RAM)603中的程序而执行各种适当的动作和处理。在RAM 603中，还存储有电子设备600操作所需的各种程序和数据。处理装置601、ROM 602以及RAM 603通过总线604彼此相连。输入/输出(I/O)接口605也连接至总线604。
通常，以下装置可以连接至I/O接口605：包括例如触摸屏、触摸板、键盘、鼠标、摄像头、麦克风、加速度计、陀螺仪等的输入装置606；包括例如液晶显示器(LCD)、扬声器、振动器等的输出装置607；包括例如磁带、硬盘等的存储装置608；以及通信装置609。通信装置609可以允许电子设备600与其他设备进行无线或有线通信以交换数据。虽然图8示出了具有各种装置的电子设备600，但是应理解的是，并不要求实施或具备所有示出的装置。可以替代地实施或具备更多或更少的装置。
特别地，根据本公开的实施例，上文参考流程图描述的过程可以被实现为计算机软件程序。例如，本公开的实施例包括一种计算机程序产品，其包括承载在非暂态计算机可读介质上的计算机程序，该计算机程序包含用于执行流程图所示的方法的程序代码。在这样的实施例中，该计算机程序可以通过通信装置609从网络上被下载和安装，或者从存储装置608被安装，或者从ROM 602被安装。在该计算机程序被处理装置601执行时，执行本公开实施例的方法中限定的上述功能。
需要说明的是，本公开上述的计算机可读介质可以是计算机可读信号介质或者计算机可读存储介质或者是上述两者的任意组合。计算机可读存储介质例如可以是——但不限于——电、磁、光、电磁、红外线、或半导体的系统、装置或器件，或者任意以上的组合。计算机可读存储介质的更具体的例子可以包括但不限于：具有一个或多个导线的电连接、便携式计算机磁盘、硬盘、随机访问存储器(RAM)、只读存储器(ROM)、可擦式可编程只读存储器(EPROM或闪存)、光纤、便携式紧凑磁盘只读存储器(CD-ROM)、光存储器件、磁存储器件、或者上述的任意合适的组合。在本公开中，计算机可读存储介质可以是任何包含或存储程序的有形介质，该程序可以被指令执行系统、装置或者器件使用或者与 其结合使用。而在本公开中，计算机可读信号介质可以包括在基带中或者作为载波一部分传播的数据信号，其中承载了计算机可读的程序代码。这种传播的数据信号可以采用多种形式，包括但不限于电磁信号、光信号或上述的任意合适的组合。计算机可读信号介质还可以是计算机可读存储介质以外的任何计算机可读介质，该计算机可读信号介质可以发送、传播或者传输用于由指令执行系统、装置或者器件使用或者与其结合使用的程序。计算机可读介质上包含的程序代码可以用任何适当的介质传输，包括但不限于：电线、光缆、RF(射频)等等，或者上述的任意合适的组合。
在一些实施方式中，客户端、服务器可以利用诸如HTTP(HyperText Transfer Protocol，超文本传输协议)之类的任何当前已知或未来研发的网络协议进行通信，并且可以与任意形式或介质的数字数据通信(例如，通信网络)互连。通信网络的示例包括局域网(“LAN”)，广域网(“WAN”)，网际网(例如，互联网)以及端对端网络(例如，ad hoc端对端网络)，以及任何当前已知或未来研发的网络。
上述计算机可读介质可以是上述电子设备中所包含的；也可以是单独存在，而未装配入该电子设备中。上述计算机可读介质承载有一个或者多个程序，当上述一个或者多个程序被该电子设备执行时，使得该电子设备执行本公开任意实施例所提供的图文排版方法。
可以以一种或多种程序设计语言或其组合来编写用于执行本公开的操作的计算机程序代码，上述程序设计语言包括但不限于面向对象的程序设计语言—诸如Java、Smalltalk、C++，还包括常规的过程式程序设计语言——诸如“C”语言或类似的程序设计语言。程序代码可以完全地在用户计算机上执行、部分地在用户计算机上执行、作为一个独立的软件包执行、部分在用户计算机上部分在远程计算机上执行、或者完全在远程计算机或服务器上执行。在涉及远程计算机的情形中，远程计算机可以通过任意种类的网络——包括局域网(LAN)或广域网(WAN)——连接到用户计算机，或者，可以连接到外部计算机(例如利用因特网服务提供商来通过因特网连接)。
附图中的流程图和框图，图示了按照本公开各种实施例的系统、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段、或代码的一部分，该模块、程序段、或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在有些作为替换的实现中，方框中所标注的功能也可以以不同于附图中所标注的顺序发生。例如，两个接连地表示的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，框图和/或流程图中的每个方框、以及框图和/或流程图中的方框的组合，可以用执行规定的功能或操作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算 机指令的组合来实现。
描述于本公开实施例中所涉及到的模块可以通过软件的方式实现，也可以通过硬件的方式来实现。其中，模块的名称在某种情况下并不构成对该模块本身的限定，例如，获取模块还可以被描述为“用于基于背景底图获得显著性图的模块”。
本文中以上描述的功能可以至少部分地由一个或多个硬件逻辑部件来执行。例如，非限制性地，可以使用的示范类型的硬件逻辑部件包括：现场可编程门阵列(FPGA)、专用集成电路(ASIC)、专用标准产品(ASSP)、片上系统(SOC)、复杂可编程逻辑设备(CPLD)等等。
在本公开的上下文中，机器可读介质可以是有形的介质，其可以包含或存储以供指令执行系统、装置或设备使用或与指令执行系统、装置或设备结合地使用的程序。机器可读介质可以是机器可读信号介质或机器可读储存介质。机器可读介质可以包括但不限于电子的、磁性的、光学的、电磁的、红外的、或半导体系统、装置或设备，或者上述内容的任何合适组合。机器可读存储介质的更具体示例会包括基于一个或多个线的电气连接、便携式计算机盘、硬盘、随机存取存储器(RAM)、只读存储器(ROM)、可擦除可编程只读存储器(EPROM或快闪存储器)、光纤、便捷式紧凑盘只读存储器(CD-ROM)、光学储存设备、磁储存设备、或上述内容的任何合适组合。
根据本公开的一个或多个实施例，示例1提供了一种图文排版方法，包括：基于背景底图获得显著性图；结合待布局素材在所述显著性图上进行查找，得到所述待布局素材的候选框；确定所述候选框的显著性值；基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置。
根据本公开的一个或多个实施例，示例2提供了示例1的方法，其中，所述待布局素材的数量为多个，所述方法还包括：按照大小优先级，从多个所述待布局素材中选取要在所述显著性图上进行查找的待布局素材；
所述结合待布局素材在所述显著性图上进行查找，包括：结合所选取的待布局素材在所述显著性图上进行查找。
根据本公开的一个或多个实施例，示例3提供了示例1的方法，其中，所述方法还包括：对所述待布局素材进行多尺度变换；
所述结合待布局素材在所述显著性图上进行查找，包括：基于多尺度变换后的待布局素材依次在所述显著性图上进行查找。
根据本公开的一个或多个实施例，示例4提供了示例1的方法，其中，所述确定所述 候选框的显著性值，包括：确定所述显著性图的积分图；基于所述积分图，确定所述候选框的显著性值。
根据本公开的一个或多个实施例，示例5提供了示例1的方法，其中，所述方法还包括：在从所述候选框中确定所述待布局素材的放置位置之后，在所述显著性图上将所确定的放置位置标记为显著。
根据本公开的一个或多个实施例，示例6提供了示例1至5中任一项的方法，其中，所述方法还包括：从待尺寸转换的海报中剪裁掉所述待布局素材，得到背景图；对所述背景图进行尺寸转换，得到所述背景底图；
所述基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置，包括：基于所述待布局素材在所述待尺寸转换的海报中的位置和尺寸，对所述显著性值进行处理；基于处理后的显著性值，从所述候选框中确定所述待布局素材的放置位置。
根据本公开的一个或多个实施例，示例7提供了示例6的方法，其中，所述基于所述待布局素材在所述待尺寸转换的海报中的位置和尺寸，对所述显著性值进行处理，包括：计算所述候选框的位置和尺寸与所述待布局素材在所述待尺寸转换的海报中的位置和尺寸之间的相似度；对所述相似度进行归一化处理；将归一化处理后的相似度与所述显著性值相乘，得到所述处理后的显著性值。
根据本公开的一个或多个实施例，示例8提供了示例1至5和7中任一项的方法，其中，所述基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置，包括：针对位置相近且所述显著性值的差值小于预设阈值的候选框，使用非极大值抑制方法从所述候选框中确定所述待布局素材的放置位置。
根据本公开的一个或多个实施例，示例9提供了一种图文排版装置，包括：获取模块，被配置为基于背景底图获得显著性图；查找模块，被配置为结合待布局素材在所述显著性图上进行查找，得到所述待布局素材的候选框；第一确定模块，被配置为确定所述候选框的显著性值；第二确定模块，被配置为基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置。
根据本公开的一个或多个实施例，示例10提供了一种计算机可读介质，其上存储有计算机程序和/或指令，该计算机程序和/或指令被处理装置执行时实现示例1-8中任一项所述方法的步骤。
根据本公开的一个或多个实施例，示例11提供了一种电子设备，包括：存储装置，其上存储有计算机程序和/或指令；处理装置，被配置为执行所述存储装置中的所述计算机程序和/或指令，以实现示例1-8中任一项所述方法的步骤。
根据本公开的一个或多个实施例，示例12提供了一种计算机程序产品，包含指令，该指令在由处理器执行时使得处理器实现示例1至8中任一项示例所述方法的步骤。
根据本公开的一个或多个实施例，示例13提供了一种计算机程序，包括程序代码，该程序代码在由处理器执行时导致实现示例1至8中任一项示例所述方法的步骤。
以上描述仅为本公开的较佳实施例以及对所运用技术原理的说明。本领域技术人员应当理解，本公开中所涉及的公开范围，并不限于上述技术特征的特定组合而成的技术方案，同时也应涵盖在不脱离上述公开构思的情况下，由上述技术特征或其等同特征进行任意组合而形成的其它技术方案。例如上述特征与本公开中公开的(但不限于)具有类似功能的技术特征进行互相替换而形成的技术方案。
此外，虽然采用特定次序描绘了各操作，但是这不应当理解为要求这些操作以所示出的特定次序或以顺序次序执行来执行。在一定环境下，多任务和并行处理可能是有利的。同样地，虽然在上面论述中包含了若干具体实现细节，但是这些不应当被解释为对本公开的范围的限制。在单独的实施例的上下文中描述的某些特征还可以组合地实现在单个实施例中。相反地，在单个实施例的上下文中描述的各种特征也可以单独地或以任何合适的子组合的方式实现在多个实施例中。
尽管已经采用特定于结构特征和/或方法逻辑动作的语言描述了本主题，但是应当理解所附权利要求书中所限定的主题未必局限于上面描述的特定特征或动作。相反，上面所描述的特定特征和动作仅仅是实现权利要求书的示例形式。关于上述实施例中的装置，其中各个模块执行操作的具体方式已经在有关该方法的实施例中进行了详细描述，此处将不做详细阐述说明。

Claims (13)

  1. 一种图文排版方法，包括：
    基于背景底图获得显著性图；
    结合待布局素材在所述显著性图上进行查找，得到所述待布局素材的候选框；
    确定所述候选框的显著性值；
    基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置。
  2. 根据权利要求1所述的方法，其中，所述待布局素材的数量为多个，所述方法还包括：按照大小优先级，从多个所述待布局素材中选取要在所述显著性图上进行查找的待布局素材；
    所述结合待布局素材在所述显著性图上进行查找，包括：结合所选取的待布局素材在所述显著性图上进行查找。
  3. 根据权利要求1所述的方法，其中，所述方法还包括：对所述待布局素材进行多尺度变换；
    所述结合待布局素材在所述显著性图上进行查找，包括：基于多尺度变换后的待布局素材依次在所述显著性图上进行查找。
  4. 根据权利要求1所述的方法，其中，所述确定所述候选框的显著性值，包括：
    确定所述显著性图的积分图；
    基于所述积分图，确定所述候选框的显著性值。
  5. 根据权利要求1所述的方法，其中，所述方法还包括：在从所述候选框中确定所述待布局素材的放置位置之后，在所述显著性图上将所确定的放置位置标记为显著。
  6. 根据权利要求1至5中任一项所述的方法，其中，所述方法还包括：从待尺寸转换的海报中剪裁掉所述待布局素材，得到背景图；对所述背景图进行尺寸转换，得到所述背景底图；
    所述基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置，包括：基于所述待布局素材在所述待尺寸转换的海报中的位置和尺寸，对所述显著性值进行处理；基于处理后的显著性值，从所述候选框中确定所述待布局素材的放置位置。
  7. 根据权利要求6所述的方法，其中，所述基于所述待布局素材在所述待尺寸转换的海报中的位置和尺寸，对所述显著性值进行处理，包括：
    计算所述候选框的位置和尺寸与所述待布局素材在所述待尺寸转换的海报中的位置和尺寸之间的相似度；
    对所述相似度进行归一化处理；
    将归一化处理后的相似度与所述显著性值相乘，得到所述处理后的显著性值。
  8. 根据权利要求1至5和7中任一项所述的方法，其中，所述基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置，包括：针对位置相近且所述显著性值的差值小于预设阈值的候选框，使用非极大值抑制方法从所述候选框中确定所述待布局素材的放置位置。
  9. 一种图文排版装置，包括：
    获取模块，被配置为基于背景底图获得显著性图；
    查找模块，被配置为结合待布局素材在所述显著性图上进行查找，得到所述待布局素材的候选框；
    第一确定模块，被配置为确定所述候选框的显著性值；
    第二确定模块，被配置为基于所述显著性值，从所述候选框中确定所述待布局素材的放置位置。
  10. 一种计算机可读介质，其上存储有计算机程序，其中，该程序被处理装置执行时实现根据权利要求1-8中任一项所述方法。
  11. 一种电子设备，其中，包括：
    存储装置，其上存储有计算机程序；
    处理装置，被配置为执行所述存储装置中的所述计算机程序，以实现根据权利要求1-8中任一项所述方法。
  12. 一种计算机程序产品，包含指令，该指令在由处理器执行时使得处理器实现根据权利要求1到8中任一项所述的方法。
  13. 一种计算机程序，包括程序代码，该程序代码在由处理器执行时导致实现根据权利要求1到8中任一项所述的方法。
PCT/CN2024/096949 2023-06-06 2024-06-03 图文排版方法、装置、介质和电子设备 Ceased WO2024251060A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310667291.9A CN116541544A (zh) 2023-06-06 2023-06-06 图文排版方法、装置、介质和电子设备
CN202310667291.9 2023-06-06

Publications (1)

Publication Number Publication Date
WO2024251060A1 true WO2024251060A1 (zh) 2024-12-12

Family

ID=87447189

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/096949 Ceased WO2024251060A1 (zh) 2023-06-06 2024-06-03 图文排版方法、装置、介质和电子设备

Country Status (2)

Country Link
CN (1) CN116541544A (zh)
WO (1) WO2024251060A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116541544A (zh) * 2023-06-06 2023-08-04 北京字跳网络技术有限公司 图文排版方法、装置、介质和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110795925A (zh) * 2019-10-12 2020-02-14 腾讯科技（深圳）有限公司 基于人工智能的图文排版方法、图文排版装置及电子设备
CN111428447A (zh) * 2020-02-28 2020-07-17 华东师范大学 一种基于显著性检测的智能图文排版方法
US20220139013A1 (en) * 2019-03-01 2022-05-05 Huawei Technologies Co., Ltd. Image and Text Typesetting Method and Related Apparatus Thereof
CN114677402A (zh) * 2022-04-07 2022-06-28 北京智源人工智能研究院 海报文本布局、海报生成方法及相关装置
CN116541544A (zh) * 2023-06-06 2023-08-04 北京字跳网络技术有限公司 图文排版方法、装置、介质和电子设备

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107545576A (zh) * 2017-07-31 2018-01-05 华南农业大学 基于构图规则的图像编辑方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20220139013A1 (en) * 2019-03-01 2022-05-05 Huawei Technologies Co., Ltd. Image and Text Typesetting Method and Related Apparatus Thereof
CN110795925A (zh) * 2019-10-12 2020-02-14 腾讯科技（深圳）有限公司 基于人工智能的图文排版方法、图文排版装置及电子设备
CN111428447A (zh) * 2020-02-28 2020-07-17 华东师范大学 一种基于显著性检测的智能图文排版方法
CN114677402A (zh) * 2022-04-07 2022-06-28 北京智源人工智能研究院 海报文本布局、海报生成方法及相关装置
CN116541544A (zh) * 2023-06-06 2023-08-04 北京字跳网络技术有限公司 图文排版方法、装置、介质和电子设备

Also Published As

Publication number Publication date
CN116541544A (zh) 2023-08-04

Similar Documents

Publication Publication Date Title
CN112487331B (zh) 页面生成方法和装置、存储介质和电子设备
CN112258622B (zh) 图像处理方法、装置、可读介质及电子设备
WO2024251228A1 (zh) 媒体内容展示方法、装置、电子设备及存储介质
CN112418232B (zh) 图像分割方法、装置、可读介质及电子设备
US20250022136A1 (en) Image cropping method and apparatus, model training method and apparatus, electronic device, and medium
US20230334880A1 (en) Hot word extraction method and apparatus, electronic device, and medium
CN112418054B (zh) 图像处理方法、装置、电子设备和计算机可读介质
CN118132056A (zh) 页面开发方法、装置、设备及存储介质
CN110619597A (zh) 一种半透明水印去除方法、装置、电子设备及存储介质
CN113255812B (zh) 视频边框检测方法、装置和电子设备
WO2025092611A1 (zh) 信息展示方法、装置、电子设备和存储介质
CN118840453A (zh) 文字颜色确定方法、装置、介质、电子设备及程序产品
CN112418233B (zh) 图像处理方法、装置、可读介质及电子设备
CN112257598B (zh) 图像中四边形的识别方法、装置、可读介质和电子设备
WO2024251060A1 (zh) 图文排版方法、装置、介质和电子设备
WO2024120446A1 (zh) 特效道具、特效图的生成方法、装置、设备及存储介质
CN113762260A (zh) 一种版面图片的处理方法、装置、设备及存储介质
CN115731313A (zh) Svg格式的图片的处理方法、装置、设备、介质及产品
WO2025108303A1 (zh) 视频生成方法、装置、电子设备和存储介质
CN111258582B (zh) 一种窗口渲染方法、装置、计算机设备及存储介质
CN114786069B (zh) 视频生成方法、装置、介质及电子设备
WO2025045169A1 (zh) 图像处理模板处理方法、装置、设备、存储介质及产品
WO2024152797A1 (zh) 视频补全方法、装置、介质和电子设备
WO2025001726A1 (zh) 封面的设置方法、装置、电子设备和存储介质
WO2024198952A1 (zh) 图像超分辨率方法、设备、存储介质及程序产品

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24818592

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE