WO2024005590A1 - Dispositif de mise à l'échelle d'image et procédé de mise à l'échelle d'image - Google Patents

Dispositif de mise à l'échelle d'image et procédé de mise à l'échelle d'image Download PDF

Info

Publication number
WO2024005590A1
WO2024005590A1 PCT/KR2023/009230 KR2023009230W WO2024005590A1 WO 2024005590 A1 WO2024005590 A1 WO 2024005590A1 KR 2023009230 W KR2023009230 W KR 2023009230W WO 2024005590 A1 WO2024005590 A1 WO 2024005590A1
Authority
WO
WIPO (PCT)
Prior art keywords
groups
value
transform
conversion
information
Prior art date
Application number
PCT/KR2023/009230
Other languages
English (en)
Korean (ko)
Inventor
허훈
박연숙
Original Assignee
주식회사 네패스
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 주식회사 네패스 filed Critical 주식회사 네패스
Publication of WO2024005590A1 publication Critical patent/WO2024005590A1/fr

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T1/00—General purpose image data processing
    • G06T1/60—Memory management
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformation in the plane of the image
    • G06T3/40—Scaling the whole image or part thereof
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes

Definitions

  • the present invention relates to an image scaling device and an image scaling method.
  • AI learning or inference accelerators that learn images as input data are being actively developed.
  • images may have different sizes, and the image resolution designed for each AI learning or inference accelerator is different, so it is necessary to convert the size of the input image to suit the design of the accelerator.
  • the purpose of the present invention is to provide an image scaling device and an image scaling method with a higher degree of freedom in conversion size.
  • the purpose of the present invention is to provide an image scaling device and an image scaling method that convert the size of an image to higher quality.
  • An image scaling device comprising: two or more memories; Receive an input image, information about the size of the input image, and information about the transformation size, and based on the size of the input image and the transformation size, a first transformation multiplier corresponding to the width and a second transformation multiplier corresponding to the height Identifying and dividing the plurality of pixels of the input image into a first plurality of groups based on any one of the first transformation multiplier and the second transformation multiplier, and sequentially providing information about the first plurality of groups. is stored in the two or more memories, and the first plurality of groups are grouped into a second plurality of groups based on the remaining one of the first conversion multiplier and the second conversion multiple. It includes a processor that generates a converted image by identifying the average value of pixels included in each group.
  • the processor includes a division design block and a fixed-point design block, and the division design block divides the size of the input image and the transformation size and divides the quotient and the remainder into the first transformation multiplier and the second transformation multiplier.
  • the conversion multiplier is identified as an integer value
  • the fixed-point design block can identify decimal values of the first conversion multiple and the second conversion multiple using the remainder and the conversion size.
  • the fixed-point design block (1) If the conversion size is less than twice the remainder, sets the most significant bit value of the decimal value to 1, and outputs a value obtained by subtracting the conversion size from two times the remainder. or, if the conversion size is not smaller than twice the remainder, the most significant bit value of the decimal value is set to 0, and a value obtained by doubling the remainder is output, and (2) the conversion size is equal to that of the previous step. If the output value is less than 2 times, the next bit value of the decimal value is set to 1, and a value obtained by subtracting the conversion size from 2 times the output value of the previous step is output, or the conversion size is equal to the output value of the previous step. If it is not less than twice, the next bit value of the decimal value is designated as 0, and a value twice the output value of the previous step is output. (3) When the decimal value has 8 binary bit values, You can repeat step (2) above until.
  • the processor groups the pixels included in each line in the width direction of the input image according to the first transform multiple, and groups the pixels included in each line in the width direction of the input image according to the first transform multiple.
  • the first plurality of groups can be generated by grouping pixels included in each line in the height direction of the input image according to the second transformation multiplier.
  • the processor is configured to perform a line in the width direction of the input image when the one transform multiple is the first transform multiple, and for each line in the height direction of the input image when the one transform multiple is the second transform multiple.
  • Information about the first plurality of groups may be separately stored in the two or more memories.
  • the information about the first plurality of groups may include information about the sum of the number of pixels included in each group and the pixel values of the pixels included in each group.
  • the processor may group the first plurality of groups into the second plurality of groups while sequentially reading information about the first plurality of groups from the two or more memories based on the remaining conversion multiple.
  • the processor stores information about the second plurality of groups, including information on the sum of the number of pixels included in each group among the second plurality of groups and pixel values of pixels included in each group, in the two or more memories. You can save it.
  • the processor sequentially reads information about the first plurality of groups as many lines as the number of lines corresponding to the integer value of the remaining conversion multiple, and groups the first plurality of groups into the second plurality of groups, The sum of the decimal values of the remaining conversion multiples is calculated as the number of the second plurality of groups, and each time the sum of the decimal values exceeds 1, one more number than the number of lines is read to form the first plurality of groups. It can be grouped into the second plurality group.
  • the processor identifies pixel values of a plurality of pixels of the input image, identifies a plurality of intermediate pixel values between each pixel and at least one neighboring pixel, and based on the first transformation multiplier, the pixel value and Store first up-scaling information including the intermediate pixel value corresponding to the number of pixels to be added in the two or more memories, and store first up-scaling information including the intermediate pixel value corresponding to the number of pixels to be added, and corresponding to the number of pixels to be added based on the first up-scaling information and the second transform multiplier.
  • the converted image may be generated by storing second up-scaling information including intermediate pixel values in the two or more memories, and sequentially reading the second up-scaling information stored in the two or more memories.
  • An image scaling method performed by an image scaling device comprising: receiving an input image, information about the size of the input image, and information about the transformation size; identifying a first transform multiplier corresponding to the width and a second transform multiplier corresponding to the height based on the size of the input image and the transform size; Dividing a plurality of pixels of the input image into a first plurality of groups based on one of the first transformation multiplier and the second transformation multiplier and sequentially storing information about the first plurality of groups ; After grouping the first plurality of groups into a second plurality of groups based on the remaining one of the first transformation multiplier and the second transformation multiplier, the average value of the pixels included in each of the second plurality of groups is calculated. It includes the step of identifying and generating a converted image.
  • the step of identifying the first transform multiplier and the second transform multiplier includes dividing the quotient obtained by dividing the size of the input image and the transform size and dividing the quotient among the remainder into integers of the first transform multiplier and the second transform multiplier. identifying by value; It may include identifying decimal values of the first transform multiple and the second transform multiple using the remainder and the transform size.
  • the step of identifying the decimal value of the first transform multiple and the second transform multiple includes (1) when the transform size is less than twice the remainder, specifying the highest bit value of the decimal value as 1, and Output a value obtained by subtracting the conversion size from 2 times the remainder, or, if the conversion size is not smaller than 2 times the remainder, specify the most significant bit value of the decimal value as 0, and output a value obtained by subtracting the conversion size from 2 times the remainder.
  • step 2 If the conversion size is less than twice the output value of the previous step, set the next bit value of the decimal value to 1, and output a value obtained by subtracting the conversion size from twice the output value of the previous step, Or, if the conversion size is not smaller than twice the output value of the previous step, designating the next bit value of the decimal value as 0 and outputting a value that is twice the output value of the previous step; (3) It may include repeating step (2) until the decimal value has 8 binary bit values.
  • the step of storing information about the first plurality of groups includes grouping pixels included in each line in the width direction of the input image according to the first transform multiplier when one of the transform multipliers is the first transform multiplier. (grouping), and when one of the transformation multipliers is the second transformation multiplier, the pixels included in each line in the height direction of the input image are grouped according to the second transformation multiplier to generate the first plurality of groups. May include steps.
  • the step of storing information about the first plurality of groups may include, if the one transform multiple is the first transform multiple, for each line in the width direction of the input image, the one transform multiple is the second transform multiple. If it is a multiple, the method may include storing information about the first plurality of groups for each line in the height direction of the input image.
  • the step of generating the conversion image includes sequentially reading information about the first plurality of groups stored based on the remaining conversion multiple and grouping the first plurality of groups into the second plurality of groups. It can be included.
  • the step of grouping the first plurality of groups into the second plurality of groups includes information on the sum of the number of pixels included in each group among the second plurality of groups and the pixel values of the pixels included in each group. It may include the step of storing information about the second plurality of groups.
  • the step of grouping the first plurality of groups includes sequentially reading information about the first plurality of groups as many lines as the number of lines corresponding to the integer value of the remaining conversion multiple, thereby dividing the first plurality of groups into the second plurality of groups. Grouping into a plurality of groups; calculating the sum of the decimal values of the remaining conversion multiples as many as the number of the second plurality of groups; The method may include grouping the first plurality of groups into the second plurality of groups by reading one more line than the number of lines whenever the sum of the decimal values exceeds 1.
  • identifying pixel values of a plurality of pixels of the input image identifying a plurality of intermediate pixel values between each pixel and at least one neighboring pixel; storing first up-scaling information including the pixel value and the intermediate pixel value corresponding to the number of pixels to be added based on the first conversion multiplier; storing second up-scaling information including an intermediate pixel value corresponding to the number of pixels to be added based on the first up-scaling information and the second transform multiplier; It may include generating the converted image by sequentially reading the stored second up-scaling information.
  • scaling can be performed faster and with fewer constraints on conversion multiples through a hardware configuration.
  • the conversion multiplier can be freely configured, so the usability of the conversion image, such as for AI learning, can be increased.
  • learning performance can be improved.
  • FIG. 1 is a diagram schematically showing an image scaling device according to an embodiment of the present invention.
  • Figure 2 is a block diagram showing the configuration of an image scaling device according to an embodiment of the present invention.
  • Figure 3 is a diagram illustrating an operation flowchart of an image scaling device according to an embodiment of the present invention.
  • FIG. 4 is a diagram illustrating an operation flowchart for identifying a conversion multiplier of an image scaling device according to an embodiment of the present invention.
  • Figure 5 is a diagram showing a downscaling operation of an image scaling device according to an embodiment of the present invention.
  • Figure 6 is a diagram showing the operation of an image scaling device according to an embodiment of the present invention.
  • Figure 7 is a diagram illustrating an upscaling operation of an image scaling device according to an embodiment of the present invention.
  • FIG. 8 is a diagram illustrating a lookup table for the number of pixels to be added according to the conversion multiplier during upscaling of the image scaling device according to an embodiment of the present invention.
  • Figure 9 is a diagram showing the operation of an image scaling device according to an embodiment of the present invention.
  • FIG. 1 is a diagram schematically showing an image scaling device according to an embodiment of the present invention.
  • the image scaling device 100 (hereinafter referred to as the scaling device 100) shown in FIG. 1 is a device that converts the size of the input image 10 into a converted image 30 according to size information 20. As such, it can be implemented with a computer, server, etc.
  • the input image 10 is not limited in format and may include dynamic images as well as static images.
  • size conversion may be applied on a frame-by-frame basis.
  • the size information 20 may include information about the size of the input image 10 and information about the size to be converted (hereinafter referred to as conversion size).
  • the scaling device 100 may perform upscaling to enlarge the image 10 and downscaling to reduce the image 10 by comparing the transformation size with the size of the input image 10.
  • Information about the size of the input image 10 may include the width and height of the input image, and information about the converted size may include the width and height of the converted image. However, information about the size of the input image 10 can be identified from the input image 10 received by the scaling device 100.
  • the converted image 30 is image data output by the scaling device 100.
  • the conversion multiple the multiple to be converted
  • the present invention proposes a scaling device 100 based on hardware design that has faster processing speed than software programming and has a high degree of freedom in conversion multiples.
  • Figure 2 is a block diagram showing the configuration of an image scaling device according to an embodiment of the present invention.
  • the scaling device 100 includes an input unit 110, a communication unit 120, a display unit 130, a memory 140, and a processor 150.
  • the input unit 110 generates input data in response to user input of the scaling device 100.
  • user input for performing image scaling such as user input regarding the conversion size, it can be applied without limitation.
  • the input unit 110 includes at least one input means.
  • the input unit 110 includes a keyboard, key pad, dome switch, touch panel, touch key, mouse, menu button, etc. may include.
  • the communication unit 120 receives the input image, information about the size of the input image, and information about the conversion size, and performs communication with external devices such as an AI learner and a server to transmit the converted image. To this end, the communication unit 120 may perform communications such as 5th generation communication (5G), long term evolution-advanced (LTE-A), long term evolution (LTE), and wireless fidelity (Wi-Fi).
  • 5G 5th generation communication
  • LTE-A long term evolution-advanced
  • LTE long term evolution
  • Wi-Fi wireless fidelity
  • the communication unit 120 includes a connector or port to which a cable capable of receiving the input image, information about the size of the input image, and information about the conversion size can be connected.
  • Connectors or ports may include USB ports, HDMI ports, DisplayPort, DVI ports, SCART, etc.
  • the communication unit 120 may include a connector or port according to a network transmission standard such as Ethernet.
  • the communication unit 120 may be implemented as a LAN card wired to a router or gateway.
  • the display unit 130 displays display data according to the operation of the scaling device 100.
  • the display unit 130 may display a screen including an input image, a screen for inputting a conversion size, a screen including a generated converted image, etc.
  • the display unit 130 includes a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, and a micro electro mechanical systems (MEMS) display. and electronic paper displays.
  • LCD liquid crystal display
  • LED light emitting diode
  • OLED organic light emitting diode
  • MEMS micro electro mechanical systems
  • the display unit 130 may be combined with the input unit 110 and implemented as a touch screen.
  • the memory 140 stores operation programs of the scaling device 100.
  • the memory 140 is a non-volatile storage that can preserve data (information) regardless of whether power is provided, and data to be processed by the processor 150 is loaded. If power is not provided, the data is stored. Includes memory with volatile properties that cannot be preserved. Storage includes flash memory, HDD (hard-disc drive), SSD (solid-state drive), ROM (Read Only Memory), etc., and memory includes buffer and RAM (Random Access Memory). etc.
  • it may include two or more memories. At this time, two or more memories store information for each line in the width direction or height direction of the image, but if the number of lines is greater than the number of memories, the memory can be used like a ring buffer to sequentially store information.
  • the memory 140 stores information about the plurality of groups or is necessary in the process of performing image scaling. Calculation programs, etc. can be stored.
  • the processor 150 may control at least one other component (eg, hardware or software component) of the scaling device 100 by executing software such as a program, and may perform various data processing or calculations.
  • software such as a program
  • the processor 150 may include a division design block 151 and a fixed point design block 152. At this time, the division design block 151 and the fixed point design block 152 may be configured as circuits electrically connected to each other.
  • the conversion multiple can be identified using the division design block 151 and the fixed point design block 152.
  • the transformation multiplier according to an embodiment of the present invention is a transformation multiplier corresponding to the width of the input image (hereinafter referred to as the first transformation multiplier) and a transformation multiplier corresponding to the height of the input image (hereinafter referred to as the second transformation multiplier). .) includes. At this time, the conversion multiple consists of an integer value and a decimal value.
  • the division design block 151 divides the size and transformation size of the input image 10 to obtain a quotient and a remainder, and identifies the obtained quotient as an integer value of the transformation multiple.
  • the decimal value may appear as a finite decimal number, but it may also appear as an infinite decimal number.
  • the width of the input image is 10 pixels and the width of the conversion size is 4 pixels
  • the integer value of the conversion multiplier is 2
  • the decimal value is 0.5
  • the conversion multiple is 1/2.5 times.
  • decimal values must be expressed as binary numbers in a circuit.
  • the scaling device 100 uses a fixed-point design block 152 designed to express the decimal value of the conversion multiple in binary bit units.
  • the fixed point design block 152 may identify decimal values of the first and second conversion multiples using the remainder and conversion size identified by the division design block 151.
  • the processor 150 receives an input image, information about the size of the input image, and information about the transformation size, and based on the size of the input image and the transformation size, a first transformation multiple corresponding to the width and a second transformation multiple corresponding to the height. Identify the transformation multiplier, divide the plurality of pixels of the input image into first plurality groups based on either the first transformation multiplier or the second transformation multiplier, and sequentially store information about the first plurality groups in two or more memories. After storing the first plurality of groups into the second plurality of groups based on the remaining one of the first transformation multiplier and the second transformation multiplier, the average value of the pixels included in each of the second plurality of groups is identified to create a converted image. creates .
  • Figure 3 is a diagram illustrating an operation flowchart of an image scaling device according to an embodiment of the present invention.
  • the processor 150 receives the input image 10, information about the size of the input image 10, and information about the conversion size (S10).
  • the input image 10 can be received from the outside, but is not limited to this and can be pre-stored in the memory 140 and used.
  • the processor 150 may identify information about the size of the input image 10 from the input image 10, and receive user input about the size or conversion size of the input image 10 through the input unit 110. , information about the conversion size can be received from the outside.
  • the processor 150 identifies a first transform multiplier corresponding to the width and a second transform multiplier corresponding to the height based on the size and transform size of the input image (S20).
  • the processor 150 includes a division design block 151 and a fixed point design block 152, and conversion is performed using the division design block 151 and the fixed point design block 152. Multiples can be identified.
  • the division design block 151 may obtain a quotient and a remainder by dividing the width of the input image 10 and the width of the transformation size, and identify the quotient as an integer value of the first transformation multiple. Likewise, the division design block 151 may obtain a quotient and a remainder by dividing the height of the input image 10 and the height of the transformation size, and identify the quotient as an integer value of the second transformation multiple.
  • the fixed point design block 152 uses the remainder and the transform size identified by the division design block 151 to identify the decimal value of the transform multiple. At this time, the fixed point design block 152 may receive the remaining information from the division design block 151.
  • the processor 150 divides a plurality of pixels of the input image into a first plurality of groups based on either a first transform multiplier or a second transform multiplier and sequentially divides the plurality of pixels into the first plurality groups.
  • Information about is stored in two or more memories (S30).
  • the scaling process consists of reduction/enlargement in the width and height directions, respectively. At this time, the scaling order in the width and height directions does not affect the final result.
  • the converted image obtained by first reducing it by 1/2.5 times in the width direction and then reducing it by 1/4.5 times in the height direction, and The converted image obtained by reducing the image by 1/4.5 times in the height direction and then reducing it by 1/2.5 times in the width direction is the same.
  • the processor 150 may divide a plurality of pixels of the input image into a first plurality of groups according to a conversion multiplier in the width direction or the height direction, whichever scales first.
  • the processor 150 selects the pixels included for each line in the width direction among the plurality of pixels forming the input image to the first transformation multiplier.
  • a first plurality of groups can be created by grouping accordingly.
  • one transformation multiplier is a second transformation multiplier, that is, when scaling is performed in the height direction first
  • the processor 150 divides the pixels included for each line in the height direction among the plurality of pixels constituting the input image into a second transformation multiplier.
  • a first plurality of groups can be created by grouping according to the conversion multiple. A specific example of performing grouping will be described with reference to FIG. 5.
  • the processor 150 may sequentially store information about the first plurality of groups in two or more memories.
  • the information about the first plurality of groups includes information about the sum of the number of pixels included in each group and the pixel values of the pixels included in each group. The process of storing in memory will be described in detail with reference to FIG. 6.
  • the processor 150 groups the first plurality of groups into a second plurality of groups based on the remaining one of the first transformation multiplier and the second transformation multiplier, and then groups each of the second plurality groups.
  • the converted image 30 is generated by identifying the average value of the pixels included in (S40).
  • the processor 150 sequentially reads information about the first plurality of groups as many lines as the number of lines corresponding to the integer value of the remaining conversion multiple, and converts the first plurality of groups into the second plurality of groups. Can be grouped into groups.
  • the processor 150 sequentially reads information about the first plurality of groups as many lines as the number of lines corresponding to the integer value of the second transformation multiplier. .
  • the processor 150 generates a second plurality of groups by reading one more line than the number of lines corresponding to the integer value each time the sum of the decimal values of the conversion multiples exceeds 1 while reading.
  • the converted image 30 is ultimately created by continuously performing the process of reducing a plurality of pixels to one pixel in the case of downscaling, and the processor 150 determines the average value of the pixels included in each group for each of the second plurality of groups.
  • the converted image 30 can be generated by identifying .
  • scaling can be performed faster and with fewer constraints on conversion multiples through a hardware configuration.
  • the conversion multiplier can be freely configured, so the usability of the conversion image, such as for AI learning, can be increased.
  • learning performance can be improved.
  • FIG. 4 is a diagram illustrating an operation flowchart for identifying a conversion multiplier of an image scaling device according to an embodiment of the present invention.
  • FIG. 4 shows an operational flow diagram of the fixed point design block 152 identifying a conversion multiple in relation to S20 of FIG. 3.
  • the input image 10 has a size of 9 pixels * 13 pixels
  • the conversion size has a size of 2 pixels * 7 pixels
  • the first image corresponding to the height of the input image 10 is assumed to be 2 pixels * 7 pixels. 2Describes the process of calculating the decimal value of a conversion multiple. At this time, the decimal value is identified in bit units as described above.
  • the fixed-point design block 152 receives information about the remainder from the division design block 151 and receives information about the transform size (S21). At this time, when 13 is divided by 7, the quotient is 1 and the remainder is 6. The corresponding transformation size is 7.
  • the fixed point design block 152 identifies whether the transform size is less than twice the remainder (S22). 7 is less than 12, which is twice 6 (Yes in S22).
  • the fixed point design block 152 specifies the highest bit value of the decimal value as 1 and outputs (remainder * 2 - conversion size). (S23).
  • decimal values will be represented by 8 bits. At this time, the highest bit value means the first bit value among 8 bits.
  • the fixed-point design block 152 sets the highest bit value of the decimal value to 0 and outputs the remainder doubled (S24).
  • the fixed point design block 152 identifies whether the transform size is less than twice the previous output value (S25).
  • the previous output value refers to the value output by the fixed-point design block 152 before each step.
  • the fixed point design block 152 sets the next bit value of the decimal value to 1 and outputs (previous output value * 2 - conversion size) ( S26).
  • the fixed point design block 152 sets the next bit value of the decimal value to 0 and outputs the previous output value by doubling it (S27).
  • the fixed-point design block 152 checks whether the bit value for the decimal value has been completed with 8 bits (S28), and if not yet completed (No in S28), returns to step S25 to identify the next bit value.
  • the integer value of the second conversion multiple is 1, and the decimal value can be expressed as 0.11011011(2) in binary.
  • Figure 5 is a diagram showing a downscaling operation of an image scaling device according to an embodiment of the present invention.
  • FIG. 5 provides a detailed explanation of dividing a plurality of pixels of an input image into a first plurality of groups and a second plurality of groups with reference to S30 and S40 of FIG. 3 .
  • the example assumes that, as in FIG. 4, the input image 10 has a size of 9 pixels*13 pixels, the transformation size has a size of 2 pixels*7 pixels, and scaling is performed first based on the first transformation multiple. do.
  • the first transformation multiple in the width direction is 1/4.5.
  • the processor 150 may divide the plurality of pixels of the input image 10 into a first plurality of groups based on the first transformation multiplier.
  • the processor 150 may generate a first plurality of groups by grouping pixels included in each line in the width direction among the plurality of pixels forming the input image 10 according to the first transformation multiplier.
  • the processor 150 groups the 9 pixels included in line 1 of the lines in the width direction of the input image 10 into one group 11 with the integer value of the first transform multiple being 4. , the remaining 5 pixels can be grouped into another group (12).
  • group 12 since the integer 1 is obtained by combining the decimal value 0.5 of the first conversion multiple of group 11 and the decimal value 0.5 of the first conversion multiple of group 12, group 12 has the integer value 4 of the first conversion multiple. It consists of 5 pixels, including one more pixel.
  • the processor 150 stores information about the first plurality of groups in the memory 140, including information on the sum of the number of pixels included in each group and the pixel values of the pixels included in each group.
  • the pixel value may include brightness information, color information, etc. of the corresponding pixel.
  • the pixel value can range between 0 and 255 by expressing 1 pixel in 8 bits. Black has a pixel value of 0 and white has a pixel value of 255.
  • the pixel value can range between 0 and 255 for each color component by expressing 1 pixel with 8 bits for each color component of R (red), G (green), and B (blue). there is.
  • black has a pixel value of (0, 0, 0)
  • white has a pixel value of (255, 255, 255).
  • the pixel value of all pixels is 10, in the case of group 11, information on 4 pixels and the total of 40 pixel values is stored in the memory.
  • the image 510 can be generated reduced in the width direction. At this time, group 11 can be converted into pixel 511 and group 12 can be converted into pixel 512.
  • the present invention does not directly scale, but sequentially stores information about the first plurality of groups in the memory 140.
  • the processor 150 sequentially reads information about the first plurality of groups as many lines as the number of lines corresponding to the integer value of the remaining second conversion multiple and groups the first plurality of groups into the second plurality of groups. You can.
  • the second transformation multiplier in the height direction is 1/1.85546875.
  • the processor 150 sequentially reads information about the first plurality of groups as many lines as the number of lines corresponding to the integer value of the second conversion multiple. At this time, the processor 150 generates a second plurality of groups by reading one more line than the number of lines corresponding to the integer value each time the sum of the decimal values of the conversion multiples exceeds 1 while reading.
  • the processor 150 can read group 11 and group it into group 31.
  • the sum of the decimal value 0.85546875 of group (11) and the decimal value of group (13) 0.85546875 exceeds 1 (1.7109375), so read the two groups (13) and group (14) with one more than 1 of the integer value. It can be grouped into groups (32).
  • the 26 first plural groups can be grouped into 14 second plural groups.
  • the processor 150 may generate the converted image 30 by identifying the average value of pixels included in each group for each of the second plurality of groups.
  • group 31 has 4 pixels and 40 pixel values, and has a pixel value of 10, which is the average value of the pixels.
  • Group 32 has 8 pixels and 80 pixel values, and has a pixel value of 10, which is the average value of the pixels.
  • Group 33 has 8 pixels and 80 pixel values, and has a pixel value of 10, which is the average value of the pixels.
  • Figure 6 is a diagram showing the operation of an image scaling device according to an embodiment of the present invention.
  • FIG. 6 shows the scaling process described with reference to FIG. 5 being performed using memory.
  • the scaling device 100 has two or more memories 140.
  • the memory 140 consists of 17 memories. At this time, the number and structure of memories are not limited by this drawing.
  • the processor 150 groups the pixels included in each line in the width direction among the plurality of pixels forming the input image 10 according to the first transformation multiplier to form a first plurality of groups. can be created.
  • the processor 150 groups 9 pixels included in each line among the lines in the width direction of the input image 10 into one group with an integer value of the first transformation multiple of 4, and groups the remaining 5 pixels into another group. Can be grouped.
  • the processor 150 may sequentially store information about the first plurality of groups in two or more memories 140 for each line in the width direction of the input image 10.
  • the processor 150 can store information about the two groups, group 11, and group 12 constituting line 1 in memory 1, and sequentially store information about the groups for each line.
  • the processor 150 sequentially reads the information about the first plurality of groups from the memory 140 based on the second conversion multiplier and The first plurality of groups can be grouped into the second plurality of groups.
  • the second conversion multiplier is 1/1.85546875
  • the information in memory 1 is read as is, and memory 2 and memory 3, and memory 4 and memory 5 are read based on the second conversion multiple.
  • memory 6 and memory 7, memory 8 and memory 9, memory 10 and memory 11, and memory 12 and memory 13 can be read together.
  • the processor 150 can immediately output the pixel value of the converted image 30 by reading information from the memories together and performing calculations immediately.
  • FIG. 7 is a diagram illustrating an upscaling operation of an image scaling device according to an embodiment of the present invention.
  • the processor 150 may identify pixel values of a plurality of pixels of the input image 10 and identify a plurality of intermediate pixel values between each pixel and at least one neighboring pixel. .
  • the pixel values of a plurality of pixels of the input image 10 are 10, 20, and 30, respectively.
  • a plurality of intermediate pixel values between a pixel with a pixel value of 10 and a neighboring pixel with a pixel value of 20 are obtained.
  • the plurality of intermediate pixel values may be an average value of pixel values and may include an intermediate value between any one pixel value and the average value. That is, the middle pixel value between 10 and 20 may be 15, which is the average value, 12, which is the middle value between 10 and 15, and 17, which is the middle value between 15 and 20.
  • the plurality of intermediate pixel values between 10 and 20 may be 12, 15, and 17. At this time, the calculation can be done more precisely depending on the number of pixels to be added.
  • the processor 150 may store upscaling information including pixel values and intermediate pixel values for each number of pixels to be added in two or more memories 140 .
  • upscaling information including 10 and 15 can be stored, and if you want to add 2 pixels, upscaling information including 10, 12, and 17 can be stored.
  • the processor 150 may list and store the number of pixels and the corresponding pixel values.
  • the memory 140 can be used as a ring buffer depending on the number of lines, and can also be used as a plurality of ring buffers divided by a certain number of memories.
  • upscaling like downscaling, identifies a conversion multiplier.
  • the processor 150 may store first up-scaling information including a pixel value based on the first transform multiplier and an intermediate pixel value corresponding to the number of pixels to be added in two or more memories. .
  • the input image 710 has a size of 3 pixels*2 pixels
  • the transformation size has a size of 9 pixels*6 pixels, and is first upscaled based on the first transformation multiplier.
  • the processor 150 uses the division design block 151 and the fixed point design block 152 as described above to achieve a first conversion multiple of 3 and a second conversion multiple of 3. A breach of trust can be identified.
  • the processor 150 identifies first up-scaling information including a pixel value and an intermediate pixel value corresponding to the number of pixels to be added based on the first transform multiplier.
  • the processor 150 outputs four intermediate pixel values 12, 15, 17, and 18 between pixels 10 and 20 in line 1, and two intermediate pixel values 23 between pixels 20 and 30. Identify 27, and in line 2 identify the four intermediate pixel values 22, 25, 27, and 28 between pixels 20 and 30 and the two intermediate pixel values 32 and 37 between pixels 30 and 40.
  • the processor 150 identifies first up-scaling information including the identified pixel values. If the processor 150 immediately performs scaling, an image 720 enlarged in the width direction can be generated. However, in the present invention, scaling is not performed immediately, and the first up-scaling information is sequentially stored in the memory 140.
  • the processor 150 identifies second up-scaling information including an intermediate pixel value corresponding to the number of pixels to be added based on the first up-scaling information and the second transform multiplier.
  • line 1 of the middle image 720 has pixels with pixel values 10, 12, 15, 17, 18, 20, 23, 27, and 30, and line 2 has pixel values 20, 22, 25, and 27. There are pixels numbered , 28, 30, 32, 37, and 40.
  • the processor 150 identifies second up-scaling information including intermediate pixel values of 4 lines between line 1 and line 2, and sequentially reads the second up-scaling information to create a converted image. Generates (730).
  • FIG. 8 is a diagram illustrating a lookup table for the number of pixels to be added according to the conversion multiplier during upscaling of the image scaling device according to an embodiment of the present invention.
  • the lookup table 800 shown in FIG. 8 indicates the number of additional pixels according to the conversion multiplier.
  • the processor 150 may identify the number of pixels to be added according to the first transform multiplier and the second transform multiplier based on the lookup table 800.
  • the first transform multiple of the input image 710 of FIG. 7 is 3 times and the second transform multiple is 3 times.
  • the lookup table 800 when the conversion multiple is 3, the number of additional pixels between initial pixels is 4, and the number of additional pixels between subsequent pixels is 2.
  • the input image has a size of 4 pixels x 4 pixels
  • the conversion size has a size of 11 pixels x 6 pixels
  • the first conversion multiple is 2.75 times
  • the second conversion multiple is 1.5 times.
  • the number of additional pixels between initial pixels is 3, and the number of additional pixels between subsequent pixels is 1 or 2.
  • the number of additional pixels between initial pixels is 3, and the number of additional pixels thereafter is set to 1 if the sum of the decimal values of the conversion multiples does not exceed 1, and 2 if the sum of the decimal values of the conversion multiples exceeds 1.
  • 3 additional pixels are required between the first and second pixels, 2 additional pixels are required between the 2nd and 3rd pixels, and 2 additional pixels are required between the 3rd and 4th pixels.
  • Figure 9 is a diagram showing the operation of an image scaling device according to an embodiment of the present invention.
  • FIG. 9 shows the scaling process described with reference to FIG. 7 being performed using memory.
  • the scaling device 100 has two or more memories 140.
  • the memory 140 consists of 17 memories. At this time, the number and structure of memories are not limited by this drawing.
  • the processor 150 generates a first pixel value including a pixel value of a plurality of pixels constituting the input image 710 and an intermediate pixel value corresponding to the number of pixels to be added based on the first transform multiplier.
  • Upscaling information can be stored in two or more memories 140.
  • the processor 150 sequentially stores first up-scaling information in the memory 140 for each line of the input image 710 of FIG. 7 .
  • the processor 150 stores the first upscaling information ⁇ (5, 10, 12, 15, 17, 18), (3, 20, 23, 27), (1, 30) ⁇ of line 1 in memory 1, and , the first upscaling information ⁇ (5,20,22,25,27,28), (3,30,33,37), (1,40) ⁇ of line 2 is stored in memory 2.
  • FIG. 9 there are 17 memories shown in FIG. 9, but they are composed of two lines, and the first up-scaling information is stored up to memory 2. If the number of lines is greater than the number of memories, the memory can be used like a ring buffer and stored in the first memory order again.
  • the processor 150 When the first up-scaling information is sequentially stored in the memory 140, the processor 150 generates a second up-scaling information including an intermediate pixel value corresponding to the number of pixels to be added based on the first up-scaling information and the second transform multiplier. Scaling information can be stored in two or more memories 140.
  • the processor 150 may store second up-scaling information corresponding to each of the six lines increased according to the number of pixels added in the height direction in memories 3 to 8.
  • the processor 150 may sequentially read the second up-scaling information stored in memories 3 to 8 to generate the converted image 730. At this time, by storing the second up-scaling information for each line in each memory, information omission that may occur when generating a converted image due to communication failure, etc. can be prevented.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Geometry (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Image Processing (AREA)
  • Editing Of Facsimile Originals (AREA)

Abstract

Un dispositif de mise à l'échelle d'image, selon un mode de réalisation de la présente invention, comprend : au moins deux mémoires; et un processeur qui reçoit une image d'entrée, des informations concernant la taille de l'image d'entrée, et des informations concernant une taille de conversion, identifie un premier multiple de conversion correspondant à la largeur et un second multiple de conversion correspondant à la hauteur en fonction de la taille de l'image d'entrée et de la taille de conversion, divise une pluralité de pixels de l'image d'entrée en une première pluralité de groupes en fonction d'un multiple de conversion quelconque parmi le premier multiple de conversion et le second multiple de conversion, et stocke consécutivement des informations concernant la première pluralité de groupes dans les au moins deux mémoires, regroupe la première pluralité de groupes en une deuxième pluralité de groupes en fonction de l'autre multiple de conversion parmi le premier multiple de conversion et le second multiple de conversion, puis identifie les valeurs moyennes des pixels inclus dans la deuxième pluralité respective de groupes, et génère ainsi une image convertie.
PCT/KR2023/009230 2022-07-01 2023-06-30 Dispositif de mise à l'échelle d'image et procédé de mise à l'échelle d'image WO2024005590A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
KR10-2022-0081126 2022-07-01
KR1020220081126A KR102611423B1 (ko) 2022-07-01 2022-07-01 이미지 스케일링 장치 및 이미지 스케일링 방법

Publications (1)

Publication Number Publication Date
WO2024005590A1 true WO2024005590A1 (fr) 2024-01-04

Family

ID=89163260

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2023/009230 WO2024005590A1 (fr) 2022-07-01 2023-06-30 Dispositif de mise à l'échelle d'image et procédé de mise à l'échelle d'image

Country Status (2)

Country Link
KR (1) KR102611423B1 (fr)
WO (1) WO2024005590A1 (fr)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100657145B1 (ko) * 2005-02-24 2006-12-13 매그나칩 반도체 유한회사 스케일러를 구비한 이미지센서 및 이미지센서의 이미지 스케일링 방법
KR20150140729A (ko) * 2013-04-08 2015-12-16 퀄컴 인코포레이티드 비트 심도에 기초한 샘플 적응형 오프셋 스케일링
KR20180083861A (ko) * 2015-11-19 2018-07-23 퀄컴 인코포레이티드 디스플레이 스트림 압축 (dsc) 에서의 고정 소수점 근사를 위한 시스템 및 방법
US20190306536A1 (en) * 2016-07-13 2019-10-03 Electronics And Telecommunications Research Institute Image encoding/decoding method and device
KR102348795B1 (ko) * 2020-11-02 2022-01-07 주식회사 바움 부동 소수점 방식에서 고정 소수점 방식으로의 변환 수행시 비트 폭 최적화 방법

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR100657145B1 (ko) * 2005-02-24 2006-12-13 매그나칩 반도체 유한회사 스케일러를 구비한 이미지센서 및 이미지센서의 이미지 스케일링 방법
KR20150140729A (ko) * 2013-04-08 2015-12-16 퀄컴 인코포레이티드 비트 심도에 기초한 샘플 적응형 오프셋 스케일링
KR20180083861A (ko) * 2015-11-19 2018-07-23 퀄컴 인코포레이티드 디스플레이 스트림 압축 (dsc) 에서의 고정 소수점 근사를 위한 시스템 및 방법
US20190306536A1 (en) * 2016-07-13 2019-10-03 Electronics And Telecommunications Research Institute Image encoding/decoding method and device
KR102348795B1 (ko) * 2020-11-02 2022-01-07 주식회사 바움 부동 소수점 방식에서 고정 소수점 방식으로의 변환 수행시 비트 폭 최적화 방법

Also Published As

Publication number Publication date
KR102611423B1 (ko) 2023-12-07

Similar Documents

Publication Publication Date Title
WO2018016720A1 (fr) Dispositif d'affichage d'image et procédé d'affichage d'image
WO2018113239A1 (fr) Procédé et système de planification de données pour un réseau neuronal convolutionnel, et dispositif informatique
WO2020242057A1 (fr) Appareil de décompression et procédé de commande de celui-ci
WO2020130496A1 (fr) Appareil d'affichage et procédé de commande associé
WO2016159518A1 (fr) Dispositif de calcul de la moyenne de données non linéaires
WO2017104934A1 (fr) Dispositif et procédé de conversion de type commun de programme de commande d'automate programmable
WO2017111197A1 (fr) Système et procédé de visualisation de mégadonnées pour l'analyse d'apprentissage
WO2024063197A1 (fr) Appareil et procédé de retouche d'image
WO2021153969A1 (fr) Procédés et systèmes de gestion du traitement d'un réseau neuronal à travers des processeurs hétérogènes
WO2021080209A1 (fr) Appareil d'affichage et son procédé de fonctionnement
WO2023171981A1 (fr) Dispositif de gestion de caméra de surveillance
WO2021054696A1 (fr) Dispositif d'affichage, serveur, et procédé de correction de dispositif d'affichage
WO2024005590A1 (fr) Dispositif de mise à l'échelle d'image et procédé de mise à l'échelle d'image
WO2019226025A1 (fr) Procédé d'affichage de contenu d'application par l'intermédiaire d'un dispositif d'affichage, et dispositif électronique
WO2020246848A1 (fr) Dispositif et procédé de tri d'un texte chiffré approximativement chiffré
WO2023042989A1 (fr) Procédé d'opération d'addition tenant compte d'une échelle de données, accélérateur matériel associé, et dispositif informatique l'utilisant
WO2023153723A1 (fr) Dispositif d'affichage pour chien et procédé d'attaque associé
WO2020135022A1 (fr) Procédé et dispositif d'optimisation d'image de panneau d'affichage et support d'informations lisible par ordinateur
WO2020135051A1 (fr) Dispositif et procédé d'optimisation d'image destinés à un panneau d'affichage, et support d'informations lisible par ordinateur
WO2021054511A1 (fr) Dispositif électronique et procédé de commande associé
WO2020130737A1 (fr) Appareil d'affichage et son procédé de commande
WO2022108114A1 (fr) Dispositif d'affichage empilé et procédé de commande associé
WO2018093198A1 (fr) Appareil de traitement d'image, et son procédé de commande
WO2016208806A1 (fr) Dispositif de génération d'image intégrale utilisant une structure de bloc et procédé associé
WO2020213885A1 (fr) Serveur et son procédé de commande

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23831948

Country of ref document: EP

Kind code of ref document: A1