WO2021200213A1 - 撮像装置、撮像システム及び撮像方法 - Google Patents

撮像装置、撮像システム及び撮像方法 Download PDF

Info

Publication number
WO2021200213A1
WO2021200213A1 PCT/JP2021/011076 JP2021011076W WO2021200213A1 WO 2021200213 A1 WO2021200213 A1 WO 2021200213A1 JP 2021011076 W JP2021011076 W JP 2021011076W WO 2021200213 A1 WO2021200213 A1 WO 2021200213A1
Authority
WO
WIPO (PCT)
Prior art keywords
line data
line
output
frame
data
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2021/011076
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
鈴木　正敏
智裕 山崎
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Semiconductor Solutions Corp
Original Assignee
Sony Semiconductor Solutions Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Semiconductor Solutions Corp filed Critical Sony Semiconductor Solutions Corp
Priority to CN202180016031.4A priority Critical patent/CN115136583A/zh
Priority to JP2022511882A priority patent/JPWO2021200213A1/ja
Priority to US17/906,667 priority patent/US20230132193A1/en
Publication of WO2021200213A1 publication Critical patent/WO2021200213A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N1/00—Scanning, transmission or reproduction of documents or the like, e.g. facsimile transmission; Details thereof
    • H04N1/024—Details of scanning heads ; Means for illuminating the original
    • H04N1/028—Details of scanning heads ; Means for illuminating the original for picture information pick-up
    • H04N1/03—Details of scanning heads ; Means for illuminating the original for picture information pick-up with photodetectors arranged in a substantially linear array
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/60—Control of cameras or camera modules
    • H04N23/63—Control of cameras or camera modules by using electronic viewfinders
    • H04N23/633—Control of cameras or camera modules by using electronic viewfinders for displaying additional information relating to control or operation of the camera
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58—Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/20—Cameras or camera modules comprising electronic image sensors; Control thereof for generating image signals from infrared radiation only

Definitions

  • the present disclosure relates to an imaging device, an imaging system, and an imaging method.
  • the imaging device outputs an captured image including information such as the start position of the line by adding additional information such as a start code indicating the beginning of the line to each line of the captured image.
  • the imaging device is, for example, a line sensor that sequentially images a moving object to be measured for each predetermined area
  • the frame can determine the beginning of the line only by the start code indicating the beginning of the line. It was difficult to determine the beginning of. Therefore, for example, an external control device that controls the image pickup device needs to detect the beginning of the frame from the captured image, which causes a problem that the processing of the control device increases.
  • the present disclosure provides an imaging device, an imaging system, and an imaging method that can reduce the processing of an external control device.
  • an imaging device includes an image pickup unit, an acquisition unit, an additional information addition unit, and an output unit.
  • the imaging unit extends linearly in the longitudinal direction.
  • the acquisition unit acquires line data from the imaging unit.
  • the additional information adding unit adds the first additional information to the line data which is the first line of the frame among the plurality of line data, and generates the output line data.
  • the output unit outputs the output line data.
  • Imaging system 2.1 Overview of imaging system 2.2. Configuration example of imaging system 2.3. Configuration example of line sensor 3. Operation example of the imaging system 3.1. Image acquisition process 3.2. Operation example of the imaging system 3.3. Operation example of line sensor 4. supplement
  • FIG. 1 is a diagram for explaining an image acquisition process using a line sensor.
  • the line sensor 100A has an imaging unit 110A having a plurality of pixels arranged so as to extend linearly in the longitudinal direction.
  • the object ob moves from right to left, and the line sensor 100A sequentially images the moving object ob at a predetermined cycle.
  • the line sensor 100A When the moving object ob is imaged by the imaging unit 110A, the line sensor 100A generates line data and outputs the line data to the control device 200A (not shown).
  • the line sensor 100A adds additional information to the line data and outputs the line data.
  • the additional information is, for example, a control signal, a fixed value parameter, or the like.
  • such additional information is also described as a synchronization code.
  • the line sensor 100A adds a head code C01 (an example of additional information) indicating the head of the line data to the head of the line data and outputs it as a synchronization code.
  • the image data M is displayed on the display device 300A (not shown) with reference to the line data synchronization code as shown in the lower figure of FIG. To generate.
  • the control device 200A needs to determine from which line data to which line data is set as one frame by referring to the pixel value of the line data. In the image acquisition process by the conventional line sensor 100A, the processing load of the control device 200A becomes high, so that the line sensor 100A capable of reducing the processing load of the control device 200A has been desired.
  • the line sensor assigns a synchronization code (an example of the first additional information) indicating the beginning of the frame to the line data which is the beginning line of the frame.
  • a synchronization code an example of the first additional information
  • FIG. 2 is a diagram for explaining an outline of an image acquisition process by the imaging system according to the embodiment of the present disclosure.
  • the imaging system includes a line sensor 100, a control device 200 (not shown), and a display device 300 (not shown), and displays an image captured by the line sensor 100 on the display device 300.
  • the line sensor 100 has an imaging unit 110 having a plurality of pixels arranged so as to extend linearly in the longitudinal direction.
  • the object ob moves from right to left, and the line sensor 100 sequentially images the moving object ob at a predetermined cycle.
  • the line sensor 100 assigns a synchronization code to the captured line data to generate output line data.
  • the line sensor 100 assigns a line head code C01 indicating the head of the line data as a synchronization code.
  • the line sensor 100 assigns a frame head code C02 (an example of the first additional information) indicating that the line data is the head of the frame as a synchronization code to the line data which is the head line of the frame.
  • the line data shown in FIG. 2 includes a line end code C11 indicating the end of the line data and a frame end code C12 indicating the end line of the frame (No. 1). (Example of 2 additional information) may be added. Further, a valid / invalid code C13 (not shown. An example of the third additional information) indicating whether or not the line data is valid data may be assigned.
  • the line sensor 100 outputs the generated output line data to the control device 200.
  • the control device 200 refers to the synchronization code and outputs from the output line data to which the frame start code C02 is added to the line data to which the frame end code C12 is added to the display device 300 as one frame.
  • the display device 300 can display one frame of image data M as shown in the lower figure of FIG.
  • control device 200 can determine the line data included in the frame by referring to the synchronization code, it is not necessary to refer to the pixel value of the line data, and the processing load can be reduced.
  • FIG. 3 is a block diagram showing a configuration example of the imaging system according to the embodiment of the present disclosure.
  • the imaging system includes a line sensor 100, a control device 200, and a display device 300.
  • the line sensor 100 is an imaging device including an imaging unit 110 (not shown) extending linearly in the longitudinal direction (hereinafter, also referred to as a horizontal direction), and outputs output line data line by line to the control device 200. .. Details of the line sensor 100 will be described later with reference to FIG. 4 and the like.
  • the control device 200 is a device that controls the entire imaging system.
  • the control device 200 is composed of, for example, a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), or a microcomputer equipped with these processors, and executes signal processing according to a predetermined program. , Controls the operation of the line sensor 100 and the display device 300.
  • the control device 200 generates display line data based on the output line data acquired from the line sensor 100 and outputs the display line data to the display device 300.
  • the display device 300 is a display device that displays the image M based on the display line data output by the control device 200.
  • the display device 300 corresponds to, for example, a liquid crystal display, an organic EL (Electro Luminescence) display, a touch panel, or the like.
  • FIG. 4 is a block diagram showing a configuration example of the line sensor 100 according to the embodiment of the present disclosure.
  • the line sensor 100 includes an imaging unit 110, a storage unit 120, a control unit 130, a communication I / F 140, and an output I / F 141.
  • the imaging unit 110 has a plurality of unit pixels 111 (hereinafter, also referred to as pixels) having a photoelectric conversion element (not shown) that photoelectrically converts the amount of electric charge according to the amount of incident light, stores it inside, and outputs it as a signal.
  • unit pixels 111 hereinafter, also referred to as pixels
  • photoelectric conversion element not shown
  • FIG. 5 is a diagram for explaining an arrangement example of unit pixels 111 included in the imaging unit 110 according to the embodiment of the present disclosure. As shown in FIG. 5, the unit pixels 111 are arranged in a V2 ⁇ H2 matrix.
  • the vertical size (vertical size) V2 may be "1" or may be “1” or more as shown in FIG.
  • the horizontal direction corresponds to the longitudinal direction described above, and the horizontal size H2 is longer than the vertical size V2.
  • FIG. 6 is a diagram for explaining the image pickup unit 110B of the area sensor.
  • the imaging unit 110B of the area sensor has pixels 111 arranged in a V1 ⁇ H1 matrix. It is assumed that V1> V2 and H1-V1> H2-V2.
  • the area sensor reads pixel signals from all the pixels of the imaging unit 110B in one imaging and generates pixel data.
  • the area sensor outputs the pixel data captured at one time as one frame.
  • the display device 300 displays the pixel data output by the area sensor as one captured image. Therefore, the display device 300 displays the captured image having a vertical size of V1 and a horizontal size of H1.
  • the area sensor outputs pixel data line by line, for example, by adding a synchronization code.
  • the synchronization code includes a code indicating the beginning of a line and a code indicating the beginning or end of a frame.
  • the area sensor since the area of V1 ⁇ H1 in which the pixels are arranged is one frame, the area sensor assigns a code indicating the beginning of the frame to the first line of the imaging unit 110B, and finally Add code to the line to indicate the end of the frame.
  • Such an area sensor does not output an area larger than the area that can be imaged by a plurality of pixels 111 at a time as one frame.
  • the area in which the imaging unit 110 can image is elongated compared to the imaging unit 110B of the area sensor.
  • the imaging system covers an area (H2 ⁇ V3, V3> V2) wider than the imageable area. It can be treated as an image M.
  • the communication I / F 140 is an interface for communicating with the control device 200.
  • the communication I / F 140 may include a general-purpose communication interface for communicating with an external device, or may include an external terminal into which a control signal from the control device 200 is input.
  • the output I / F 141 outputs the line data acquired by the line sensor 100 as output line data.
  • the output I / F 141 according to the present embodiment outputs output line data to the control device 200.
  • the storage unit 120 stores an image captured by the image pickup unit 110 and various data (for example, a synchronization code) necessary for processing executed by the control unit 130. Further, the storage unit 120 has an STB register 121.
  • the STB register 121 stores whether the line sensor 100 is in an active state in which imaging is possible or in a standby state in which imaging is stopped.
  • the STB register 121 of the present embodiment transitions the stored state by the control of the control unit 130, and also transitions the state by the control of the control device 200. Details of this point will be described later.
  • the control unit 130 controls the entire line sensor 100.
  • the control unit 130 is composed of, for example, a control circuit, and controls the operation of the line sensor 100 by referring to the internal parameters stored in the storage unit 120.
  • the control unit 130 is composed of, for example, a processor such as a CPU or DSP, or a microcomputer equipped with these processors, and controls the operation of the line sensor 100 by executing signal processing according to a predetermined program. You may try to do it.
  • the control unit 130 includes an acquisition unit 131, a code assigning unit 132, and an output control unit 133, and realizes or executes an information processing function or operation described below.
  • the internal configuration of the control unit 130 is not limited to the configuration shown in FIG. 5, and may be another configuration as long as it is a configuration for performing information processing described later. Further, the connection relationship of each unit of the control unit 130 is not limited to the connection relationship shown in FIG. 5, and may be another connection relationship.
  • the acquisition unit 131 acquires an image captured from the image capturing unit 110, generates line data, and outputs the line data to the code adding unit 132.
  • the acquisition unit 131 may integrate a plurality of captured images to generate one line data.
  • the acquisition unit 131 suppresses a decrease in the brightness value of the captured image by integrating the pixel values of the pixels 111 of different rows that image the same region of the object ob to generate one line data.
  • a line sensor that integrates captured images in this way is also called a TDI (Time Delay Integration) line sensor.
  • the code assigning unit 132 assigns a synchronization code to the line data output by the acquisition unit 131 to generate output line data.
  • the code assigning unit 132 outputs the generated output line data to the output control unit 133.
  • the code assigning unit 132 assigns a frame head code C02 indicating that the line data is the head of the frame to the head of the line data. Further, the code assigning unit 132 assigns a frame termination code C12 indicating a frame termination line and a valid / invalid code C13 indicating whether or not the line data is valid data. Further, the code assigning unit 132 assigns a line termination code C11 indicating the termination of the line data to the termination of the line data.
  • the valid data refers to the data obtained by integrating the captured images of a predetermined number of pixel rows when, for example, the acquisition unit 131 integrates the captured images of a plurality of pixel rows.
  • invalid data that is not valid data refers to data obtained by integrating captured images of a predetermined number of pixel rows.
  • the valid data is displayed on the display device 300, or is used for image processing in a subsequent stage such as detection of a subject.
  • the code assigning unit 132 assigns the frame start code C02 and the frame end code C12 based on the control signal of the control device 200.
  • the output control unit 133 outputs the output line data generated by the code assigning unit 132 to the control device 200 via the output I / F 141.
  • FIG. 7 is a diagram for explaining an image acquisition process by the imaging system according to the comparative example.
  • the line sensor 100A of the imaging system according to the comparative example assigns the line head code C01 to the acquired line data to generate the output line data D01A.
  • the line sensor 100A outputs 300 lines of output line data D01A.
  • the control device 200A acquires the output line data D01A
  • the control device 200A generates the display line data D02 in order to connect the output line data D01A in the vertical direction and display one image M on the display device 300A.
  • the output line data D01A of the line sensor 100A includes valid data to be displayed on the display device 300A and invalid data not to be displayed on the display device 300A. In FIG. 7, valid data are shaded.
  • the control device 200A determines whether the output line data D01A is valid data or invalid data simply by referring to the synchronization code. Can not do it.
  • control device 200A refers to the data included in the output line data D01A, and determines whether or not the output line data D01A is valid data from the luminance value, edge detection, and the like.
  • the control device 200 refers to the data included in the valid output line data D01A, performs edge detection and the like, and performs edge detection and the like. From the acquired output line data D01A, the output line data D01A at the beginning of the frame is determined. In the example of FIG. 7, the control device 200A determines the line data # 100, which is valid data, at the beginning of the frame.
  • control device 200A determines the end of the frame according to the image size that can be displayed on the display device 300A. In the example of FIG. 7, the control device 200A determines the line data # 299, which is valid data, at the end of the frame.
  • the control device 200A deletes the synchronization code from the output line data D01A included in the determined frame, generates the display line data D02, and outputs the display line data D02 to the display device 300A.
  • FIG. 8 is a diagram for explaining an image acquisition process by the imaging system according to the embodiment of the present disclosure.
  • the line sensor 100 assigns a synchronization code to the acquired line data.
  • the line sensor 100 assigns a valid / invalid code C13 indicating invalid data to the beginning of the line data # 1, a line termination code C11 to the end, and outputs line data D01.
  • the line sensor 100 integrates the captured images of a plurality of pixel rows to acquire the line data
  • the line sensor 100 assigns the valid / invalid code C13 according to the integrated amount. More specifically, when the integrated amount of captured images is less than a predetermined number, the line sensor 100 assigns a valid / invalid code C13 indicating that the acquired line data is invalid.
  • the line sensor 100 assigns an valid / invalid code C13 indicating that the acquired line data is valid when the integrated amount of captured images is a predetermined number.
  • the line sensor 100 also adds a frame start code C02 to the beginning of the line data # 100 and a line end code C11 to the end to generate the output line data D01.
  • the line sensor 100 assigns a valid / invalid code C13 indicating that the line data # 200 is valid data at the beginning and a line termination code C11 at the end to generate output line data D01.
  • the line sensor 100 assigns the frame end code C12 to the beginning of the line data # 300 and adds the line end code C11 to the end to generate the output line data D01.
  • FIG. 8 shows a case where the line sensor 100 assigns one synchronization code to the beginning of the line data, but the present invention is not limited to this.
  • the line sensor 100 may add two or more synchronization codes to the beginning of the line data.
  • the line sensor 100 may assign the valid / invalid code C13 and the frame start code C02 or the frame end code C12 to the beginning of the same line data.
  • the line sensor 100 assigns one of the frame start code C02, the frame end code C12, and the valid / invalid code C13 to the beginning of the line data. Therefore, in FIG. 8, the line head code C01 indicating the head of the line data is omitted.
  • the control device 200 that has acquired the output line data D01 refers to the synchronization code, and discards the output line data D01 to which the valid / invalid code C13 indicating that the data is invalid, for example.
  • the control device 200 determines from the output line data D01 to which the frame start code C02 is assigned to the output line data D01 immediately before the output line data D01 to which the frame end code C12 is assigned as one frame.
  • the output line data D01 to which the frame end code C12 is attached is not included in the frame here, the output line data D01 to which the frame end code C12 is attached may also be included in one frame. ..
  • the control device 200 deletes the synchronization code of the output line data D01 included in the determined frame, generates display line data, and outputs the display line data to the display device 300.
  • FIG. 9 is a diagram for explaining an operation example of the imaging system according to the comparative example.
  • the line sensor 100A when the line sensor 100A receives the data output control signal TX from the control device 200A, the line sensor 100A controls the line data output timing by the data output control signal TX (step S11).
  • the line sensor 100A outputs the generated output line data D01 to the control device 200A based on the line data output timing (step S12).
  • the control device 200A determines the frame information based on the line data (pixel data) included in the output line data D01 (step S13). More specifically, the control device 200A determines from where to where the output line data D01 is included in one frame.
  • the control device 200A generates display line data from the output line data D01, outputs the generated display line data to the display device 300A (step S14), and outputs the synchronization signals VD and HD to the display device 300A (step S15). ).
  • the display device 300A displays the image M based on the display line data and the synchronization signals VD and HD (step S16).
  • FIG. 10 is a diagram for explaining a control signal generated by the control device 200A according to the comparative example.
  • the control signal generated by the control device 200A includes a data output control signal TX, a synchronization signal VD, and HD.
  • the data output control signal TX shown in FIG. 10A is a control signal output by the control device 200A when the line data output timing is adjusted.
  • the control device 200A determines the output timing of the output line data D01 while changing the output timing of the output line data D01.
  • the control device 200A performs, for example, edge detection of the line data of the output line data D01 continuously acquired, and determines whether or not the pixel values are continuous. When the pixel values are not continuous, the control device 200A changes the output timing of the output line data D01 and determines the output timing so that the edges are continuous.
  • the control device 200A can display the image M having a size equal to or larger than the vertical size V2 of the imaging unit 110A on the display device 300A.
  • Such adjustment is performed, for example, when the imaging system is introduced or when the imaging system is started.
  • FIG. 10B is a diagram showing a data output control signal TX after adjusting the output timing of line data.
  • the adjusted data output control signal TX is repeatedly set to High at predetermined intervals, and the line sensor 100A synchronizes with, for example, the fall of the data output control signal TX, and the output line data D01. Is output.
  • FIG. 10 (c) is a diagram showing synchronization signals VD and HD.
  • the synchronization signal VD shown in FIG. 10C is a signal indicating the synchronization timing in the vertical direction, and the display line data included in the high period is the line data included in one frame.
  • the synchronization signal HD is a signal indicating the synchronization timing in the horizontal direction, and the display line data included in the high period is the line data displayed on the display device 300A as one line.
  • FIG. 11 is a diagram for explaining an operation example of the imaging system according to the embodiment of the present disclosure.
  • the line sensor 100A adds a synchronization code including frame information (for example, frame head code C02) to the line data to generate output line data D01 (step S21).
  • the frame information is given based on the frame control signal CS output by the control device 200 or the state of the STB register 121.
  • the state of the STB register 121 changes based on, for example, the register control signal RS output by the control device 200.
  • the line sensor 100 outputs the generated output line data D01 to the control device 200 based on the data output control signal TX (step S22).
  • the control device 200 controls the synchronization signal VD based on the frame information obtained by referring to the synchronization code (step S23).
  • the control device 200 generates display line data from the output line data D01, outputs the generated display line data to the display device 300 (step S24), and outputs the synchronization signals VD and HD to the display device 300 (step S25). ).
  • the display device 300 displays the image M based on the display line data and the synchronization signals VD and HD (step S26).
  • step S23 the control device 200 can control the synchronization signal VD only by referring to the synchronization code without referring to the pixel value of the line data. In this way, the processing load of the control device 200 can be reduced.
  • control device 200 controls the STB register 121 by using the register control signal RS, but the present invention is not limited to this.
  • the control device 200 may directly control the STB register 121 via the communication IF without using the register control signal RS.
  • FIG. 12 is a diagram for explaining a control signal generated by the control device 200 according to the embodiment of the present disclosure.
  • the control signal generated by the control device 200 includes a data output control signal TX, a synchronization signal VD, and HD.
  • the data output control signal TX shown in FIG. 12A is a control signal output by the control device 200 when the line data output timing is adjusted.
  • the control device 200 determines the output timing of the output line data D01 while changing the output timing of the output line data D01.
  • the control device 200 determines whether or not the pixel values are continuous by, for example, performing edge detection of the line data of the output line data D01 continuously acquired. When the pixel values are not continuous, the control device 200 changes the output timing of the output line data D01 and determines the output timing so that the edges are continuous.
  • control device 200 determines the output line data D01 to be the head of the frame by detecting the edge of the output line data D01.
  • the control device 200 can display the image M having a size equal to or larger than the vertical size V2 of the imaging unit 110 on the display device 300. Further, the control device 200 can determine the head of the frame. Such adjustment is performed, for example, when the imaging system is introduced or when the imaging system is started. Further, the control device 200 can determine the end of the frame by determining the beginning of the frame. The control device 200 determines, for example, the number of effective output line data D01 required for one frame from the beginning of the frame according to the image size displayed on the display device 200. When the control device 200 acquires the determined number of valid output line data D01, it can determine that the output line data D01 has been acquired up to the end of the frame. In this way, the size of the frame is determined according to the size of the image M displayed on the display device 300.
  • FIG. 12B is a diagram showing a data output control signal TX after adjusting the output timing of line data.
  • the adjusted data output control signal TX is repeatedly set to High at predetermined intervals, and the line sensor 100 synchronizes with, for example, when the data output control signal TX falls, the output line data D01. Is output.
  • FIG. 12 (c) is a diagram showing synchronization signals VD and HD.
  • the synchronization signal VD shown in FIG. 12C is a signal indicating the synchronization timing in the vertical direction, and the display line data included in the high period is the line data included in one frame.
  • the synchronization signal HD is a signal indicating the synchronization timing in the horizontal direction, and the display line data included in the high period is the line data displayed on the display device 300 as one line.
  • FIG. 13 is a diagram for explaining a register control signal RS generated by the control device 200 according to the embodiment of the present disclosure.
  • FIG. 13 shows an example of the register control signal RS, the data output control signal TX, and the output line data D01.
  • the register control signal RS is a signal that shifts the state of the line sensor 100. More specifically, when the high register control signal RS is input to the line sensor 100, the STB register 121 is released or set, and the state of the line sensor 100 changes from the standby state to the active state or from the active state to the standby state. Transition to the state.
  • the control device 200 directly releases / sets the STB register 121 of the storage unit 120 via the communication I / F 140 by using the register control signal RS. In other words, the control device 200 can write directly to the STB register 121 using the register control signal RS.
  • the data output control signal TX is a signal instructing the output timing of the output line data D01 as described above, and the line sensor 100 outputs the output line data D01 in synchronization with the data output control signal TX.
  • the output line data D01 shown in FIG. 13 corresponds to the frame start code C02 as a header, corresponds to the FS (frame start) indicating the beginning of the frame, and corresponds to the frame end code C12, and corresponds to the FE (frame end) indicating the end of the frame. including.
  • the header includes, for example, a Valid (corresponding to the valid / invalid code C13) indicating valid / invalid of the line data. The header is added at the beginning of DATA (corresponding to line data).
  • a line termination code indicating the termination of the line is added as a footer after DATA.
  • the STB register 121 is set (for example, "1" is written), and the state of the line sensor 100 becomes the standby state.
  • the data output control signal TX which becomes High at a predetermined cycle is input from the control device 200.
  • the line sensor 100 transitions to the active state at time t10, the line sensor 100 sequentially outputs the output line data D01 in synchronization with the data output control signal TX.
  • FS 0, it indicates that the output line data D01 is not the beginning of the frame.
  • the line sensor 100 can determine the start and end of the frame, and the start and end of the frame can be determined.
  • the indicated synchronization code can be added to the line data.
  • control device 200 can determine the start and end of the frame without referring to the data of the output line data D01, and the processing load of the control device 200 can be suppressed.
  • control device 200 determines the start timing of the first frame, for example, when adjusting the output timing of the line data described above. Further, the control device 200 may determine the end timing of the first frame according to, for example, the number of acquisitions of valid line data included in the first frame. That is, the control device 200 determines that the first frame is finished at the timing when the line data for the number of lines to be displayed on the display device 300 is acquired after the first frame is started. By determining the start and end timings of the second and subsequent frames in the same manner, the control device 200 can determine the frame information from the data of the output line data D01 without performing processing such as edge detection.
  • FIG. 14 is a diagram for explaining a frame control signal CS generated by the control device 200 according to the embodiment of the present disclosure.
  • the frame control signal CS shown in FIG. 14 is input to the line sensor 100 from the control device 200 via, for example, an external terminal XVS (not shown).
  • the control device 200 sends the high frame control signal CS to the line sensor 100 at time t20 when the high data output control signal TX is input a predetermined number of times (four times in FIG. 14) after the line sensor 100 is activated. input.
  • the control device 200 acquires the number of output line data D01 required for displaying on the display device 300 from the line sensor 100, the control device 200 outputs the frame control signal CS.
  • the line sensor 100 determines the start and end of the frame, and changes the values of FS and FE indicating the start or end of the frame.
  • the line sensor 100 can add frame information to the line data by designating the start and end timings of the frames by using the frame control signal CS by the control device 200.
  • the register control signal RS described above sets or cancels the STB register 121 in the case of a high signal, but the present invention is not limited to this.
  • the register control signal RS when the register control signal RS is a High signal, the line sensor 100 may be put into a standby state, and when it is a Low signal, it may be put into an active state.
  • the register control signal RS when the register control signal RS is a High signal, "1" is written in the STB register 121, and when it is a Low signal, "0" is written in the STB register 121. Become.
  • FIG. 15 is a diagram for explaining another example of the register control signal RS generated by the control device 200 according to the embodiment of the present disclosure.
  • the control device 200 when the control device 200 acquires the number of output line data D01 required to be displayed on the display device 300 from the line sensor 100, the control device 200 switches the register control signal RS from Low to High. As a result, the line sensor 100 determines the end of the frame, and then outputs the output line data D01 in which the FE value indicating the end of the frame is changed at the timing of synchronizing with the data output control signal TX, and enters the standby state. Transition.
  • control device 200 can specify the start and end timings of the frame by controlling the STB register at the rising and falling timings of the register control signal RS.
  • FIG. 16 is a flowchart showing an example of line data output processing by the line sensor 100 according to the embodiment of the present disclosure.
  • the line sensor 100 acquires line data based on the captured image captured by the imaging unit 110 (step S101).
  • the line sensor 100 assigns a synchronization code to the acquired line data (step S102). For example, the line sensor 100 assigns a synchronization code (additional information) indicating the beginning of the frame to the line data at the beginning of the frame. As a result, the line sensor 100 generates the output line data D01 including the frame information.
  • the line sensor 100 outputs the generated output line data D01 to the control device 200 (step S103).
  • the line sensor 100 can output the output line data D01 including the frame information to the control device 200.
  • the control device 200 can reduce the processing load by displaying the line data on the display device 300 based on the frame information.
  • each component of each device shown in the figure is a functional concept, and does not necessarily have to be physically configured as shown in the figure. That is, the specific form of distribution / integration of each device is not limited to the one shown in the figure, and all or part of the device is functionally or physically dispersed / physically distributed in arbitrary units according to various loads and usage conditions. Can be integrated and configured.
  • An imaging unit that extends linearly in the longitudinal direction, An acquisition unit that acquires line data from the imaging unit, Of the plurality of line data, an additional information addition unit that adds first additional information to the line data that is the first line of the frame to generate output line data, and An output unit that outputs the output line data and An imaging device comprising.
  • the additional information adding unit adds at least one of the first additional information and the second additional information based on a control signal from the control device.
  • the imaging device wherein the control signal is a signal that specifies at least one of a start line and an end line of the frame.
  • the control signal is a signal instructing the release or setting of the imaging device from the standby state.
  • the additional information adding unit adds a third additional information indicating whether or not the line data is valid data.
  • the acquisition unit integrates the captured images captured by the imaging unit to acquire the line data.
  • the additional information adding unit adds the third additional information to the line data according to the integrated amount of the captured image.
  • the imaging device according to (6).
  • Imaging device (9) Imaging device and A display device that displays an image captured by the image pickup device, and a display device that displays the captured image.
  • a control device that controls the image pickup device and the display device, With The image pickup device An imaging unit that extends linearly in the longitudinal direction, An acquisition unit that acquires line data from the imaging unit, Of the plurality of line data, an additional information addition unit that adds first additional information to the line data that is the first line of the frame to generate output line data, and An output unit that outputs the output line data and Imaging system with.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Studio Devices (AREA)
PCT/JP2021/011076 2020-03-30 2021-03-18 撮像装置、撮像システム及び撮像方法 Ceased WO2021200213A1 (ja)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202180016031.4A CN115136583A (zh) 2020-03-30 2021-03-18 成像装置、成像系统和成像方法
JP2022511882A JPWO2021200213A1 (https=) 2020-03-30 2021-03-18
US17/906,667 US20230132193A1 (en) 2020-03-30 2021-03-18 Image capturing device, image capturing system, and image capturing method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2020-061581 2020-03-30
JP2020061581 2020-03-30

Publications (1)

Publication Number Publication Date
WO2021200213A1 true WO2021200213A1 (ja) 2021-10-07

Family

ID=77927777

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2021/011076 Ceased WO2021200213A1 (ja) 2020-03-30 2021-03-18 撮像装置、撮像システム及び撮像方法

Country Status (4)

Country Link
US (1) US20230132193A1 (https=)
JP (1) JPWO2021200213A1 (https=)
CN (1) CN115136583A (https=)
WO (1) WO2021200213A1 (https=)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002511718A (ja) * 1998-04-10 2002-04-16 リンクス システム デベロッパーズ インコーポレイテッド 対象物認識機能及びデータ出力を有するカメラ

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3732398B2 (ja) * 2000-10-11 2006-01-05 三菱電機株式会社 乗員保護装置
JP4154157B2 (ja) * 2002-02-25 2008-09-24 株式会社東芝 撮像装置
ATE386951T1 (de) * 2002-08-05 2008-03-15 Elbit Systems Ltd Nachtsichtabbildungssystem und -verfahren zur montage in einem fahrzeug
JP3846410B2 (ja) * 2002-11-21 2006-11-15 ソニー株式会社 画像処理装置および方法、記録媒体、並びにプログラム
US7881496B2 (en) * 2004-09-30 2011-02-01 Donnelly Corporation Vision system for vehicle
JP4452161B2 (ja) * 2004-11-12 2010-04-21 パナソニック株式会社 撮像装置
EP2420954B8 (en) * 2004-12-01 2017-04-12 Datalogic USA, Inc. Data reader with automatic exposure adjustment and methods of operating a data reader
KR100745837B1 (ko) * 2005-11-30 2007-08-02 엠텍비젼 주식회사 여유 데이터가 부가된 화소 데이터 출력 방법 및 장치
GB2435974A (en) * 2006-03-07 2007-09-12 I S G Thermal Systems Ltd Imaging system including detectors using plural sensitivity levels
JP4131284B2 (ja) * 2006-07-14 2008-08-13 ソニー株式会社 ビデオ信号処理装置、ビデオ信号処理方法
US7855740B2 (en) * 2007-07-20 2010-12-21 Eastman Kodak Company Multiple component readout of image sensor
JP2009152669A (ja) * 2007-12-18 2009-07-09 Sony Corp 撮像装置、撮像処理方法及び撮像制御プログラム
JP5040776B2 (ja) * 2008-03-31 2012-10-03 アイシン精機株式会社 撮像装置
US20100149393A1 (en) * 2008-05-22 2010-06-17 Panavision Imaging, Llc Increasing the resolution of color sub-pixel arrays
US8717417B2 (en) * 2009-04-16 2014-05-06 Primesense Ltd. Three-dimensional mapping and imaging
US9749607B2 (en) * 2009-07-16 2017-08-29 Digimarc Corporation Coordinated illumination and image signal capture for enhanced signal detection
JP2011109397A (ja) * 2009-11-17 2011-06-02 Sony Corp 画像送信方法、画像受信方法、画像送信装置、画像受信装置、及び画像伝送システム
US8305456B1 (en) * 2011-05-11 2012-11-06 Pelican Imaging Corporation Systems and methods for transmitting and receiving array camera image data
JP2013239961A (ja) * 2012-05-16 2013-11-28 Sony Corp 動画撮像装置、動画像電子ズーム方法
JP5935779B2 (ja) * 2013-09-30 2016-06-15 カシオ計算機株式会社 画像処理装置、画像処理方法及びプログラム
JP2015220690A (ja) * 2014-05-20 2015-12-07 株式会社東芝 カメラモジュール及びイメージセンサ
US10009544B2 (en) * 2014-09-02 2018-06-26 Canon Kabushiki Kaisha Image capturing apparatus, control method of the same and image sensor
JP2016152497A (ja) * 2015-02-17 2016-08-22 株式会社ブレインズ ラインセンサカメラおよびラインセンサカメラ用信号変換装置
JP6602162B2 (ja) * 2015-10-30 2019-11-06 キヤノン株式会社 画像処理装置、画像処理方法及びプログラム
JP2018117220A (ja) * 2017-01-17 2018-07-26 ソニーセミコンダクタソリューションズ株式会社 撮像装置、および、撮像装置の制御方法
US12198049B2 (en) * 2021-03-25 2025-01-14 Intel Corporation Vehicle data relation device and methods therefor

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002511718A (ja) * 1998-04-10 2002-04-16 リンクス システム デベロッパーズ インコーポレイテッド 対象物認識機能及びデータ出力を有するカメラ

Also Published As

Publication number Publication date
JPWO2021200213A1 (https=) 2021-10-07
US20230132193A1 (en) 2023-04-27
CN115136583A (zh) 2022-09-30

Similar Documents

Publication Publication Date Title
KR101796481B1 (ko) 저전력 셔터랙 제거 방법, 카메라 모듈 및 이를 구비하는 모바일 기기
US10997689B1 (en) High dynamic range sensor system with row increment operation
US9832382B2 (en) Imaging apparatus and imaging method for outputting image based on motion
KR100601460B1 (ko) 화상 센서와 영상 처리기 간의 인터페이스 장치 및 방법
JP2008054136A (ja) 撮像装置及び駆動制御方法
US20160006942A1 (en) Image signal processing apparatus and a control method thereof, and an image pickup apparatus and a control method thereof
WO2020207427A1 (en) Method for image-processingand electronic device
US20230239570A1 (en) Reception apparatus and transmission apparatus
US20070188645A1 (en) Image output apparatus, method and program thereof, and imaging apparatus
US11356603B2 (en) Image capturing apparatus and control method therefor
KR20100007628A (ko) 이미지 센서 인터페이스 장치 및 이를 구비하는 디지털촬영 장치
US20140133781A1 (en) Image processing device and image processing method
JP2013135410A (ja) 撮像装置および評価値生成装置
WO2021200213A1 (ja) 撮像装置、撮像システム及び撮像方法
CN106941591B (zh) 半导体装置、视频显示系统以及视频信号的输出方法
CN113119866A (zh) 一种基于流媒体的后视镜显示方法及显示装置
JP2015033020A (ja) 撮像装置、撮像方法、プログラム、及び記録媒体
CN114500767A (zh) 输入视频源调节方法、装置、视频输入卡和视频处理设备
CN107920198A (zh) 图像处理设备和方法
US11368624B2 (en) Image capturing apparatus and control method thereof
JP4965971B2 (ja) メモリ制御装置、撮像装置およびメモリ制御方法
US9043514B2 (en) Data transfer device and method
JP6091216B2 (ja) 画像信号処理装置およびその制御方法、並びに撮像装置
WO2020077513A1 (en) Method, apparatus, program and recording medium for processing image data
US10044963B2 (en) Imaging apparatus

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21781162

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2022511882

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21781162

Country of ref document: EP

Kind code of ref document: A1