I. INTRODUCTION

Section:

ChooseTop of pageABSTRACTI. INTRODUCTION <<II. SYSTEM DESIGN AND SIM...III. EXPERIMENTAL DEMONST...IV. CONCLUSION AND DISCUS...SUPPLEMENTARY MATERIALREFERENCESMultimode fibers (MMFs) are promising for minimally invasive endoscopic applications since they can have high resolutions (∼1μm) with diameters (<0.2mm) much smaller than conventional endoscopes [e.g., fiber bundles11. Y. Zhou, B. Xiong, W. Song, X. Zhang, G. Zheng, Q. Dai, and X. Cao, “Light-field micro-endoscopy using a fiber bundle: A snapshot 3D epi-fluorescence endoscope,” Photonics Res. 10(9), 2247–2260 (2022). https://doi.org/10.1364/prj.464051 or gradient index (GRIN) lenses22. G. Oh, E. Chung, and S. H. Yun, “Optical fibers for high-resolution in vivo microendoscopic fluorescence imaging,” Opt. Fiber Technol. 19(6), 760–771 (2013). https://doi.org/10.1016/j.yofte.2013.07.008]. With an ultra-thin diameter, an MMF is capable of reaching the organs that are difficult to access, such as deep brains,3,43. S. Turtaev, I. T. Leite, T. Altwegg-Boussac, J. M. Pakan, N. L. Rochefort, and T. Čižmár, “High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging,” Light: Sci. Appl. 7(1), 92 (2018). https://doi.org/10.1038/s41377-018-0094-x4. B. Lochocki, M. V. Verweg, J. J. M. Hoozemans, J. F. de Boer, and L. V. Amitonova, “Epi-fluorescence imaging of the human brain through a multimode fiber,” APL Photonics 7(7), 071301 (2022). https://doi.org/10.1063/5.0080672 cochleas,55. Y. Pu, X. Yang, I. Papadopoulos, S. Farahi, C.-L. Hsieh, C. A. Ong, C. Moser, D. Psaltis, and K. M. Stankovic, “Imaging of the mouse cochlea with two-photon microscopy and multimode fiber-based microendoscopy,” paper presented at the Biomedical Optics, . or other regions deep inside bodies.66. I. T. Leite, S. Turtaev, D. E. Boonzajer Flaes, and T. Čižmár, “Observing distant objects with a multimode fiber-based holographic endoscope,” APL Photonics 6(3), 036112 (2021). https://doi.org/10.1063/5.0038367 The presence of intermodal dispersion and coupling can significantly distort the image arriving at the distal end of the MMF. Nevertheless, recent research has shown that the distorted image can be restored by using computational methods. There have been several reports of endoscopy systems using MMFs, including systems for fluorescence imaging,3,43. S. Turtaev, I. T. Leite, T. Altwegg-Boussac, J. M. Pakan, N. L. Rochefort, and T. Čižmár, “High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging,” Light: Sci. Appl. 7(1), 92 (2018). https://doi.org/10.1038/s41377-018-0094-x4. B. Lochocki, M. V. Verweg, J. J. M. Hoozemans, J. F. de Boer, and L. V. Amitonova, “Epi-fluorescence imaging of the human brain through a multimode fiber,” APL Photonics 7(7), 071301 (2022). https://doi.org/10.1063/5.0080672 two-photon imaging,5,75. Y. Pu, X. Yang, I. Papadopoulos, S. Farahi, C.-L. Hsieh, C. A. Ong, C. Moser, D. Psaltis, and K. M. Stankovic, “Imaging of the mouse cochlea with two-photon microscopy and multimode fiber-based microendoscopy,” paper presented at the Biomedical Optics, .7. E. E. Morales-Delgado, D. Psaltis, and C. Moser, “Two-photon imaging through a multimode fiber,” Opt. Express 23(25), 32158–32170 (2015). https://doi.org/10.1364/oe.23.032158 time-of-flight 3D imaging,88. D. Stellinga, D. B. Phillips, S. P. Mekhail, A. Selyem, S. Turtaev, T. Čižmár, and M. J. Padgett, “Time-of-flight 3D imaging through multimode optical fibers,” Science 374(6573), 1395–1399 (2021). https://doi.org/10.1126/science.abl3771 angle-resolved imaging,9,109. H. Zhang, Z. A. Steelman, S. Ceballos, K. K. Chu, and A. Wax, “Reconstruction of angle-resolved backscattering through a multimode fiber for cell nuclei and particle size determination,” APL Photonics 5(7), 076105 (2020). https://doi.org/10.1063/5.001150010. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109(20), 203901 (2012). https://doi.org/10.1103/physrevlett.109.203901 confocal imaging,1111. S.-Y. Lee, V. J. Parot, B. E. Bouma, and M. Villiger, “Confocal 3D reflectance imaging through multimode fiber without wavefront shaping,” Optica 9(1), 112–120 (2022). https://doi.org/10.1364/optica.446178 and photoacoustic imaging.1212. A. M. Caravaca-Aguirre, S. Singh, S. Labouesse, M. V. Baratta, R. Piestun, and E. Bossy, “Hybrid photoacoustic-fluorescence microendoscopy through a multimode fiber using speckle illumination,” APL Photonics 4(9), 096103 (2019). https://doi.org/10.1063/1.5113476The most common way to reconstruct images transmitted in an MMF is the transmission matrix method (TMM),3,5–113. S. Turtaev, I. T. Leite, T. Altwegg-Boussac, J. M. Pakan, N. L. Rochefort, and T. Čižmár, “High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging,” Light: Sci. Appl. 7(1), 92 (2018). https://doi.org/10.1038/s41377-018-0094-x5. Y. Pu, X. Yang, I. Papadopoulos, S. Farahi, C.-L. Hsieh, C. A. Ong, C. Moser, D. Psaltis, and K. M. Stankovic, “Imaging of the mouse cochlea with two-photon microscopy and multimode fiber-based microendoscopy,” paper presented at the Biomedical Optics, .6. I. T. Leite, S. Turtaev, D. E. Boonzajer Flaes, and T. Čižmár, “Observing distant objects with a multimode fiber-based holographic endoscope,” APL Photonics 6(3), 036112 (2021). https://doi.org/10.1063/5.00383677. E. E. Morales-Delgado, D. Psaltis, and C. Moser, “Two-photon imaging through a multimode fiber,” Opt. Express 23(25), 32158–32170 (2015). https://doi.org/10.1364/oe.23.0321588. D. Stellinga, D. B. Phillips, S. P. Mekhail, A. Selyem, S. Turtaev, T. Čižmár, and M. J. Padgett, “Time-of-flight 3D imaging through multimode optical fibers,” Science 374(6573), 1395–1399 (2021). https://doi.org/10.1126/science.abl37719. H. Zhang, Z. A. Steelman, S. Ceballos, K. K. Chu, and A. Wax, “Reconstruction of angle-resolved backscattering through a multimode fiber for cell nuclei and particle size determination,” APL Photonics 5(7), 076105 (2020). https://doi.org/10.1063/5.001150010. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109(20), 203901 (2012). https://doi.org/10.1103/physrevlett.109.20390111. S.-Y. Lee, V. J. Parot, B. E. Bouma, and M. Villiger, “Confocal 3D reflectance imaging through multimode fiber without wavefront shaping,” Optica 9(1), 112–120 (2022). https://doi.org/10.1364/optica.446178 in which the MMF is modeled as a linear system described by a complex matrix.13,1413. S.-Y. Lee, V. J. Parot, B. E. Bouma, and M. Villiger, “Reciprocity-induced symmetry in the round-trip transmission through complex systems,” APL Photonics 5(10), 106104 (2020). https://doi.org/10.1063/5.002128514. M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics 9(8), 529–535 (2015). https://doi.org/10.1038/nphoton.2015.112 The image can be reconstructed by finding the inverse of the matrix9–119. H. Zhang, Z. A. Steelman, S. Ceballos, K. K. Chu, and A. Wax, “Reconstruction of angle-resolved backscattering through a multimode fiber for cell nuclei and particle size determination,” APL Photonics 5(7), 076105 (2020). https://doi.org/10.1063/5.001150010. Y. Choi, C. Yoon, M. Kim, T. D. Yang, C. Fang-Yen, R. R. Dasari, K. J. Lee, and W. Choi, “Scanner-free and wide-field endoscopic imaging by using a single multimode optical fiber,” Phys. Rev. Lett. 109(20), 203901 (2012). https://doi.org/10.1103/physrevlett.109.20390111. S.-Y. Lee, V. J. Parot, B. E. Bouma, and M. Villiger, “Confocal 3D reflectance imaging through multimode fiber without wavefront shaping,” Optica 9(1), 112–120 (2022). https://doi.org/10.1364/optica.446178 or by focusing the fiber output to a point that can be scanned by controlling the phases of the optical fields at the proximal facet of the MMF.3,5–83. S. Turtaev, I. T. Leite, T. Altwegg-Boussac, J. M. Pakan, N. L. Rochefort, and T. Čižmár, “High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging,” Light: Sci. Appl. 7(1), 92 (2018). https://doi.org/10.1038/s41377-018-0094-x5. Y. Pu, X. Yang, I. Papadopoulos, S. Farahi, C.-L. Hsieh, C. A. Ong, C. Moser, D. Psaltis, and K. M. Stankovic, “Imaging of the mouse cochlea with two-photon microscopy and multimode fiber-based microendoscopy,” paper presented at the Biomedical Optics, .6. I. T. Leite, S. Turtaev, D. E. Boonzajer Flaes, and T. Čižmár, “Observing distant objects with a multimode fiber-based holographic endoscope,” APL Photonics 6(3), 036112 (2021). https://doi.org/10.1063/5.00383677. E. E. Morales-Delgado, D. Psaltis, and C. Moser, “Two-photon imaging through a multimode fiber,” Opt. Express 23(25), 32158–32170 (2015). https://doi.org/10.1364/oe.23.0321588. D. Stellinga, D. B. Phillips, S. P. Mekhail, A. Selyem, S. Turtaev, T. Čižmár, and M. J. Padgett, “Time-of-flight 3D imaging through multimode optical fibers,” Science 374(6573), 1395–1399 (2021). https://doi.org/10.1126/science.abl3771 However, a holographic setup with a reference beam and full calibration is necessary to retrieve the complex matrix. The machine learning method (MLM) is an alternative approach that can directly reconstruct the input image from the intensity distribution of the output speckle pattern using a deep neural network. Although impressive performance with high frame rates15,1615. L. Wang, Y. Yang, Z. Liu, J. Tian, Y. Meng, T. Qi, T. He, D. Li, P. Yan,M. Gong, Q. Liu, and Q. Xiao, “High‐speed all‐fiber micro‐imaging with large depth of field,” Laser Photonics Rev. 16, 2100724 (2022). https://doi.org/10.1002/lpor.20210072416. Z. Liu, L. Wang, Y. Meng, T. He, S. He, Y. Yang, L. Wang, J. Tian, D. Li, and P. Yan, “All-fiber high-speed image detection enabled by deep learning,” Nat. Commun. 13(1), 1433 (2022). https://doi.org/10.1038/s41467-022-29178-8 and long-term stability17,1817. S. Resisi, S. M. Popoff, and Y. Bromberg, “Image transmission through a dynamically perturbed multimode fiber by deep learning,” Laser Photonics Rev. 15(10), 2000553 (2021). https://doi.org/10.1002/lpor.20200055318. P. Fan, M. Ruddlesden, Y. Wang, L. Zhao, C. Lu, and L. Su, “Learning enabled continuous transmission of spatially distributed information through multimode fibers,” Laser Photonics Rev. 15(4), 2000348 (2021). https://doi.org/10.1002/lpor.202000348 has been demonstrated, the MLM requires large volumes of experimental data to train the neural network for image reconstruction. A third approach is the speckle imaging (SI) method,4,12,194. B. Lochocki, M. V. Verweg, J. J. M. Hoozemans, J. F. de Boer, and L. V. Amitonova, “Epi-fluorescence imaging of the human brain through a multimode fiber,” APL Photonics 7(7), 071301 (2022). https://doi.org/10.1063/5.008067212. A. M. Caravaca-Aguirre, S. Singh, S. Labouesse, M. V. Baratta, R. Piestun, and E. Bossy, “Hybrid photoacoustic-fluorescence microendoscopy through a multimode fiber using speckle illumination,” APL Photonics 4(9), 096103 (2019). https://doi.org/10.1063/1.511347619. L. V. Amitonova and J. F. de Boer, “Endo-microscopy beyond the Abbe and Nyquist limits,” Light: Sci. Appl. 9(1), 81 (2020). https://doi.org/10.1038/s41377-020-0308-x which combines the advantages of TMM and MLM. It only needs a calibration of the intensity distribution of speckle patterns produced by the MMF, and the reconstruction problem is mathematically solvable. Moreover, the SI method has a two-fold better spatial resolution compared to the focus point scanning method,2020. R. N. Mahalati, R. Y. Gu, and J. M. Kahn, “Resolution limits for imaging through multi-mode fiber,” Opt. Express 21(2), 1656–1668 (2013). https://doi.org/10.1364/oe.21.001656 and is compatible with compressive sensing techniques,2121. J. Romberg, “Imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 14–20 (2008). https://doi.org/10.1109/msp.2007.914729 enabling imaging beyond the Nyquist2222. L. V. Amitonova and J. F. De Boer, “Compressive imaging through a multimode fiber,” Opt. Lett. 43(21), 5427–5430 (2018). https://doi.org/10.1364/ol.43.005427 and Abbe limits.1919. L. V. Amitonova and J. F. de Boer, “Endo-microscopy beyond the Abbe and Nyquist limits,” Light: Sci. Appl. 9(1), 81 (2020). https://doi.org/10.1038/s41377-020-0308-x To generate uncorrelated speckle patterns for SI, it is essential to control the optical field distribution at the proximal end of the MMF. Previously, spatial light modulators (SLMs) and free-space optical systems were used to control the phase distribution of the optical field before launching the light into the MMF to produce different speckle patterns. However, such free-space systems are quite bulky, expensive, and sensitive to optical misalignment and mechanical vibrations.Silicon photonic integrated circuits (PICs) are scalable to low-cost, high-volume manufacturing using the standard processes developed for CMOS microelectronics.23,2423. W. Bogaerts and L. Chrostowski, “Silicon photonics circuit design: Methods, tools and challenges,” Laser Photonics Rev. 12(4), 1700237 (2018). https://doi.org/10.1002/lpor.20170023724. G. T. Reed and A. P. Knights, Silicon Photonics: An Introduction (John Wiley and Sons, 2004). Compact devices capable of high-speed operation25,2625. J. Witzens, “High-speed silicon photonics modulators,” Proc. IEEE 106(12), 2158–2182 (2018). https://doi.org/10.1109/jproc.2018.287763626. G. Kang, S.-H. Kim, J.-B. You, D.-S. Lee, H. Yoon, Y.-G. Ha, J.-H. Kim, D.-E. Yoo, D.-W. Lee, and C.-H. Youn, “Silicon-based optical phased array using electro-optic p-i-n phase shifters,” IEEE Photonics Technol. Lett. 31(21), 1685–1688 (2019). https://doi.org/10.1109/lpt.2019.2939550 with high reliability and low-cost can be offered by integrating a bulky system on a single chip. Spatial division multiplexing has been widely investigated using PICs with single-mode fibers,2727. D. W. U. Chan, G. Zhou, X. Wu, Y. Tong, J. Zhang, C. Lu, A. P. T. Lau, and H. K. Tsang, “A compact 112-Gbaud PAM-4 silicon photonics transceiver for short-reach interconnects,” J. Lightwave Technol. 40(8), 2265–2273 (2022). https://doi.org/10.1109/jlt.2022.3141906 multicore fibers,2828. Y. Tong, G.-H. Chen, Y. Wang, Z. Zhang, D. W. U. Chan, C.-W. Chow, and H. K. Tsang, “1.12-Tbit/s PAM-4 enabled by a silicon photonic transmitter bridged with a 7-channel MCF,” IEEE Photonics Technol. Lett. 32(16), 987–990 (2020). https://doi.org/10.1109/lpt.2020.3007665 and multimode fibers29,3029. T. Watanabe, B. I. Bitachon, Y. Fedoryshyn, B. Baeuerle, P. Ma, and J. Leuthold, “Coherent few mode demultiplexer realized as a 2D grating coupler array in silicon,” Opt. Express 28(24), 36009–36019 (2020). https://doi.org/10.1364/oe.40625130. X. Zhou and H. K. Tsang, “High efficiency multimode waveguide grating coupler for few-mode fibers,” IEEE Photonics J. 14(4), 1–5 (2022). https://doi.org/10.1109/jphot.2022.3188800 for applications in high-capacity optical communications. Despite this, the use of PICs with highly multimode fibers (>100 modes) for imaging applications is relatively unexplored. Silicon waveguides can be integrated with thermo-optic3131. M. Jacques, A. Samani, E. El-Fiky, D. Patel, Z. Xing, and D. V. Plant, “Optimization of thermo-optic phase-shifter design and mitigation of thermal crosstalk on the SOI platform,” Opt. Express 27(8), 10456–10471 (2019). https://doi.org/10.1364/oe.27.010456 or electro-optic2626. G. Kang, S.-H. Kim, J.-B. You, D.-S. Lee, H. Yoon, Y.-G. Ha, J.-H. Kim, D.-E. Yoo, D.-W. Lee, and C.-H. Youn, “Silicon-based optical phased array using electro-optic p-i-n phase shifters,” IEEE Photonics Technol. Lett. 31(21), 1685–1688 (2019). https://doi.org/10.1109/lpt.2019.2939550 phase modulators to control the phase of light coming out of each waveguide and used as an optical phased array (OPA)3232. J. Sun, E. Timurdogan, A. Yaacobi, E. S. Hosseini, and M. R. Watts, “Large-scale nanophotonic phased array,” Nature 493(7431), 195–199 (2013). https://doi.org/10.1038/nature11727 for wavefront shaping in the far field. Such integrated OPAs have many potential applications, including beam steering,3333. C. V. Poulton, A. Yaacobi, D. B. Cole, M. J. Byrd, M. Raval, D. Vermeulen, and M. R. Watts, “Coherent solid-state LIDAR with silicon photonic optical phased arrays,” Opt. Lett. 42(20), 4091–4094 (2017). https://doi.org/10.1364/ol.42.004091 free-space communications,3434. C. V. Poulton, M. J. Byrd, P. Russo, E. Timurdogan, M. Khandaker, D. Vermeulen, and M. R. Watts, “Long-range LiDAR and free-space data communication with high-performance optical phased arrays,” IEEE J. Sel. Top. Quantum Electron. 25(5), 7700108 (2019). https://doi.org/10.1109/jstqe.2019.2908555 and coherent projectors.3232. J. Sun, E. Timurdogan, A. Yaacobi, E. S. Hosseini, and M. R. Watts, “Large-scale nanophotonic phased array,” Nature 493(7431), 195–199 (2013). https://doi.org/10.1038/nature11727 OPAs are also promising alternatives to the free-space SLMs previously used in MMF imaging systems. Recently, a 128-channel OPA packaged with a laser-written 3D waveguide is used in an MMF imaging system.3535. T. Fukui, Y. Kohno, R. Tang, Y. Nakano, and T. Tanemura, “Single-pixel imaging using multimode fiber and silicon photonic phased array,” J. Lightwave Technol. 39(3), 839–844 (2020). https://doi.org/10.1109/JLT.2020.3008968 For such a system, 6.7 µm wide line pairs with a 13.4 µm pitch in a field of view of 105 µm can be experimentally resolved, and the number of resolvable points is about 1000, based on the point spread function analysis.3535. T. Fukui, Y. Kohno, R. Tang, Y. Nakano, and T. Tanemura, “Single-pixel imaging using multimode fiber and silicon photonic phased array,” J. Lightwave Technol. 39(3), 839–844 (2020). https://doi.org/10.1109/JLT.2020.3008968 Although higher spatial resolutions have been reported using SLM-based MMF imaging systems,3,4,193. S. Turtaev, I. T. Leite, T. Altwegg-Boussac, J. M. Pakan, N. L. Rochefort, and T. Čižmár, “High-fidelity multimode fibre-based endoscopy for deep brain in vivo imaging,” Light: Sci. Appl. 7(1), 92 (2018). https://doi.org/10.1038/s41377-018-0094-x4. B. Lochocki, M. V. Verweg, J. J. M. Hoozemans, J. F. de Boer, and L. V. Amitonova, “Epi-fluorescence imaging of the human brain through a multimode fiber,” APL Photonics 7(7), 071301 (2022). https://doi.org/10.1063/5.008067219. L. V. Amitonova and J. F. de Boer, “Endo-microscopy beyond the Abbe and Nyquist limits,” Light: Sci. Appl. 9(1), 81 (2020). https://doi.org/10.1038/s41377-020-0308-x PIC-based MMF imaging systems have not yet attained the same level of spatial resolutions.2020. R. N. Mahalati, R. Y. Gu, and J. M. Kahn, “Resolution limits for imaging through multi-mode fiber,” Opt. Express 21(2), 1656–1668 (2013). https://doi.org/10.1364/oe.21.001656We previously presented a conference paper on a chip-scale OPA, integrated with an array of nanoantennas to excite different modes in the MMF.3636. G. Hu, K. Zhong, Y. Qin, and H. K. Tsang, “Silicon photonics optical phase array and optical nano-antenna array for a multimode fiber imaging system,” paper presented at the TENCON 2022 IEEE Region 10 Conference (TENCON), . We now report the detailed theoretical modeling, experimental validation, and investigations on the image reconstruction using the PIC-based MMF imaging system. Our system can reach the theoretical resolution limit determined by the number of modes in the MMF. The designed PIC has a 45-channel OPA for controlling the phases of light that are launched from an array of optical nanoantennas into the MMF. The nanoantennas, being photolithographically defined, are compatible with low-cost, large-volume CMOS manufacturing and can excite a large number of spatial modes in the MMF with less than 3 dB mode-group dependent loss (MGDL). Numerical simulations show the importance of having a low MGDL: similar losses across all mode groups will maximize the finesse and number of unique speckle patterns that can be produced from the MMF and yield a high spatial resolution in the imaging system. An equivalent resolution of 1.75 µm is experimentally measured in a field of view (FOV) of 105 µm, yielding 3000 resolvable points, which is solely limited by the number of modes in the MMF2020. R. N. Mahalati, R. Y. Gu, and J. M. Kahn, “Resolution limits for imaging through multi-mode fiber,” Opt. Express 21(2), 1656–1668 (2013). https://doi.org/10.1364/oe.21.001656 and is greater than the square of the number of phase shifters. This work represents the PIC-based MMF imaging system with the highest resolution.

II. SYSTEM DESIGN AND SIMULATION RESULTS

Section:

ChooseTop of pageABSTRACTI. INTRODUCTIONII. SYSTEM DESIGN AND SIM... <<III. EXPERIMENTAL DEMONST...IV. CONCLUSION AND DISCUS...SUPPLEMENTARY MATERIALREFERENCESFigure 1(a) schematically illustrates the proposed MMF imaging system. The architecture is based on a ghost imaging (GI) scheme,37,3837. M. P. Edgar, G. M. Gibson, and M. J. Padgett, “Principles and prospects for single-pixel imaging,” Nat. Photonics 13(1), 13–20 (2019). https://doi.org/10.1038/s41566-018-0300-738. Y. Bromberg, O. Katz, and Y. Silberberg, “Ghost imaging with a single detector,” Phys. Rev. A 79(5), 053840 (2009). https://doi.org/10.1103/physreva.79.053840 with the SLM3737. M. P. Edgar, G. M. Gibson, and M. J. Padgett, “Principles and prospects for single-pixel imaging,” Nat. Photonics 13(1), 13–20 (2019). https://doi.org/10.1038/s41566-018-0300-7 or rotating scatter3939. J. H. Shapiro, “Computational ghost imaging,” Phys. Rev. A 78(6), 061802 (2008). https://doi.org/10.1103/physreva.78.061802 replaced by the silicon photonic chip and MMF. Figures 1(b) and 1(c) show the 3D structure of the silicon photonic OPA chip and the coupling interface between the chip and MMF using the nanoantenna array. The incident light is split into 45 waveguides using a star coupler. Each waveguide is independently modulated by a phase shifter. An array of waveguide-grating-based nanoantennas serves to couple the light from the PIC to the MMF. The target to be imaged is placed at the distal end of the MMF and is illuminated by the different speckle patterns from the MMF. Total transmitted power is collected by a single optical power meter acting as a bucket detector. The target can thus be reconstructed from the recorded power signals S and pre-calibrated illumination patterns I by solving the least squares problem below,where T is the transmittance distribution of the target to be resolved. More details about the principle of image reconstruction can be found in the supplementary material, Sec. I.It has been theoretically demonstrated that only 4M OPA channels are required to realize the theoretical resolution limit for an MMF with M modes for a single polarization.4040. T. Fukui, Y. Nakano, and T. Tanemura, “Resolution limit of single-pixel speckle imaging using multimode fiber and optical phased array,” J. Opt. Soc. Am. B 38(2), 379–386 (2021). https://doi.org/10.1364/josab.408985 However, to the best of our knowledge, PIC-based MMF imaging systems have not yet reached this resolution limit, even in systems using a larger number of OPA channels. The main reason can be attributed to the difficulty in exciting all the modes in the MMF. MMF modes in higher-order mode groups typically have a higher spatial frequency and are essential for high-resolution imaging. High MGDL during the coupling will result in the high-spatial-frequency information carried by high-order modes being masked by the interference of the low-order modes with higher powers, leading to a loss of spatial resolution.To solve this problem and realize high-resolution imaging, limited only by the number of modes supported by the MMF, we designed an array of nanoantennas that could excite a wide range of spatial modes in the MMF with a modest MGDL. The 3D view of the nanoantenna is shown in Fig. 2(a). A 70 nm-depth partial etch is applied to the first grating groove to improve the upward diffraction efficiency, and the remaining groove is fully etched. The 3D finite-difference time-domain (3D-FDTD) method is used to simulate the diffraction efficiency and diffracted optical field of the nanoantenna. The pitch and duty cycle of each groove are optimized using the genetic algorithm to optimize the vertical upward diffraction efficiency. The upward diffraction efficiency reaches 48% at 1310 nm after optimization. Figures 2(b) and 2(c) show the near-field and far-field diffraction patterns. The far-field diffraction is designed to overfill the receiving angle of the chosen MMF (Thorlabs FG105LCA) , which has a core diameter of 105 µm and a numerical aperture (NA) of 0.22, to make sure that modes in MMF can be fully excited. This is at the sacrifice of coupling efficiency, in that only 19% of the upward diffracted field is within the receiving angle of the MMF. The total coupling efficiency from the chip into the MMF is thus calculated to be 9%. Although this efficiency is much lower than conventional systems using galvo mirrors,4141. K. Abrashitova and L. V. Amitonova, “High-speed label-free multimode-fiber-based compressive imaging beyond the diffraction limit,” Opt. Express 30(7), 10456–10469 (2022). https://doi.org/10.1364/oe.444796 it is still acceptable for imaging applications and comparable to SLM-based systems.4242. S. Li, S. A. R. Horsley, T. Tyc, T. Čižmár, and D. B. Phillips, “Memory effect assisted imaging through multimode optical fibres,” Nat. Commun. 12(1), 3751 (2021). https://doi.org/10.1038/s41467-021-23729-1 The number of OPA channels is set to 45, and there are 45 nanoantennas within the core area of MMF, shown schematically in Fig. 1(c).Additionally, the star coupler for power splitting is specifically designed to improve the power uniformity in the 45 output waveguides. The widths of different output waveguides are tailored to account for the reduction in the optical field amplitude at large angles from the center optical axis. The variational FDTD (varFDTD) solver is utilized to do the simulation and design. The insertion loss and non-uniformity of the star coupler are simulated to be −1.27 and 1 dB, respectively. More detailed information can be found in the supplementary material, Sec. VI. The phase shifter is a straight 160 μm-long titanium nitride (TiN) heater with a thickness of 100 nm and width of 3.5 µm. The heater is located at an 800 nm distance above the silicon waveguide. The heat transport solver and finite-difference eigenmode (FDE) solver, provided by Lumerical, are utilized to model the phase shifter. The necessary power for a π phase shift is simulated to be ∼17 mW, with a response time of 12 µs.Further numerical simulations validate the use of nanoantennas to produce different speckle patterns for imaging at the distal facet of MMF. The imaging performance of the GI system is fundamentally limited by the finesse and number of linearly independent speckle patterns that can be generated. The simulation is based on the eigenmode expansion (EME) and the transfer matrix of MMF, with the detailed analytical model in the supplementary material, Sec. II. The modeling of MMF is under the weakly guiding approximation, and linearly polarized (LP) modes at 1310 nm are calculated. The simulated near-field diffraction pattern of the nanoantenna is used in the simulation. The use of a 3D array of waveguides with a mode field diameter (MFD) of 6.5 µm3535. T. Fukui, Y. Kohno, R. Tang, Y. Nakano, and T. Tanemura, “Single-pixel imaging using multimode fiber and silicon photonic phased array,” J. Lightwave Technol. 39(3), 839–844 (2020). https://doi.org/10.1109/JLT.2020.3008968 is also simulated for comparison. At the coupling interface with the MMF, the 3D waveguide array is considered to have the same position distribution as our nanoantenna array.The simulation parameters are all set based on the experimental setup. The number of OPA channels is 45. A 2 m-long step-index MMF with a core diameter of 105 µm and NA of 0.22 is used in the simulation, and two different conditions are considered when generating the transfer matrix. (1) There is no mode crosstalk in the fiber, i.e., the fiber is perfectly straight and has no perturbation to induce mode crosstalk. (2) Inter-modal coupling is present, and the MMF has a minimum bend radius of 3 cm, which is reasonable for the fiber we use. The transfer matrix of MMF is generated based on perturbation theory.1414. M. Plöschner, T. Tyc, and T. Čižmár, “Seeing through chaos in multimode fibres,” Nat. Photonics 9(8), 529–535 (2015). https://doi.org/10.1038/nphoton.2015.112 More detailed information about the transfer matrix we use in the simulation can be found in Sec. III of the supplementary material. The MMF-limited imaging performance is simulated under the condition that the phase and intensity of all pixels at the proximal facet of MMF can be independently controlled and the transfer matrix of MMF is a random unitary matrix.2020. R. N. Mahalati, R. Y. Gu, and J. M. Kahn, “Resolution limits for imaging through multi-mode fiber,” Opt. Express 21(2), 1656–1668 (2013). https://doi.org/10.1364/oe.21.001656Figures 2(d)–2(f) show the simulated results. Figure 2(d) shows the normalized average power in each mode group under different coupling strategies and MMF mode coupling conditions. LP modes with the same mode group index (l + 2m + 1) are sorted in the same mode group for the similar propagation constant. MGDL is calculated to be greater than 20 dB when the 3D waveguide is applied to excite the modes in MMF. Though the existence of mode coupling can improve the excitation of higher order mode groups, fiber modes with a mode group index greater than 20 still cannot be well excited. In comparison, our designed nanoantenna can excite both low and high order mode groups almost equally. The MGDL is simulated to be ∼3 dB for the first 40 mode groups, independent of the mode coupling condition of MMF. Though the last ten mode groups still have a much higher loss, the imaging performance is not degraded because the first 40 mode groups already contain the modes with the shortest self-correlation length, as shown in Fig. S3 in the supplementary material. It is well-known that the spatial resolution of the GI system is theoretically limited by the self-correlation function of speckle patterns for illumination.4343. F. Ferri, D. Magatti, A. Gatti, M. Bache, E. Brambilla, and L. A. Lugiato, “High-resolution ghost image and ghost diffraction experiments with thermal light,” Phys. Rev. Lett. 94(18), 183602 (2005). https://doi.org/10.1103/physrevlett.94.183602 We calculate the self-correlation function of the speckle patterns using the definition,CΔx=IΦi,xIΦi,x+ΔxxIΦi,xxIΦi,x+Δxx−1i,(2)where IΦi,x is the optical intensity of speckle patterns at location x with a certain phase distribution Φi at OPA, and ⋯x represents the average over x. As shown in Fig. 2(e), the nanoantenna-based system can always reach the limit given by the MMF, independent of the mode coupling condition. The length at half-maximum (δx) is around 1.7 µm, which is an indicator of the resolution of the imaging system. For comparison, this value is in the range of 2.5–4 µm in the imaging system using the 3D waveguide array for launching the light into the MMF because of its incomplete excitation of all modes. In addition, this leads to the degradation of singular values (SVs) of illumination matrix I, as shown in Fig. 2(f), which can estimate the orthogonality of generated speckle patterns and is indicative of the number of linearly independent measurements possible in the image reconstruction. Though SVs are cut off at the same index of N2 − N + 1 = 1981 with N = 45, as predicted by the theoretical derivation,4040. T. Fukui, Y. Nakano, and T. Tanemura, “Resolution limit of single-pixel speckle imaging using multimode fiber and optical phased array,” J. Opt. Soc. Am. B 38(2), 379–386 (2021). https://doi.org/10.1364/josab.408985 the magnitude of normalized SVs is much smaller for the 3D waveguide vs the nanoantenna array, and the information with small SVs will be hidden under the noise floor. The noise floor is around 10−3 in the experiment. The number of useful SVs above the noise floor is around 1000 for the 3D waveguide, which is only half of the predicted value. For the nanoantenna system, the SVs are 10 dB larger, and non-zero SVs are all above the noise floor. The existence of a small gap between the MMF limit is due to the restricted number of OPA channels and can be further improved by increasing the channel number to 56, as discussed in the supplementary material, Sec. V. A smaller number of non-zero SVs will lead to under-sampling2121. J. Romberg, “Imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 14–20 (2008). https://doi.org/10.1109/msp.2007.914729 during the imaging process, and the under-sampling ratio equals 66% in our condition. Nevertheless, the target can still be reconstructed by utilizing compressive sensing techniques and making use of the sparsity of the target.21,2221. J. Romberg, “Imaging via compressive sampling,” IEEE Signal Process. Mag. 25(2), 14–20 (2008). https://doi.org/10.1109/msp.2007.91472922. L. V. Amitonova and J. F. De Boer, “Compressive imaging through a multimode fiber,” Opt. Lett. 43(21), 5427–5430 (2018). https://doi.org/10.1364/ol.43.005427The ability of image reconstruction is further simulated, as shown in Fig. 3. An array of vertical and horizontal line pairs with 8, 6, 4, and 3 µm pitch is used to analyze the resolution. The simplest truncated singular value decomposition (TSVD) algorithm and pseudo-inverse of the matrix I are used to solve the least squares problem4444. S. Duan, B. Yang, F. Wang, and G. Liu, “Determination of singular value truncation threshold for regularization in ill-posed problems,” Inverse Probl. Sci. Eng. 29(8), 1127–1157 (2021). https://doi.org/10.1080/17415977.2020.1832090 to avoid the influence of the reconstruction algorithm. 3000 different speckle patterns are generated numerically by loading random phases on 45 OPA channels, and SVs are truncated at the condition value of κI<500, which is the same as the experiment. One can clearly see that the nanoantenna-based system can always resolve the 4 μm-pitch line pairs, and the 3 μm-pitch line pairs can barely be resolved, whereas the one reconstructed using the 3D-waveguide-based system can resolve neither of them for the long self-correlation length (δx). In addition, numerical simulations validate that the 45-channel OPA approaches the MMF limit with o