基于并行Transformer和CNN的图像压缩感知重构网络

张新岩; 祝勇俊; 吴宏杰; 周凡利

doi:10.3981/j.issn.1000-7857.2023.12.01823

科技导报 >

2025 , Vol. 43 >Issue 2: 108 - 116

DOI: https://doi.org/10.3981/j.issn.1000-7857.2023.12.01823

研究论文

基于并行Transformer和CNN的图像压缩感知重构网络

张新岩 ^,¹ ,
祝勇俊 ^,¹^,²^,³^,* ,
吴宏杰 ¹ ,
周凡利 ³

展开

1. 苏州科技大学电子与信息工程学院, 苏州 215009
2. 南京航空航天大学电子信息工程学院, 南京 211106
3. 苏州同元软控信息技术有限公司, 苏州 215123

祝勇俊（通信作者），高级实验师，研究方向为图像信号处理、智能楼宇与智慧交通，电子信箱：zyj@mail.usts.edu.cn

张新岩，硕士研究生，研究方向为图像处理，电子信箱：2213041047@post.usts.edu.cn

收稿日期: 2023-12-03

网络出版日期: 2025-02-19

基金资助

国家自然科学基金项目(62073231)

版权

收起

A parallel Transformer-CNN network for image compression sensing reconstruction

Xinyan ZHANG ^,¹ ,
Yongjun ZHU ^,¹^,²^,³^,* ,
Hongjie WU ¹ ,
Fanli ZHOU ³

Expand

1. School of Electronic and Information Engineering, Suzhou University of Science and Technology, Suzhou 215009, China
2. College of Electronic and Information Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 211106, China
3. Suzhou Tongyuan Software & Control Technology Company, Suzhou 215123, China

Received date: 2023-12-03

Online published: 2025-02-19

Copyright

Fold

摘要

图像压缩感知是一种能够在低采样率下实现高效信号采样与重构的技术，但在实现高质量图像重构时，面临局部与全局特征难以有效融合的问题。为此，提出一种结合Transformer与卷积神经网络（convolutional neural networks，CNN）优点的图像压缩感知重构框架（transformer-CNN mixture transformer，TCMformer）。该框架充分利用CNN的局部建模能力和Transformer的全局特征捕捉能力；设计了一种特征融合模块（TCM Block），有效桥接局部与全局特征，从而提升特征表示效率；同时，为降低模型复杂度并控制计算成本，框架采用基于窗口的Transformer结构，通过分块实现高效的全局建模。此外，引入渐进式重建策略，利用多尺度特征图逐步优化重建质量。实验结果表明，TCMformer在峰值信噪比、结构相似性和视觉效果上相较于主流的压缩感知重构算法表现更优，为实现高质量的图像重建提供了一种有效的解决方案。

关键词： 压缩感知; Transformer; 卷积神经网络; 图像重建

本文引用格式

张新岩 , 祝勇俊 , 吴宏杰 , 周凡利 . 基于并行Transformer和CNN的图像压缩感知重构网络[J]. 科技导报, 2025 , 43(2) : 108 -116 . DOI: 10.3981/j.issn.1000-7857.2023.12.01823

参考文献

原文顺序 | 文献年度倒序 | 文中引用次数倒序

1	Donoho D L . Compressed sensing[J]. IEEE Transactions on Information Theory, 2006, 52 (4): 1289- 1306. DOI

2	张华, 曹良才, 金国藩, 等. 压缩成像技术的应用与挑战[J]. 科技导报, 2018, 36 (10): 20- 29. DOI

3	Duarte M F , Davenport M A , Takhar D , et al. Single-pixel imaging via compressive sampling[J]. IEEE Signal Processing Magazine, 2008, 25 (2): 83- 91. DOI

4	Zhu L , Wu X , Sun Z Y , et al. Compressed-sensing accelerated 3-dimensional magnetic resonance cholangiopancreatography: Application in suspected pancreatic diseases[J]. Investigative Radiology, 2018, 53 (3): 150- 157. DOI

5	Yuan X , Brady D J , Katsaggelos A K . Snapshot compressive imaging: Theory, algorithms, and applications[J]. IEEE Signal Processing Magazine, 2021, 38 (2): 65- 88. DOI

6	Zhang J , Zhao D B , Gao W . Group-based sparse representation for image restoration[J]. IEEE Transactions on Image Processing, 2014, 23 (8): 3336- 3351. DOI

7	Dong W S , Shi G M , Li X , et al. Compressive sensing via nonlocal low-rank regularization[J]. IEEE Transactions on Image Processing, 2014, 23 (8): 3618- 3632. DOI

8	山世光, 阚美娜, 刘昕, 等. 深度学习: 多层神经网络的复兴与变革[J]. 科技导报, 2016, 34 (14): 60- 70. DOI

9	Shi W Z , Jiang F , Liu S H , et al. Image compressed sensing using convolutional neural network[J]. IEEE Transactions on Image Processing, 2019, 29: 375- 388.

10	Zhang Z , Liu Y , Liu J , et al. AMP-Net: Denoising-based deep unfolding for compressive image sensing[J]. IEEE Transactions on Image Processing, 2020, 30: 1487- 1500.

11	Zhang J, Ghanem B. ISTA-net: Interpretable optimization-inspired deep network for image compressive sensing[C]// Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2018: 1828-1837.

12	Kabkab M , Samangouei P , Chellappa R . Task-aware compressed sensing with generative adversarial networks[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2018, 32 (1): 1- 8.

13	Chen J W, Sun Y B, Liu Q S, et al. Learning memory augmented cascading network for compressed sensing of images[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2020: 513-529.

14	Sun Y B , Yang Y , Liu Q S , et al. Learning non-locally regularized compressed sensing network with half-quadratic splitting[J]. IEEE Transactions on Multimedia, 2020, 22 (12): 3236- 3248. DOI

15	Sun Y B , Chen J W , Liu Q S , et al. Dual-path attention network for compressed sensing image reconstruction[J]. IEEE Transactions on Image Processing, 2020, 29: 9482- 9495. DOI

16	Vaswani A, Shazeer N, Parmar N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6000-6010.

17	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[J]. arXiv preprint arXiv: 2010.11929, 2020.

18	Wang Z D, Cun X D, Bao J M, et al. Uformer: A general U-shaped transformer for image restoration[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2022: 17683-17693.

19	Jiang Y , Chang S , Wang Z . Transgan: Two pure transformers can make one strong gan, and that can scale up[J]. Advances in Neural Information Processing Systems, 2021, 34: 14745- 14758.

20	Shen M H , Gan H P , Ning C , et al. TransCS: A transformer-based hybrid architecture for image compressed sensing[J]. IEEE Transactions on Image Processing, 2022, 31: 6991- 7005. DOI

21	Gan L. Block compressed sensing of natural images[C]//Proceedings of 15th International Conference on Digital Signal Processing. Piscataway, NJ: IEEE, 2007: 403-406.

22	Arbeláez P , Maire M , Fowlkes C , et al. Contour detection and hierarchical image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33 (5): 898- 916. DOI

23	Kulkarni K, Lohit S, Turaga P, et al. ReconNet: Non-iterative reconstruction of images from compressively sensed measurements[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2016: 449-458.

24	Dong W S , Wang P Y , Yin W T , et al. Denoising prior driven deep neural network for image restoration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41 (10): 2305- 2318. DOI

25	Chen W J, Yang C L, Yang X. FSOINET: Feature-space optimization-inspired network for image compressive sensing[C]//Proceedings of ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). Piscataway, NJ: IEEE, 2022: 2460-2464.

26	Chen B , Zhang J . Content-aware scalable deep compressed sensing[J]. IEEE Transactions on Image Processing, 2022, 31: 5412- 5426.

Options

文章导航

摘要

本文引用格式

参考文献

联系我们

访问统计

模态框（Modal）标题

摘要

本文引用格式

参考文献

联系我们

访问统计