Group-based Sparse Representation for Image Restoration

Jian Zhang1, Debin Zhao1, Wen Gao2

1School of Computer Science and Technology, Harbin Institute of Technology
2National Engineering Laboratory for Video Technology, Peking University

Abstract—Traditional patch-based sparse representation modeling of natural images usually suffer from two problems. First, it has to solve a large-scale optimization problem with high computational complexity in dictionary learning. Second, each patch is considered independently in dictionary learning and sparse coding, which ignores the relationship among patches, resulting in inaccurate sparse coding coefficients. In this paper, instead of using patch as the basic unit of sparse representation, we exploit the concept of group as the basic unit of sparse representation, which is composed of nonlocal patches with similar structures, and establish a novel sparse representation modeling of natural images, called group-based sparse representation (GSR). The proposed GSR is able to sparsely represent natural images in the domain of group, which enforces the intrinsic local sparsity and nonlocal self-similarity of images simultaneously in a unified framework. Moreover, an effective self-adaptive dictionary learning method for each group with low complexity is designed, rather than dictionary learning from natural images. To make GSR tractable and robust, a split Bregman based technique is developed to solve the proposed GSR-driven L0 minimization problem for image restoration efficiently. Extensive experiments on image inpainting, image deblurring and image compressive sensing recovery manifest that the proposed GSR modeling outperforms many current state-of-the-art schemes in both PSNR and visual perception.

Paper:

Group-based Sparse Representation for Image Restoration
J. Zhang, D. Zhao, W. Gao
IEEE Transactions on Image Processing
[PDF]　[Matlab Code]   

Instructions: click on the thumbnail image to see the corresponding results from various algorithms.

PSNR (dB) Comparison of Experimental Results:

1. Image Inpainting
2. Image Deblurring
3. Image Compressive Sensing Recovery

1. Image Inpainting

Image Restoration from Partial Random Samples

Input

Ground Truth

 

Barbara in the case of Ratio=20%

Barbara20

HSR (28.83 dB)

SKR (21.92 dB)

NLTV (23.46 dB)

SAIST (29.68 dB)

BPFA (25. 70 dB)

Proposed (31.32 dB)


Input

Ground Truth

 

 Parrots in the case of Ratio=20%

Parrots20

HSR (28.63 dB)

SKR (28.79 dB)

NLTV (27.58 dB)

SAIST (29.41 dB)

BPFA (27.63 dB)

Proposed (29.83 dB)


Input

Ground Truth

 

 House in the case of Ratio=20%

House20

HSR (32.35 dB)

SKR (30.40 dB)

NLTV (31.19 dB)

SAIST (35.73 dB)

BPFA (30.89 dB)

Proposed (35.61 dB)

 

Text Removal

Input

Ground Truth

 

Barbara

BarbaraText

HSR (38.86 dB)

SKR (30.81 dB)

NLTV (32.60 dB)

SAIST (39.00 dB)

BPFA (34.28 dB)

Proposed (40.86 dB)


Input

Ground Truth

 

 House

HouseText

HSR (38.65 dB)

SKR (38.65 dB)

NLTV (38.44 dB)

SAIST (41.20 dB)

BPFA (39.01 dB)

Proposed (42.51 dB)



2. Image Deblurring

Input

Ground Truth

 

 Bike
Uniform Kernel: 9x9 with sigma=0.5

BikeDeblur1

TVMM (26.51 dB)

L0_ABS (26.78 dB)


IDDBM3D (28.45 dB)

NCSR (27.92 dB)

Proposed (28.61 dB)

 

Input

Ground Truth

 

 Barbara
Gaussian Kernel: fspecial('Gaussian', [7 7], 8) with sigma=0.5

BarbaraDeblur2

TVMM (27.79 dB)

L0_ABS (28.39 dB)


IDDBM3D (31.73 dB)

NCSR (30.37 dB)

Proposed (33.52 dB)

 

Input

Ground Truth

 

 Leaves
Motion Kernel: fspecial('motion', 20, 45) with sigma=0.5

LeavesDeblur3

TVMM (30.60 dB)

L0_ABS (29.44 dB)

 

IDDBM3D (34.40 dB)

NCSR (34.23 dB)

Proposed (34.54 dB)

 

Input

Ground Truth

 

 Barbara (256x256)
Uniform Kernel: 9x9 with sigma=sqrt(2)

BarbaraDeblur7

TVMM (26.00 dB)

L0_ABS (26.41 dB)

 

IDDBM3D (27.98 dB)

NCSR (28.10 dB)

Proposed (28.95 dB)

 

Input

Ground Truth

 

 House (256x256)
Gaussian Kernel: fspecial('Gaussian', 25, 1.6) with sigma=sqrt(2)

HouseDeblur8

TVMM (33.01 dB)

L0_ABS (33.07 dB)

 

IDDBM3D (34.08 dB)

NCSR (33.63 dB)

Proposed (34.45 dB)

 

Table 1: Six Typical Deblurring Experiments

All the six typical experiments achieved by GSR can be downloaded here: Download

Input

Ground Truth

 Barbara (512x512) (Scenario 2 in Table 1)

BarbaraDeblur4

TVMM (ISNR=1.33 dB)

NCSR (ISNR=3.64 dB)

IDDBM3D (ISNR=3.96 dB)

Proposed (ISNR=4.80 dB)



3. Image Compressive Sensing Recovery

Ground Truth

 Barbara (in the case of CS Ratio=0.2)

BarbaraCS

DWT (23.96 dB)

MH (31.09 dB)

TV (23.79 dB)

CoS (26.60 dB)

Proposed (34.59 dB)


Ground Truth

 Vessels (in the case of CS Ratio=0.2)

VesselsCS

DWT (21.14 dB)

MH (24.95 dB)

TV (22.04 dB)

CoS (26.71 dB)

Proposed (31.58 dB)


References

[1] SKR: H. Takeda, S. Farsiu, and P. Milanfar, “Kernel regression for image processing and reconstruction,” IEEE Trans. on Image Process., vol. 16, no. 2, pp. 349–366, Feb. 2007.
[2] HSR: X. Li, “Image recovery from hybrid sparse representation: a deterministic annealing approach,” IEEE J. of Selected Topics in Signal Processing, vol. 5, no. 5, pp. 953–962, Sep. 2011.
[3] NLTV: X. Zhang, M. Burger, X. Bresson and S. Osher, “Bregmanized nonlocal regularization for deconvolution and sparse reconstruction,” SIAM J. Imaging Sci., vol. 3, no. 3, pp. 253–276, 2010.
 [4] BPFA: M. Zhou, H. Chen, J. Paisley, L. Ren, L. Li, Z. Xing, D. Dunson, G. Sapiro and L. Carin, “Nonparametric Bayesian dictionary learning for analysis of noisy and incomplete images,” IEEE Trans. Image Processing, vol. 21, no. 1, pp. 130–144, Jan. 2012.
[5] SAIST: W. Dong, G. Shi, and X. Li, “Nonlocal image restoration with bilateral variance estimation: a low-rank approach,” IEEE Trans. On Image Processing, vol. 22, no. 2, pp. 700–711, Feb. 2013.
[6] MH: C. Chen, E. W. Tramel, and J. E. Fowler, “Compressed-Sensing Recovery of Images and Video Using Multi-hypothesis Predictions,” Proc. of the 45th Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, pp. 1193–1198, Nov. 2011.
[7] TV: C. Li, W. Yin, and Y. Zhang, “TVAL3: TV Minimization by Augmented Lagrangian and Alternating Direction Algorithm,” 2009.
[8] CoS: J. Zhang, D. Zhao, C. Zhao, R. Xiong, S. Ma and W. Gao, “Image compressive sensing recovery via collaborative sparsity,” IEEE Journal on Emerging and Selected Topics in Circuits and Systems, vol. 2, no. 3, pp. 380–391, Sep. 2012.
[9] TVMM: J. Bioucas-Dias, M. Figueiredo, J. Oliveira, “Total-variation image deconvolution: A majorization-minimization approach,” Proc. of IEEE Int. Conf. on Acoustics, Speech, and Signal Processing, Toulouse, France, 2006.
[10] L0_ABS: J. Portilla, “Image restoration through l0 analysis-based sparse optimization in tight frames,” Proc. of IEEE Int. conf. Image Process., pp. 3909–3912, Nov. 2009.
[11] IDDBM3D: A. Danielyan, V. Katkovnik, and K. Egiazarian, “BM3D frames and variational image deblurring,” IEEE Trans. Image Process., vol. 21,no. 4, pp. 1715–1728, Apr. 2012.
[12] NCSR: W. Dong, L. Zhang, G. Shi and X. Li, “Nonlocally Centralized Sparse Representation for Image Restoration,” IEEE Trans. on Image Processing, vol. 22, no. 4, pp. 1620–1630, Apr. 2013.

Last updated: April 16, 2014.

TOP