Translator Disclaimer
15 November 2016 Nonconvex compressive video sensing
Author Affiliations +
Abstract
High-speed cameras explore more details than normal cameras in the time sequence, while the conventional video sampling suffers from the trade-off between temporal and spatial resolutions due to the sensor’s physical limitation. Compressive sensing overcomes this obstacle by combining the sampling and compression procedures together. A single-pixel-based real-time video acquisition is proposed to record dynamic scenes, and a fast nonconvex algorithm for the nonconvex sorted 1 regularization is applied to reconstruct frame differences using few numbers of measurements. Then, an edge-detection-based denoising method is employed to reduce the error in the frame difference image. The experimental results show that the proposed algorithm together with the single-pixel imaging system makes compressive video cameras available.

1.

Introduction

Video acquisition captures time-dependent natural scenes and brings real-time images directly to screens for immediate observation. It not only serves for the live television (TV) production, but also for security, military, and industrial operations including professional video cameras, camcorders, closed circuit TV, webcams, camera phones, and special camera systems. In traditional video acquisition, e.g., H.261, H.265, and MPEG series, the sampling and compression procedures are implemented in sequential order. The Nyquist–Shannon sampling theorem requires the sampling rate to be at least twice that of the signal frequency for guaranteed exact recovery. The compression procedure is implemented by video compression chipsets1 or separate software.2

Although state-of-the-art video cameras can record most nature scenes, they do not work for very high-resolution images or high fps videos because the growth in data storage, communication, and processing is far behind the growth in data generation. In space exploration, an image of the shuttle discovery flight deck could be 2.74 gigapixels,3 and a bubble dynamics research needs a 500-fps video microscopy.4 More importantly, commercialized high-performance video cameras are extremely expensive, e.g., the price of a basic model with 7500 fps, one-megapixel resolution, and 12-bit color depth (FASTCAM SA5 from Photron) is around $100,000.

The limitation comes from weak light irradiation and the readout bandwidth when capturing high-speed objects at a high resolution. As shown in Fig. 1 and Eq. (1), the reflected illumination is collected by sensor arrays in a limited space–time volume

Eq. (1)

J=1015F2tIsrcRqΔ2.
The number of electrons (J) accumulated on each pixel is reversely proportional to the square of the ratio of the focal length to the aperture of the lens (F), but proportional to exposure time (t), incident illumination (Isrc), scene reflectivity (R), quantum efficiency (q), and the pixel size (Δ2).5 In video sensing, the exposure time (t) corresponds to the temporal resolution and the pixel size (Δ2) is related to the spatial resolution. In other words, the temporal and spatial resolutions are mutual restraint in conventional video cameras due to the imaging sensors’ requirement on the minimum number of accumulated electrons and the fixed number of total electrons. The spatial resolution will decrease when the temporal resolution increases. Another limitation is the sensor’s readout speed. The readout timing includes an analog-to-digital conversion, clear charge from the parallel register, and shutter delay, e.g., a one-megapixel, 1000 fps, and 16-bit color camera will need a 4-GB/s readout circuit.

Fig. 1

Light illumination in single-lens reflex cameras.

JEI_25_6_063003_f001.png

To obtain high-resolution images and high fps videos, the sampling rate has to be reduced, and compressive sensing technique can be applied. Compressive sensing6 allows combining both sampling and compression procedures together. This paradigm directly samples the signal in a compressed form such that the sampling rate can be significantly reduced. Compressive sensing has attracted extreme interest in imaging,7 geophysical data analysis,8 control and robotics,9 communication,10 and medical imaging processing.11

Compressive sensing has been applied in compressive video sensing since 2006, when the single-pixel camera setup was first used for video sampling.12 In this first approach, the three-dimensional (3-D) video was reconstructed with all the measurements together using 3-D wavelets as a sparse representation. This method cannot be used for real-time video streaming without incurring latency and delay because all the measurements have to be obtained before the reconstruction starts. Since then, in order to reconstruct the frames one by one for the purpose of real-time streaming, most approaches reconstruct or sample reference frames with more measurements and find the differences between two consecutive frames with fewer measurements. There are mainly two types of strategies: sampling the frame and sampling the difference between frames. In the first sampling method, in order to obtain a continuous video, motion estimation techniques are applied to recover frames from reference frames. For example, the evolution of dynamic textured scenes was modeled as a linear dynamical system.13 A multiframe motion estimation algorithm was proposed.14 The latest compressive video sensing research learned a linear mapping between video sequences and corresponding measured frames.15 In addition, the correlation between consecutive frames in the frequency domain16 and other transform domains17 was also used.

There are also several approaches in sampling the difference between two frames. For example, Stankovic et al.18 split the video frame into nonoverlapping blocks of equal size, and compressive sampling was performed on sparse blocks determined by predicting sparsities based on previous reference frames, which were sampled conventionally. The remaining blocks were sampled fully. It would be time-consuming to determine the sparse blocks because every block has to be tested. In addition, directly sampling the difference between two consecutive frames was employed19 to save the sampling time.

Though compressive sensing techniques are used in video sensing, most of the approaches use the convex 1 minimization to approximate the nonconvex 0 minimization, which is a nondeterministic polynomial-time (NP)-hard and difficult to solve. The compressive sensing theorem can reduce the number of measurements using the 1 minimization. However, with nonconvex regularizations, it can reduce the number of measurements and thus the sampling rate further so as to achieve real-time video capturing. Recently, there are many nonconvex regularizations proposed to obtain better performance than the 1 norm in compressive sensing.20,21,22

In this paper, a single-pixel compressive video sensing framework based on the nonconvex sorted 1 regularization is proposed for fast and super resolution video. In this framework, we sample reference frames using the spatial sparsity (individual image sparsity) and the difference between two frames using the temporal sparsity. In Sec. 2, we first give a short review about compressive sensing and nonconvex solvers. Then, we propose our nonconvex compressive video sensing framework. The experimental results are depicted in Sec. 3.

2.

Compressive Video Sensing

2.1.

Compressive Sensing

The core of compressive sensing is recovering the sparse vector xRn from a small number of linear measurements y=Φx, where ΦRm×n is the measurement matrix (mn). There are many solutions for the underdetermined linear system if y is in the range of Φ, and we are interested in finding the sparsest one among all the solutions. However, finding the sparsest solution is NP-hard. Therefore, instead of solving the NP-hard problem, people are looking into alternative approaches. Convex approaches are of great interest because there are lots of algorithms for solving these convex problems and it is easy to analyze the solutions of the convex problems. If x is sparse and Φ satisfies some conditions such as the null space property,23 the incoherence condition,24 and the restricted isometry property,25 the following problem is equivalent for finding the sparest solution:

Eq. (2)

x˜=argminxx1subject to  Φx=y.
When there is noise in the measurements, i.e., Φx+n=y with n being the white Gaussian noise, we solve the following problem instead:

Eq. (3)

x˜=argminxx1+λ2Φxy2,
where λ is a parameter for balancing the data fitting term and the regularization term. In order to solve these convex 1 problems, many algorithms are proposed.26,27

Although the 1 minimization is fully understood and stable with theoretical guarantee, the number of required measurements is still high, and the performance is not good in many applications with a small number of measurements. For example, radiologists want to reduce more projections and thus radiation than that required for 1 minimization in computed tomography. For the difference between two frames in a video, we want to decrease the number of measurements further such that it can realize higher fps videos than current cameras can produce. In order to recover signals from even fewer measurements, nonconvex regularizations are applied, and a short review will be given in Sec. 2.2.

2.2.

Nonconvex Optimization Problems for Compressive Sensing

In this section, we review several nonconvex regularizations for compressive sensing and their corresponding algorithms. Denote x=(x1,x2,,xn)Rn, the truth sparse signal as x0, and xl as the l’th iteration.

The p (0p1) term is commonly used,28 and it has 0 and 1 as special cases. Because of the nonconvexity, it recovers sparse signals with even fewer measurements than the convex counterpart, 1. To solve the nonconvex problems, there are several approaches. We describe three of them on both the noise-free and noisy cases. First, two reweighted algorithms for the following noise-free case are presented:

Eq. (4)

x˜=argminxxpsubject to  Φx=y.

The iteratively reweighted 1 minimization (IRL1)20 replaces the p term using a weighted 1 term with the weights depending on the previous iteration. The iteration is expressed as

Eq. (5)

xl+1=argminxi=1n1(|xil|+ϵ)1p|xi|subject to  Φx=y.
For every iteration, a weighted 1 minimization problem has to be solved and iterative algorithms are applied.

Similarly, the iteratively reweighted least squares21,22 replace the p term using a weighted least squares term with the weights depending on the previous iteration. The iteration is expressed as

Eq. (6)

xl+1=argminxi=1n1(|xil|2+ϵ)1p/2|xi|2subject to  Φx=y.
In this case, there is an analytical solution for the weighted 2 minimization problem, since it is equivalent to a least squares problem.

Except for these two reweighted algorithms for solving p minimization problems, some algorithms for solving convex optimization problems are applied to solve nonconvex problems with general nonconvex regularizations.29 One example is the forward–backward iteration. In each forward–backward iteration, for solving

Eq. (7)

x˜=argminxr(x)+λ2Φxy2,
where r(x) is a nonconvex regularization term including xp and the following mentioned nonconvex sorted 1 as special cases, a proximal mapping of the nonconvex regularization term follows a gradient descent on the data fidelity term, i.e.,

Eq. (8)

xl+1=argminxτr(x)+12x(xlτλΦT(Φxly)2.
However, for p minimization, there are only analytical solutions when p=0, 1/2, 2/3, and 1.30

The success of p minimization and both iterative algorithms for solving p minimization problems depicts that it is better to assign small weights for components with large absolute values and large weights for zero components and components with small absolute values. A nonconvex sorted 1 that assigns weights based on the ranking of absolute values was developed by Huang et al.31 Let the coefficients {ωi}i=1n be a nondecreasing sequence of nonnegative real numbers, i.e., 0ω1ωn0. The nonconvex sorted 1 regularization is defined as

Eq. (9)

rω(x1,x2,,xn)=ω1|x[1]|+ω2|x[2]|++ωn|x[n]|,
where |x[1]||x[n]| are the absolute values of the components in x ranked in decreasing order. Two special cases of nonconvex sorted 1 are 2-level 1 with w1=w2==wk=a1<1=wk+1==wn and iterative support detection (ISD) with w1=w2==wk=0<1=wk+1==wn. In addition, Huang et al. suggested a way for adaptively changing the weights during the iteration instead of having a fixed set of weights for better performance. The proposed update rule is

Eq. (10)

ωil={1,if  i>Kl,er(Kli)/Kl,otherwise,
where r controls the rate of decreasing ωi from 1 to 0 and Kl is the smallest i such that |x[i+1]lx[i]l|>xl/β with some positive β.32

2.3.

Video Compressive Sampling

A video can be considered as a series of images, as shown in Fig. 2 (left), where the coordinate space (x,y,t) consists both the spatial domain (x,y) and the temporal domain (t). Each frame could be realized as a static natural image that is redundant because natural images are intrinsically sparse in a specific domain.24,33 Another redundancy happens between similar frames in the temporal domain. As shown in Fig. 3, more than 85% of the pixels have no significant changes. Therefore, difference coding34 in MPEG and H.265 series reuses existing frames and updates only the pixels with significant changes.

Fig. 2

Sparsity in videos.

JEI_25_6_063003_f002.png

Fig. 3

Difference between two consecutive frames. The difference between the left and middle images is shown on the right. We can see that most pixels are unchanged in these two figures.

JEI_25_6_063003_f003.png

As discussed in Sec. 1, the objective of compressive video sensing is to combine both compression and sampling procedures to achieve the signal compression in hardware. In our proposed compressive video sensing, there are two types of image frames: intraframes (I-frames in H.264 or reference frames) and interframes (P-frames in H.264), shown in Fig. 4. The compressive sampling is applied on both I-frames and P-frames, where P-frames are reconstructed by the difference between P-frames and their previous frames.

Fig. 4

A frame sequence with one I-frame (reference frame) and three P-frames.

JEI_25_6_063003_f004.png

Since I-frames are considered as static images and the image compressive sampling has already been studied for single-pixel cameras,7,35 a total variation algorithm36 is applied to recover intraframes from the I-frame sampling. For the P-frames, because the difference between similar frames is sparse, a nonconvex regularization is adopted to reduce the number of samples and thus increase the compression ratio. We compare the performance of four different nonconvex regularizations numerically and choose the best in the experiment. The four regularizations are: p with IRL1, ISD, 2-level, and the nonconvex sorted 1 (m-level). In IRL1, the weights are updated by

Eq. (11)

ωil=1|xi|+max(0.5l1,0.88).
For 2-level, we choose a=0.6. For m-level, we choose β=7 and r=0.1.

We compare the runtimes, root-mean-square error (RMSE), and the peak signal-to-noise ratio (PSNR) for these four algorithms on the difference between two consecutive frames (64×64) in Fig. 5. The difference between the left and the middle images in Fig. 5 is shown on the right. We choose the measurement matrices to be randomized Bernoulli matrices with ±1 entries. The sampling rate (the number of measurements/the number of pixels) is changed from 6% to 35%. The comparison result is shown in Fig. 6, where the x-axis represents the sampling rate. When the number of measurements is small, nonconvex algorithms are unstable because they can easily be trapped at stationary points and the strategy for adaptively updating weights may not work so well. Overall, m-level is the most efficient and effective algorithm among all these four algorithms. Therefore, we choose m-level in our experiments in Sec. 3.

Fig. 5

Difference image of two consecutive frames; the difference between the left and middle images is shown in the right.

JEI_25_6_063003_f005.png

Fig. 6

Comparison of four nonconvex algorithms for signal recovery at different sampling rates. Overall, the m-level is the most efficient and effective algorithm.

JEI_25_6_063003_f006.png

Though nonconvex algorithms are able to recover sparse signals accurately from a small number of linear measurements, there is still error due to the hardware noise and the modeling error. For example, there is noise in the measurements and the algorithms cannot recover the sparse signals exactly. In Fig. 7, we show the exact difference image between two frames on the left and compare it with that recovered using the nonconvex sorted 1 on the middle. It is noticed that there are many isolated pixels with small nonzero values in the recovered difference image, and these pixels are supposed to have zero values. In order to improve this, we develop a simple and effective method to remove these pixels and update only the pixels in the areas with significant changes.

Fig. 7

Frame difference recovery comparison: (a) ground truth, (b) recovery by nonconvex algorithm, and (c) after denoising.

JEI_25_6_063003_f007.png

We apply the Sobel operator with a pair of 3×3 convolution masks on the recovered difference image to find the edges since the Sobel kernels compute the gradient with smoothing in both the horizontal and vertical directions. Then a threshold is selected to obtain a binary mask that indicates the pixels with large gradient values. However, it does not delineate the outline of the changing area of interest. Then the binary gradient mask is dilated using the vertical structuring element followed by the horizontal structuring element for a better outline. Because the mask shows only the edges of the difference image and the areas with significant changes are inside the edges, the whole areas with significant changes are obtained via filling the holes inside the edges using a flood fill operation via the MATLAB® function “imfill.” This method keeps the most significant changes and removes error on the difference image so as to reduce the reconstruction error in P-frames. Figure 7(c) shows the performance of this postprocessing (denoising) procedure. The flow chart for this procedure is described in Fig. 8.

Fig. 8

Flow chart of denoising using image segmentation.

JEI_25_6_063003_f008.png

Due to the frame difference sensing mechanism, the reconstruction error accumulates because every time we reconstruct P-frames using the difference between two consecutive frames. The error in the first P-frame is accumulated to the second P-frame. Therefore, the reconstruction of the first P-frame after I-frames is very important, and an improvement on this frame also improves following P-frames. On the other hand, if the number of P-frames between two consecutive I-frames is small, we can compute the difference image between the P-frame and the previous I-frame instead to avoid the accumulated error from previous P-frames.

The next numerical experiment shows that we can apply the simple denoising procedure to improve the reconstruction results of the first P-frame and all the P-frames after that. In this numerical experiment, there are five P-frames after one I-frame. In Fig. 9, all five P-frames are plotted. The first row has five ground true frames (P01 to P05). For the second and third rows, we show the reconstruction results using the difference image between two consecutive frames, and the reconstruction results using the difference image between P-frames and the I-frame are shown in the fourth and fifth rows. The reconstruction results using m-level without the denoising step are shown in the second row (P11 to P15) and the fourth row (P31 to P35). The reconstruction results with the denoising step are shown in the third row (P21 to P25) and the fifth row (P41 to P45). The PSNR and RMSE values are shown in Tables 1 and 2. From both tables, we can see that the PSNR value is decreasing and the RMSE value is increasing for the five P-frames, if the difference images between two consecutive frames are used and the denoising step improves all P-frames, especially the first P-frame. However, if all the P-frames are compared with the I-frame, the improvement of the denoising step is large for all five P-frames. This numerical experiment suggests that we may choose to compare P-frames with the previous I-frame instead of the previous frame because the error in the previous P-frames will be accumulated.

Table 1

PSNR values for the five reconstructed P-frames with four methods: difference images between two consecutive images without the denoising step (m-level); difference images between two consecutive images with the denoising step (denoising); difference images between P-frames and the I-frame without the denoising step (m-level*); and difference images between P-frames and the I-frame with the denoising step (denoising*).

P01P02P03P04P05
m-level40.898737.458736.274535.632335.0012
Denoising42.338237.583936.692835.937135.0856
m-level*40.898739.538640.212839.334139.5685
Denoising*42.338240.598441.524040.700841.0858

Table 2

PSNR values for the five reconstructed P-frames with four methods: difference images between two consecutive images without the denoising step (m-level); difference images between two consecutive images with the denoising step (denoising); difference images between P-frames and the I-frame without the denoising step (m-level*); and difference images between P-frames and the I-frame with the denoising step (denoising*).

P01P02P03P04P05
m-level2.29943.41673.91574.21634.5340
Denoising1.94823.36783.73174.07094.4901
m-level*2.29942.68912.48832.75322.6799
Denoising*1.94822.38022.13962.35232.2504

Fig. 9

Accumulation error, ground true frames (P01 to P05), m-level without the denoising (P11 to P15), m-level with the denoising (P21 to P25), m-level directly to I-frame without the denoising (P31 to P35), m-level directly to I-frame with the denoising (P41 to P45).

JEI_25_6_063003_f009.png

The whole algorithm for P-frames reconstruction is depicted in Table 3. The steps (a) to (c) show the nonconvex sorted 1 calculation process, while steps (d) to (e) demonstrate the edge-detection denoising procedure to reduce the error in the compressive video sensing.

Table 3

P-frames reconstruction algorithm.

Algorithm
Initialize x0, β, r and τ
forl= 1: maxit
a. Compute Kl
b. Update ωl
c. Apply one forward–backward iteration and check stopping rules.
end
d. Find the areas with significant changes
e. Reconstruct the P-frame by updating only the pixels values in the areas identified in the previous step.

3.

Experiments

The projection measurement matrices can be implemented by spatial light modulators such as the digital micromirror device (DMD) and the liquid crystal on silicon. The DMD runs as fast as 32,000 Hz, and we use a DMD with 6000 Hz in the experiments. A DMD chip has several thousand microscopic mirrors arranged in a rectangular array on its surface. These mirrors correspond to the pixels in the image to be reconstructed. The mirrors can be individually rotated ±12  deg to an on or off state. These two states correspond to ±1 in the Bernoulli matrix. During the sampling process, the measurement matrix is sent to the DMD controller row by row. The matrices for P-frames are selected from the rear end of the matrix for the previous I-frame, e.g., if the previous I-frame measurement matrix is ΦRm×n, then the P-frame measurement matrix will be Φ(mp+1:m,:)Rp×n with pm. During the experiments, the irradiator (THORLABS LIU850A) is 850 nm near the IR source, and a silicon photodiode (THORLABS FDS1010) is chosen as the receiver sensor.

We validate the proposed nonconvex compressive video sensing system using two experiments: a linear moving object and a rotating object. In the first experiment with a linear moving airplane in Fig. 10, the frame rate is 10 fps. There is only one P-frame between two consecutive I-frames, i.e., t00,t02,,t16 are I-frames, while t01,t03,,t17 are P-frames. The sampling ratios are 18% and 8.5% for I-frames and P-frames, respectively. The proposed system records the whole scene in real time.

Fig. 10

Moving object video recording.

JEI_25_6_063003_f010.png

The second experiment is to capture the rotation of a fan. As shown in Fig. 11, each blade is designed with a different length for easy identification. There are three P-frames between two consecutive I-frames, and each row in Fig. 11 shows one I-frame on the first column and three P-frames after the I-frame on the last three columns. The frame rate is 18 fps, and the sampling ratios are 20% and 9% for I-frames and P-frames, respectively.

Fig. 11

Rotating object video recording.

JEI_25_6_063003_f011.png

4.

Conclusions

Nonconvex compressive sensing algorithms require a fewer number of linear measurements to reconstruct a sparse signal than convex algorithms. In this work, the nonconvex sorted 1 approach is employed to reconstruct the difference images, which are sparse, and decrease the sampling rate. Furthermore, an edge-detection-based denoising step is applied to reduce the error on the difference image. Thus, it requires a smaller number of measurements compared to the conventional compressive video sensing. We tested our algorithm on the real-time video reconstruction in the experiments. Though the frame rate in the experiments is only 18 fps, it can reach up to 105 fps based on current DMD mirror speed (maximum 32,000 Hz).

Acknowledgments

This research work was partially supported under National Science Foundation Grants Nos. IIS-0713346 and DMS-1621798, Office of Naval Research Grants Nos. N00014-04-1-0799 and N00014-07-1-0935, the U. S. Army Research Laboratory, and the U. S. Army Research Office under Grant No. W911NF-14-1-0327.

References

1. 

M. Irvin, T. Kitazawa and T. Suzuki, “A new generation of MPEG-2 video encoder ASIC and its application to new technology markets,” in Int. Broadcasting Convention Conf., 391 –396 (1996). http://dx.doi.org/10.1049/cp:19960840 Google Scholar

2. 

G. J. Sullivan and T. Wiegand, “Video compression from concepts to the H.264/AVC standard,” Proc. IEEE, 93 (1), 18 –31 (2005). http://dx.doi.org/10.1109/JPROC.2004.839617 IEEPAD 0018-9219 Google Scholar

3. 

, “Space shuttle discovery flight deck by national geographic,” (2015) http://www.gigapan.com/gigapans/102753 ( July ). 2015). Google Scholar

4. 

M. Hepher, D. Duckett and A. Loening, “High-speed video microscopy and computer enhanced imagery in the pursuit of bubble dynamics,” Ultrason. Sonochem., 7 229 –233 (2000). http://dx.doi.org/10.1016/S1350-4177(00)00058-4 Google Scholar

5. 

O. Cossairt, M. Gupta and S. K. Nayar, “When does computational imaging improve performance?,” IEEE Trans. Image Process., 22 (2), 447 –458 (2013). http://dx.doi.org/10.1109/TIP.2012.2216538 IIPRE4 1057-7149 Google Scholar

6. 

D. L. Donoho, “Compressed sensing,” IEEE Trans. Inf. Theory, 52 (4), 1289 –1306 (2006). http://dx.doi.org/10.1109/TIT.2006.871582 IETTAW 0018-9448 Google Scholar

7. 

H. Chen et al., “Infrared camera using a single nano-photodetector,” IEEE Sens. J., 13 (3), 949 –958 (2013). http://dx.doi.org/10.1109/JSEN.2012.2225424 ISJEAZ 1530-437X Google Scholar

8. 

Y. Wang, J. Cao and C. Yang, “Recovery of seismic wavefields based on compressive sensing by an 1 -norm constrained trust region method and the piecewise random sub-sampling,” Geophys. J. Int., 187 (1), 199 –213 (2011). http://dx.doi.org/10.1111/j.1365-246X.2011.05130.x GJINEA 0956-540X Google Scholar

9. 

B. Song et al., “Compressive feedback-based motion control for nanomanipulation: theory and applications,” IEEE Trans. Rob., 30 (1), 103 –114 (2014). http://dx.doi.org/10.1109/TRO.2013.2291619 ITREAE 1552-3098 Google Scholar

10. 

P. Zhang et al., “A compressed sensing based ultra-wideband communication system,” in IEEE Int. Conf. on Communications, 1 –5 (2009). http://dx.doi.org/10.1109/ICC.2009.5198584 Google Scholar

11. 

M. Lustig et al., “Compressed sensing MRI,” IEEE Signal Process. Mag., 25 (2), 72 –82 (2008). http://dx.doi.org/10.1109/MSP.2007.914728 ISPRE6 1053-5888 Google Scholar

12. 

M. B. Wakin et al., “Compressive imaging for video representation and coding,” in Proc. of Picture Coding Symp., 1 –6 (2006). Google Scholar

13. 

A. C. Sankaranarayanan et al., “Compressive acquisition of dynamic scenes,” in European Conf. on Computer Vision, 129 –142 (2010). Google Scholar

14. 

S. Bi et al., “Compressive video recovery using block match multi-frame motion estimation based on single pixel cameras,” Sensors, 16 (3), 1 –8 (2016). http://dx.doi.org/10.3390/s16030318 SNSRES 0746-9462 Google Scholar

15. 

M. Iliadis, L. Spinoulas and A. K. Katsaggelos, “Deep fully-connected networks for video compressive sensing,” (2016). Google Scholar

16. 

J. Chen et al., “Residual distributed compressive video sensing based on double side information,” Acta Autom. Sin., 40 (10), 2316 –2323 (2014). http://dx.doi.org/10.1016/S1874-1029(14)60363-3 THHPAY 0254-4156 Google Scholar

17. 

N. Eslahi, A. Aghagolzadeh and S. Mehdi, “Image/video compressive sensing recovery using joint adaptive sparsity measure,” Neurocomputing, 200 88 –109 (2016). http://dx.doi.org/10.1016/j.neucom.2016.03.013 Google Scholar

18. 

V. Stankovic, L. Stankovi and S. Cheng, “Compressive video sampling,” in 16th European Signal Processing Conf., 1 –6 (2008). http://dx.doi.org/10.1109/CISP.2008.476 Google Scholar

19. 

J. Zheng and E. L. Jacobs, “Video compressive sensing using spatial domain sparsity,” Opt. Eng., 48 087006 (2009). http://dx.doi.org/10.1117/1.3206733 Google Scholar

20. 

E. Candes, M. Wakin and S. Boyd, “Enhancing sparsity by reweighted 1 minimization,” J. Fourier Anal. Appl., 14 (5), 877 –905 (2008). http://dx.doi.org/10.1007/s00041-008-9045-x Google Scholar

21. 

R. Chartrand and W. Yin, “Iteratively reweighted algorithms for compressive sensing,” 2008 IEEE International Conference on Acoustics, Speech and Signal Processing, 3869 –3872 IEEE(2008). http://dx.doi.org/10.1109/ICASSP.2008.4518498 Google Scholar

22. 

I. Daubechies et al., “Iteratively reweighted least squares minimization for sparse recovery,” Commun. Pure Appl. Math., 63 (1), 1 –38 (2010). Google Scholar

23. 

A. Cohen, W. Dahmen and R. DeVore, “Compressed sensing and best k-term approximation,” J. Am. Math. Soc., 22 (1), 211 –231 (2009). http://dx.doi.org/10.1090/S0894-0347-08-00610-3 0894-0347 Google Scholar

24. 

J. A. Tropp, “Greed is good: algorithmic results for sparse approximation,” IEEE Trans. Inf. Theory, 50 (10), 2231 –2242 (2004). http://dx.doi.org/10.1109/TIT.2004.834793 IETTAW 0018-9448 Google Scholar

25. 

E. Candes, J. Romberg and T. Tao, “Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information,” IEEE Trans. Inf. Theory, 52 (2), 489 –509 (2006). http://dx.doi.org/10.1109/TIT.2005.862083 IETTAW 0018-9448 Google Scholar

26. 

D. Needell and J. Tropp, “CoSaMP: iterative signal recovery from incomplete and inaccurate samples,” Appl. Comput. Harmon. Anal., 53 (12), 93 –100 (2010). http://dx.doi.org/10.1145/1859204.1859229 ACOHE9 1063-5203 Google Scholar

27. 

J. Yang and X. Yuan, “Linearized augmented Lagrangian and alternating direction methods for nuclear norm minimization,” Math. Comput., 82 (281), 301 –329 (2013). http://dx.doi.org/10.1090/mcom/2013-82-281 MCMPAF 0025-5718 Google Scholar

28. 

R. Chartrand, “Exact reconstruction of sparse signals via nonconvex minimization,” IEEE Signal Process. Lett., 14 (10), 707 –710 (2007). http://dx.doi.org/10.1109/LSP.2007.898300 Google Scholar

29. 

, “Fast L1-L2 minimization via a proximal operator,” (2016). Google Scholar

30. 

Z. Xu et al., “l1/2 regularization: a thresholding representation theory and a fast solver,” IEEE Trans. Neural Networks Learn. Syst., 23 (7), 1013 –1027 (2012). http://dx.doi.org/10.1109/TNNLS.2012.2197412 Google Scholar

31. 

X. L. Huang, L. Shi and M. Yan, “Nonconvex sorted 1 minimization for sparse approximation,” J. Oper. Res. Soc. China, 3 207 –229 (2015). http://dx.doi.org/10.1007/s40305-014-0069-4 Google Scholar

32. 

Y. Wang and W. Yin, “Sparse signal reconstruction via iterative support detection,” SIAM J. Imaging Sci., 3 (3), 462 –491 (2010). http://dx.doi.org/10.1137/090772447 Google Scholar

33. 

B. Olshausen and D. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” Nature, 381 607 –609 (1996). http://dx.doi.org/10.1038/381607a0 Google Scholar

34. 

D. N. Hein and N. Ahmed, “Video compression using conditional replenishment and motion prediction,” IEEE Trans. Electromagn. Compat., EMC-26 (3), 134 –142 (1984). http://dx.doi.org/10.1109/TEMC.1984.304204 IEMCAE 0018-9375 Google Scholar

35. 

M. Duarte et al., “Single-pixel imaging via compressive sampling,” IEEE Signal Process. Mag., 25 (2), 83 –91 (2008). http://dx.doi.org/10.1109/MSP.2007.914730 ISPRE6 1053-5888 Google Scholar

36. 

C. Li, “An efficient algorithm for total variation regularization with applications to the single pixel camera and compressive sensing,” 10 –80 Rice University, (2009). Google Scholar

Biography

Liangliang Chen received his bachelor’s and master’s degrees in electrical engineering from the Huazhong University of Science and Technology, Wuhan, China, in 2009 and 2007, respectively. Currently, he is pursuing his PhD at Michigan State University, East Lansing. His research interests include infrared sensor and imaging, ultraweak signal detection in nanosensors, signal processing, analog circuits, and carbon nanotube/graphene nanosensors.

Ming Yan received his PhD from the University of California, Los Angeles, in 2012. He is an assistant professor at the Department of Computational Mathematics, Science and Engineering and the Department of Mathematics, Michigan State University. His research interests include signal and image processing, optimization, and parallel and distributed methods for large-scale datasets.

Chunqi Qian received his BS degree in chemistry from Nanjing University and his PhD in physical chemistry from the University of California, Berkeley, in 2007. Following postdoctoral trainings at the National High Magnetic Field Laboratory and the National Institutes of Health, he joined Michigan State University as an assistant professor in radiology. His research interest includes the development and application of imaging technology in biomedical research.

Ning Xi received his DSc degree in systems science and mathematics from Washington University in St. Louis, Missouri, USA, in 1993. Currently, he is the chair professor of robotics and automation at the Department of Industrial and Manufacturing System, and director of Emerging Technologies Institute of the University of Hong Kong. He is a fellow of the Institute of Electrical and Electronics Engineers (IEEE). His research interests include robotics, manufacturing automation, micro/nanomanufacturing, nanosensors and devices, and intelligent control and systems.

Zhanxin Zhou received her bachelor’s and master’s degrees in control engineering from the Second Artillery Engneering College, Xi’an, China, in 1992 and 1997, respectively. She received her PhD in control engineering from Beijing Institute of Technology, Beijing, China, in 2008. Her research interests include infrared imaging, imaging enhancement, nonlinear filter and optimal control.

Yongliang Yang received his BS degree in mechanical engineering from Harbin Engineering University, Harbin, China, in 2005. He received his MS and PhD degrees from the University of Arizona, Tucson, USA, in 2012 and 2014, respectively. He has been a research associate at Michigan State University since 2014. His research interests include micro/nanorobotics and their application in biomedicine.

Bo Song received his BEng degree in mechanical engineering from Dalian University of Technology, Dalian, China, in 2005, and his MEng degree in electrical engineering from the University of Science and Technology of China, Hefei, China, in 2009. Currently, he is pursuing his PhD at the Department of Electrical and Computer Engineering, Michigan State University, East Lansing. His research interests include nanorobotics, nonvector space control, compressive sensing, and biomechanics.

Lixin Dong received his BS and MS degrees in mechanical engineering from Xi’an University of Technology, Xi’an, China, in 1989 and 1992, respectively, and his PhD in microsystems engineering from Nagoya University, Nagoya, Japan, in 2003. He is an associate professor at Michigan State University. His research interests include nanorobotics, nanoelectromechnical systems, mechatronics, mechanochemistry, and nanobiomedical devices. He is a senior editor of the IEEE Transactions on Nanotechnology.

© 2016 SPIE and IS&T 1017-9909/2016/$25.00 © 2016 SPIE and IS&T
Liangliang Chen, Ming Yan, Chunqi Qian, Ning Xi, Zhanxin Zhou, Yongliang Yang, Bo Song, and Lixin Dong "Nonconvex compressive video sensing," Journal of Electronic Imaging 25(6), 063003 (15 November 2016). https://doi.org/10.1117/1.JEI.25.6.063003
Published: 15 November 2016
JOURNAL ARTICLE
9 PAGES


SHARE
Advertisement
Advertisement

CHORUS Article. This article was made freely available starting 15 November 2017

Back to Top