Spike Detection and Removal of Raman Spectra by the Method Combined Nearest Neighbor Comparison with Median Filter

Abstract

Abstract:

Laser Raman spectroscopy is widely applied in marine chemical environmental detection and analysis of substances, as an in-situ, real-time, multi- substances detection sensor technology. Charge-coupled devices (CCD) as the detector of Raman spectrometer are easily interfered by cosmic ray so that some spikes that have random location ,intensity and peak width occur on Raman spectra. The presence of such spikes can damage spectral presentation, complicate interpretation, and impede further analyses. The method combined nearest neighbor comparison method（NNC）with median filter was used to detect and remove spike from the measured spectral data of seawater in laboratory. The Raman spectra were divided into two areas firstly: Raman peaks area and the area that has no Raman peaks. The NNC processed the area that has Raman peaks and the median filter processed the area that has no Raman peaks. The result shows that the combined method can remove spike of the full spectra completely on the premise that the intensity of Raman peaks remain unchanged. The method contributes to improving the accuracy and stability of qualitative and quantitative analysis for Raman spectra.

Key words: Raman spectra, spike, nearest neighbor comparison method, median filter

CLC Number:

O433

YAN Jing-Wen, LI Ying, CHEN Jing, YANG De-Wang, ZHENG Rong-Er. Spike Detection and Removal of Raman Spectra by the Method Combined Nearest Neighbor Comparison with Median Filter[J]. Journal of Atmospheric and Environmental Optics, 2017, 12(2): 128-135.

References

[1] Brewer P G, Malby G, Pasteris J D, et al. Development of a laser Raman spectrometer for deep-ocean science [J]. Deep Sea Res. PT., 2004, 51(5): 739-753.
[2] White S N, Dunk R M, Peltzer E T, et al. In situ Raman analyses of deep‐sea hydrothermal and cold seep systems (Gorda Ridge and Hydrate Ridge) [J]. Geochem., Geophy., Geosy., 2006, 7(5): 5023.
[3] Du Z, Li Y, Chen J, et al. Feasibility investigation on deep ocean compact autonomous Raman spectrometer developed for in-situ detection of acid radical ions [J]. Chin. J. Oceanol. Limnol., 2015, 33(2): 545-550.
[4] Zhang D M, Ben-Amotz D. Removal of cosmic spikes from hyper-spectral images using a hybrid upper-bound spectrum method [J]. Appl. Spectr., 2002, 56(1): 91-98.
[5] Schulze H G, Turner R F B. A two-dimensionally coincident second difference cosmic ray spike removal method for the fully automated processing of Raman spectra [J]. Appl. Spectr., 2014, 68(2): 185-191.
[6] Bussian B M, H?rdle W. Robust smoothing applied to white noise and single outlier contaminated Raman spectra [J]. Appl. Spectr., 1984, 38(3): 309-313.
[7] Katsumoto Y, Ozaki Y. Practical algorithm for reducing convex spike noises on a spectrum [J]. Appl. Spectr., 2003, 57(3): 317-322.
[8] Hill W, Rogalla D. Spike-correction of weak signals from charge-coupled devices and its application to Raman spectroscopy [J]. Anal. Chem., 1992, 64(21): 2575-2579.
[9] Phillips G R, Harris J M. Polynomial filters for data sets with outlying or missing observations: application to charge-coupled-device-detected Raman spectra contaminated by cosmic rays [J]. Anal. Chem., 1990, 62(21): 2351-2357.
[10] Schulze H G, Turner R F B. A fast, automated, polynomial-based cosmic ray spike-removal method for the high-throughput processing of Raman spectra [J]. Appl. Spectr., 2013, 67(4): 457-462.
[11] Takeuchi H, Hashimoto S, Harada I. Simple and efficient method to eliminate spike noise from spectra recorded on charge-coupled device detectors [J]. Appl. Spectr., 1993, 47(1): 129-131.
[12] Zhang D M, Jallad K N, Ben-Amotz D. Stripping of cosmic spike spectral artifacts using a new upper-bound spectrum algorithm [J]. Appl. Spectr., 2001, 55(11): 1523-1531.
[13] Behrend C J, Tarnowski C P, Morris M D. Identification of outliers in hyperspectral Raman image data by nearest neighbor comparison [J]. Appl. Spectr., 2002, 56(11): 1458-1461.
[14] Zhang L, Henson M J. A practical algorithm to remove cosmic spikes in Raman imaging data for pharmaceutical applications [J]. Appl. Spectr., 2007, 61(9): 1015-1020.
[15] Cappel U B, Bell I M, Pickard L K. Removing cosmic ray features from Raman map data by a refined nearest neighbor comparison method as a precursor for chemometric analysis [J]. Appl. Spectr., 2010, 64(2): 195-200.
[16] Li Sheng，Dai Liankui. A simple spike removal algorithm for online Raman monitoring [J]. Chinese Journal of Light Scattering, 2011, 23(3): 188-194(in Chinese).
李晟, 戴连奎. 一种简单的在线拉曼光谱 spike 剔除方法 [J]. 光散射学报, 2011, 23(3): 188-194.
[17] Li S, Dai L K. An improved algorithm to remove cosmic spikes in Raman spectra for online monitoring [J]. Appl. Spectr., 2011, 65(11): 1300-1306.
[18] Mozharov S, Nordon A, Littlejohn D, et al. Automated cosmic spike filter optimized for process Raman spectroscopy [J]. Appl. Spectr., 2012, 66(11): 1326-1333.