w=nt_find_outliers(x,toobig1,toobig2) - find outliers (glitches, etc.). w: mask matrix (0: bad, 1: good) x: data toobig1: absolute threshold for glitches toobig2: relative threshold for outliers Outliers are found independently for each channel. Data can be 2D or 3D. If 3D, data are folded and variance stats are calculated over folded (concatenated) columns. W is same size as X. TOOBIG1 is an absolute threshold that applies to absolute value. TOOBIG2 is a threshold that applies to absolute value relative to mean absolute value. For any value above threshold the mask is set to zero, else one. NoiseTools
0001 function w=nt_find_outliers(x,toobig1,toobig2); 0002 %w=nt_find_outliers(x,toobig1,toobig2) - find outliers (glitches, etc.). 0003 % 0004 % w: mask matrix (0: bad, 1: good) 0005 % 0006 % x: data 0007 % toobig1: absolute threshold for glitches 0008 % toobig2: relative threshold for outliers 0009 % 0010 % Outliers are found independently for each channel. 0011 % 0012 % Data can be 2D or 3D. If 3D, data are folded and variance stats are 0013 % calculated over folded (concatenated) columns. W is same size as X. 0014 % 0015 % TOOBIG1 is an absolute threshold that applies to absolute value. TOOBIG2 0016 % is a threshold that applies to absolute value relative to mean absolute value. 0017 % For any value above threshold the mask is set to zero, else one. 0018 % 0019 % NoiseTools 0020 0021 if nargin<2; error('!'); return; end 0022 if nargin<3; toobig2=[]; end 0023 0024 [m,n,o]=size(x); 0025 x=nt_unfold(x); 0026 0027 % remove mean 0028 x=nt_demean(x); 0029 0030 % apply absolute threshold: 0031 w=ones(size(x)); 0032 if ~ isempty(toobig1); 0033 w(find(abs(x)>toobig1))=0; 0034 x=nt_demean(x,w); 0035 w(find(abs(x)>toobig1))=0; 0036 x=nt_demean(x,w); 0037 w(find(abs(x)>toobig1))=0; 0038 x=nt_demean(x,w); 0039 else 0040 w=ones(size(x)); 0041 end 0042 0043 0044 % apply relative threshold 0045 if ~isempty(toobig2); 0046 X=nt_wmean(x.^2,w); 0047 X=repmat(X,size(x,1),1); 0048 idx=find(x.^2>(X*toobig2)); 0049 w(idx)=0; 0050 end 0051 0052 w=nt_fold(w,m);