Home > NoiseTools > nt_find_outliers.m

# nt_find_outliers

## PURPOSE

w=nt_find_outliers(x,toobig1,toobig2) - find outliers (glitches, etc.).

## SYNOPSIS

function w=nt_find_outliers(x,toobig1,toobig2);

## DESCRIPTION

    w=nt_find_outliers(x,toobig1,toobig2) - find outliers (glitches, etc.).

      w: mask matrix (0: bad, 1: good)

      x: data
      toobig1: absolute threshold for glitches
      toobig2: relative threshold for outliers

     Outliers are found independently for each channel.

     Data can be 2D or 3D.  If 3D, data are folded and variance stats are
     calculated over folded (concatenated) columns.  W is same size as X.

     TOOBIG1 is an absolute threshold that applies to absolute value.  TOOBIG2
     is a threshold that applies to absolute value relative to mean absolute value.
     For any value above threshold the mask is set to zero, else one.

     NoiseTools

## CROSS-REFERENCE INFORMATION

This function calls:

- nt_demean [y,mn]=nt_demean(x,w) - remove weighted mean over cols
- nt_fold y=fold(x,epochsize) - fold 2D to 3D
- nt_unfold y=nt_fold(x) - unfold 3D to 2D
- nt_wmean y=nt_wmean(x,w,dim) - weighted average

This function is called by:

## SOURCE CODE

    0001 function w=nt_find_outliers(x,toobig1,toobig2);
    0002 %w=nt_find_outliers(x,toobig1,toobig2) - find outliers (glitches, etc.).
    0003 %
    0004 %  w: mask matrix (0: bad, 1: good)
    0005 %
    0006 %  x: data
    0007 %  toobig1: absolute threshold for glitches
    0008 %  toobig2: relative threshold for outliers
    0009 %
    0010 % Outliers are found independently for each channel.
    0011 %
    0012 % Data can be 2D or 3D.  If 3D, data are folded and variance stats are
    0013 % calculated over folded (concatenated) columns.  W is same size as X.
    0014 %
    0015 % TOOBIG1 is an absolute threshold that applies to absolute value.  TOOBIG2
    0016 % is a threshold that applies to absolute value relative to mean absolute value.
    0017 % For any value above threshold the mask is set to zero, else one.
    0018 %
    0019 % NoiseTools
    0020
    0021 if nargin<2; error('!'); return; end
    0022 if nargin<3; toobig2=[]; end
    0023
    0024 [m,n,o]=size(x);
    0025 x=nt_unfold(x);
    0026
    0027 % remove mean
    0028 x=nt_demean(x);
    0029
    0030 % apply absolute threshold:
    0031 w=ones(size(x));
    0032 if ~ isempty(toobig1);
    0033     w(find(abs(x)>toobig1))=0;
    0034     x=nt_demean(x,w);
    0035     w(find(abs(x)>toobig1))=0;
    0036     x=nt_demean(x,w);
    0037     w(find(abs(x)>toobig1))=0;
    0038     x=nt_demean(x,w);
    0039 else
    0040     w=ones(size(x));
    0041 end
    0042
    0043
    0044 % apply relative threshold
    0045 if ~isempty(toobig2);
    0046     X=nt_wmean(x.^2,w);
    0047     X=repmat(X,size(x,1),1);
    0048     idx=find(x.^2>(X*toobig2));
    0049     w(idx)=0;
    0050 end
    0051
    0052 w=nt_fold(w,m);