How can I preprocess a dataset that is too large to fit into memory?

If your dataset is too large to fit into memory at once, you can preprocess one channel at a time With ft_preprocessing, immediately combined with ft_resampledata. After gathering all downsampled channels, you can combine them again into a normal multi-channel data structure.

In summary, the code would look like this

for i=1:nchans
  cfgp         = [];
  cfgp.dataset = 'yourfile.dat';
  % instead of specifying channel names, you are allowed to use channel numbers
  cfgp.channel = i;
  datp         = ft_preprocessing(cfgp);
  
  cfgr            = [];
  cfgr.resamplefs = 250; 
  datr{i}         = ft_resampledata(cfgr, datp);
  
  clear datp
end

cfg = [];
datall = ft_appenddata(cfg, datr{:}); % this expands all cells into input variables
faq/how_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory.txt · Last modified: 2012/06/24 16:53 by 85.223.67.170

You are here: startfaqhow_can_i_preprocess_a_dataset_that_is_too_large_to_fit_into_memory
CC Attribution-Share Alike 3.0 Unported
www.chimeric.de Valid CSS Driven by DokuWiki do yourself a favour and use a real browser - get firefox!! Recent changes RSS feed Valid XHTML 1.0
This DokuWiki features an Anymorphic Webdesign theme, customised by Eelke Spaak and Stephen Whitmarsh.
Mobile Analytics Website Security Test