DiskBasedFolderMixin

class Stoner.folders.mixins.DiskBasedFolderMixin(*args, **kargs)[source]

Bases: object

A Mixin class that implmenets reading metadataObjects from disc.

type

the type ob object to sotre in the folder (defaults to Stoner.Core.Data)

Type

Stoner.Core.metadataObject

extra_args

Extra arguments to use when instantiatoing the contents of the folder from a file on disk.

Type

dict

pattern

A filename globbing pattern that matches the contents of the folder. If a regular expression is provided then any named groups are used to construct additional metadata entryies from the filename. Default is . to match all files with an extension.

Type

str or regexp

exclude

A filename globbing pattern that matches files to exclude from the folder. Default is *.tdms_index to exclude all tdms index files.

Type

str or regexp

read_means

If true, additional metatdata keys are added that return the mean value of each column of the data. This can hep in grouping files where one column of data contains a constant value for the experimental state. Default is False

Type

bool

recursive

Specifies whether to search recurisvely in a whole directory tree. Default is True.

Type

bool

flatten

Specify where to present subdirectories as spearate groups in the folder (False) or as a single group (True). Default is False. The DiskBasedFolderMixin.flatten() method has the equivalent effect and DiskBasedFolderMixin.unflatten() reverses it.

Type

bool

discard_earlier

If there are several files with the same filename apart from !#### being appended just before the extension, then discard all except the one with the largest value of #### when collecting the list of files.

Type

bool

directory

The root directory on disc for the folder - by default this is the current working directory.

Type

str

multifile

Whether to select individual files manually that are not (necessarily) in a common directory structure.

Type

boo

readlist

Whether to read the directory immediately on creation. Default is True

Type

bool

Attributes Summary

basenames

Return a list of just the filename parts of the objectFolder.

directory

Just alias directory to root now.

key

Override the parent class key to use the directory attribute.

not_loaded

Return an array of True/False for whether we've loaded a metadataObject yet.

pattern

Provide support for getting the pattern attribute.

Methods Summary

fetch([futures])

Preload the contents of the DiskBasedFolderMixin.

getlist(**kargs)

Scan the current directory, optionally recursively to build a list of filenames.

keep_latest()

Filter out earlier revisions of files with the same name.

on_load_process(tmp)

Carry out processing on a newly loaded file to set means and extra metadata.

save([root])

Save the entire data folder out to disc using the groups as a directory tree.

unload([name])

Remove the instance from memory without losing the name in the Folder.

Attributes Documentation

basenames

Return a list of just the filename parts of the objectFolder.

directory

Just alias directory to root now.

key

Override the parent class key to use the directory attribute.

not_loaded

Return an array of True/False for whether we’ve loaded a metadataObject yet.

pattern

Provide support for getting the pattern attribute.

Methods Documentation

fetch(futures=False)[source]

Preload the contents of the DiskBasedFolderMixin.

Keyword Arguments

futures (bool) – If True, then return te concurrent.futures object otherwise return the folder.

With multiprocess enabled this will parallel load the contents of the folder into memory.

getlist(**kargs)[source]

Scan the current directory, optionally recursively to build a list of filenames.

Keyword Arguments
  • recursive (bool) – Do a walk through all the directories for files

  • directory (string or False) – Either a string path to a new directory or False to open a dialog box or not set in which case existing directory is used.

  • flatten (bool) – After scanning the directory tree, flaten all the subgroupos to make a flat file list. (this is the previous behaviour of objectFolder.getlist())

Returns

A copy of the current DataFoder directory with the files stored in the files attribute

getlist() scans a directory tree finding files that match the pattern. By default it will recurse through the entire directory tree finding sub directories and creating groups in the data folder for each sub directory.

keep_latest()[source]

Filter out earlier revisions of files with the same name.

The CM group LabVIEW software will avoid overwirting files when measuring by inserting !#### where #### is an integer revision number just before the filename extension. This method will look for instances of several files which differ in name only by the presence of the revision number and will kepp only the highest revision number. This is useful if several measurements of the same experiment have been carried out, but only the last file is the correct one.

Returns

A copy of the DataFolder.

on_load_process(tmp)[source]

Carry out processing on a newly loaded file to set means and extra metadata.

save(root=None)[source]

Save the entire data folder out to disc using the groups as a directory tree.

Calls the save method for each file in turn.

Parameters

root (string) – The root directory to start creating files and subdirectories under. If set to None or not specified, the current folder’s diretory attribute will be used.

Returns

A list of the saved files

unload(name=None)[source]

Remove the instance from memory without losing the name in the Folder.

Parameters

name (string,int or None) – Specifies the entry to unload from memeory. If set to None all loaded entries are unloaded.

Returns

(DataFolder) – returns a copy of itself.