Stoner.DataFolder

class Stoner.DataFolder(*args, **kargs)

Provide an interface to manipulating lots of data files stored within a directory structure on disc.

By default, the members of the DataFolder are instances of Stoner.Data. The DataFolder emplys a lazy open strategy, so that files are only read in from disc when actually needed.

Methods

__init__(*args, **kargs)	Set the default type before creating the DataFolder.
add_group(key)	Add a new group to the current baseFolder with the given key.
all()	Iterate over all the files in the Folder and all it's sub Folders recursely.
append(value)	Append an item to the folder object.
clear()	Clear the subgroups.
compress([base, key, keep_terminal])	Compresses all empty groups from the root up until the first non-empty group is located.
concatenate([sort, reverse])	Concatenates all the files in a objectFolder into a single metadataObject like object.
count(value)	Provide a count method like a sequence.
extend(values)	S.extend(iterable) -- extend sequence by appending elements from the iterable
extract(*metadata, **kargs)	Extract metadata from each of the files in the terminal group.
fetch()	Preload the contents of the DiskBasedFolderMixin.
file(name, value[, create, pathsplit])	recursely add groups in order to put the named value into a virtual tree of baseFolder.
filter([filter, invert, copy, recurse, prune])	Filter the current set of files by some criterion.
filterout(filter[, copy, recurse, prune])	Synonym for self.filter(filter,invert=True).
flatten([depth])	Compresses all the groups and sub-groups iunto a single flat file list.
gather([xcol, ycol])	Collect xy and y columns from the subfiles in the final group in the tree.
get(name[, default])	Return either a sub-group or named object from this folder.
getlist(**kargs)	Scan the current directory, optionally recursively to build a list of filenames.
group(key)	Sort Files into a series of objectFolders according to the value of the key.
index(value[, start, end])	Provide an index method like a sequence.
insert(index, value)	Implement the insert method with the option to append as well.
items()	Return the key,value pairs for the subbroups of this folder.
keep_latest()	Filter out earlier revisions of files with the same name.
keys()	Return the keys used to access the sub-=groups of this folder.
make_name([value])	Construct a name from the value object if possible.
on_load_process(tmp)	Carry out processing on a newly loaded file to set means and extra metadata.
pop([name, default])	Return and remove either a subgroup or named object from this folder.
popitem()	Return the most recent subgroup from this folder.
prune([name])	Remove any empty groups from the objectFolder (and subgroups).
remove(value)	S.remove(value) -- remove first occurrence of value.
reverse()	S.reverse() -- reverse IN PLACE
save([root])	Save the entire data folder out to disc using the groups as a directory tree.
select(*args, **kargs)	Select a subset of the objects in the folder based on flexible search criteria on the metadata.
setdefault(k[, d])	Return or set a subgroup or named object.
slice_metadata(key[, output])	Return an array of the metadata values for each item/file in the top level group.
sort([key, reverse, recurse])	Sort the files by some key.
unflatten()	Take the file list an unflattens them according to the file paths.
unload([name])	Remove the instance from memory without losing the name in the Folder.
update(other)	Update this folder with a dictionary or another folder.
values()	Return the sub-groups of this folder.
walk_groups(walker, **kargs)	Walk through a hierarchy of groups and calls walker for each file.
zip_groups(groups)	Return a list of tuples of metadataObjects drawn from the specified groups.

Attributes

basenames	Return a list of just the filename parts of the objectFolder.
clone	Clone just does a deepcopy as a property for compatibility with Stoner.Core.DataFile.
debug	Just read the local debug value.
defaults	Build a single list of all of our defaults by iterating over the __mro__, caching the result.
depth	Give the maximum number of levels of group below the current objectFolder.
directory	Just alias directory to root now.
each	Return a Stoner.folders.each.item proxy object.
files	Return an iterator of potentially unloaded named objects.
groups	Subfolders are held in an ordered dictionary of groups.
instance	Return a default instance of the type of object in the folder.
is_empty	Return True if the folder is empty.
key	Override the parent class key to use the directory attribute.
layout	Return a tuple that describes the number of files and groups in the folder.
loaded	Iterate only over those members of the folder in memory.
loader	Return a callable that will load the files on demand.
ls	List just the names of the objects in the folder.
lsgrp	Return a list of the groups as a generator.
metadata	Return a Stoner.folders.metadata.MetadataProxy object.
mindepth	Give the minimum number of levels of group below the current objectFolder.
not_empty	Iterate over the objectFolder that checks whether the loaded metadataObject objects have any data.
not_loaded	Return an array of True/False for whether we've loaded a metadataObject yet.
objects	Return the objects in the folder are stored in a regexpDict.
pattern	Provide support for getting the pattern attribute.
root	Return the real folder root.
setas	Return the proxy for the setas attribute for each object in the folder.
shape	Return a data structure that is characteristic of the objectFolder's shape.
trunkdepth	Return the number of levels of group before a group with files is found.
type	Return the (sub)class of the Stoner.Core.metadataObject instances.