************************** Working with Images ************************** .. py:currentmodule:: Stoner.Image Introduction ============ The :mod:`Stoner.Image` package provides a means to carry out image processing functions in a smilar way that :mod:`Stoner.Core` and :class:`Stoner.Data` and :class:`Stoner.DataFolder` do. :class:`ImageFile` ------------------ In the Stoner package, a :class:`ImageFile` represent a single image along with associated metadata from when the image was taken. The actual image data is stored in the :attr:`ImageFile.image` as a 2D numpy array. This is actually a masked array - the mask is useful for ignoring certain portions of the image when carrying out operations. As well as the image data, a :class:`ImageFile` also has a :attr:`ImageFile.metadata` attribute that stores the associated image metadata as a set of key-value pairs in a dictionary. As with :class:`Stoner.Data`, :class:`ImageFile` has a rich array of methods that can carry out different image analysis tasks - in common with the philosophy of the Stoner package, these methods operate *in place* and also return the modified :class:`ImageFile` allowing a sequence of operations to be completed by chaining the method calls together. The array of methods available to a :class:`ImageFile` is particularly rich as the class automatically wraps functions from the popular scikit-image package (:py:mod:`skimage`) as well as the :py:mod:`scipy.ndimage` module. In general such functions take an image as their first argument and :class:`ImageFile` passes its own image data as that argument to whatever function is being wrapped. If the function Returns image data of the same size that the original image, then the wrapper replaces the :class:`ImageFile`'s image data with the new image data. As well as the wrapped scikit-image and scipy.ndimage functions, :class:`ImageFile` provides a set of its own functions for carrying out some common image processing operations. By default :class:`ImageFile` can load and save both .tiff and .png files and will embed its metadata into custom tags in those formats. Subclasses of the :class:`ImageFile` are provided for reading the .png files produced by the CM group's Evico Kerr Microscopes as well as a :class:`Stoner.HDF5.STXMImage` class that reads the SLS Pollux Beamline's STXM images. Loading an Image ---------------- The :class:`ImageFile` constructor supports taking a string argument which is interpreted as a filename of an image format recognised by PIL. The resulting image data is used to form the contents of the :attr:`ImageFile.image` which holds the image data.:: from Stoner import ImageFile im = Image.ImageFile("kermit.png") .. image:: ../../sample-data/kermit.png Like :class:`Stoner.Data` :class:`ImageFile` supports image metadata. Where this can be stored in the file, e.g. in png and tiff images, this is read in automatically. This metadata is stored as a :class:`Stoner.Core.typeHintedDict` dictionary. This metadata can be set directly in the construction of the :class:`ImageFile`:: im = ImageFile(np.arange(10000).reshape(100,100), metadata={'myarray':1}) Examining and manipulating the ImageFile ----------------------------------------- IF you are using an ipython console or Jupyter Notebook, then the :class:`ImageFile` supports rich format outputs and it will show you a picture of the image data as its default representation. ImageFile Attributes and Properties ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The key attributes and properties of the :class:`ImageFile` are: - :py:attr:`ImageFile.image`: This is the actual numpy array of data that is the image. - :attr:`ImageFile.metadata`: This is the dictionary that contains the metadata associated with the image. This is normally parameters and information about the measurement or data that is encoded within the the measurement file, but can be supplemented by information added by the user directly or by methods and functions. Each item of metadata is normally referred to by a string *key*. - :attr:`ImageFile.filename`: This is the name of the file from which the image was loaded (as a string). It is also used as a default title when displaying the image. - :attr:`ImageFile.mask`: This represents the mask data for an image file. By default it is entirely False, meaning all parts of the image are visible. It supports a variety of different ways of manipulating which parts of the image are masked - see the section below. - :attr:`ImageFile.draw`: THis is a special attribute that lets you draw simple geometric shapes on the image. :class:`ImageFile`s use the appropriate data type for the underlying image format - integers for png and either integers or floating point numbers for tiffs (STXM files are always floating point). The :meth:`ImageFile.convert` method can be used to convert the data to a more appropriate format:: im.convert(float) im.asfloat() im.asint() It also has simpler :meth:`ImageFile.asfloat` and :meth:`ImageFile.asint` methods for converting to floating point or integer formats as required. There are a number of other attributes that can tell you information about the :class:`ImageFile` such as: - :attr:`ImageFile.centre`: the coordinates of the centre of the image - :attr:`ImageFile.aspect`: the image aspect ratio - :attr:`ImageFile.shape`: the size of the underlying numpy array for the image. To make it easier to quickly get a rotated image, the attributes :attr:`ImageFile.CW` and :attr:`ImageFile.CCW` return copies of the :class:`ImageFile` that are rotate 90 degrees clockwise and counter-clockwise. In many cases the default behaviour of modifying the image data in place may not be desired - to get a copy of the :class:`ImageFile` you use the :attr:`ImageFile.clone` attribute:: new_im = im.clone.gaussian(4.0) # create a copy of im and then apply a gaussian blur .. image:: figures/kermit-blurred.png An :class:`ImageFile` can be indexed like an array to return a new :class:`ImageFile` object which contains only a subset of the original image data.:: im[:,10:50] # Return a vertical section of the image .. image:: figures/kermit_vslice.png :: im[::-1,:] # Flip the image vertically .. image:: figures/kermit_flipped.png Indexing an :class:`ImageFile` with a string will instead access the metadata stored with the object.:: im["Loaded as"] >>> 'KermitPNGFile' The :class:`ImageFile` inherits from a dictionary like class - so all the usual Python dictionary methods work on the metadata of the image.:: for metadata_key in im: print(f"{key}->{im[keuy}") averages = im.pop("Averages",1) if "Message" in im: orint(f"Message = {im['Message']}") Like the :class:`Stoner.Data` class, the metadata dictionary will fall back to matching keys with regular expressions if not exact match is found. In this case, if multiple metadata items match then the result of indexing with a string may return a dictionary with all the matching keys. The metadata associated with an image will always be stored in the best Python type that can be found - where possible metadata is stored in files with a hint as to the native type of that data and this type hint is used to map to an appropriate Python type. Unmappable types are retained as a string representation. Common Functions ---------------- It is not possible to provide an exhaustive guid to all the functions made available from scikit-image, scipy.ndimage and opencv2, but this section will cover some of the commonly used functionality. Specify a box ^^^^^^^^^^^^^ Many of the functions can be used with a *_box* parameter to limit their operation to a specified region of the image. Additionally the :meth:`ImageFile.crop` method will discard the image outside a region specified in a similar way. The working box can be given as follows: - A single integer: This is interpreted to exclude a region of n pixels from all sides of the image. - A single floating point number between 0.0 and 0.5: This is interpreted to exclude this corresponding fraction of the width and height from each side of the image - A tuple of 4 numbers, or 4 arguments: This is interpreeted as a sequence of pxiel coordinates for (left-x, right-x, top-y, bottom-y). If any of the numbers are None, then this is take as the minimum or maximum extents of the width or height (depending on whether the None value substitutes for the left, right top or bottom coordinate). - A single string: The argument is interpreted as the name of a metadata item that will define the box to be used. - A sigle value False: This is equivalent to the whole image (i.e. to not specify a box) - A single None value: In this case a copy of the image is shown to the user and they are invited to draw the box with the mouse and then press the key to confirm their selection. .. image:: figures/kermit-crop.png Aligning Two Images ^^^^^^^^^^^^^^^^^^^ The :meth:`Stoner.ImageFile.align` method can ve used to align an image to a reference image. It offers a variety of different algorithms which may be better or worse depending on the nature of the image. The options are: - chi2_shift: this uses the image-registration module to carry out a chi-squared analysis of shifting the two images relative to each other. - imreg_dft: this uses the imreg_dft module to carry out the image registration. In essence it takes a fourier transform of the two images and then compares the phases within the fourier transforms to calculate the necessary shift. - scharr: this is the default method used. It first of all applies a Scharr edge detection filter and uses the imreg_dft method to find the translation vector. - cv2: this method uses the opencv2 package's alignment algorithm. Align also takes a *_box* keyword parameter to confine the section of the image used for the alignment to a specific region (this can make the operation more efficient if much of the images are featureless), and a *scale* parameter that will upscale the image before attempting to do the alignment. This may improve sub-pixel alignment corrections. As well as returning the shifted images, :meth:`ImageFile.align` will record the translation vector used in the metadata item *tvec*.:: im.align(ref,method="imreg_dft", scale=4) print(f"Translation Vector={im['tvec']}") Align also calculates a *translation-limits* metadata item that represents the maximum extent of the image where the pixels have not been added to accommodate the translation for the alignment. Cropping an Image ^^^^^^^^^^^^^^^^^ The :meth:`ImageFile.crop` will crop the image to the box you specify. If called with no parameters, then it will show the current image and allow you to draw the crop box on it. Pressing return will select the region to crop. The combination of the *translation_limits* metadata item from the :meth:`ImageFile.align` method and the specification of the box for cropping allows images to be aligned and then cropped to match.:: im.align(ref,method="imreg_dft", scale=4) # Crop the image according to the translation limits worked out be align. im.crop("translation_limits") # Now crop the reference image, using the revsed translation limits ref.crop(im.translation_limits("tvec", reverse=True)) Contrast Normalisation ^^^^^^^^^^^^^^^^^^^^^^ If the absolute value of the image data is not important, then normalising the image can improve the contrast. The :meth:`ImageFile.normalise` will do this.By default it will remap the image intensity values to the range -1 to +1. This can be changed with the *limits* keyword parameter. The area of the image used to calculate the mapping between the old and new intensity values can be set with the *sample* parameter. If there are a few pixels with extreme values in the image then this can reduce the contrast for the bulk of the image. By using the *limits* parameter you can restrict the portion of the input range to be rescaled over. For example, *limits=(0.1,0.9)* will map the first 10% and the last 10% of the pixels to -1 or +1 and the middle 80% between the two extremes.:: im.normalise(limits=(0.01,0.99)) Image Filtering ^^^^^^^^^^^^^^^ Filtering is frequently used to remove npise and other artefacts from images. Most of the filters that can be applied are functions within the sckit-image.filters module. A good general purpose option is to use a gaussian filter - this will convolute the image with a 2-D gaussian function with a user-definable width. This is effective for removing high frequency noise and speckle.:: im.gaussian(1.0) For band-pass filt4ering the scikit-image.filters method *difference_of_gaussians* can be used. This filters the image with two different gaussian blurs and then takes the difference between them - the smaller gaussian blur removes high frequency noise whilst the large gaussian removes low spatial frequency variations.:: im.difference_of_gaussians(1.0,2.0) .. image:: figures/kermit-bandpass.png Another filtering approach is to us a Savitsky-Golay filter - this fits a polynomial surface locally over the data to smooth or differentiate the date. This sort of filtering is good for preserving feature sizes in the original data set.:: im:sgolay2d(pints=5) Fourier transforms ^^^^^^^^^^^^^^^^^^ Fast Fourier transforms are oftern used when we need to examain the frequency spectra of image data. The :meth:`ImageFile.fft` method provides a convenient one-stop method for generating the fft that can also take care of some of the artefacts that can result.:: fft=im.clone.fft(replace_dc=True, window="hamming") .. image:: figures/kermit-fft.png In this example, a copy of the image is transformed to the mangitude of its fourier transform. THe fourier transform is shifted so that the central pixels are the 0-ffrequency componennts. The optional keywrod *remove_dc* replaces the 0 frequency data with the mean value of the FFT to avoid a large spike int he FFT from the mean value of the image. The *window* parameter tells the method to multiply the image by the corresponding window function (from scikit-image.filters.window) before calculating the FFT. This avoids artefacts caused by the discontinuities at the edges of the image. The shifting of the FFT to align the dc componentns to the centre of the image can be controlled with the *shift* keyword parameter, whilst the output is controlled by the *phase* parameter - False gives the magnitude, True returns the phase angle in radians and None returns the full complex FFT. To aid with analyhsing radial distributions in FFTs (or images), the :meth:`ImageFile.radial_profile` method can be used. This will compute a profile from a given centre outwards - either integrating over all angles, or restricting to specific angles. At its simpletst one can just do:: profile=fft.radial_profile() Which will return a :class:`Stoner.Data` object with columns for the radial distance, mean pixel value at the corresponding radius, standard deviation and number of pixels counted. The optional *angle* keyword parameter will select either one angle (float) or a rangle of angles (tuple of two floats). This can be easily plotted since the :class:`Stoner.Data` object is created with the appropriate columns setup as x oand y data columns.:: fft.radial_profile(angle=(-0.04,0.04)).plot(plotter=semilogy) .. image:: figures/kermit-fft-profile.png Further functions ^^^^^^^^^^^^^^^^^ Further functions that could be useful: - im.threshold_minmax(0.2,0.8) Returns a binary image - im.plot_histogram() Plot a histogram of the pixel intensities - im.level_image() Flatten a skewed image background - im.subtract_image(otherim) Subtract another image and enhance contrast - im.align(otherim) Translate image to line up with other im Scikit-Image & scipy.ndimage Functions ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ While many local analysis functions have been added to ImageFile one of the big benefits is that function proxy methods have been added to make the entire scikit-image library and scipy routines available. Function requests will first search local image functions and secondly look up any function from the external libraries. The proxy will pass the :attr:`ImageFile.image` attribute as the first argument to any external call to a scikit-image or scipy function. The return value for such calls is handled a bit carefully: #. If the return value is a 2d numpy.ndarray like type that has the same size as the original, or if the *_* keyword argument is set to **True** then the original :attr:`ImageFile.image` is replaced with the returned result. #. If the return value is anything else then it is simply passed back to the calling program. In this way, many operations can be carried out 'in-place' on a :class:`ImageFile`. For example:: ImageFile Representation ------------------------ By default, the representation of an ImageFile is just a short textual description, however if the *short_repr& and *short_img_repr* options are both set to False and a graphical console is in use with an ipython kernel, then th special _repr_png_ method will show a picture of the contents of the ImageFile instead.:: i = Stopner.Image.ImageFile("kermit.png") i >>> kermit.png() of shape (479, 359) (uint16) and 53 items of metadata from Stoner import Options Options.short_repr=False Options.shoft_img_repr=False i >>> .. image:: ../../sample-data/kermit.png Alternatively the :meth:`ImageArray.imshow` method (accessible to :class:`ImagerFile`) will show the image data in a matplotlib window. :class:`ImageArray`: A numpy array like class ============================================= Somewhat analogous to :class:`Stoner.Core.DataArray`, the :class:`ImageArray` is a specialised subclass of :class:`numpy.ma.MaskedArray` used to store the image data in ImageFile. The numpy.ndarray like data can be accessed at any point via either :attr:`ImageFile.image` or :attr:`ImageFile.data` and will be accepted by functions that take an numpy.ndarray as an argument. Working with Lots of Images: :class:`ImageFolder` and :class:`ImageStack` ========================================================================== Just as :class:`Stoner.DataFolder` allows you to efficiently process lots of separate :class:`Stoner.Data` files, :class:`ImageFolder` does the same for lots of :class:`ImageFile` files. It is based on the same parent :class:`Stoner.Fodlers.baseFolder` class - so has similar abilities to iterate, form into sub-folders and so on. In addition, an :class:`Imagefolder` has additional attributes and methods for working with multiple images. Due to the potentially large amount of data involved in processing images it is good to take advantage of native numpy's speed wherever possible. To this end :class:`Stoner.Image.ImageStack` is now available. This works very similarly to ImageFolder but internally represents the image stack as a 3d numpy array. For example:: imst = ImageStack('pathtomyfolder', pattern='*.tif') #directory is held in memory but images are not loaded yet imst = imst['subfolder'] #take advantage of :class:`DiskBasedFolder` grouping abilities imst.translate(5,3) #instantiate the stack and translate all images You can request and manipulate this 3d array directly with the imarray property, alternatively you can ask for any function accepted by the underlying ImageFile (including the scikit-image and scipy library).