Table Of Contents

Release notes

Version 0.13.0 (beta)

  • The class classification.SVM adapt the scikit-learn implementation of Support Vector Machines to the analysis of hyperspectral cubes.
  • An example of using the SVC class with the Pine Creek cube.

version 0.12.2 (beta)

This is a maintenance version: code review, bugs fix, classes refactoring and many small improvements that have not a direct impact on the functionality.

New

  • Full Python 3.3 compatibility. However, there is a drawback, as the SPy software ENVI file reader is not ported to Python 3.3, instead, pysptools use a JSON version for the USGS spectral library. See the spectro module documentation.
  • The scikit-learn KMeans class is wrapped to make it user-friendly when applied to a HS cube. This new class live in the classification module.
  • A new Pine Creek example that illustrate the use of KMeans with unmixing.

Reorganisation

The module classifiers is renamed classification (Yes, again! Sorry)

version 0.12.1 (beta)

New

  • This version adds compatibility to the IPython Notebook. A display method is introduced for many classes. This one use matplotlib and have the same role than the plot method. Calling display show figures embedded in the Notebook.
  • The source is ported to Pyhton 3.3. This porting is not integral. Because the SPy library run on Python <= 2.7, the spectro module is not part of the porting.

New examples

This version adds compatibility to the IPython Notebook. Five examples present this feature.

  • Methanol gas
  • Hematite drill core
  • Convex hull
  • Pine Creek 1
  • Pine Creek 2

Fix

  • Function distance.SID: the division at the line “p = (s1 / np.sum(s1)) and (s2 / np.sum(s2))” is now always real with the inclusion of a “__future__ division” declaration. That was not the case before, but this is a minor problem as cubes are always made of real.
  • Function denoise.whiten: the line “S_1_2 = S**-1/2” is changed to “S_1_2 = S**(-0.5)”. This is a bug and have a direct impact on the classes Whiten and MNF.

version 0.12.0 (beta)

New

  • A first version of the HySime hyperspectral signal subspace estimation.
  • SAM, SID and NormXCorr classifiers classes can now classify one spectrum at a time. Previously, they asked for two or more spectra to be classified.
  • A parameter noise_whitening was added to the HfcVd class. When set to True a whiten is applied before calling HfcVd and this way implement the NWHFC function.

Fix

  • The variance calculation for HfcVd was wrong - it was integer based but it is assume to be float based - fixed.
  • The call to normalize() inside the count() method that belong to the HfcVd class was removed. Now, data conditioning needs to be done before calling HfcVd. It is simpler this way.

Reorganisation

  • test_HfcVd.py is renamed test_vd.py

Examples

  • The examples Hematite drill core and Smokestack was updated.
  • A new one: Convex hull.

version 0.11.0 (beta)

This version support a noise reduction module called ‘noise’. The first version of this module is composed by:

  • Savitzky-Golay filter, a convolution based low-pass denoising algorithm, this algorithm can be applied on bands or on spectra;
  • withening;
  • MNF (maximum noise fraction), this is a first version.

To be compatible with the noise reduction module, ATGP and NFINDR needed some adjustments and they was updated accordingly:

  • the internal call to HfcVd is removed, the number of endmembers to extract is now mandatory;
  • a ‘transform’ parameter is added to NFINDR; it accept a transformed HSI cube; when a transformed cube is submited to NFINDR, the built-in PCA call is bypassed.

version 0.10.1 (beta)

This is a patch level version for the 0.10 version. The function FCLS is fixed. To run this new FCLS you need to install CVXOPT. Note that you need CVXOPT only if your intent is to run FCLS. See the Christoph Gohlke web site, it support CVXOPT for Windows.

version 0.10 (beta)

  • For all the classifiers classes: a new constrained parameter and a new get_single_map() method.
  • For the unmixing classes NFINDR and ATGP: a new mask parameter
  • A comprehensive exception mechanism
  • Unit tests
  • Two examples, one with a hematite drill core and the other with a smokestack

Reorganization

  • The unmix module is renamed eea (for endmembers extraction algorithms)
  • The unmix module method ‘unmix’ is renamed ‘extract’

fixes

  • classifiers.SAM, SID and NormXCorr: the current plot_single_map method accept 0 for the lib_idx parameter, the expected value is 1. – fixed
  • classifiers.NormXCorr: the colorbar is inverted and the threshold value is inverted from the expected behavior. – fixed

version 0.09 (alpha)

  • The main improvement for this version is a threshold parameter added to the classifiers classes. The threshold can be single value or multiple values. In the later case, there is one individual threshold for each signal to match. Another sweet improvement to the classifiers classes is the capacity to plot one map by each signal to match with the threshold of your choice.
  • A new module receives all the classes and functions that work around the USGS library and the convex hull removal. This new module name is ‘spectro’. This move leaves only one function in the sigproc module. A new class add get and search functionality to the USGS library.
  • Module abundance_map is renamed abundance_maps.
  • Module classification is renamed classifiers.
  • A new example that use the Telops Hyper-Cam instrument.

version 0.08 (alpha)

The two majors improvements are a substantial speed up for NFINR and an interface to the convex_hull_removal function.

In details:

NFINDR exist in two versions now. The pure python version have a speed gain of 4x against the version 0.07. The code is the same, the only difference is an almost direct call to lapack. Something that we can do with numpy and scipy. The second version is a cython rewrite without a call to lapack. It give a 8x speedup compared to the 0.07 version. This result seems abnormal, beating MKL blas/lapack is difficult but not impossible. It depends on the context, check the unmixing module documentation.

Speedup:
GLRT: 2x against the 0.07 version
New:
class interface to the convex_hull_removal function (see the sigproc module) continuum based features extractor with a tetracorder style (see the sigproc module) Distutils setup
Renamed:
files tests/ex_*.py renamed to test_*.py
Updated:
test_hull.py

version 0.07 (alpha)

The more important improvements are a documentation, a NormXCorr classifier and a fix to NFINDR.

New:
documentation OSP target detection NormXCorr distance measure chebyshev distance measure NormXCorr classifier corr stat function cov stat function
Fixed:
NFINDR

NFINDR have a new feature. You can use ATGP to initialize the first endmembers set.

convert2d_signal_last is renamed to convert2D the signal module is renamed to sigproc

version 0.06 (alpha)

Many improvements to UCLS, NNLS, FCLS, ACE, MatchedFilter, HfcVd, NFINDR (all now in C-order). In overall this version do a better use of numpy.

Speedup improvements to ACE, SAM and SID.

New functions: hyperCem -> detextion.CEM hyperGLRT -> detection.GLRT eia.FIPPI -> unmixing.FIPPI

Added a get_idx method to the classes PPI, NFINDR, ATGP, FIPPI. This method return a index list corresponding to the induced endmembers.

Added a legend to the plot function for the classes SAM and SID.

Unlike the previous version, Cython was not use for this one. Exploring the power of numpy gave very good speedup to SID and SAM.

Fix: NFINDR: np.zeros((p), dtype=np.int16) become IDX = np.zeros((p), dtype=np.long)

In conclusion, almost all the code was revisited and improved.

version 0.05 (alpha)

The library was reorganized in many modules: abundance_map, classification, detection, formatting, material_count, signal, tests, unmixing and util. The functions as been sorted into them and in some case a class interface to the function was added. See the tests examples, they are the only documentation for now.

Cython has been introduced. For this version, only the SAM classifier is compiled with cython.

Three distributions come with this version: - the source - one with SAM classifier compiled for Windows 7 32 bits and Python 2.7 and - one with SAM classifier compiled for Windows 7 64 bits and Python 2.7

You can use the source package, it fall back on the python version of the SAM classifier. To use, unzip the package and place the checkout directory on your PYTHONPATH.

ex_clsf.py is renamed to ex_clf.py

For the class interfaces, the format of the HSI data cube is (m x n x p), m and n are the spatial indices and p is the pixel. And for a library of signals the format is (N x p), N is the indice and p the pixel.

version 0.04 (alpha)

Note : The SPy library use the C-order for the numpy representation of the data. The signal is in the innermost loop. That’s the best representation for almost all algorithms that we can apply to this kind of data. In a first attemp I tried to simulate the Matlab way of doing for hyperspectral processing (usin a F-order shape). But for this version I am revising my position and started of doing things in a more pythonic approach. For the listed functions the signal is always last. This give a gain of around 20% in speed improvement :

SAM, SID, SAM_classifier, SID_classifier

For the next versions, more functions will be migrated to the C-order, signal last.

Added from the Matlab Hyperspectral Toolbox: hyperMatchedFilter -> MatchedFilter (with a good optimisation from numpy) hyperAce -> ACE

A test program for the detections algorithms -> ex_detect.py

New functions, not from the current sources, but from articles : SID and SAM and helper functions SID_classifier and SAM_classifier

A test program for the classification algorithms -> ex_clsf.py

version 0.03 (alpha)

Added from the Matlab Hyperspectral Toolbox: hyperNnls -> NNLS

From the Endmember Induction Algorithms toolbox (EIA): EIA_ATGP -> ATGP

Patch to convert3d(): it can now accept a 1D vector as input

version 0.02 (alpha)

Added from the Matlab Hyperspectral Toolbox: hyperConvexHullRemoval -> convex_hull_removal hyperPpi -> PPI

From the Endmember Induction Algorithms toolbox (EIA): EIA_NFINDR -> NFINDR

Added two new examples, one for the convex_hull_removal, ex_hull.py and one for NFIND, ex_eia.py

A ligthweight data for the tests.

Many improvements to the tests programs.

version 0.01 (alpha)

First version!

From the Matlab Hyperspectral Toolbox:

hyperConvert2D -> convert2d_signal_first, convert2d_signal_last

hyperConvert3D -> convert3d

hyperNormalize -> normalize

hyperAtgp -> ATGP

hyperHfcVd -> HfcVd

hyperPct -> PCT, but use scikit.learn PCA

hyperUcls -> UCLS

hyperFcls -> FCLS

hyperCov -> use numpy.cov

hyperCorr -> use numpy.corrcoef

From the The piecewise constant toolbox: pwc_bilateral -> bilateral

Bug fixes (old)

  • The N-FINDR algorithm have a problem that make it unusable. This is true for version 0.06 and previous. The pixel that give an expansion of the simplex was not saved and NFINDR never converge to a solution (2013-11-09).
  • mht.ATPG signal a singular matrix and exit when the end members asked is more than, say, 30. It’s a random bug. Fix: this function is deprecated, it was replaced with the eia ATGP (2013-08-05)