marytts.signalproc.analysis

Class EnergyAnalyser

java.lang.Object

marytts.signalproc.process.FrameProvider

marytts.signalproc.analysis.FrameBasedAnalyser<Double>

marytts.signalproc.analysis.EnergyAnalyser

Direct Known Subclasses:

EnergyAnalyser_dB

public class EnergyAnalyser
extends FrameBasedAnalyser<Double>

Author:

Marc Schröder A class that analyses the energy distribution, and computes a silence cutoff threshold, in the linear energy domain.

Nested Class Summary

Nested classes/interfaces inherited from class marytts.signalproc.analysis.FrameBasedAnalyser

FrameBasedAnalyser.FrameAnalysisResult<T>

Field Summary

Fields

Modifier and Type	Field and Description
protected int	DEFAULT_MAXSIZE
protected double[]	frameEnergies array of frame energies, for further analysis
protected int	len Length of valid data, counting from offset.
protected int	maxSize maximum size of the double[] storing the frame energies
protected int	offset Beginning of valid data in frameEnergies; will be >0 only after more than maxSize frames have been read.

Fields inherited from class marytts.signalproc.analysis.FrameBasedAnalyser

analysisResults

Fields inherited from class marytts.signalproc.process.FrameProvider

frame, frameLength, frameShift, frameStart, nextFrameStart, processor, samplingRate, signal, totalRead, validSamplesInFrame

Constructor Summary

Constructors

Constructor and Description
EnergyAnalyser(DoubleDataSource signal, int framelength, int samplingRate)
EnergyAnalyser(DoubleDataSource signal, int framelength, int frameShift, int samplingRate)
EnergyAnalyser(DoubleDataSource signal, int framelength, int frameShift, int samplingRate, int maxSize)

Method Summary

Methods

Modifier and Type	Method and Description
Double	analyse(double[] frame) Apply this FrameBasedAnalyser to the given data.
static void	energySegmentation(String[] args) Segment a WAVE file by energy, ideally one word per segment (the result might contain more); the result is saved in a file in transcriber format so the segmentation can be easily inspected and corrected.
double[]	getEnergyHistogram() Compute a histogram of energies found in the data.
double[]	getEnergyHistogram(int nbins) Compute a histogram of energies found in the data.
double	getMaxFrameEnergy() Compute the overall maximum energy in all frames.
double	getMeanFrameEnergy() Compute the overall mean energy in all frames.
double	getMinFrameEnergy() Compute the overall minimum energy in all frames.
double	getSilenceCutoff() Determine the energy level below which to find silence.
double	getSilenceCutoffFromKMeansClustering(double shiftFromMinimumEnergyCenter, int numClusters)
double	getSilenceCutoffFromSortedEnergies(FrameBasedAnalyser.FrameAnalysisResult[] far, double silenceThreshold)
double[][]	getSpeechStretches() For the current audio data and the automatically calculated silence cutoff, compute a list of start and end times representing speech stretches within the file.
double[][]	getSpeechStretchesUsingEnergyHistory(int energyBufferLength, double speechStartLikelihood, double speechEndLikelihood, double shiftFromMinimumEnergyCenter, int numClusters) The latest version uses K-Means clustering to cluster energy values into 3 separate clusters.
static void	main(String[] args)
protected void	rememberFrameEnergy(double energy)

Methods inherited from class marytts.signalproc.analysis.FrameBasedAnalyser

analyseAllFrames, analyseAvailableFrames, analyseNextFrame, constructAnalysisResult

Methods inherited from class marytts.signalproc.process.FrameProvider

getCurrentFrame, getData, getFrameLengthSamples, getFrameLengthTime, getFrameShiftSamples, getFrameShiftTime, getFrameStartSamples, getFrameStartTime, getNextFrame, getSamplingRate, hasMoreData, resetInternalTimer, stopWhenTouchingEnd, validSamplesInFrame

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

DEFAULT_MAXSIZE

protected final int DEFAULT_MAXSIZE

See Also:

Constant Field Values

frameEnergies

protected double[] frameEnergies

array of frame energies, for further analysis

offset

protected int offset

Beginning of valid data in frameEnergies; will be >0 only after more than maxSize frames have been read.

len

protected int len

Length of valid data, counting from offset. This will count up to maxSize and then stay equal to maxSize.

maxSize

protected int maxSize

maximum size of the double[] storing the frame energies

Constructor Detail

EnergyAnalyser

public EnergyAnalyser(DoubleDataSource signal,
              int framelength,
              int samplingRate)

EnergyAnalyser

public EnergyAnalyser(DoubleDataSource signal,
              int framelength,
              int frameShift,
              int samplingRate)

EnergyAnalyser

public EnergyAnalyser(DoubleDataSource signal,
              int framelength,
              int frameShift,
              int samplingRate,
              int maxSize)

Method Detail

analyse

public Double analyse(double[] frame)

Apply this FrameBasedAnalyser to the given data.

Specified by:

analyse in class FrameBasedAnalyser<Double>

Parameters:

frame - the data to analyse, which must be of the length prescribed by this FrameBasedAnalyser, i.e. by works like FrameProvider.getFrameLengthSamples() .

Returns:

a Double representing the total energy in the frame.

Throws:

IllegalArgumentException - if frame does not have the prescribed length

rememberFrameEnergy

protected void rememberFrameEnergy(double energy)

getMeanFrameEnergy

public double getMeanFrameEnergy()

Compute the overall mean energy in all frames.

Returns:

a double representing the mean energy (non-normalised, i.e. in units of square sample amplitudes).

getMaxFrameEnergy

public double getMaxFrameEnergy()

Compute the overall maximum energy in all frames.

Returns:

a double representing the maximum energy (non-normalised, i.e. in units of square sample amplitudes).

getMinFrameEnergy

public double getMinFrameEnergy()

Compute the overall minimum energy in all frames.

Returns:

a double representing the minimum energy (non-normalised, i.e. in units of square sample amplitudes).

getEnergyHistogram

public double[] getEnergyHistogram()

Compute a histogram of energies found in the data. Bin sizes are automatically determined based on the min and max frame energies, such that the interval between min and max energy is split into 100 bins.

Returns:

an array of doubles of length nbins, representing percentage distribution across bins.

getEnergyHistogram

public double[] getEnergyHistogram(int nbins)

Compute a histogram of energies found in the data. Bin sizes are automatically determined based on the min and max frame energies, such that the interval between min and max energy is split into nbins bins.

Parameters:

nbins - the number of bins to compute, e.g. 100

Returns:

an array of doubles of length nbins, representing percentage distribution across bins.

getSilenceCutoff

public double getSilenceCutoff()

Determine the energy level below which to find silence. This is based on the energy histogram.

Returns:

the energy below which is silence.

getSilenceCutoffFromSortedEnergies

public double getSilenceCutoffFromSortedEnergies(FrameBasedAnalyser.FrameAnalysisResult[] far,
                                        double silenceThreshold)

getSpeechStretches

public double[][] getSpeechStretches()

For the current audio data and the automatically calculated silence cutoff, compute a list of start and end times representing speech stretches within the file. This method will take the following System properties into account:

• signalproc.minsilenceduration (default: 0.1 (seconds))
• signalproc.minspeechduration (default: 0.1 (seconds))

Silence or speech stretches shorter than these values will be ignored.

Returns:

an array of double pairs, representing start and end times (in seconds) for each speech stretch.

getSilenceCutoffFromKMeansClustering

public double getSilenceCutoffFromKMeansClustering(double shiftFromMinimumEnergyCenter,
                                          int numClusters)

getSpeechStretchesUsingEnergyHistory

public double[][] getSpeechStretchesUsingEnergyHistory(int energyBufferLength,
                                              double speechStartLikelihood,
                                              double speechEndLikelihood,
                                              double shiftFromMinimumEnergyCenter,
                                              int numClusters)

The latest version uses K-Means clustering to cluster energy values into 3 separate clusters. Then, the energy threshold is selected using the lowest and highest energy cluster centers

Parameters:

energyBufferLength - energyBufferLength

speechStartLikelihood - speechStartLikelihood

speechEndLikelihood - speechEndLikelihood

shiftFromMinimumEnergyCenter - shiftFromMinimumEnergyCenter

numClusters - numClusters

Returns:

stretches.toArray(new double[0][0])

energySegmentation

public static void energySegmentation(String[] args)
                               throws Exception

Segment a WAVE file by energy, ideally one word per segment (the result might contain more); the result is saved in a file in transcriber format so the segmentation can be easily inspected and corrected. The parameters in: EnergyAnalyser.getSpeechStretchesUsingEnergyHistory(): signalproc.minsilenceduration signalproc.minspeechduration can be tuned to get better segmentation.

Parameters:

args - : first argument is the directory where the wav files are, next arguments in the list are the files for segmenting.

Throws:

Exception - : IOException, UnsupportedAudioFile exception and IllegalArgumentException when the file is not mono, it just handles mono audio signals.

main

public static void main(String[] args)
                 throws Exception

Throws:

Exception