FrontEnd is a wrapper class for the chain of front end processors. It provides methods for manipulating and
navigating the processors.
The front end is modeled as a series of data processors, each of which performs a specific signal processing
function. For example, a processor performs Fast-Fourier Transform (FFT) on input data, another processor performs
high-pass filtering. Figure 1 below describes how the front end looks like:
Figure 1: The Sphinx4 front end.
Each such data processor implements the
edu.cmu.sphinx.frontend.DataProcessor interface. Objects that
implements the
edu.cmu.sphinx.frontend.Data interface enters and exits the front end, and go between the
processors in the front end. The input data to the front end is typically audio data, but this front end allows any
input type. Similarly, the output data is typically features, but this front end allows any output type. You can
configure the front end to accept any input type and return any output type. We will describe the configuration of
the front end in more detail below.
The Pull Model of the Front End
The front end uses a pull model. To obtain output from the front end, one would call the method:
FrontEnd frontend = ... // see how to obtain the front end below
Data output = frontend.getData();
Calling
#getData() on the front end would in turn call the getData() method on the last
DataProcessor, which in turn calls the getData() method on the second last DataProcessor, and so on, until the
getData() method on the first DataProcessor is called, which reads Data objects from the input. The input to the
front end is actually another DataProcessor, and is usually (though not necessarily) part of the front end and is not
shown in the figure above. If you want to maintain some control of the input DataProcessor, you can create it
separately, and use the
#setDataSource(edu.cmu.sphinx.frontend.DataProcessor) method to set it
as the input DataProcessor. In that case, the input DataProcessor will be prepended to the existing chain of
DataProcessors. One common input DataProcessor is the
edu.cmu.sphinx.frontend.util.Microphone, which
implements the DataProcessor interface.
DataProcessor microphone = new Microphone();
microphone.initialize(...);
frontend.setDataSource(microphone);
Another common input DataProcessor is the
edu.cmu.sphinx.frontend.util.StreamDataSource. It turns a Java
java.io.InputStream into Data objects. It is usually used in batch mode decoding.
Configuring the front end
The front end must be configured through the Sphinx properties file. For details about configuring the front end,
refer to the document Configuring the Front End.
Current state-of-the-art front ends generate features that contain Mel-frequency cepstral coefficients (MFCC). To
specify such a front end (called a 'pipeline') in Sphinx-4, insert the following lines in the Sphinx-4 configuration
file:
<component name="mfcFrontEnd" type="edu.cmu.sphinx.frontend.FrontEnd">
<propertylist name="pipeline">
<item>preemphasizer</item>
<item>windower</item>
<item>dft</item>
<item>melFilterBank</item>
<item>dct</item>
<item>batchCMN</item>
<item>featureExtractor</item>
</propertylist>
</component>
<component name="preemphasizer" type="
edu.cmu.sphinx.frontend.filter.Preemphasizer"/>
<component name="windower" type="
edu.cmu.sphinx.frontend.window.RaisedCosineWindower"/>
<component name="dft" type="
edu.cmu.sphinx.frontend.transform.DiscreteFourierTransform"/>
<component name="melFilterBank" type="
edu.cmu.sphinx.frontend.frequencywarp.MelFrequencyFilterBank2"/>
<component name="dct" type="
edu.cmu.sphinx.frontend.transform.DiscreteCosineTransform"/>
<component name="batchCMN" type="
edu.cmu.sphinx.frontend.feature.BatchCMN"/>
<component name="featureExtractor" type="
edu.cmu.sphinx.frontend.feature.DeltasFeatureExtractor"/>
Note: In this example, 'mfcFrontEnd' becomes the name of the front end.
Sphinx-4 also allows you to:
- specify multiple front end pipelines
- specify multiple instance of the
same DataProcessor in the same pipeline
For details on how to do this, refer to the document Configuring the
Front End.
Obtaining a Front End
In order to obtain a front end, it must be specified in the configuration file. The Sphinx-4 front end is connected
to the rest of the system via the scorer. We will continue with the above example to show how the scorer will obtain
the front end. In the configuration file, the scorer should be specified as follows:
<component name="scorer" type="edu.cmu.sphinx.decoder.scorer.SimpleAcousticScorer">
<property name="frontend" value="mfcFrontEnd"/>
</component>
In the SimpleAcousticScorer, the front end is obtained in the
edu.cmu.sphinx.util.props.Configurable#newProperties method as follows:
public void newProperties(PropertySheet ps) throws PropertyException {
FrontEnd frontend = (FrontEnd) ps.getComponent("frontend", FrontEnd.class);
}