How to use
TokenSequenceMatchDataAndTarget
in
cc.mallet.pipe

Best Java code snippets using cc.mallet.pipe.TokenSequenceMatchDataAndTarget (Showing top 2 results out of 315)

Add the Codota plugin to your IDE and get smart completions

private void myMethod () {
 Gson g =new Gson()
GsonBuilder gsonBuilder;gsonBuilder.create()
new GsonBuilder().create()
Smart code suggestions by Tabnine
}

public void testStartState() {
  Pipe p = new SerialPipes(new Pipe[] {
      new LineGroupString2TokenSequence(),
      new TokenSequenceMatchDataAndTarget(Pattern
          .compile("^(\\S+) (.*)"), 2, 1),
      new TokenSequenceParseFeatureString(false), new TokenText(),
      new TokenSequence2FeatureVectorSequence(true, false),
      new Target2LabelSequence(), new PrintInputAndTarget(), });
  InstanceList data = new InstanceList(p);
  data.addThruPipe(new LineGroupIterator(new StringReader(toy), Pattern
      .compile("\n"), true));
  CRF crf = new CRF(p, null);
  crf.print();
  crf.addStatesForLabelsConnectedAsIn(data);
  crf.addStartState();
  CRFTrainerByLabelLikelihood crft = new CRFTrainerByLabelLikelihood(crf);
  Optimizable.ByGradientValue maxable = crft.getOptimizableCRF(data);
  assertEquals(-1.3862, maxable.getValue(), 1e-4);
  crf = new CRF(p, null);
  crf
      .addOrderNStates(data, new int[] { 1 }, null, "A", null, null,
          false);
  crf.print();
  crft = new CRFTrainerByLabelLikelihood(crf);
  maxable = crft.getOptimizableCRF(data);
  assertEquals(-3.09104245335831, maxable.getValue(), 1e-4);
}

public void testStartState() {
  Pipe p = new SerialPipes(new Pipe[] {
      new LineGroupString2TokenSequence(),
      new TokenSequenceMatchDataAndTarget(Pattern
          .compile("^(\\S+) (.*)"), 2, 1),
      new TokenSequenceParseFeatureString(false), new TokenText(),
      new TokenSequence2FeatureVectorSequence(true, false),
      new Target2LabelSequence(), new PrintInputAndTarget(), });
  InstanceList data = new InstanceList(p);
  data.addThruPipe(new LineGroupIterator(new StringReader(toy), Pattern
      .compile("\n"), true));
  CRF crf = new CRF(p, null);
  crf.print();
  crf.addStatesForLabelsConnectedAsIn(data);
  crf.addStartState();
  CRFTrainerByLabelLikelihood crft = new CRFTrainerByLabelLikelihood(crf);
  Optimizable.ByGradientValue maxable = crft.getOptimizableCRF(data);
  assertEquals(-1.3862, maxable.getValue(), 1e-4);
  crf = new CRF(p, null);
  crf
      .addOrderNStates(data, new int[] { 1 }, null, "A", null, null,
          false);
  crf.print();
  crft = new CRFTrainerByLabelLikelihood(crf);
  maxable = crft.getOptimizableCRF(data);
  assertEquals(-3.09104245335831, maxable.getValue(), 1e-4);
}

Javadoc

Run a regular expression over the text of each token; replace the text with the substring matching one regex group; create a target TokenSequence from the text matching another regex group.

For example, if you have a data file containing one line per token, and the label also appears on that line, you can first get a TokenSequence in which the text of each line is the Token.getText() of each token, then run this pipe, and separate the target information from the data information. For example to process the following,

 
BACKGROUND Then 
PERSON Mr. 
PERSON Smith 
BACKGROUND said 
...

use new TokenSequenceMatchDataAndTarget (Pattern.compile ("([A-Z]+) (.*)"), 2, 1).

Most used methods

<init>

Popular in Java

Making http post requests using okhttp
setContentView (Activity)
setRequestProperty (URLConnection)
getOriginalFilename (MultipartFile)
Return the original filename in the client's filesystem.This may contain path information depending
Semaphore (java.util.concurrent)
A counting semaphore. Conceptually, a semaphore maintains a set of permits. Each #acquire blocks if
Stream (java.util.stream)
A sequence of elements supporting sequential and parallel aggregate operations. The following exampl
ServletException (javax.servlet)
Defines a general exception a servlet can throw when it encounters difficulty.
JFileChooser (javax.swing)
Options (org.apache.commons.cli)
Main entry-point into the library. Options represents a collection of Option objects, which describ
Base64 (org.apache.commons.codec.binary)
Provides Base64 encoding and decoding as defined by RFC 2045.This class implements section 6.8. Base
Top 12 Jupyter Notebook Extensions

How to useTokenSequenceMatchDataAndTarget in cc.mallet.pipe

Best Java code snippets using cc.mallet.pipe.TokenSequenceMatchDataAndTarget (Showing top 2 results out of 315)

How to use
TokenSequenceMatchDataAndTarget
in
cc.mallet.pipe