Tabnine Logo
JapaneseTokenizer.setReader
Code IndexAdd Tabnine to your IDE (free)

How to use
setReader
method
in
org.apache.lucene.analysis.ja.JapaneseTokenizer

Best Java code snippets using org.apache.lucene.analysis.ja.JapaneseTokenizer.setReader (Showing top 3 results out of 315)

origin: org.codelibs/elasticsearch-analysis-ja

tokenizer.setReader(inputPending);
origin: omegat-org/omegat

@SuppressWarnings("resource")
@Override
protected TokenStream getTokenStream(String strOrig, boolean stemsAllowed, boolean stopWordsAllowed)
    throws IOException {
  if (stemsAllowed) {
    // Blank out tags when stemming only
    strOrig = blankOutTags(strOrig);
    CharArraySet stopWords = stopWordsAllowed ? JapaneseAnalyzer.getDefaultStopSet() : CharArraySet.EMPTY_SET;
    Set<String> stopTags = stopWordsAllowed ? JapaneseAnalyzer.getDefaultStopTags() : Collections.emptySet();
    return new JapaneseAnalyzer(null, Mode.SEARCH, stopWords, stopTags).tokenStream("",
        new StringReader(strOrig));
  } else {
    JapaneseTokenizer tokenizer = new JapaneseTokenizer(null, false, Mode.NORMAL);
    tokenizer.setReader(new StringReader(strOrig));
    return new TagJoiningFilter(tokenizer);
  }
}
origin: org.apache.lucene/lucene-analyzers-kuromoji

setReader(new StringReader(inText));
reset();
try {
org.apache.lucene.analysis.jaJapaneseTokenizersetReader

Popular methods of JapaneseTokenizer

  • <init>
    Create a new JapaneseTokenizer.
  • calcNBestCost
  • clearAttributes
  • close
  • correctOffset
  • end
  • incrementToken
  • reset
  • setNBestCost
  • add
  • backtrace
  • backtraceNBest
  • backtrace,
  • backtraceNBest,
  • computePenalty,
  • computeSecondBestThreshold,
  • equals,
  • fixupPendingList,
  • getDict,
  • hashCode,
  • isPunctuation

Popular in Java

  • Reading from database using SQL prepared statement
  • setRequestProperty (URLConnection)
  • getApplicationContext (Context)
  • startActivity (Activity)
  • HttpServer (com.sun.net.httpserver)
    This class implements a simple HTTP server. A HttpServer is bound to an IP address and port number a
  • Component (java.awt)
    A component is an object having a graphical representation that can be displayed on the screen and t
  • SocketTimeoutException (java.net)
    This exception is thrown when a timeout expired on a socket read or accept operation.
  • Selector (java.nio.channels)
    A controller for the selection of SelectableChannel objects. Selectable channels can be registered w
  • Enumeration (java.util)
    A legacy iteration interface.New code should use Iterator instead. Iterator replaces the enumeration
  • TimeUnit (java.util.concurrent)
    A TimeUnit represents time durations at a given unit of granularity and provides utility methods to
  • Top 12 Jupyter Notebook extensions
Tabnine Logo
  • Products

    Search for Java codeSearch for JavaScript code
  • IDE Plugins

    IntelliJ IDEAWebStormVisual StudioAndroid StudioEclipseVisual Studio CodePyCharmSublime TextPhpStormVimGoLandRubyMineEmacsJupyter NotebookJupyter LabRiderDataGripAppCode
  • Company

    About UsContact UsCareers
  • Resources

    FAQBlogTabnine AcademyTerms of usePrivacy policyJava Code IndexJavascript Code Index
Get Tabnine for your IDE now