Semantic

Posted on November 20th, 2018

Tabnine was created as a language-agnostic autocompleter. The core idea behind Tabnine is that by indexing your code and detecting statistical patterns within it, Tabnine can make better suggestions than a completer which understands your programming language. This brings additional gains in responsiveness, reliability, and ease of configuration because Tabnine doesn’t need to compile your code.

Nonetheless, there are some situations where you would rather use a semantic completer than Tabnine. For example, if you don’t know what methods are contained in the class Math, you could type Math. and see what suggestions are provided by the semantic completer. Also, semantic completers can provide documentation and type signatures together with their suggestions.

That’s why Tabnine now supports semantic completion. Tabnine continues to be language-agnostic: semantic completion is provided by external software which Tabnine communicates with using the Language Server Protocol. Tabnine comes with default install scripts for several common language servers, and this is fully configurable, so you can use a different language server or add semantic completion for a new language if you wish. You can read more about how to configure it here.

Here’s an example:

Here, Tabnine uses the language server to provide type signatures and documentation for the suggestions. Also note that Tabnine is using its index to sort the language server’s suggestions in order of frequency, placing new and with_capacity at the top.

The skeptical reader asks: why use Tabnine together with a language server when I could simply use the language server instead? Here are three ways in which Tabnine improves the user experience over the language server alone:

1. Many language servers are slow. If the language server is slow, Tabnine will provide its own results while querying the language server in the background. Tabnine typically returns its results in 20 milliseconds or so, while Jedi, a completion engine for Python, usually takes around 500 milliseconds. Rather than waiting for Jedi, Tabnine will provide its results immediately and cache Jedi’s suggestions to be returned instantly once you type more characters.
2. Language servers don’t always have the right completion. When you use an autocompleter, you give it half-written code that may contain syntax errors, type errors, or references to functions that you haven’t written yet. These cases are challenging for language servers because they need to make long chains of inferences to determine what completions to suggest, so breaking any link in this chain can disrupt the entire process. Tabnine is robust to half-written code because it does not parse the code, so there are no long-distance effects.
3. Multi-token completion. Tabnine can improve the language server’s suggestions by suggesting multiple tokens at once. For example:
   (image)
   In this example, Tabnine uses Jedi to infer that z is a string and has an encode method. Then Tabnine uses its own multi-token completion engine to suggest encode(‘utf8’) rather than simply encode.

Semantic completion is part of Tabnine 1.0.7. You can read the full changelog here.