Learning About the Language Server Protocol

As part of my programming language project, I’ve been writing a language server. I’m familiar with the very basics of the LSP from integrating Neovim with projects like rust-analyzer, but this is the first time I’ve dug into its internals.

The protocol consists of JSON messages exchanged^[1] between a server and a client. The server understands the programming language and does all the processing. The client^[2] interacts with the server in a high-level way, abstracting the difference between programming languages. Most exchanges involve a request from the client and a response from the server;

some go the other direction (the server requests and the client responds).

Still others are one-way messages with no response; these are called “notifications” in the LSP spec.

Different clients and servers can have different “capabilities”, which they exchange during initial connection.^[3] For example, there’s a capability definitionProvider which the server may or may not have. If a server doesn’t advertise this capability, the client’s interface shouldn’t show “go to definition” controls.

At various points in the document lifecycle, the client notifies the server of changes. This is why rust-analyzer can typecheck your files even before you save; your editor is sending your changes to the server as notifications. Depending on what capabilities the server advertises, it can receive events like “documented opened”, “document changed”, “document closed”, “document renamed”, etc. It’s expected to track changes internally, as future commands will reference these unsaved versions of the documents.

There are all sorts of features that servers can support. Some are user-initiated actions, like:

• Goto definition
• Find references
• Highlight instances of a symbol
• Retrieve hover information
• Run “code actions” (like importing unknown names or refactoring code blocks)
• Rename a given symbol

There are other requests that the client makes without user input, such as:

• Inlay hints (used for type annotations in rust-analyzer, I don’t think they’re in Neovim)
• Inline values (not sure what these are used for, they’re not in Neovim I think)
• Completion options (this is technically kicked off by user input but rarely a specific command in the interface)
• Semantic syntax highlighting

You’ll notice that the line between “user actions” and “editor actions” is blurry. The LSP spec itself makes no distinction between user-initiated and automatic requests; they all come from the client and the server handles them the same way.

Inline errors (“diagnostics” in the spec) are usually pushed from the server to the client as a notification. The spec makes no requirement of when this notification can be made, which means that compilation and type-checking can happen asynchronously. Servers are responsible for clearing out diagnostics if a compilation succeeds, by sending a “publish diagnostics” notification with an empty body.

I still have more to learn about the spec! While browsing I’ve seen references to CodeLens and Workspaces. I’m not sure these features exist in my editor? I’ve never encountered them at least. If I want to provide excellent VSCode support for my language, I’ll revisit them and try to learn what they are.

In a future post, I plan to talk through the language server I’ve created and the design decisions I’m making as I work through it.

Thanks to Tim Likarish Ellis for feedback on a draft of this post.