{
"$type": "site.standard.document",
"content": {
"$type": "site.standard.document#content",
"markdown": "\n### What is a Language Server?\n\nIf you're unfamiliar, the Language Server Protocol is a protocol by which a client (usually a code editor) can talk to a language server and get information about an open file and/or workspace. This is made use of in editors like VS Code and Neovim (my editor of choice, by the way). The protocol passes data around in JSON which essentially allows for an RPC where the client tells the server to do something or vice versa. These can be built in any language, as long as said language supports whatever transport method you choose for the JSON RPC (usually stdout and stdin, but TCP is another example of an option). For this language server I built it in Go using stdout and stdin.\n\n### What is a Chess PGN?\n\nThis may be the one fewer reading this have heard of, so what is a PGN? PGN (Portable Game Notation) is the format in which most chess games are stored digitally. A PGN file can hold a single game or an entire database of games, but for this basic LSP implementation I have assumed that the user is editing single game files, which is a big limiting factor, so be warned if you decide to use this. The PGN standard consists of two main parts: tag pairs and moves. The tag pairs store metadata about the game, while the moves are understandably the moves of the game. There are 7 required tags in the standard to define a game, and the moves are recorded in Standard Algebraic Notation (i.e. Ke2). I won't go too heavily into the specification of the standard here because I decided against writing my own parser (although not before creating a lexer when my initial playing around with existing parsers frustrated me), but if you want to read up on it, it is an interesting spec in and of itself.\n\n### What can this language server do?\n\nIn its current basic form, it can parse a PGN of a single chess game, report errors with the parsing, and suggest legal moves as completions.\n\n## Making the Server\n\n### Notes and Issues\n\nSince this was my first time ever creating a language server, I no doubt made some mistakes, and I'm also fairly new to Go, so I have no doubt that will have contributed to any issues as well, but all that being said, this went over pretty painlessly. I had some hiccups with the parser itself, but as far as implementing the protocol it went off without much of a hitch. The only real warning I'd give to people following suit on this is that you should probably design your analysis tool before doing much else because the way I did it, I felt a lot like I was jumping around my codebase continuously adding and removing from the LSP to fit the demands of the analysis tool. That might seem self-evident, especially if you're writing a language server for a programming language, but I figured I should mention it.\n\n### The Protocol\n\nI started with the help of TJ Devries' [educationalsp repo](https://github.com/tjdevries/educationalsp) creating some helper functions for the RPC:\n\n```go\n// rpc/rpc.go\npackage rpc\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"strconv\"\n)\n\nfunc EncodeMessage(msg any) string {\n\tcontent, err := json.Marshal(msg)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\treturn fmt.Sprintf(\"Content-Length: %d\\r\\n\\r\\n%s\", len(content), content)\n}\n\ntype BaseMessage struct {\n\tMethod string `json:\"method\"`\n}\n\nfunc DecodeMessage(msg []byte) (string, []byte, error) {\n\theader, content, found := bytes.Cut(msg, []byte{'\\r', '\\n', '\\r', '\\n'})\n\tif !found {\n\t\treturn \"\", nil, errors.New(\"Did not find separator\")\n\t}\n\n\t// Content-Length: <number>\n\tcontentLengthBytes := header[len(\"Content-Length: \"):]\n\tcontentLength, err := strconv.Atoi(string(contentLengthBytes))\n\tif err != nil {\n\t\treturn \"\", nil, err\n\t}\n\n\tvar baseMessage BaseMessage\n\tif err := json.Unmarshal(content[:contentLength], &baseMessage); err != nil {\n\t\treturn \"\", nil, err\n\t}\n\n\treturn baseMessage.Method, content[:contentLength], nil\n}\n\n// type SplitFunc func(data []byte, atEOF bool) (advance int, token []byte, err error)\nfunc Split(data []byte, _ bool) (advance int, token []byte, err error) {\n\theader, content, found := bytes.Cut(data, []byte{'\\r', '\\n', '\\r', '\\n'})\n\tif !found {\n\t\treturn 0, nil, nil\n\t}\n\n\t// Content-Length: <number>\n\tcontentLengthBytes := header[len(\"Content-Length: \"):]\n\tcontentLength, err := strconv.Atoi(string(contentLengthBytes))\n\tif err != nil {\n\t\treturn 0, nil, err\n\t}\n\n\tif len(content) < contentLength {\n\t\treturn 0, nil, nil\n\t}\n\n\ttotalLength := len(header) + 4 + contentLength\n\treturn totalLength, data[:totalLength], nil\n}\n```\n\n```go\n// main.go\npackage main\n\nimport (\n\t\"bufio\"\n\t\"chesslsp/analysis\"\n\t\"chesslsp/lsp\"\n\t\"chesslsp/rpc\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n)\n\nfunc main() {\n\tscanner := bufio.NewScanner(os.Stdin)\n\tscanner.Split(rpc.Split)\n\tw := os.Stdout\n\tstate := analysis.NewState()\n\n\tfor scanner.Scan() {\n\t\tmsg := scanner.Bytes()\n\t\tmethod, contents, err := rpc.DecodeMessage(msg)\n\t\tif err != nil {\n\t\t\tlog.Printf(\"Error decoding message: %s\", err)\n\t\t}\n\n\t\thandleMessage(w, &state, method, contents)\n\t}\n}\n\nfunc handleMessage(w io.Writer, state *analysis.State, method string, contents []byte) {\n\tswitch method {\n\tcase \"initialize\":\n\t\tvar request lsp.InitializeRequest\n\t\tif err := json.Unmarshal(contents, &request); err != nil {\n\t\t\treturn\n\t\t}\n\t\tmsg := lsp.NewInitializeResponse(request.ID)\n\t\twriteResponse(w, msg)\n\t}\n}\n\nfunc writeResponse(w io.Writer, msg any) {\n\treply := rpc.EncodeMessage(msg)\n\tw.Write([]byte(reply))\n}\n```\n\nThe `main.go` code has been truncated to only show the initialize event, but I left in the bits of it (i.e. the `state` parameter of the `handleMessage` function that won't become clear just yet).\nIf you don't fully understand the code in `rpc.go`, I recommend watching the beginning portion of [TJ's video about the LSP spec](https://youtu.be/YsdlcQoHqPY?si=Sq9mlljv4PgBLpMI).\nThe real meat of this, however, comes in the analysis portion, as that's where everything actually happens. Before we look at that though, let's look at some of the implementations for the actual Language Server Protocol's JSON messages.\n\n```go\n// lsp/message.go\npackage lsp\n\ntype Request struct {\n\tRPC string `json:\"jsonrpc\"`\n\tID int `json:\"id\"`\n\tMethod string `json:\"method\"`\n}\n\ntype Response struct {\n\tRPC string `json:\"jsonrpc\"`\n\tID int `json:\"id,omitempty\"`\n}\n\ntype Notification struct {\n\tRPC string `json:\"jsonrpc\"`\n\tMethod string `json:\"method\"`\n}\n```\n\nThese are the basic structures for the types of messages, an example of which is the `DidOpenTextDocumentNotification`:\n\n```go\ntype DidOpenTextDocumentNotification struct {\n\tNotification\n\tParams DidOpenTextDocumentParams `json:\"params\"`\n}\n```\n\nSince Go doesn't have traditional inheritance, the `Notification` is just passed as a field in this struct.\nThe `Params` are then a type of another struct `DidOpenTextDocumentParams`:\n\n```go\ntype DidOpenTextDocumentParams struct {\n\tTextDocument TextDocumentItem `json:\"textDocument\"`\n}\n```\n\nThe `TextDocumentItem` struct is what describes the actual file:\n\n```go\ntype TextDocumentItem struct {\n\tURI string `json:\"uri\"`\n\tLanguageID string `json:\"languageID\"`\n\tVersion int `json:\"version\"`\n\tText string `json:\"text\"`\n}\n```\n\nThere are other similar structs that are used for other events, but this `Notification` is sent when a file is, shocker, opened.\nAs an LSP is expanded you can add more and more of these. The most important `Request` and `Response`, however, are the ones for the initialize event. These define what both the client and server are capable of. The `IntializeResponse` can be viewed here:\n\n```go\ntype InitializeResponse struct {\n\tResponse\n\tResult InitializeResult `json:\"result\"`\n}\ntype InitializeResult struct {\n\tCapabilities ServerCapabilities `json:\"capabilities\"`\n\tServerInfo ServerInfo `json:\"serverInfo\"`\n}\n```\n\nWhen the response is given, we give the client these `ServerCapabilities`:\n\n```go\nCapabilities:\nServerCapabilities{\n\tTextDocumentSync: 2,\n\tCompletionProvider: map[string]any{},\n}\n```\n\n### Analysis\n\nWhen actually doing the analysis I went through three stages:\n\n1. I will use someone's existing PGN parser\n2. I'm having some trouble making use of these existing parsers, I'm going to design one myself\n3. Man, just writing that lexer was a lot, I should try someone else's parser again.\n The one that stuck was one from 5 years ago [by malbrecht](https://github.com/malbrecht/chess), but there are multiple others that exist, and if I continue working on this, I'll probably go back to building my own as I require more customizability. For now though, this parser works out just fine.\n The two features I wanted for this were diagnostics and completions, and this parser (obviously) returns errors if it can't parse the PGN, and is part of a larger package that allows for looking up legal moves from a position.\n The actual analysis tool runs by having a `State` struct that stores the text documents (in this case just the one) and a database of PGNs from the parser (in this case just the one). There are then some functions that are called every time an event happens, like opening a document, updating a document, or asking for completions. I won't put any actual code here, but I'll run you through the basic order in which things happen when running the LSP in the editor.\n4. The Initialize Request is sent, the server responds telling info about the server and its capabilities.\n5. The `textDocument/didOpen` notification is sent, and the server loads the text of the document into the state of the analysis tool. This also sends back any diagnostic information the parser returns.\n6. If the user makes any changes to the PGN, the `textDocument/didChange` notification is sent, and the server loads the text of the document in the state changes to reflect these changes. This also updates the diagnostic information.\n7. Presumably, these changes cause a completion request to be sent, and the server looks through the legal moves at the position at the cursor in the file, and returns them as a list of completion items.\n\n## Conclusion\n\nHopefully this was a cool look into what the LSP can do for you, and why it's such a cool technology. I didn't go into a whole lot of detail here, but if you want to take a look at the code for this, it's all on [GitHub](https://github.com/sammyshear/chesslsp). Feel free to make issues or pull requests if you want, as there are definitely problems, I just haven't found them yet.\n"
},
"links": [
{
"$type": "site.standard.document#alternateLink",
"label": "sammyshear.com",
"url": "https://sammyshear.com/blog/writing-my-own-lsp-in-go"
}
],
"path": "/blog/writing-my-own-lsp-in-go",
"publishedAt": "2024-06-26T00:00:00.000Z",
"site": "https://sshear.dev",
"tags": [
"Programming"
],
"textContent": "What is a Language Server?\n\nIf you're unfamiliar, the Language Server Protocol is a protocol by which a client (usually a code editor) can talk to a language server and get information about an open file and/or workspace. This is made use of in editors like VS Code and Neovim (my editor of choice, by the way). The protocol passes data around in JSON which essentially allows for an RPC where the client tells the server to do something or vice versa. These can be built in any language, as long as said language supports whatever transport method you choose for the JSON RPC (usually stdout and stdin, but TCP is another example of an option). For this language server I built it in Go using stdout and stdin.\n\nWhat is a Chess PGN?\n\nThis may be the one fewer reading this have heard of, so what is a PGN? PGN (Portable Game Notation) is the format in which most chess games are stored digitally. A PGN file can hold a single game or an entire database of games, but for this basic LSP implementation I have assumed that the user is editing single game files, which is a big limiting factor, so be warned if you decide to use this. The PGN standard consists of two main parts: tag pairs and moves. The tag pairs store metadata about the game, while the moves are understandably the moves of the game. There are 7 required tags in the standard to define a game, and the moves are recorded in Standard Algebraic Notation (i.e. Ke2). I won't go too heavily into the specification of the standard here because I decided against writing my own parser (although not before creating a lexer when my initial playing around with existing parsers frustrated me), but if you want to read up on it, it is an interesting spec in and of itself.\n\nWhat can this language server do?\n\nIn its current basic form, it can parse a PGN of a single chess game, report errors with the parsing, and suggest legal moves as completions.\n\nMaking the Server\n\nNotes and Issues\n\nSince this was my first time ever creating a language server, I no doubt made some mistakes, and I'm also fairly new to Go, so I have no doubt that will have contributed to any issues as well, but all that being said, this went over pretty painlessly. I had some hiccups with the parser itself, but as far as implementing the protocol it went off without much of a hitch. The only real warning I'd give to people following suit on this is that you should probably design your analysis tool before doing much else because the way I did it, I felt a lot like I was jumping around my codebase continuously adding and removing from the LSP to fit the demands of the analysis tool. That might seem self-evident, especially if you're writing a language server for a programming language, but I figured I should mention it.\n\nThe Protocol\n\nI started with the help of TJ Devries' educationalsp repo creating some helper functions for the RPC:\n\ngo\n// rpc/rpc.go\npackage rpc\n\nimport (\n\t\"bytes\"\n\t\"encoding/json\"\n\t\"errors\"\n\t\"fmt\"\n\t\"strconv\"\n)\n\nfunc EncodeMessage(msg any) string {\n\tcontent, err := json.Marshal(msg)\n\tif err != nil {\n\t\tpanic(err)\n\t}\n\n\treturn fmt.Sprintf(\"Content-Length: %d\\r\\n\\r\\n%s\", len(content), content)\n}\n\ntype BaseMessage struct {\n\tMethod string json:\"method\"\n}\n\nfunc DecodeMessage(msg []byte) (string, []byte, error) {\n\theader, content, found := bytes.Cut(msg, []byte{'\\r', '\\n', '\\r', '\\n'})\n\tif !found {\n\t\treturn \"\", nil, errors.New(\"Did not find separator\")\n\t}\n\n\t// Content-Length: <number>\n\tcontentLengthBytes := header[len(\"Content-Length: \"):]\n\tcontentLength, err := strconv.Atoi(string(contentLengthBytes))\n\tif err != nil {\n\t\treturn \"\", nil, err\n\t}\n\n\tvar baseMessage BaseMessage\n\tif err := json.Unmarshal(content[:contentLength], &baseMessage); err != nil {\n\t\treturn \"\", nil, err\n\t}\n\n\treturn baseMessage.Method, content[:contentLength], nil\n}\n\n// type SplitFunc func(data []byte, atEOF bool) (advance int, token []byte, err error)\nfunc Split(data []byte, bool) (advance int, token []byte, err error) {\n\theader, content, found := bytes.Cut(data, []byte{'\\r', '\\n', '\\r', '\\n'})\n\tif !found {\n\t\treturn 0, nil, nil\n\t}\n\n\t// Content-Length: <number>\n\tcontentLengthBytes := header[len(\"Content-Length: \"):]\n\tcontentLength, err := strconv.Atoi(string(contentLengthBytes))\n\tif err != nil {\n\t\treturn 0, nil, err\n\t}\n\n\tif len(content) < contentLength {\n\t\treturn 0, nil, nil\n\t}\n\n\ttotalLength := len(header) + 4 + contentLength\n\treturn totalLength, data[:totalLength], nil\n}\n\ngo\n// main.go\npackage main\n\nimport (\n\t\"bufio\"\n\t\"chesslsp/analysis\"\n\t\"chesslsp/lsp\"\n\t\"chesslsp/rpc\"\n\t\"encoding/json\"\n\t\"io\"\n\t\"log\"\n\t\"os\"\n)\n\nfunc main() {\n\tscanner := bufio.NewScanner(os.Stdin)\n\tscanner.Split(rpc.Split)\n\tw := os.Stdout\n\tstate := analysis.NewState()\n\n\tfor scanner.Scan() {\n\t\tmsg := scanner.Bytes()\n\t\tmethod, contents, err := rpc.DecodeMessage(msg)\n\t\tif err != nil {\n\t\t\tlog.Printf(\"Error decoding message: %s\", err)\n\t\t}\n\n\t\thandleMessage(w, &state, method, contents)\n\t}\n}\n\nfunc handleMessage(w io.Writer, state analysis.State, method string, contents []byte) {\n\tswitch method {\n\tcase \"initialize\":\n\t\tvar request lsp.InitializeRequest\n\t\tif err := json.Unmarshal(contents, &request); err != nil {\n\t\t\treturn\n\t\t}\n\t\tmsg := lsp.NewInitializeResponse(request.ID)\n\t\twriteResponse(w, msg)\n\t}\n}\n\nfunc writeResponse(w io.Writer, msg any) {\n\treply := rpc.EncodeMessage(msg)\n\tw.Write([]byte(reply))\n}\n\nThe main.go code has been truncated to only show the initialize event, but I left in the bits of it (i.e. the state parameter of the handleMessage function that won't become clear just yet).\nIf you don't fully understand the code in rpc.go, I recommend watching the beginning portion of TJ's video about the LSP spec.\nThe real meat of this, however, comes in the analysis portion, as that's where everything actually happens. Before we look at that though, let's look at some of the implementations for the actual Language Server Protocol's JSON messages.\n\ngo\n// lsp/message.go\npackage lsp\n\ntype Request struct {\n\tRPC string json:\"jsonrpc\"\n\tID int json:\"id\"\n\tMethod string json:\"method\"\n}\n\ntype Response struct {\n\tRPC string json:\"jsonrpc\"\n\tID int json:\"id,omitempty\"\n}\n\ntype Notification struct {\n\tRPC string json:\"jsonrpc\"\n\tMethod string json:\"method\"\n}\n\nThese are the basic structures for the types of messages, an example of which is the DidOpenTextDocumentNotification:\n\ngo\ntype DidOpenTextDocumentNotification struct {\n\tNotification\n\tParams DidOpenTextDocumentParams json:\"params\"\n}\n\nSince Go doesn't have traditional inheritance, the Notification is just passed as a field in this struct.\nThe Params are then a type of another struct DidOpenTextDocumentParams:\n\ngo\ntype DidOpenTextDocumentParams struct {\n\tTextDocument TextDocumentItem json:\"textDocument\"\n}\n\nThe TextDocumentItem struct is what describes the actual file:\n\ngo\ntype TextDocumentItem struct {\n\tURI string json:\"uri\"\n\tLanguageID string json:\"languageID\"\n\tVersion int json:\"version\"\n\tText string json:\"text\"\n}\n\nThere are other similar structs that are used for other events, but this Notification is sent when a file is, shocker, opened.\nAs an LSP is expanded you can add more and more of these. The most important Request and Response, however, are the ones for the initialize event. These define what both the client and server are capable of. The IntializeResponse can be viewed here:\n\ngo\ntype InitializeResponse struct {\n\tResponse\n\tResult InitializeResult json:\"result\"\n}\ntype InitializeResult struct {\n\tCapabilities ServerCapabilities json:\"capabilities\"\n\tServerInfo ServerInfo json:\"serverInfo\"\n}\n\nWhen the response is given, we give the client these ServerCapabilities:\n\ngo\nCapabilities:\nServerCapabilities{\n\tTextDocumentSync: 2,\n\tCompletionProvider: map[string]any{},\n}\n\nAnalysis\n\nWhen actually doing the analysis I went through three stages:\n\n1. I will use someone's existing PGN parser\n2. I'm having some trouble making use of these existing parsers, I'm going to design one myself\n3. Man, just writing that lexer was a lot, I should try someone else's parser again.\n The one that stuck was one from 5 years ago by malbrecht, but there are multiple others that exist, and if I continue working on this, I'll probably go back to building my own as I require more customizability. For now though, this parser works out just fine.\n The two features I wanted for this were diagnostics and completions, and this parser (obviously) returns errors if it can't parse the PGN, and is part of a larger package that allows for looking up legal moves from a position.\n The actual analysis tool runs by having a State struct that stores the text documents (in this case just the one) and a database of PGNs from the parser (in this case just the one). There are then some functions that are called every time an event happens, like opening a document, updating a document, or asking for completions. I won't put any actual code here, but I'll run you through the basic order in which things happen when running the LSP in the editor.\n4. The Initialize Request is sent, the server responds telling info about the server and its capabilities.\n5. The textDocument/didOpen notification is sent, and the server loads the text of the document into the state of the analysis tool. This also sends back any diagnostic information the parser returns.\n6. If the user makes any changes to the PGN, the textDocument/didChange notification is sent, and the server loads the text of the document in the state changes to reflect these changes. This also updates the diagnostic information.\n7. Presumably, these changes cause a completion request to be sent, and the server looks through the legal moves at the position at the cursor in the file, and returns them as a list of completion items.\n\nConclusion\n\nHopefully this was a cool look into what the LSP can do for you, and why it's such a cool technology. I didn't go into a whole lot of detail here, but if you want to take a look at the code for this, it's all on GitHub. Feel free to make issues or pull requests if you want, as there are definitely problems, I just haven't found them yet.",
"title": "Writing My Own Language Server in Go (to Parse Chess PGNs)"
}