{
"path": "/blog/2012-07-18-memory-checking-in-low-level-javascript",
"site": "at://did:plc:4vjd3fe2cgzq5d24j4f3zvar/site.standard.publication/3mjz2bzni752x",
"$type": "site.standard.document",
"title": "Memory Checking in Low-Level JavaScript",
"content": {
"$type": "dev.disnet.blog.content.markdown",
"markdown": "\nSo this past month I've been helping out on the\n[Low-Level JavaScript][lljs] (LLJS) project. Bit of a change for me since I usually\nhide out at\nas [high a level as I can possibly get][abstraction] :)\n\nLLJS is an experiment in adding low-level features like pointers and\nmanual memory management to a dialect of JavaScript.\nThese low-level features are pretty cool and can get you some nice\nperformance wins, but they also come at a cost since\nyou are always one\nnull pointer dereference or memory leak away from oblivion.\n\nOne way C/C++ programmers have handled this danger is by using an\nanalysis tool like [Valgrind][valgrind] to detect memory errors\nthat happen while the program is running.\nSince memory checking has proven to be useful in the C world, we\nfigured it might be helpful for LLJS. So, I've hacked up a\nvalgrind-like memory analysis for LLJS. It can detect four kinds\nof memory errors:\n\n* reads/writes to unallocated memory (pointers we haven't `malloc`'ed)\n* reads from undefined memory (`malloc`'ed memory we haven't\n written to)\n* bad frees (calling `free` on memory that hasn't been `malloc`'ed)\n* leaks (any unfreed memory at the end of the program run)\n\n\n## How to use\n\nGo grab the latest version of LLJS from its github [repo][lljsrepo].\nThen add the following snippet to the end of your script:\n\n```js\nlet m = require('memory');\nconsole.log(m.memcheck.report());\n```\n\n\n\nThis pulls in the memory module and prints out the result of the\nmemory checking (this snippet is for node.js, so use `load` if you're\nusing SpiderMonkey or grab `memory` off the global object if you're in\nthe browser).\n\nThen compile your LLJS code with the `-m` flag:\n\n```sh\nbin/ljc -m -o your_file.js your_file.ljs\n```\n\nand run it:\n\n```sh\nnode --harmony-proxies your_file.js\n```\n\nAs you can see the memory checker uses [Proxies][proxy], which node/V8\nhides behind a flag (no flag needed on SpiderMonkey).\n\nFor example, the following script:\n\n```js\nextern console;\n\nfunction test_unallocated() {\n let uint *x;\n // not allocated yet!\n return *x;\n}\ntest_unallocated();\n\nfunction test_leak() {\n // leak!\n let uint *leak = new uint;\n}\ntest_leak();\n\nfunction test_free() {\n let uint *x = new uint;\n delete x;\n delete x;\n}\ntest_free();\n\nlet m = require('memory');\nconsole.log(m.memcheck.report());\n```\n\n\nwill print out the following report:\n\n Unallocated:\n 0 at:\n test_unallocated:3:0\n\n Undefined:\n 0 at:\n test_unallocated:3:0\n\n Bad frees:\n 8184 at:\n test_free:16:0\n\n Leaks:\n 8200 at:\n test_leak:10:0\n\n## How it works\n\nMemory allocation in LLJS is implemented as a big [typed array][tarray]\nwhere pointers are really just array indexes into the array.\n\nTo do memory checking we create a second *shadow memory* typed array\nthat stores the allocation status of each byte of real memory (if the byte\nhas been allocated, initialized, etc.). Calls to `malloc` and `free`\nchange the allocation status of the bytes in shadow memory. The real memory\narray is\nwrapped in a Proxy that\nintercepts each get and set operation and\nchecks the shadow memory\nfor a possible error.\n\nFor example, say we we allocate an int and try to do some pointer\narithmetic and dereference unallocated memory:\n\n```js\nlet int *x = new int;\n*(x+1)\n```\n\nWhen the dereference of `x+1` happens, the Proxy wrapping real memory\nwill check in shadow memory to see if `x+1` has been allocated. Since\nit hasn't, an unallocated memory access error is recorded.\n\nThis approach is basically a simplified version of how\nValgrind [does memory checking][memcheck].\n\n\n[lljs]: http://lljs.org/ \"Low-Level JavaScript\"\n[lljsrepo]: https://github.com/mbebenita/LLJS \"LLJS on github\"\n[valgrind]: http://valgrind.org/ \"Valgrind\"\n[proxy]: https://developer.mozilla.org/en/JavaScript/Reference/Global_Objects/Proxy \"Proxy\"\n[memcheck]: http://valgrind.org/docs/shadow-memory2007.pdf \"Memcheck\"\n[tarray]: https://developer.mozilla.org/en/JavaScript_typed_arrays \"Typed Arrays\"\n[abstraction]: http://www.quickmeme.com/meme/358lli/ \"Abstract all the things!\"\n",
"sourceFormat": "markdown"
},
"publishedAt": "2012-07-18T00:00:00.000Z",
"textContent": "So this past month I've been helping out on the [Low Level JavaScript][lljs] (LLJS) project. Bit of a change for me since I usually hide out at as [high a level as I can possibly get][abstraction] :) LLJS is an experiment in adding low level features like pointers and manual memory management to a dialect of JavaScript. These low level features are pretty cool and can get you some nice performance wins, but they also come at a cost since you are always one null pointer dereference or memory leak away from oblivion. One way C/C++ programmers have handled this danger is by using an analysis tool like [Valgrind][valgrind] to detect memory errors that happen while the program is running. Since memory checking has proven to be useful in the C world, we figured it might be helpful for LLJS. So, I've hacked up a valgrind like memory analysis for LLJS. It can detect four kinds of memory errors: reads/writes to unallocated memory (pointers we haven't malloc'ed) reads from undefined memory (malloc'ed memory we haven't written to) bad frees (calling free on memory that hasn't been malloc'ed) leaks (any unfreed memory at the end of the program run) How to use Go grab the latest version of LLJS from its github [repo][lljsrepo]. Then add the following snippet to the end of your script: This pulls in the memory module and prints out the result of the memory checking (this snippet is for node.js, so use load if you're using SpiderMonkey or grab memory off the global object if you're in the browser). Then compile your LLJS code with the m flag: and run it: As you can see the memory checker uses [Proxies][proxy], which node/V8 hides behind a flag (no flag needed on SpiderMonkey). For example, the following script: will print out the following report: Unallocated: 0 at: test unallocated:3:0 Undefined: 0 at: test unallocated:3:0 Bad frees: 8184 at: test free:16:0 Leaks: 8200 at: test leak:10:0 How it works Memory allocation in LLJS is implemented as a big [typed array][tarray] where pointers are really just array indexes into the array. To do memory checking we create a second shadow memory typed array that stores the allocation status of each byte of real memory (if the byte has been allocated, initialized, etc.). Calls to malloc and free change the allocation status of the bytes in shadow memory. The real memory array is wrapped in a Proxy that intercepts each get and set operation and checks the shadow memory for a possible error. For example, say we we allocate an int and try to do some pointer arithmetic and dereference unallocated memory: When the dereference of x+1 happens, the Proxy wrapping real memory will check in shadow memory to see if x+1 has been allocated. Since it hasn't, an unallocated memory access error is recorded. This approach is basically a simplified version of how Valgrind [does memory checking][memcheck]. [lljs]: http://lljs.org/ \"Low Level JavaScript\" [lljsrepo]: https://github.com/mbebenita/LLJS \"LLJS on github\" [valgrind]: http://valgrind.org/ \"Valgrind\" [proxy]: https://developer.mozilla.org/en/JavaScript/Reference/Global Objects/Proxy \"Proxy\" [memcheck]: http://valgrind.org/docs/shadow memory2007.pdf \"Memcheck\" [tarray]: https://developer.mozilla.org/en/JavaScript typed arrays \"Typed Arrays\" [abstraction]: http://www.quickmeme.com/meme/358lli/ \"Abstract all the things!\""
}