{
"$type": "site.standard.document",
"content": {
"$type": "site.standard.content.markdown",
"text": "I have been revisiting the infrastructure which supports themes at my day job where the number of tokens has exploded into the hundreds; thousands if you count the possibilities of defining custom ones for special needs. This makes the token curation process unwieldly when theming because assigning values to over 500 semantic tokens is very tedious.\n\nThe trouble is that while a certain brand or theme might have a few base values which give it personality, the final collection of tokens to define a theme will require slight variations of these base values for a properly curated presentation.\n\nWhat if we could instead allow for a few values to be set in a theme to inform the rest of the values in some specified way?\n\n## Operations\n\n[First suggested within the Design Tokens Community Group by Jerome Farnum](https://github.com/design-tokens/community-group/issues/88#issuecomment-1029905903) we could consider a special key on the specification that expects an order of operations to be performed ([example from James Nash](https://github.com/design-tokens/community-group/issues/88#issuecomment-1073374383)):\n\n```\n{\n \"some-token\": {\n \"$value\": \"#ff0000\",\n \"$type\": \"color\",\n \"$operations\": [\n { \"hue\": -20 },\n { \"alpha\": 0.3 } \n ]\n }\n}\n```\n\nThis is a good start but I saw two major flaws with this approach:\n\n### Freeform naming\n\nThe words `hue` and `alpha` are meant to reference some external function that must exist in the token processing in order to transform to a final value. The algorithms used to determine the final output might be different or not exist at all. Each vendor must implement their own `hue` and `alpha` functions in order for these to be useful. This would require the specification to outline *every* kind of possible transform a theme author might want to use.\n\n### Custom functions\n\nFurthermore, theme curators could be interested in creating their own custom functions which would need to be executed in vendors’ processes. In this format, there’s no clear way to define this. We can’t even support multiple arguments in the current suggestion.\n\n## Step-by-step\n\nThe solution I have for this is to implement a low-level language that can be built upon to make more complex functions. Here’s an example of what a token file could look like with a new take on `$operations`:\n\n```\n{\n \"primary-color-overlay\": {\n \"$type\": \"color\",\n \"$value\": \"#fffc00\",\n \"$operations\": [\n 0.5,\n [\"String.match\", \"$value\", \"#([0-9A-Fa-f]{2})\"],\n [\"String.match\", \"$value\", \"#(?:[0-9A-Fa-f]{2})([0-9A-Fa-f]{2})\"],\n [\"String.match\", \"$value\", \"#(?:[0-9A-Fa-f]{4})([0-9A-Fa-f]{2})\"],\n [\"Math.parseInt\", \"$1\", 16],\n [\"Math.parseInt\", \"$2\", 16],\n [\"Math.parseInt\", \"$3\", 16],\n [\"String.concat\", \",\", \"$4\", \"$5\", \"$6\", \"$0\"],\n [\"String.concat\", \"\", \"rgba(\", \"$7\", \")\"]\n ]\n }\n}\n```\n\n[See minimal prototype in action.](https://replit.com/@fauxserious/TokenOperations)\n\nThe `$operations` key is still an array which is necessary to ensure we can have a sequential step. The first concept is that each item (except for the first in this example) is an **operation**. It has the following signature:\n\n```\n[\"OperationReference\", \"argument1\", \"argument2\", \"argument3\",...]\n```\n\nThe first item in each operation array references some well-known function that can be implemented across token processing tools and vendors. For this to work, I believe we’ll need at least `String` and `Math` with a few additional methods to support standard operations (eg., `Math.add`). This is something that can be included in the tokens specification if agreeable.\n\nThe result of each operation is stored in a special variable reference related to its index in the operations array. In the example above, the first operation (defined at index `1`) result is stored at `\"$1\"`. This result is used as an argument in the fifth operation (the first to use `Math.parseInt` above). Primitive values in the operations array are simply stored as results to the index. Here’s the example again, with comments marking how results are stored:\n\n```\n{\n \"primary-color-overlay\": {\n \"$type\": \"color\",\n \"$value\": \"#fffc00\",\n \"$operations\": [\n 0.5, // 0.5 stored at $0\n [\"String.match\", \"$value\", \"#([0-9A-Fa-f]{2})\"], // 'ff' stored at $1\n [\"String.match\", \"$value\", \"#(?:[0-9A-Fa-f]{2})([0-9A-Fa-f]{2})\"], // 'fc' stored at $2\n [\"String.match\", \"$value\", \"#(?:[0-9A-Fa-f]{4})([0-9A-Fa-f]{2})\"], // '00' stored at $3\n [\"Math.parseInt\", \"$1\", 16], // 255 stored at $4\n [\"Math.parseInt\", \"$2\", 16], // 252 stored at $5\n [\"Math.parseInt\", \"$3\", 16], // 0 stored at $6\n [\"String.concat\", \",\", \"$4\", \"$5\", \"$6\", \"$0\"], // 255,252,0,0.5 stored at $7\n [\"String.concat\", \"\", \"rgba(\", \"$7\", \")\"] // rgba(255,252,0,0.5) stored at $8\n ]\n }\n}\n```\n\nThis would also include a special `$value` which refers to the original token value.\n\nOne of the larger benefits to this approach is that the low-level set of operations don’t receive a name, which avoids contention about naming of operations from within the specification (ie., should it be `opacity` or `alpha`). Names would only be needed when sharing sets of operations between token authors.\n\n## Reusing common operations\n\nClearly, we would not want to write all of these operations in order to apply opacity to a given token each time. We’ll need a way to reference a set of operations. This could be supported by allowing an import syntax within the specification. [I’ve suggested this idea before in the community group.](https://github.com/design-tokens/community-group/issues/210#issuecomment-1501037423) It could look like this for operations:\n\n```\n{\n \"primary-color-overlay\": {\n \"$type\": \"color\",\n \"$value\": \"#fffc00\",\n \"$operations\": [ \n // All items after the import are used as arguments for the nested operation in order\n // 0.5 stored at $0 for @token-operations/hex-opacity operations\n [\"@token-operations/hex-opacity\", 0.5 ]\n ]\n }\n}\n```\n\nIn the above example `@token-operations/hex-opacity` is completely arbitrary and could be any package registry or url that has an array as the export. The expectation here is that folks might write projects that abstract low-level collections of operations into more user-friendly exports. This is similar to the purpose of the [Lit framework](https://lit.dev/) for component authoring; to make web component development more accessible with a level of abstraction.\n\nThe special sauce would be within the operations parser. Since the first argument in an operation is meant to be a reference to a function (eg., `String.concat`), we could also check for a potential file to import here or have some other special syntax. The result would be to inject the operations array related to this reference in place as a nested operation with its own scoped step indicies. The trailing arguments of the import are then placed in the beginning of the imported operation set since each curated operation has an expected signature.\n\n```\n{\n \"primary-color-overlay\": {\n \"$type\": \"color\",\n \"$value\": \"#fffc00\",\n \"$operations\": [\n // First level of operations\n [\n // Second level of operations (nested), final output returns to parent operation result in place\n 0.5, // First argument after import operation, placed at $0 in nested operation\n [\"String.match\", \"$value\", \"#([0-9A-Fa-f]{2})\"],\n [\"String.match\", \"$value\", \"#(?:[0-9A-Fa-f]{2})([0-9A-Fa-f]{2})\"],\n [\"String.match\", \"$value\", \"#(?:[0-9A-Fa-f]{4})([0-9A-Fa-f]{2})\"],\n [\"Math.parseInt\", \"$1\", 16],\n [\"Math.parseInt\", \"$2\", 16],\n [\"Math.parseInt\", \"$3\", 16],\n [\"String.concat\", \",\", \"$4\", \"$5\", \"$6\", \"$0\"],\n [\"String.concat\", \"\", \"rgba(\", \"$7\", \")\"]\n ]\n ]\n }\n}\n```\n\nThe purpose of this is so imported operations can be chained while maintaining the positional references for each step. In other words, we want to maintain the meaning of `$0` related to the rest of the steps in place.\n\nIn this way, as long as token processors include the low-level operational functions and the standardized way to execute them, you can define any kind of transform you need right within the token file.",
"version": "1.0"
},
"coverImage": {
"$type": "blob",
"ref": {
"$link": "bafkreigkb664v7ngvcv6qdmjc7mxvtworbiprbvyphcpzb5aqdn3emmhgu"
},
"mimeType": "image/png",
"size": 255458
},
"description": "Defining transforms into a specification.",
"path": "/posts/token-operations",
"publishedAt": "2023-05-31T00:00:00.000Z",
"site": "https://blog.damato.design",
"tags": [
"css",
"naming",
"tokens"
],
"textContent": "I have been revisiting the infrastructure which supports themes at my day job where the number of tokens has exploded into the hundreds; thousands if you count the possibilities of defining custom ones for special needs. This makes the token curation process unwieldly when theming because assigning values to over 500 semantic tokens is very tedious.\n\nThe trouble is that while a certain brand or theme might have a few base values which give it personality, the final collection of tokens to define a theme will require slight variations of these base values for a properly curated presentation.\n\nWhat if we could instead allow for a few values to be set in a theme to inform the rest of the values in some specified way?\n\nOperations\n\nFirst suggested within the Design Tokens Community Group by Jerome Farnum we could consider a special key on the specification that expects an order of operations to be performed (example from James Nash):\n\nThis is a good start but I saw two major flaws with this approach:\n\nFreeform naming\n\nThe words and are meant to reference some external function that must exist in the token processing in order to transform to a final value. The algorithms used to determine the final output might be different or not exist at all. Each vendor must implement their own and functions in order for these to be useful. This would require the specification to outline every kind of possible transform a theme author might want to use.\n\nCustom functions\n\nFurthermore, theme curators could be interested in creating their own custom functions which would need to be executed in vendors’ processes. In this format, there’s no clear way to define this. We can’t even support multiple arguments in the current suggestion.\n\nStep-by-step\n\nThe solution I have for this is to implement a low-level language that can be built upon to make more complex functions. Here’s an example of what a token file could look like with a new take on :\n\nSee minimal prototype in action.\n\nThe key is still an array which is necessary to ensure we can have a sequential step. The first concept is that each item (except for the first in this example) is an operation. It has the following signature:\n\nThe first item in each operation array references some well-known function that can be implemented across token processing tools and vendors. For this to work, I believe we’ll need at least and with a few additional methods to support standard operations (eg., ). This is something that can be included in the tokens specification if agreeable.\n\nThe result of each operation is stored in a special variable reference related to its index in the operations array. In the example above, the first operation (defined at index ) result is stored at . This result is used as an argument in the fifth operation (the first to use above). Primitive values in the operations array are simply stored as results to the index. Here’s the example again, with comments marking how results are stored:\n\nThis would also include a special which refers to the original token value.\n\nOne of the larger benefits to this approach is that the low-level set of operations don’t receive a name, which avoids contention about naming of operations from within the specification (ie., should it be or ). Names would only be needed when sharing sets of operations between token authors.\n\nReusing common operations\n\nClearly, we would not want to write all of these operations in order to apply opacity to a given token each time. We’ll need a way to reference a set of operations. This could be supported by allowing an import syntax within the specification. I’ve suggested this idea before in the community group. It could look like this for operations:\n\nIn the above example is completely arbitrary and could be any package registry or url that has an array as the export. The expectation here is that folks might write projects that abstract low-level collections of operations into more user-friendly exports. This is similar to the purpose of the Lit framework for component authoring; to make web component development more accessible with a level of abstraction.\n\nThe special sauce would be within the operations parser. Since the first argument in an operation is meant to be a reference to a function (eg., ), we could also check for a potential file to import here or have some other special syntax. The result would be to inject the operations array related to this reference in place as a nested operation with its own scoped step indicies. The trailing arguments of the import are then placed in the beginning of the imported operation set since each curated operation has an expected signature.\n\nThe purpose of this is so imported operations can be chained while maintaining the positional references for each step. In other words, we want to maintain the meaning of related to the rest of the steps in place.\n\nIn this way, as long as token processors include the low-level operational functions and the standardized way to execute them, you can define any kind of transform you need right within the token file.",
"title": "Token operations",
"updatedAt": "2026-06-13T19:14:07.413Z"
}