\n Common Actions Interface\n
\n\n
\n\n## Development\n\nThis codebase is built around TypeScript, Node.js/Browser, pnpm, and zod for parsing JSON input. All central functions should have an object passed as input, which zod parses and validates. We automatically generate the zod types using `pnpm tsx make`, which reads from `code/**/source.ts` for a bunch of \"type\" definitions written in JavaScript/JSON, and it then takes those definitions and generates TypeScript types and zod parsers. This way we don't have to write 2 or 3 times the same type definition (once in TypeScript, once in zod, once for an API somewhere else, etc.), you just write the source types, and it generates the types from that.\n\nThen there are 3 parts:\n\n1. Node.js API\n2. CLI\n3. Browser API (not really started yet)\n\nThe Node.js API is all Promise based, using the object input pattern. The CLI wraps that Node.js interface and logs some stuff to the terminal, not too much.\n\nTo setup the command locally, just do `pnpm link -g`.\n\n### Key Commands\n\n```bash\npnpm install # install the packages to get prettier/eslint autocomplete in vscode!\npnpm tsx make # regenerate the typescript and zod files frouce source.ts files.\npnpm test # if you want to play around with the test commands, it's not finished yet.\n```\n\nThat's it! When you run `pnpm tsx make`, it regenerates the types in `code/type/*.ts`. You don't really have to pay much attention to those, but if you have VSCode and the prettier/eslint stuff setup (should be configured after `pnpm install`, if not, let me know), then you can start typing the type name in some file and it will autoimport it for you. Nice stuff.\n\n### Key Files\n\nAll the code lives in the `code` folder, that plus the `make` folder has a command.\n\n```bash\n# where type definitions are specified, used to generate typescript and zod files\ncode/action/**/source.ts\n# node and browser compatible code\ncode/action/**/shared.ts\n# node.js only code\ncode/action/**/node.ts\n# browser only code\ncode/action/browser.ts\n```\n\nCall in browser.\n\n- if remote\n - file.path must be remote\n - file.content can exist\n - file.sha256 must exist if content exists\n- if not remote\n - file.path must be remote\n - fetch file through server\n - file.content can exist\n - file.sha256 not necessary\n\n```ts\nimport {\n ConvertFontWithFontForgeBrowserInput,\n ConvertFontWithFontForgeBrowserInputParser,\n ConvertFontWithFontForgeBrowserLocalInput,\n ConvertFontWithFontForgeBrowserOutputParser,\n ConvertFontWithFontForgeBrowserRemoteInput,\n} from '~/code/type/index.js'\nimport { buildRequestToConvert } from '../shared.js'\nimport { resolveWorkFileAsBlob } from '~/code/tool/shared/work.js'\nimport kink from '~/code/tool/shared/kink.js'\n\nexport async function convertFontWithFontForgeBrowser(\n source: ConvertFontWithFontForgeBrowserInput,\n) {\n const input = ConvertFontWithFontForgeBrowserInputParser().parse(source)\n\n switch (input.handle) {\n case 'remote':\n return await convertFontWithFontForgeBrowserRemote(input)\n default:\n return await convertFontWithFontForgeBrowserLocal(input)\n }\n}\n\nexport async function convertFontWithFontForgeBrowserRemote(\n input: ConvertFontWithFontForgeBrowserRemoteInput,\n) {\n const request = buildRequestToConvert(input)\n const content = await resolveWorkFileAsBlob(request)\n\n return ConvertFontWithFontForgeBrowserOutputParser().parse({\n file: {\n content,\n },\n })\n}\n\nexport async function convertFontWithFontForgeBrowserLocal(\n input: ConvertFontWithFontForgeBrowserLocalInput,\n) {\n throw kink('task_not_implemented', {\n task: 'convertFontWithFontForgeBrowserLocal',\n })\n}\n```\n\nCall in nodejs.\n\n- if remote\n - file.path can be local or remote\n - file.content can exist\n - file.sha256 must exist if content exists\n- if not remote\n - if external\n - file.path must be remote\n - file.content can exist\n - file.sha256 if content exists\n - if not external\n - file.path can be local or remote\n - file.content can exist\n - file.sha256 if content exists\n\n```ts\nimport {\n ConvertFontWithFontForgeNodeInput,\n ConvertFontWithFontForgeNodeInputParser,\n ConvertFontWithFontForgeNodeOutputParser,\n ConvertFontWithFontForgeNodeLocalInternalInput,\n ConvertFontWithFontForgeNodeLocalExternalInput,\n ConvertFontWithFontForgeNodeLocalInputParser,\n ConvertFontWithFontForgeNodeRemoteInput,\n ConvertFontWithFontForgeNodeClientInputParser,\n} from '~/code/type/index.js'\nimport { buildCommandToConvertFontWithFontForge } from './shared.js'\nimport { runCommandSequence } from '~/code/tool/node/command.js'\nimport {\n resolveInputForConvertLocalNode,\n resolveInputForConvertRemoteNode,\n} from '../tool/node.js'\nimport { extend } from '~/code/tool/shared/object.js'\nimport { buildRequestToConvert } from '../shared.js'\nimport { resolveWorkFileNode } from '~/code/tool/node/request.js'\n\nexport async function convertFontWithFontForgeNode(\n source: ConvertFontWithFontForgeNodeInput,\n) {\n const input = ConvertFontWithFontForgeNodeInputParser().parse(source)\n\n switch (input.handle) {\n case 'remote':\n return await convertFontWithFontForgeNodeRemote(input)\n case 'external':\n return await convertFontWithFontForgeNodeLocalExternal(input)\n default:\n return await convertFontWithFontForgeNodeLocalInternal(input)\n }\n}\n\nasync function convertFontWithFontForgeNodeLocalExternal(\n source: ConvertFontWithFontForgeNodeLocalExternalInput,\n) {\n const input = await resolveInputForConvertLocalNode(source)\n return await convertFontWithFontForgeNodeLocal(input)\n}\n\nasync function convertFontWithFontForgeNodeLocalInternal(\n source: ConvertFontWithFontForgeNodeLocalInternalInput,\n) {\n const input = await resolveInputForConvertLocalNode(source)\n return await convertFontWithFontForgeNodeLocal(input)\n}\n\nexport async function convertFontWithFontForgeNodeRemote(\n source: ConvertFontWithFontForgeNodeRemoteInput,\n) {\n const input = await resolveInputForConvertRemoteNode(source)\n const clientInput =\n ConvertFontWithFontForgeNodeClientInputParser().parse(\n extend(input, { handle: 'client' }),\n )\n\n const request = buildRequestToConvert(clientInput)\n await resolveWorkFileNode(request, input.output.file.path)\n\n return ConvertFontWithFontForgeNodeOutputParser().parse({\n file: {\n path: input.output.file.path,\n },\n })\n}\n\nexport async function convertFontWithFontForgeNodeLocal(input) {\n const localInput =\n ConvertFontWithFontForgeNodeLocalInputParser().parse(input)\n\n const sequence =\n await buildCommandToConvertFontWithFontForge(localInput)\n\n await runCommandSequence(sequence)\n\n return ConvertFontWithFontForgeNodeOutputParser().parse({\n file: {\n path: localInput.output.file.path,\n },\n })\n}\n```\n\n### Task Organization\n\nEach task in Node.js basically starts from one of the top simple action methods:\n\n- `compile`\n- `format`\n- `convert`\n- etc.\n\nFirst it takes the `input` from the top-level call, and parses the input and passes the parsed input to the implementation task like `convertImageWithImageMagick`. Then that function checks for the `surf` argument, and if present, it branches to make a remote API call against `task.surf`. This serializes all local file paths into readable streams for upload, but keeps remote file paths unchanged. No further input parsing occurs after the first two top-level parsings.\n\nIf the `surf` parameter is not present, then it branche into the \"local\" API call, to the file system or a system command. So we have basically:\n\n```\nconvert(source)\n input = parse(source)\n convertImageWithImageMagick(input)\n if input.surf\n convertImageWithImageMagickRemote(input)\n request = buildRequestToConvertWithImageMagickRemote(input)\n if input.show\n return request\n return makeRequest(request)\n else\n convertImageWithImageMagickLocal(input)\n command = buildCommandToConvertWithImageMagickLocal(input)\n if input.show\n return command\n return runCommand(command)\n```\n\nThe remote method such as `convertImageWithImageMagickRemote` converts the local file paths to streams, and updates some input properties. Likewise, `convertImageWithImageMagickLocal` takes the input and converts some file data to local paths.\n\nIf the top-level command gets a `show` property, then it returns the `buildX` result instead. The top-level `code` property is to pass an auth token to requests.\n\nThe functions are stored in different places:\n\n- `convert`: `code/action/convert/node.ts`\n- `convertInternal`: `code/action/convert/node.ts`\n- `convertImageWithImageMagick`: `code/action/convert/image/node.ts`\n- `convertImageWithImageMagickRemote`: `code/action/convert/image/node.ts`\n- `convertImageWithImageMagickLocal`: `code/action/convert/image/node.ts`\n- `buildRequestToConvertWithImageMagickRemote`: `code/action/convert/image/shared.ts`\n- `buildCommandToConvertWithImageMagickLocal`: `code/action/convert/image/shared.ts`\n- `convert_image_with_image_magick`: `code/action/convert/image/source.ts`\n\nIn addition, it is actually called `convertInternal`, beccause of the way we need a TypeScript interface to everything with `convert` and under the hood it uses the parser with `convertInternal`. Then there are \"source\" types for defining type definitions for zod and TypeScript.\n\n### Adding a new Task\n\nTo add a new task, just place it in either of the shared/no/browser folders, and add a source type definition for the input. Run `pnpm tsx make` to generate the types. Then just write the code to implement the command. If the command invokes a CLI tool, you can create two functions:\n\n1. Build the command.\n2. Run the command.\n\nThe building part just creates an array of CLI arguments. The Run command takes those args and runs them and interprets any CLI output if there is some.\n\nThere are some basic tests against files in the `test` directory, just manual tests pretty much at the moment.\n\nYou don't need to run Docker to develop this, you can just install the tools on your computer to develop locally.\n\n### Choco\n\nTo trigger this build and publishing, the build needs to be run manually.\n\n- [Choco package creation discord](https://discord.com/channels/778552361454141460/897088817293574154)\n\n## TODO\n\n- add unarchive commands\n - see `unar`, `atool`, `zip`, `patool`, and `7z` CLI tool docs\n- add archive creation commands\n - same tools as unarchiving\n- add minify commands\n - css: https://github.com/css/csso [benchmarks](https://github.com/GoalSmashers/css-minification-benchmark)\n - js: probably `@swc/core` from [here](https://github.com/privatenumber/minification-benchmarks?tab=readme-ov-file)\n - just use the docs to define the input option types. If it's not a command, just define a function with the input types and call the library function.\n - could support multiple minifiers later.\n- sanitize html command (started this one, just search for \"sanitize\")\n - https://www.npmjs.com/package/sanitize-html\n- add format code commands\n - black for python\n - config files for ktfmt and clang-format\n - prettier API options for javascript and others\n - etc.\n- add compile commands (these are mostly done)\n- add spreadsheet conversion commands\n - xlsx.js API\n- add disassemble command\n - `objdump` CLI: https://man7.org/linux/man-pages/man1/objdump.1.html\n - https://unix.stackexchange.com/questions/763259/what-are-the-possible-objdump-demangle-styles\n- add crypto commands\n - see the `node-forge` package APIs\n- add decompress commands\n - see the `fflate` package APIs, started in `code/action/archive/loc/browser.ts`\n- add compress commands\n - fflate\n\n### TODO (Choco)\n\n- add support for\n - swift\n - clang-format\n - rustfmt\n - asmfmt\n - shfmt\n - the pip installed commands\n - rubocop\n\nCompare `load/choco/base.nuspec` with the `Dockerfile` to see all what's missing. If you know how to install those, please feel free to add.\n\n### TODO (Ubuntu)\n\n- create an [ubuntu package](https://earthly.dev/blog/creating-and-hosting-your-own-deb-packages-and-apt-repo/)\n\n## License\n\nMIT\n\n## TermSurf\n\nThis is being developed by the folks at [TermSurf](https://term.surf), a\nCalifornia-based project for helping humanity master information and\ncomputation. Find us on [Twitter](https://twitter.com/termsurf),\n[LinkedIn](https://www.linkedin.com/company/termsurf), and\n[Facebook](https://www.facebook.com/termsurf). Check out our other\n[GitHub projects](https://github.com/termsurf) as well!\n","readmeFilename":"readme.md"}