Skip to main content

缓存的工作原理

¥How Caching Works

在运行任何任务之前,Lerna 都会计算其计算哈希值。只要计算哈希相同,运行任务的输出就相同。

¥Before running any task, Lerna computes its computation hash. As long as the computation hash is the same, the output of running the task is the same.

默认情况下,计算哈希为 - say - lerna run test --scope=remixapp 包括:

¥By default, the computation hash for - say - lerna run test --scope=remixapp includes:

  • remixapp 的所有源文件及其依赖

    ¥All the source files of remixapp and its dependencies

  • 相关全局配置

    ¥Relevant global configuration

  • 外部依赖的版本

    ¥Versions of external dependencies

  • 用户配置的运行时值,例如 Node 的版本

    ¥Runtime values provisioned by the user such as the version of Node

  • CLI 命令标志

    ¥CLI Command flags

computation-hashing

此行为是可定制的。例如,lint 检查可能仅依赖于项目的源代码和全局配置。构建可以依赖于已编译库的 dts 文件而不是其源。

¥This behavior is customizable. For instance, lint checks may only depend on the source code of the project and global configs. Builds can depend on the dts files of the compiled libs instead of their source.

Lerna 计算任务的哈希值后,它会检查之前是否运行过这个精确的计算。首先,它在本地进行检查,然后如果丢失,如果配置了远程缓存,则进行远程检查。

¥After Lerna computes the hash for a task, it then checks if it ran this exact computation before. First, it checks locally, and then if it is missing, and if a remote cache is configured, it checks remotely.

如果 Lerna 找到计算,Lerna 会检索它并重播它。Lerna 将正确的文件放入正确的文件夹中并打印终端输出。从用户的角度来看,命令运行相同,只是速度快得多。

¥If Lerna finds the computation, Lerna retrieves it and replays it. Lerna places the right files in the right folders and prints the terminal output. From the user’s point of view, the command ran the same, just a lot faster.

cache

如果 Lerna 没有找到相应的计算哈希,Lerna 就会运行该任务,完成后,它会获取输出和终端日志并将它们存储在本地(如果也远程配置)。所有这一切都是透明发生的,因此你不必担心。

¥If Lerna doesn’t find a corresponding computation hash, Lerna runs the task, and after it completes, it takes the outputs and the terminal logs and stores them locally (and if configured remotely as well). All of this happens transparently, so you don’t have to worry about it.

尽管从概念上讲这相当简单,但 Lerna 对其进行了优化,以使这种体验对你有利。以 Lerna 为例:

¥Although conceptually this is fairly straightforward, Lerna optimizes this to make this experience good for you. For instance, Lerna:

  • 捕获 stdout 和 stderr 以确保重放的输出看起来相同,包括在 Windows 上。

    ¥Captures stdout and stderr to make sure the replayed output looks the same, including on Windows.

  • 通过记住哪些文件在哪里重播来最小化 IO。

    ¥Minimizes the IO by remembering what files are replayed where.

  • 仅在处理大型任务图时显示相关输出。

    ¥Only shows relevant output when processing a large task graph.

  • 提供对缓存未命中进行故障排除的功能。以及许多其他优化。

    ¥Provides affordances for troubleshooting cache misses. And many other optimizations.

随着工作区的增长,任务图看起来更像这样:

¥As your workspace grows, the task graph looks more like this:

cache

所有这些优化对于使 Lerna 可用于任何重要的工作区至关重要。仅发生最少量的工作。其余部分要么保持原样,要么从缓存中恢复。

¥All of these optimizations are crucial for making Lerna usable for any non-trivial workspace. Only the minimum amount of work happens. The rest is either left as is or restored from the cache.

源代码哈希输入

¥Source Code Hash Inputs

构建/测试应用或库的结果取决于该项目的源代码以及它所依赖的所有库的所有源代码(直接或间接)。

¥The result of building/testing an application or a library depends on the source code of that project and all the source codes of all the libraries it depends on (directly or indirectly).

默认情况下,Lerna 是保守的。比如说,运行时,lerna run test --scope=remixapp Lerna 将考虑 remixapp 目录中的所有文件以及 headerfooter 目录中的所有文件(remixapp 依赖)。这将导致不必要的缓存未命中。例如,我们知道更改 footer 的规范文件不会更改上述测试命令的结果。

¥By default, Lerna is conservative. When running, say, lerna run test --scope=remixapp Lerna will consider all the files in the remixapp directory and all the files in the header and footer directories (remixapp dependencies). This would result in unnecessary cache misses. For instance, we know that changing a footer's spec file will not change the result of the test command above.

我们可以定义更精确的配置,如下所示:

¥We can define a more precise configuration as follows:

注意:"{projectRoot}" 和 "{workspaceRoot}" 是任务运行程序支持的特殊语法,当命令运行时,它们将在内部进行适当的插值。因此,你不应将 "{projectRoot}" 或 "{workspaceRoot}" 替换为固定路径,因为这会降低你的配置灵活性。

¥NOTE: "{projectRoot}" and "{workspaceRoot}" are special syntax supported by the task-runner, which will be appropriately interpolated internally when the command runs. You should therefore not replace "{projectRoot}" or "{workspaceRoot}" with fixed paths as this makes your configuration less flexible.

nx.json
{
"namedInputs": {
"default": ["{projectRoot}/**/*"],
"prod": ["!{projectRoot}/**/*.spec.tsx"]
},
"targetDefaults": {
"build": {
"inputs": ["prod", "^prod"]
},
"test": {
"inputs": ["default", "^prod", "{workspaceRoot}/jest.config.ts"]
}
}
}

通过此配置,构建脚本将仅考虑 remixappheaderfooter 的非测试文件。测试脚本将考虑被测项目的所有源文件以及其依赖的非测试文件。测试脚本还将考虑工作区根目录下的 jest 配置文件。

¥With this configuration, the build script will only consider the non-test files of remixapp, header and footer. The test script will consider all the source files for the project under test and only non-test files of its dependencies. The test script will also consider the jest config file at the root of the workspace.

运行时哈希输入

¥Runtime Hash Inputs

你的目标还可以取决于运行时值。

¥Your targets can also depend on runtime values.

nx.json
{
"targetDefaults": {
"build": {
"inputs": [{ "env": "MY_ENV_NAME" }, { "runtime": "node -v" }]
}
}
}

Args 哈希输入

¥Args Hash Inputs

最后,除了源代码哈希输入和运行时哈希输入之外,Lerna 还需要考虑参数:例如,lerna run build --scope=remixapplerna run build --scope=remixapp -- --flag=true 会产生不同的结果。

¥Finally, in addition to Source Code Hash Inputs and Runtime Hash Inputs, Lerna needs to consider the arguments: For example, lerna run build --scope=remixapp and lerna run build --scope=remixapp -- --flag=true produce different results.

请注意,只有传递给 npm 脚本本身的标志会影响计算结果。例如,从缓存的角度来看,以下命令是相同的。

¥Note, only the flags passed to the npm scripts itself affect results of the computation. For instance, the following commands are identical from the caching perspective.

npx lerna run build --scope=remixapp
npx lerna run build --ignore=header,footer

换句话说,Lerna 不会缓存开发者在终端中输入的内容。

¥In other words, Lerna does not cache what the developer types into the terminal.

如果你构建/测试/lint…多个项目,每个单独的构建都有自己的哈希值,并且将从缓存中检索或运行。这意味着从缓存的角度来看,以下命令:

¥If you build/test/lint… multiple projects, each individual build has its own hash value and will either be retrieved from cache or run. This means that from the caching point of view, the following command:

npx lerna run build --scope=header,footer

与以下两个命令相同:

¥is identical to the following two commands:

npx lerna run build --scope=header
npx lerna run build --scope=footer

什么是缓存

¥What is Cached

Lerna 在流程层面上工作。无论用于构建/测试/lint/等的工具如何。你的项目,结果被缓存。

¥Lerna works on the process level. Regardless of the tools used to build/test/lint/etc.. your project, the results are cached.

Lerna 在运行命令之前设置钩子来收集 stdout/stderr。所有输出都会被缓存,然后在缓存命中期间重播。

¥Lerna sets up hooks to collect stdout/stderr before running the command. All the output is cached and then replayed during a cache hit.

Lerna 还缓存命令生成的文件。文件/文件夹列表列在项目的 package.jsonoutputs 属性中:

¥Lerna also caches the files generated by a command. The list of files/folders is listed in the outputs property of the project's package.json:

注意:"{projectRoot}" 和 "{workspaceRoot}" 是任务运行程序支持的特殊语法,当命令运行时,它们将在内部进行适当的插值。因此,你不应将 "{projectRoot}" 或 "{workspaceRoot}" 替换为固定路径,因为这会降低你的配置灵活性。

¥NOTE: "{projectRoot}" and "{workspaceRoot}" are special syntax supported by the task-runner, which will be appropriately interpolated internally when the command runs. You should therefore not replace "{projectRoot}" or "{workspaceRoot}" with fixed paths as this makes your configuration less flexible.

E.g. packages/my-project/package.json
{
"nx": {
"targets": {
"build": {
"outputs": ["{projectRoot}/build", "{projectRoot}/public/build"]
}
}
}
}

如果给定目标的 outputs 属性未在项目的 package.json 文件中定义,Lerna 将查看 nx.jsontargetDefaults 部分:

¥If the outputs property for a given target isn't defined in the project' s package.json file, Lerna will look at the targetDefaults section of nx.json:

nx.json
{
...
"targetDefaults": {
"build": {
"dependsOn": [
"^build"
],
"outputs": [
"{projectRoot}/dist",
"{projectRoot}/build",
"{projectRoot}/public/build"
]
}
}
}

如果两者均未定义,Lerna 默认将 distbuild 缓存在存储库的根目录中。

¥If neither is defined, Lerna defaults to caching dist and build at the root of the repository.

跳过缓存

¥Skipping Cache

有时你想跳过缓存。例如,如果你正在测量命令的性能,则可以使用 --skip-nx-cache 标志来跳过检查计算缓存。

¥Sometimes you want to skip the cache. If, for example, you are measuring the performance of a command, you can use the --skip-nx-cache flag to skip checking the computation cache.

npx lerna run build --skip-nx-cache
npx lerna run test --skip-nx-cache

附加配置

¥Additional Configuration

有关配置任务和缓存的其他方法,请参阅相关的 NX 文档

¥For additional ways to configure tasks and caching, see the relevant Nx documentation.