Skip to main content

分布式任务执行指南

¥Distributed Task Execution Guide

本指南中的插图由 Nrwlian Nicole Oliver 创建

¥The illustrations in this guide are created by Nrwlian Nicole Oliver

how does distributed task execution work in Nx Cloud?

什么是任务?

¥What's a Task?

what's a task? project + target (i.e. shared-product-ui + test).  each run contains many tasks.  affected contains shared-product-ui, product-page, shared-e2e-util and shared-ui

从 Lerna 的角度来看,任务是项目上运行的目标。即在项目 shared-product-ui 上运行的目标 test 是一个任务。有关任务的更多信息,请参阅 运行任务文章

¥A task, from Lerna's perspective, is a target running on a project. i.e. The target test running on the project shared-product-ui is a task. For more information about tasks, see the Run Tasks article.

Nx Cloud 自动安排你的 CI 任务

¥Nx Cloud Schedules Your CI Tasks Automatically

when you turn on DTE, Nx will schedule your commands' tasks in CI.  for example, say you want to run these 3 commands to verify your code: nx affected --target=lint, nx affected --target=test and nx affected --target=build

让我们想象一下,对于 CI 中的每个 PR,你都希望检查、测试和构建所有受影响的项目。当你编写 CI 工作流程时,你无法知道每个 PR 会影响多少项目或每个任务需要多长时间。无论你设置得多么仔细,如果你手动分配静态数量的代理机器来进行检查、测试和构建,都会浪费时间。这种方法称为分箱。

¥Let's imagine for every PR in CI, you want to lint, test and build all affected projects. When you write your CI workflow, you have no way of knowing how many projects will be affected by each PR or how long each task will take. No matter how carefully you set things up, there will be wasted time if you manually assign a static number of agent machines for linting, testing and building. This approach is called binning.

幸运的是,通过分布式任务执行,Nx Cloud 可以在任务可用时动态地将任务分配给代理。

¥Luckily, with distributed task execution, Nx Cloud can dynamically assign tasks to agents as they become available.

Nx Cloud 高效协调代理

¥Nx Cloud Efficiently Orchestrates Agents

Nx will automatically schedule tasks w/agents you assign in CI.

设置 DTE 时,你可以定义 (1) 要运行的任务以及 (2) 可供 Nx Cloud 使用的代理数量。然后,Nx Cloud Orchestrator 将任务高效地分配给代理 - 以便所有代理都得到充分利用,并且你的 CI 流程尽快完成。

¥When you set up DTE, you define (1) the tasks that you want to run and (2) the number of agents that are available for Nx Cloud to use. Then the Nx Cloud orchestrator distributes tasks to agents efficiently - so that all the agents are being fully utilized and your CI process finishes as soon as possible.

任务执行顺序很重要

¥Task Execution Order Matters

but don't some tasks depend on others' results?  Yep! Nx knows about your dependency tree, so it will execute tasks in the right order and make sure the results are available where they're needed.

有些任务需要在其他任务之前执行,但 Nx Cloud 在向代理分配任务时会考虑到这一点。有关定义这些依赖的更详细信息,请阅读 运行任务文章

¥There are some tasks that need to be executed before other tasks, but Nx Cloud takes that into account when it assigns tasks to agents. For a more detailed look at defining those dependencies, read the Run Tasks article.

为什么要分配任务?

¥Why Distribute Tasks?

Result: Faster Builds!

跨多个代理有效地并行化 CI 流程可以显着加快 CI 速度,从而帮助开发者更快地发现问题并完成更多工作。

¥Efficiently parallelizing your CI process across many agents can dramatically speed up your CI, which helps developers identify problems faster and get more work done.

成本是多少?

¥What Does It Cost?

Nx Cloud 对于开源项目是免费的。联系 cloud-support@nrwl.io 进行设置。

¥Nx Cloud is FREE for open source projects. Contact cloud-support@nrwl.io to get set up.

对于闭源项目,每月前 500 个计算小时是免费的。大多数工作区不会超过这个数量。不需要信用卡。500 小时后,每计算小时的成本为 1 美元。

¥For closed source projects, the first 500 computation hours per month are free. Most workspaces don't exceed this amount. No credit card is required. After 500 hours, the cost is $1 per computation hour.

欲了解更多详情,请参阅 Nx 云定价页面

¥For more details, see the Nx Cloud pricing page.

安全

¥Security

你的实际代码并不存储在云中,但任务的哈希输入和缓存结果却存储在云中。可以对该数据启用端到端加密,这样没有你的密钥,任何人都无法查看该信息。此外,如果你想在自己的服务器上托管 Nx Cloud,你可以注册 Nx Private Cloud。

¥Your actual code is not stored in the cloud, but the hashed inputs and cached results of your tasks are. It is possible to enable end to end encryption of that data so that no one can view that information without your key. Also, if you want to host Nx Cloud on your own servers, you can sign up for Nx Private Cloud.

示例

¥Example

这是一个示例存储库 显示了设置分布式任务执行是多么容易,显示了性能增益,并与分片/分箱进行了比较。

¥This is an example repo showing how easy it is to set up distributed task execution, showing the performance gains, and comparing to sharding/binning.

插图

¥Illustration

这是 Nicole Oliver 制作的完整图解说明页:

¥Here is the full illustrated explanation page that Nicole Oliver made:

How Does DTE Work Explainer