+2 votes
in Apache Storm by
reopened by
What makes a running topology: worker processes, executors and tasks?

1 Answer

0 votes
by
selected by
 
Best answer

Storm distinguishes between the following three main entities that are used to actually run a topology in a Storm cluster:

  1. Worker processes
  2. Executors (threads)
  3. Tasks

A worker process executes a subset of a topology. A worker process belongs to a specific topology and may run one or more executors for one or more components (spouts or bolts) of this topology. A running topology consists of many such processes running on many machines within a Storm cluster.

An executor is a thread that is spawned by a worker process. It may run one or more tasks for the same component (spout or bolt).

A task performs the actual data processing — each spout or bolt that you implement in your code executes as many tasks across the cluster. The number of tasks for a component is always the same throughout the lifetime of a topology, but the number of executors (threads) for a component can change over time.

Related questions

+2 votes
asked Jul 1, 2021 in Apache Storm by rajeshsharma
+2 votes
asked Jul 2, 2021 in Apache Storm by rajeshsharma
...