mohiit1502/Automata
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat(core): Coordinate manual workflow activation and deactivation in multi-main scenario (#7643)

Browse Source 
Followup to #7566 | Story: https://linear.app/n8n/issue/PAY-926

### Manual workflow activation and deactivation

In a multi-main scenario, if the user manually activates or deactivates
a workflow, the process (whether leader or follower) that handles the
PATCH request and updates its internal state should send a message into
the command channel, so that all other main processes update their
internal state accordingly:

- Add to `ActiveWorkflows` if activating
- Remove from `ActiveWorkflows` if deactivating
- Remove and re-add to `ActiveWorkflows` if the update did not change
activation status.

After updating their internal state, if activating or deactivating, the
recipient main processes should push a message to all connected
frontends so that these can update their stores and so reflect the value
in the UI.

### Workflow activation errors

On failure to activate a workflow, the main instance should record the
error in Redis - main instances should always pull activation errors
from Redis in a multi-main scenario.

### Leadership change

On leadership change...

- The old leader should stop pruning and the new leader should start
pruning.
- The old leader should remove trigger- and poller-based workflows and
the new leader should add them.
This commit is contained in:
Iván Ovejero
2023-11-17 15:58:50 +01:00
committed by GitHub
parent b3a3f16bc2
commit 4c4082503c
33 changed files with 639 additions and 336 deletions
Download Patch File Download Diff File Expand all files Collapse all files

55
packages/cli/test/integration/commands/start.cmd.test.ts
View File

@@ -1,55 +0,0 @@
import * as Config from '@oclif/config';
import { DataSource } from 'typeorm';
import { Start } from '@/commands/start';
import { BaseCommand } from '@/commands/BaseCommand';
import config from '@/config';
import { License } from '@/License';
import { ExternalSecretsManager } from '@/ExternalSecrets/ExternalSecretsManager.ee';
import { MultiMainInstancePublisher } from '@/services/orchestration/main/MultiMainInstance.publisher.ee';
import { ActiveWorkflowRunner } from '@/ActiveWorkflowRunner';
import { WorkflowHistoryManager } from '@/workflows/workflowHistory/workflowHistoryManager.ee';
import { RedisService } from '@/services/redis.service';
import { RedisServicePubSubPublisher } from '@/services/redis/RedisServicePubSubPublisher';
import { RedisServicePubSubSubscriber } from '@/services/redis/RedisServicePubSubSubscriber';
import { OrchestrationHandlerMainService } from '@/services/orchestration/main/orchestration.handler.main.service';
import { mockInstance } from '../../shared/mocking';
const oclifConfig: Config.IConfig = new Config.Config({ root: __dirname });
beforeAll(() => {
mockInstance(DataSource);
mockInstance(ExternalSecretsManager);
mockInstance(ActiveWorkflowRunner);
mockInstance(WorkflowHistoryManager);
mockInstance(RedisService);
mockInstance(RedisServicePubSubPublisher);
mockInstance(RedisServicePubSubSubscriber);
mockInstance(MultiMainInstancePublisher);
mockInstance(OrchestrationHandlerMainService);
});
afterEach(() => {
config.load(config.default);
jest.restoreAllMocks();
});
test('should not init license if instance is follower in multi-main scenario', async () => {
config.set('executions.mode', 'queue');
config.set('endpoints.disableUi', true);
config.set('leaderSelection.enabled', true);
jest.spyOn(MultiMainInstancePublisher.prototype, 'isFollower', 'get').mockReturnValue(true);
jest.spyOn(BaseCommand.prototype, 'init').mockImplementation(async () => {});
const licenseMock = mockInstance(License, {
isMultipleMainInstancesLicensed: jest.fn().mockReturnValue(true),
});
const startCmd = new Start([], oclifConfig);
await startCmd.init();
expect(licenseMock.init).not.toHaveBeenCalled();
});

2
packages/cli/test/integration/commands/worker.cmd.test.ts
View File

@@ -16,6 +16,7 @@ import { PostHogClient } from '@/posthog';
import { RedisService } from '@/services/redis.service';
import { OrchestrationHandlerWorkerService } from '@/services/orchestration/worker/orchestration.handler.worker.service';
import { OrchestrationWorkerService } from '@/services/orchestration/worker/orchestration.worker.service';
import { MultiMainSetup } from '@/services/orchestration/main/MultiMainSetup.ee';
import { mockInstance } from '../../shared/mocking';
@@ -37,6 +38,7 @@ beforeAll(async () => {
mockInstance(RedisService);
mockInstance(RedisServicePubSubPublisher);
mockInstance(RedisServicePubSubSubscriber);
mockInstance(MultiMainSetup);
});
test('worker initializes all its components', async () => {