Commit Graph

174 Commits

Author SHA1 Message Date
कारतोफ्फेलस्क्रिप्ट™
b6c8a0c413 refactor(core): Reduce code duplication in DB config (no-changelog) (#8679) 2024-02-20 14:28:53 +01:00
कारतोफ्फेलस्क्रिप्ट™
a2a3ca160f fix(core): Improve the startup error when EXECUTIONS_PROCESS is set (#8630)
Co-authored-by: Omar Ajoue <krynble@gmail.com>
2024-02-14 15:35:42 +01:00
कारतोफ्फेलस्क्रिप्ट™
8e392cfc1d feat(core): Migrate to n8n's typeorm fork (#8590) 2024-02-08 15:13:29 +01:00
कारतोफ्फेलस्क्रिप्ट™
c04f92f7fd refactor(core): Couple of refactors on WorkflowRunner and ActiveExecutions (no-changelog) (#8487) 2024-02-06 18:09:50 +01:00
Iván Ovejero
c4e39451db refactor(core): Continue breaking dependency cycles (no-changelog) (#8545) 2024-02-06 10:08:46 +01:00
Iván Ovejero
28aee7c425 refactor(core): Lock webhook process out of multi-main setup (no-changelog) (#8549) 2024-02-05 18:06:54 +01:00
Iván Ovejero
dc5ec8f946 refactor(core): Streamline flows in multi-main mode (no-changelog) (#8446) 2024-02-05 09:26:55 +01:00
Iván Ovejero
cc41fc7c80 fix(core): Fix new graceful shutdown env being always overridden by deprecated env (#8503) 2024-02-01 11:10:40 +01:00
कारतोफ्फेलस्क्रिप्ट™
121a55b691 feat(core): Remove own execution-process mode (#8490) 2024-01-30 12:51:40 +01:00
कारतोफ्फेलस्क्रिप्ट™
5cb55270b7 refactor(core): Replace promisify-d node calls with native promises (no-changelog) (#8464) 2024-01-30 09:49:23 +01:00
कारतोफ्फेलस्क्रिप्ट™
c70fa66e76 refactor(core): Use DI for WorkflowRunner (no-changelog) (#8372) 2024-01-26 13:49:39 +01:00
कारतोफ्फेलस्क्रिप्ट™
7c49004018 refactor(core): Use DI for eventBus code - Part 1 (no-changelog) (#8434) 2024-01-26 12:21:15 +01:00
कारतोफ्फेलस्क्रिप्ट™
cc2f0ada76 ci: Enforce noUnusedLocals on all backend and nodes packages (no-changelog) (#8428) 2024-01-25 14:33:35 +01:00
कारतोफ्फेलस्क्रिप्ट™
d6deceacde refactor(core): Remove roleId indirection (no-changelog) (#8413) 2024-01-24 13:38:57 +01:00
Danny Martini
ae06fdeb62 fix(core): Fix update workflow cli command being unable to activate all workflows (#8412)
Co-authored-by: Daniel Schröder <daniel.schroeder@skriptfabrik.com>
2024-01-23 10:59:06 +01:00
कारतोफ्फेलस्क्रिप्ट™
913c8c6b0c feat(core): Upgrade oclif (no-changelog) (#8381) 2024-01-22 18:25:36 +01:00
Iván Ovejero
f35d4fcbd8 refactor(core): Simplify OrchestrationService (no-changelog) (#8364) 2024-01-22 11:16:29 +01:00
Tomi Turtiainen
9a1cc56806 fix: Set '@typescript-eslint/return-await' rule to 'always' for node code (no-changelog) (#8363)
Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2024-01-17 17:08:50 +02:00
Iván Ovejero
d4c93b1607 fix(core): Prevent issues with missing or mismatching encryption key (#8332) 2024-01-16 18:25:53 +01:00
कारतोफ्फेलस्क्रिप्ट™
64ceb16af6 refactor(core): Use DI in PermissionChecker (no-changelog) (#8344) 2024-01-16 14:15:29 +01:00
Omar Ajoue
e1acb5911a refactor: Make execution IDs mandatory in BE (#8299)
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
2024-01-16 09:53:17 +00:00
Tomi Turtiainen
3b996a7da0 feat(core): Validate shutdown handlers on startup (#8260) 2024-01-08 17:46:45 +02:00
Iván Ovejero
40c1eeeddd refactor(core): Continue moving typeorm operators to repositories (no-changelog) (#8186)
Follow-up to: #8163
2024-01-02 17:53:24 +01:00
Iván Ovejero
e418d42450 refactor(core): Move typeorm operators from various sources into repositories (no-changelog) (#8174)
Follow-up to: #8165
2023-12-28 13:14:10 +01:00
कारतोफ्फेलस्क्रिप्ट™
f69ddcd796 refactor(core): Use Dependency Injection for all Controller classes (no-changelog) (#8146)
## Review / Merge checklist
- [x] PR title and summary are descriptive
2023-12-27 11:50:43 +01:00
कारतोफ्फेलस्क्रिप्ट™
baee47a276 refactor(core): Move all base URLs to UrlService (no-changelog) (#8141)
This change kept coming up in #6713, #7773, and #8135. 
So this PR moves the existing code without actually changing anything,
to help get rid of some of the circular dependencies.


## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-22 15:19:50 +01:00
Tomi Turtiainen
3a881be6c2 feat(core): Unify application components shutdown (#8097)
## Summary

Add `ShutdownService` and `OnShutdown` decorator for more unified way to
shutdown different components. Use this new way in the following
components:

- HTTP(S) server
- Pruning service
- Push connection
- License

---------

Co-authored-by: कारतोफ्फेलस्क्रिप्ट™ <aditya@netroy.in>
2023-12-22 12:39:58 +02:00
कारतोफ्फेलस्क्रिप्ट™
c158ca2471 refactor(core): Upgrade more dependencies to remove axios 0.x (no-changelog) (#8105)
Had to [fork localtunnel](https://github.com/n8n-io/localtunnel) to get
the axios upgrade, since localtunnel doesn't seem to be actively maintained.
2023-12-22 11:39:20 +01:00
कारतोफ्फेलस्क्रिप्ट™
464b565283 ci: Remove unnecessary async/await, enable await-thenable linting rule (no-changelog) (#8076)
## Summary
We accidentally made some functions `async` in
https://github.com/n8n-io/n8n/pull/7846
This PR reverts that change. 

## Review / Merge checklist
- [x] PR title and summary are descriptive.
2023-12-19 13:52:42 +01:00
Tomi Turtiainen
614f488386 feat(core): Add N8N_GRACEFUL_SHUTDOWN_TIMEOUT env var (#8068)
Add generic N8N_GRACEFUL_SHUTDOWN_TIMEOUT which controls how long n8n
process will wait for graceful exit before exitting forcefully. This
variables replaces the QUEUE_WORKER_TIMEOUT variable that was used for
worker process.

DEPRECATED: QUEUE_WORKER_TIMEOUT deprected

QUEUE_WORKER_TIMEOUT environment variable has been replaced with
N8N_GRACEFUL_SHUTDOWN_TIMEOUT.
2023-12-18 14:04:19 +02:00
Tomi Turtiainen
4cae976a3b fix(core): Consider timeout in shutdown an error (#8050)
If the process doesn't shutdown within a time limit, exit with error
code.

1. conceptually something timing out is an error.
2. on successful exit we close down the DB connection gracefully. On an
exit timeout we rather not do that, since it will wait for any active
connections to close and would possible block the exit.
2023-12-18 10:53:34 +02:00
Tomi Turtiainen
6ae2f5efea fix(core): Fix shutdown if terminating before hooks are initialized (#8047)
If the app receives termination signal before hooks have been
initialised, the would be objet is undefined error. This PR fixes that.
2023-12-18 09:23:10 +02:00
Tomi Turtiainen
67bd8ad698 fix(core): Handle multiple termination signals correctly (#8046)
Prevent possible multiple termination signals initiating the shutdown
process multiple times.
2023-12-15 17:35:22 +02:00
Tomi Turtiainen
e69707efd4 fix(core): Close db connection gracefully when exiting (#8045)
Close db connection gracefully when exiting
2023-12-15 17:16:35 +02:00
Iván Ovejero
2da15d0264 refactor(core): Remove ID-less workflow reporting (no-changelog) (#8038)
We added ID-less workflow reporting at #8031, which has already produced
multiple reports coming from internal, enough info to tackle [this
story](https://linear.app/n8n/issue/PAY-1147). To prevent an
overwhelming number of reports from cloud, this PR removes the reporting
for now.
2023-12-15 12:50:09 +01:00
Iván Ovejero
f18bc5f4b7 refactor(core): Warn on sqlite DB detected during init on queue mode (#8034)
When setting up queue mode, it is easy to overlook that not exporting
Postgres env vars will default the worker to use sqlite, which will fail
during execution with a non-obvious error. Hence add warnings when
starting a worker with an incompatible DB type.
2023-12-15 10:56:35 +01:00
Iván Ovejero
c5e6ba8cdd fix(core): Restore workflow ID during execution creation (#8031)
## Summary
Restore workflow ID during execution creation removed by [this
PR](https://github.com/n8n-io/n8n/pull/8002/files#diff-c8cbb62ca9ab2ae45e5f565cd8c63fff6475809a6241ea0b90acc575615224af).
The missing workflow ID, and more generally the fact that `workflow.id`
is optional when it should not be, causes `PermissionChecker.check` to
misreport a credential as inaccessible when it should be accessible.

More generally, start reporting ID-less workflows so we can root them
out and prevent this at type level.

## Related tickets and issues

https://n8nio.slack.com/archives/C035KBDA917/p1702539465555529
2023-12-14 18:13:12 +01:00
Iván Ovejero
53c0b49d15 fix(core): Initialize queue once in queue mode (#8025)
We're initializing the queue twice because of a [bad
merge](2c63474538).
No associated known bugs but no need to init the queue twice. We should
follow up by investigating if any pending bugs can be associated to
this.
2023-12-14 16:16:12 +01:00
Iván Ovejero
7b5d0a9546 refactor(core): Introduce import service (no-changelog) (#8001)
Consolidate import logic into import service.

Also fixes:
- https://linear.app/n8n/issue/PAY-1086
- https://github.com/n8n-io/n8n/issues/7881
-
https://community.n8n.io/t/cli-workflow-imports-failing-after-upgrade-to-v1-18-0/33780
- https://linear.app/n8n/issue/PAY-221
- https://github.com/n8n-io/n8n/issues/5477
- https://community.n8n.io/t/export-workflows-with-tags-got-created/6161
2023-12-13 10:00:21 +01:00
Iván Ovejero
f49aebb740 refactor(core): Move instance owner retrieval to ownership service (no-changelog) (#7980)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.
Continue breaking down `UserManagementHelper.ts`
...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-11 13:35:09 +01:00
Iván Ovejero
c48850d74f fix(core): Fix expression evaluator filename typo (no-changelog) (#7960)
## Summary
Provide details about your pull request and what it adds, fixes, or
changes. Photos and videos are recommended.

...

#### How to test the change:
1. ...


## Issues fixed
Include links to Github issue or Community forum post or **Linear
ticket**:
> Important in order to close automatically and provide context to
reviewers

...


## Review / Merge checklist
- [ ] PR title and summary are descriptive. **Remember, the title
automatically goes into the changelog. Use `(no-changelog)` otherwise.**
([conventions](https://github.com/n8n-io/n8n/blob/master/.github/pull_request_title_conventions.md))
- [ ] [Docs updated](https://github.com/n8n-io/n8n-docs) or follow-up
ticket created.
- [ ] Tests included.
> A bug is not considered fixed, unless a test is added to prevent it
from happening again. A feature is not complete without tests.
  >
> *(internal)* You can use Slack commands to trigger [e2e
tests](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#a39f9e5ba64a48b58a71d81c837e8227)
or [deploy test
instance](https://www.notion.so/n8n/How-to-use-Test-Instances-d65f49dfc51f441ea44367fb6f67eb0a?pvs=4#f6a177d32bde4b57ae2da0b8e454bfce)
or [deploy early access version on
Cloud](https://www.notion.so/n8n/Cloudbot-3dbe779836004972b7057bc989526998?pvs=4#fef2d36ab02247e1a0f65a74f6fb534e).
2023-12-07 17:50:49 +01:00
Iván Ovejero
07f6662aba refactor(core): Add log about removed TTL keys for binary data (#7892)
Context:
https://n8nio.slack.com/archives/C035KBDA917/p1701418556761549?thread_ts=1701411854.517989&cid=C035KBDA917
2023-12-01 12:51:45 +01:00
Iván Ovejero
c08c5cc37b refactor(core): Switch plain errors in cli to ApplicationError (#7857)
Ensure all errors in `cli` are `ApplicationError` or children of it and
contain no variables in the message, to continue normalizing all the
errors we report to Sentry

Follow-up to: https://github.com/n8n-io/n8n/pull/7839
2023-11-29 12:25:10 +01:00
Iván Ovejero
1c6178759c refactor(core): Reorganize error hierarchy in cli package (no-changelog) (#7839)
Ensure all errors in `cli` inherit from `ApplicationError` to continue
normalizing all the errors we report to Sentry

Follow-up to: https://github.com/n8n-io/n8n/pull/7820
2023-11-28 10:19:27 +01:00
Csaba Tuncsik
152883eed1 build: Upgrade lint related packages (#7790) 2023-11-27 09:11:52 +01:00
Iván Ovejero
fdb2c18ecc fix(core): Tighten checks for multi-main setup usage (#7788)
https://n8nio.slack.com/archives/C05HRPLSGTT/p1700731476321999?thread_ts=1700729359.746899&cid=C05HRPLSGTT
2023-11-23 12:18:39 +01:00
Iván Ovejero
4c4082503c feat(core): Coordinate manual workflow activation and deactivation in multi-main scenario (#7643)
Followup to #7566 | Story: https://linear.app/n8n/issue/PAY-926

### Manual workflow activation and deactivation

In a multi-main scenario, if the user manually activates or deactivates
a workflow, the process (whether leader or follower) that handles the
PATCH request and updates its internal state should send a message into
the command channel, so that all other main processes update their
internal state accordingly:

- Add to `ActiveWorkflows` if activating
- Remove from `ActiveWorkflows` if deactivating
- Remove and re-add to `ActiveWorkflows` if the update did not change
activation status.

After updating their internal state, if activating or deactivating, the
recipient main processes should push a message to all connected
frontends so that these can update their stores and so reflect the value
in the UI.

### Workflow activation errors

On failure to activate a workflow, the main instance should record the
error in Redis - main instances should always pull activation errors
from Redis in a multi-main scenario.

### Leadership change

On leadership change...

- The old leader should stop pruning and the new leader should start
pruning.
- The old leader should remove trigger- and poller-based workflows and
the new leader should add them.
2023-11-17 15:58:50 +01:00
कारतोफ्फेलस्क्रिप्ट™
3460eb5eeb fix(core): Initialize JWT Secret before it's used anywhere (#7707)
HELP-394
2023-11-15 12:17:18 +01:00
Iván Ovejero
b2ca050031 perf(core): Lazyload security audit reporters (#7696)
Also converting to service.

Followup to https://github.com/n8n-io/n8n/pull/7663
2023-11-13 11:50:43 +01:00
कारतोफ्फेलस्क्रिप्ट™
000e76e3b4 ci(core): Reduce memory usage in tests (part-2) (no-changelog) (#7671)
This also gets rid of `Db.collection`, which was another source of
circular dependencies.
2023-11-10 15:04:26 +01:00