feat: Make it possible to dynamically load community nodes (#2849)

* ✨ Make it possible to dynamically load node packages

* ⚡ Fix comment

* ✨ Make possible to dynamically install nodes from npm

* Created migration for sqlite regarding community nodes

* Saving to db whenever a package with nodes is installed

* Created endpoint to fetch installed packages

* WIP - uninstall package with nodes

* Fix lint issues

* Updating nodes via API

* Lint and improvement fixes

* Created community node helpers and removed packages taht do not contain nodes

* Check for package updates when fetching installed packages

* Blocked access to non-owner and preventing incorrect install of packages

* Added auto healing process

* Unit tests for helpers

* Finishing tests for helpers

* Improved unit tests, refactored more helpers and created integration tests for GET

* Implemented detection of missing packages on init and added warning to frontend settings

* Add check for banned packages and fix broken tests

* Create migrations for other db systems

* Updated with latest changes from master

* Fixed conflict errors

* Improved unit tests, refactored more helpers and created integration tests for GET

* Implemented detection of missing packages on init and added warning to frontend settings

* 🔥 Removing access check for the Settings sidebar item

* ✨ Added inital community nodes settings screen

* ⚡Added executionMode flag to settings

* ✨ Implemented N8N-callout component

* 💄Updating Callout component template propery names

* 💄 Updating Callout component styling.

* 💄Updating Callout component sizing and colors.

* ✔️ Updating Callout component test snapshots after styling changes

* ✨ Updating the `ActionBox` component so it supports callouts and conditional button rendering

* 💄 Removing duplicate callout theme validation in the `ActionBox` component. Adding a selection control for it in the storybook.

* ✨ Added warning message if instance is in the queue mode. Updated colors based on the new design.

* ⚡ Added a custom permission support to router

* 🔨 Implemented UM detection as a custom permission.

* 👌Updating route permission logic.

* ✨ Implemented installed community packages list in the settings view

* 👌 Updating settings routes rules and community nodes setting view.

* Allow installation of packages that failed to load

* 👌 Updating `ActionBox`, `CommuntyPackageCard` components and settings loading logic.

* 👌 Fixing community nodes loading state and sidebar icon spacing.

* ✨ Implemented loading skeletons for community package cards

* 👌 Handling errrors while loading installed package list. Updating spacing.

* 👌 Updating community nodes error messages.

* Added disable flag

* 🐛 Fixing a community nodes update detection bug when there are missing packages. (#3497)

* ✨ Added front-end support for community nodes feature flag

* ✨ Implemented community package installation modal dialog

* 💄 Community nodes installation modal updates: Moved links to constants and used them in translations, disabling inputs in loading state.

* ✨ Implemented community packages install flow

* Standardize error codes (#3501)

* Standardize error: 400 for request issues such as invalid package name and 500 for installation problems

* Fix http status code for when package is not found

* ✨ Implemented community package installation modal dialog

* 💄 Community nodes installation modal updates: Moved links to constants and used them in translations, disabling inputs in loading state.

* ✨ Implemented community packages install flow

* ✨ Updated error handling based on the response codes

* ✨ Implemented community package installation modal dialog

* ✨ Implemented community package uninstall flow.

* ✨ Finished update confirm modal UI

* 💄 Replaced community nodes tooltip image with the one exported from figma.

* ✨ Implemented community package update process

* ✨ Updating community nodes list after successful package update

* 🔒 Updating public API setting route to use new access rules. Updating express app definition in community nodes tests

* ✨ Implemented community package installation modal dialog

* 💄 Community nodes installation modal updates: Moved links to constants and used them in translations, disabling inputs in loading state.

* ✨ Implemented community packages install flow

* ✨ Updated error handling based on the response codes

* Change output for installation request

* Improve payload for update requests

* 👌 Updating community nodes install modal UI

* 👌 Updating community nodes confirm modal logic

* 👌 Refactoring community nodes confirm modal dialog

* 👌 Separating community nodes components loading states

* 💄 Updating community nodes install modal spacing.

* Fix behavior for installing already installed packages

* 💡 Commenting community nodes install process

* 🔥 Removing leftover commits of deleted Vue mutations

* ✨ Updated node list to identify community nodes and handle node name clash

* ✨ Implemented missing community node dialog.

* 💄 Updating n8n-tabs component to support tooltips

* ✨ Updating node details with community node details.

* 🔨 Using back-end response when updating community packages

* 👌 Updating tabs component and refactoring community nodes store mutations

* 👌 Adding community node flag to node type descriptions and using it to identify community nodes

* 👌 Hiding unnecessary elements from missing node details panel.

* 👌 Updating missing node type descriptions for custom and community nodes

* 👌 Updating community node package name detection logic

* 👌 Removing communityNode flag from node description

* ✨ Adding `force` flag to credentials fetching (#3527)

* ✨ Adding `force` flag to credentials fetching which can be used to skip check when loading credentials

* ✨ Forcing credentials loading when opening nodeView

* 👌 Minor updates to community nodes details panel

* tests for post endpoint

* duplicate comments

* Add Patch and Delete enpoints tests

* 🔒 Using `pageCategory`prop to assemble the list of settings routes instead of hard-coded array (#3562)

* 📈 Added front-end telemetry events for community nodes

* 📈 Updating community nodes telemetry events

* 💄 Updating community nodes settings UI elements based on product/design review

* 💄 Updating node view & node details view for community nodes based on product/design feedback

* 💄 Fixing community node text capitalisation

* ✨ Adding community node install error message under the package name input field

* Fixed and improved tests

* Fix lint issue

* feat: Migrated to npm release of riot-tmpl fork.

* 📈 Updating community nodes telemetry events based on the product review

* 💄 Updating community nodes UI based on the design feedback

* 🔀 Merging recent node draggable panels changes

* Implement self healing process

* Improve error messages for package name requirement and disk space

* 💄 Removing front-end error message override since appropriate response is available from the back-end

* Fix lint issues

* Fix installed node name

* 💄 Removed additional node name parsing

* 📈 Updating community nodes telemetry events

* Fix postgres migration for cascading nodes when package is removed

* Remove postman mock for banned packages

* 📈 Adding missing telemetry event for community node documentation click

* 🐛 Fixing community nodes UI bugs reported during the bug bash

* Fix issue with uninstalling packages not reflecting UI

* 🐛 Fixing a missing node type bug when trying to run a workflow.

* Improve error detection for installing packages

* 💄 Updating community nodes components styling and wording based on the product feedback

* Implement telemetry be events

* Add author name and email to packages

* Fix telemetry be events for community packages

* 📈 Updating front-end telemetry events with community nodes author data

* 💄 Updating credentials documentation link logic to handle community nodes credentials

* 🐛 Fixing draggable panels logic

* Fix duplicate wrong import

* 💄 Hiding community nodes credentials documentation links when they don't contain an absolute URL

* Fix issue with detection of missing packages

* 💄 Adding the `Docs` tab to community nodes

* 💄 Adding a failed loading indicator to community nodes list

* Prevent n8n from crashing on startup

* Refactor and improve code quality

* ⚡ Remove not needed depenedency

Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: Milorad Filipović <milorad@n8n.io>
Co-authored-by: Milorad FIlipović <miloradfilipovic19@gmail.com>
Co-authored-by: agobrech <ael.gobrecht@gmail.com>
Co-authored-by: Alex Grozav <alex@grozav.com>

packages/editor-ui/src/plugins/i18n/locales/en.json

@@ -2,11 +2,16 @@
 	"_reusableBaseText.cancel": "Cancel",
 	"_reusableBaseText.name": "Name",
 	"_reusableBaseText.save": "Save",
+	"_reusableDynamicText.readMore": "Read more",
+	"_reusableDynamicText.learnMore": "Learn more",
+	"_reusableDynamicText.moreInfo": "More info",
 	"_reusableDynamicText.oauth2.clientId": "Client ID",
 	"_reusableDynamicText.oauth2.clientSecret": "Client Secret",
 	"generic.learnMore": "Learn more",
 	"generic.confirm": "Confirm",
 	"generic.cancel": "Cancel",
+	"generic.communityNode": "Community Node",
+	"generic.communityNode.tooltip": "This is a node from our community. It's part of the {packageName} package. <a href=\"{docURL}\" target=\"_blank\" title=\"Read the n8n docs\">Learn more</a>",
 	"generic.delete": "Delete",
 	"generic.copy": "Copy",
 	"generic.clickToCopy": "Click to copy",
@@ -445,11 +450,16 @@
 	"nodeSettings.notesInFlow.description": "If active, the note above will display in the flow as a subtitle",
 	"nodeSettings.notesInFlow.displayName": "Display Note in Flow?",
 	"nodeSettings.parameters": "Parameters",
+	"nodeSettings.communityNodeTooltip": "This is a <a href=\"{docUrl}\" target=\"_blank\"/>community node</a>",
 	"nodeSettings.retryOnFail.description": "If active, the node tries to execute again when it fails",
 	"nodeSettings.retryOnFail.displayName": "Retry On Fail",
 	"nodeSettings.scopes.expandedNoticeWithScopes": "<a data-key=\"toggle-expand\">{count} scope</a> available for {activeCredential} credentials<br>{scopes}<br><a data-key=\"show-less\">Show less</a> | <a data-key=\"toggle-expand\">{count} scopes</a> available for {activeCredential} credentials<br>{scopes}<br><a data-key=\"show-less\">Show less</a>",
 	"nodeSettings.scopes.notice": "<a data-key=\"toggle-expand\">{count} scope</a> available for {activeCredential} credentials | <a data-key=\"toggle-expand\">{count} scopes</a> available for {activeCredential} credentials",
 	"nodeSettings.theNodeIsNotValidAsItsTypeIsUnknown": "The node is not valid as its type ({nodeType}) is unknown",
+	"nodeSettings.communityNodeUnknown.title": "Install this node to use it",
+	"nodeSettings.communityNodeUnknown.description": "This node is not currently installed. It's part of the <a href=\"https://www.npmjs.com/package/{packageName}\" target=\"_blank\"/>{packageName}</a> community package.",
+	"nodeSettings.communityNodeUnknown.installLink.text": "How to install community nodes",
+	"nodeSettings.nodeTypeUnknown.description": "This node is not currently installed. It is either from a newer version of n8n, a <a href=\"{docURL}\" target=\"_blank\"/>custom node</a>, or has an invalid structure",
 	"nodeSettings.thisNodeDoesNotHaveAnyParameters": "This node does not have any parameters",
 	"nodeSettings.useTheHttpRequestNode": "Use the <b>HTTP Request</b> node to make a custom API call. We'll take care of the {nodeTypeDisplayName} auth for you. <a target=\"_blank\" href=\"https://docs.n8n.io/integrations/custom-operations/\">Learn more</a>",
 	"nodeSettings.waitBetweenTries.description": "How long to wait between each attempt (in milliseconds)",
@@ -658,6 +668,46 @@
 	"saveButton.saved": "Saved",
 	"saveButton.saving": "Saving",
 	"settings": "Settings",
+	"settings.communityNodes": "Community nodes",
+	"settings.communityNodes.empty.title": "Supercharge your workflows with community nodes",
+	"settings.communityNodes.empty.description": "Install over {count} node packages contributed by our community. <br /><a href=\"{docURL}\" target=\"_blank\" title=\"Read the n8n docs\">More info</a>",
+	"settings.communityNodes.empty.description.no-packages": "Install node packages contributed by our community. <br /><a href=\"{docURL}\" target=\"_blank\" title=\"Read the n8n docs\">More info</a>",
+	"settings.communityNodes.empty.installPackageLabel": "Install a community node",
+	"settings.communityNodes.queueMode.warning": "You need to install community nodes manually because your instance is running in queue mode. <a href=\"{docURL}\" target=\"_blank\" title=\"Read the n8n docs\">More info</a>",
+	"settings.communityNodes.packageNodes.label": "{count} node | {count} nodes",
+	"settings.communityNodes.updateAvailable.tooltip": "A newer version is available",
+	"settings.communityNodes.viewDocsAction.label": "Documentation",
+	"settings.communityNodes.uninstallAction.label": "Uninstall package",
+	"settings.communityNodes.upToDate.tooltip": "You are up to date",
+	"settings.communityNodes.failedToLoad.tooltip": "There is a problem with this package, try uninstalling it then reinstalling to resolve this issue",
+	"settings.communityNodes.fetchError.title": "Problem fetching installed packages",
+	"settings.communityNodes.fetchError.message": "There may be a problem with your internet connection or your n8n instance",
+	"settings.communityNodes.installModal.title": "Install community nodes",
+	"settings.communityNodes.installModal.description": "Find community nodes to add on the npm public registry.",
+	"settings.communityNodes.browseButton.label": "Browse",
+	"settings.communityNodes.installModal.packageName.label": "npm Package Name",
+	"settings.communityNodes.installModal.packageName.tooltip": "<img src='/static/community_package_tooltip_img.png'/><p>This is the title of the package on <a href='{npmURL}'>npmjs.com</a></p><p>Install a specific version by adding it after @, e.g. <code>package-name@0.15.0</code></p>",
+	"settings.communityNodes.installModal.packageName.placeholder": "e.g. n8n-nodes-chatwork",
+	"settings.communityNodes.installModal.checkbox.label": "I understand the risks of installing unverified code from a public source.",
+	"settings.communityNodes.installModal.installButton.label": "Install",
+	"settings.communityNodes.installModal.installButton.label.loading": "Installing",
+	"settings.communityNodes.installModal.error.packageNameNotValid": "Package name must start with n8n-nodes-",
+	"settings.communityNodes.messages.install.success": "Package installed",
+	"settings.communityNodes.messages.install.error": "Error installing new package",
+	"settings.communityNodes.messages.uninstall.error": "Problem uninstalling package",
+	"settings.communityNodes.messages.uninstall.success.title": "Package uninstalled",
+	"settings.communityNodes.messages.update.success.title": "Package updated",
+	"settings.communityNodes.messages.update.success.message": "{packageName} updated to version {version}",
+	"settings.communityNodes.messages.update.error.title": "Problem updating package",
+	"settings.communityNodes.confirmModal.uninstall.title": "Uninstall package?",
+	"settings.communityNodes.confirmModal.uninstall.message": "Any workflows that use nodes from the {packageName} package won't be able to run. Are you sure?",
+	"settings.communityNodes.confirmModal.uninstall.buttonLabel": "Uninstall package",
+	"settings.communityNodes.confirmModal.uninstall.buttonLoadingLabel": "Uninstalling",
+	"settings.communityNodes.confirmModal.update.title": "Update community node package?",
+	"settings.communityNodes.confirmModal.update.message": "You are about to update {packageName} to version {version}",
+	"settings.communityNodes.confirmModal.update.description": "We recommend you deactivate workflows that use any of the package's nodes and reactivate them once the update is completed",
+	"settings.communityNodes.confirmModal.update.buttonLabel": "Update package",
+	"settings.communityNodes.confirmModal.update.buttonLoadingLabel": "Updating...",
 	"settings.goBack": "Go back",
 	"settings.personal": "Personal",
 	"settings.personal.basicInformation": "Basic Information",

packages/editor-ui/src/plugins/icons.ts

@@ -28,6 +28,7 @@ import {
 	faCloud,
 	faCloudDownloadAlt,
 	faCopy,
+	faCube,
 	faCut,
 	faDotCircle,
 	faEdit,
@@ -92,6 +93,7 @@ import {
 	faTerminal,
 	faThLarge,
 	faTimes,
+	faTimesCircle,
 	faTrash,
 	faUndo,
 	faUnlink,
@@ -136,6 +138,7 @@ addIcon(faClone);
 addIcon(faCloud);
 addIcon(faCloudDownloadAlt);
 addIcon(faCopy);
+addIcon(faCube);
 addIcon(faCut);
 addIcon(faDotCircle);
 addIcon(faGripVertical);
@@ -202,6 +205,7 @@ addIcon(faTasks);
 addIcon(faTerminal);
 addIcon(faThLarge);
 addIcon(faTimes);
+addIcon(faTimesCircle);
 addIcon(faTrash);
 addIcon(faUndo);
 addIcon(faUnlink);