Initial commit to release

This commit is contained in:
Jan Oberhauser
2019-06-23 12:35:23 +02:00
commit 9cb9804eee
257 changed files with 42436 additions and 0 deletions

230
packages/cli/LICENSE Normal file
View File

@@ -0,0 +1,230 @@
“Commons Clause” License Condition v1.0
The Software is provided to you by the Licensor under the
License, as defined below, subject to the following condition.
Without limiting other conditions in the License, the grant
of rights under the License will not include, and the License
does not grant to you, the right to Sell the Software.
For purposes of the foregoing, “Sell” means practicing any or
all of the rights granted to you under the License to provide
to third parties, for a fee or other consideration (including
without limitation fees for hosting or consulting/ support
services related to the Software), a product or service whose
value derives, entirely or substantially, from the functionality
of the Software. Any license notice or attribution required by
the License must also include this Commons Clause License
Condition notice.
Software: n8n
License: Apache 2.0
Licensor: Jan Oberhauser
---------------------------------------------------------------------
Apache License
Version 2.0, January 2004
http://www.apache.org/licenses/
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
1. Definitions.
"License" shall mean the terms and conditions for use, reproduction,
and distribution as defined by Sections 1 through 9 of this document.
"Licensor" shall mean the copyright owner or entity authorized by
the copyright owner that is granting the License.
"Legal Entity" shall mean the union of the acting entity and all
other entities that control, are controlled by, or are under common
control with that entity. For the purposes of this definition,
"control" means (i) the power, direct or indirect, to cause the
direction or management of such entity, whether by contract or
otherwise, or (ii) ownership of fifty percent (50%) or more of the
outstanding shares, or (iii) beneficial ownership of such entity.
"You" (or "Your") shall mean an individual or Legal Entity
exercising permissions granted by this License.
"Source" form shall mean the preferred form for making modifications,
including but not limited to software source code, documentation
source, and configuration files.
"Object" form shall mean any form resulting from mechanical
transformation or translation of a Source form, including but
not limited to compiled object code, generated documentation,
and conversions to other media types.
"Work" shall mean the work of authorship, whether in Source or
Object form, made available under the License, as indicated by a
copyright notice that is included in or attached to the work
(an example is provided in the Appendix below).
"Derivative Works" shall mean any work, whether in Source or Object
form, that is based on (or derived from) the Work and for which the
editorial revisions, annotations, elaborations, or other modifications
represent, as a whole, an original work of authorship. For the purposes
of this License, Derivative Works shall not include works that remain
separable from, or merely link (or bind by name) to the interfaces of,
the Work and Derivative Works thereof.
"Contribution" shall mean any work of authorship, including
the original version of the Work and any modifications or additions
to that Work or Derivative Works thereof, that is intentionally
submitted to Licensor for inclusion in the Work by the copyright owner
or by an individual or Legal Entity authorized to submit on behalf of
the copyright owner. For the purposes of this definition, "submitted"
means any form of electronic, verbal, or written communication sent
to the Licensor or its representatives, including but not limited to
communication on electronic mailing lists, source code control systems,
and issue tracking systems that are managed by, or on behalf of, the
Licensor for the purpose of discussing and improving the Work, but
excluding communication that is conspicuously marked or otherwise
designated in writing by the copyright owner as "Not a Contribution."
"Contributor" shall mean Licensor and any individual or Legal Entity
on behalf of whom a Contribution has been received by Licensor and
subsequently incorporated within the Work.
2. Grant of Copyright License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
copyright license to reproduce, prepare Derivative Works of,
publicly display, publicly perform, sublicense, and distribute the
Work and such Derivative Works in Source or Object form.
3. Grant of Patent License. Subject to the terms and conditions of
this License, each Contributor hereby grants to You a perpetual,
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
(except as stated in this section) patent license to make, have made,
use, offer to sell, sell, import, and otherwise transfer the Work,
where such license applies only to those patent claims licensable
by such Contributor that are necessarily infringed by their
Contribution(s) alone or by combination of their Contribution(s)
with the Work to which such Contribution(s) was submitted. If You
institute patent litigation against any entity (including a
cross-claim or counterclaim in a lawsuit) alleging that the Work
or a Contribution incorporated within the Work constitutes direct
or contributory patent infringement, then any patent licenses
granted to You under this License for that Work shall terminate
as of the date such litigation is filed.
4. Redistribution. You may reproduce and distribute copies of the
Work or Derivative Works thereof in any medium, with or without
modifications, and in Source or Object form, provided that You
meet the following conditions:
(a) You must give any other recipients of the Work or
Derivative Works a copy of this License; and
(b) You must cause any modified files to carry prominent notices
stating that You changed the files; and
(c) You must retain, in the Source form of any Derivative Works
that You distribute, all copyright, patent, trademark, and
attribution notices from the Source form of the Work,
excluding those notices that do not pertain to any part of
the Derivative Works; and
(d) If the Work includes a "NOTICE" text file as part of its
distribution, then any Derivative Works that You distribute must
include a readable copy of the attribution notices contained
within such NOTICE file, excluding those notices that do not
pertain to any part of the Derivative Works, in at least one
of the following places: within a NOTICE text file distributed
as part of the Derivative Works; within the Source form or
documentation, if provided along with the Derivative Works; or,
within a display generated by the Derivative Works, if and
wherever such third-party notices normally appear. The contents
of the NOTICE file are for informational purposes only and
do not modify the License. You may add Your own attribution
notices within Derivative Works that You distribute, alongside
or as an addendum to the NOTICE text from the Work, provided
that such additional attribution notices cannot be construed
as modifying the License.
You may add Your own copyright statement to Your modifications and
may provide additional or different license terms and conditions
for use, reproduction, or distribution of Your modifications, or
for any such Derivative Works as a whole, provided Your use,
reproduction, and distribution of the Work otherwise complies with
the conditions stated in this License.
5. Submission of Contributions. Unless You explicitly state otherwise,
any Contribution intentionally submitted for inclusion in the Work
by You to the Licensor shall be under the terms and conditions of
this License, without any additional terms or conditions.
Notwithstanding the above, nothing herein shall supersede or modify
the terms of any separate license agreement you may have executed
with Licensor regarding such Contributions.
6. Trademarks. This License does not grant permission to use the trade
names, trademarks, service marks, or product names of the Licensor,
except as required for reasonable and customary use in describing the
origin of the Work and reproducing the content of the NOTICE file.
7. Disclaimer of Warranty. Unless required by applicable law or
agreed to in writing, Licensor provides the Work (and each
Contributor provides its Contributions) on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
implied, including, without limitation, any warranties or conditions
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
PARTICULAR PURPOSE. You are solely responsible for determining the
appropriateness of using or redistributing the Work and assume any
risks associated with Your exercise of permissions under this License.
8. Limitation of Liability. In no event and under no legal theory,
whether in tort (including negligence), contract, or otherwise,
unless required by applicable law (such as deliberate and grossly
negligent acts) or agreed to in writing, shall any Contributor be
liable to You for damages, including any direct, indirect, special,
incidental, or consequential damages of any character arising as a
result of this License or out of the use or inability to use the
Work (including but not limited to damages for loss of goodwill,
work stoppage, computer failure or malfunction, or any and all
other commercial damages or losses), even if such Contributor
has been advised of the possibility of such damages.
9. Accepting Warranty or Additional Liability. While redistributing
the Work or Derivative Works thereof, You may choose to offer,
and charge a fee for, acceptance of support, warranty, indemnity,
or other liability obligations and/or rights consistent with this
License. However, in accepting such obligations, You may act only
on Your own behalf and on Your sole responsibility, not on behalf
of any other Contributor, and only if You agree to indemnify,
defend, and hold each Contributor harmless for any liability
incurred by, or claims asserted against, such Contributor by reason
of your accepting any such warranty or additional liability.
END OF TERMS AND CONDITIONS
APPENDIX: How to apply the Apache License to your work.
To apply the Apache License to your work, attach the following
boilerplate notice, with the fields enclosed by brackets "[]"
replaced with your own identifying information. (Don't include
the brackets!) The text should be enclosed in the appropriate
comment syntax for the file format. We also recommend that a
file or class name and description of purpose be included on the
same "printed page" as the copyright notice for easier
identification within third-party archives.
Copyright [yyyy] [name of copyright owner]
Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.

47
packages/cli/README.md Normal file
View File

@@ -0,0 +1,47 @@
# n8n - Workflow Automation Tool
![n8n.io - Workflow Automation](https://n8n.io/n8n-logo.png)
n8n is a tool which allows to easily and fast automate different taks.
Is still in beta so can not guarantee that everything works perfectly. Also
is there currently not much documentation. That will hopefully change soon.
## Give n8n a spin
To simply spin up n8n to have a look and give it spin you can simply run:
```
npx n8n
```
It will then download everything which is needed and start n8n.
You can then access n8n by opening:
[http://localhost:5678](http://localhost:5678)
## Installation
To fully install n8n globally execute:
```
npm install n8n -g
```
After the installation n8n can be started by simply typing in:
```
n8n
```
## License
[Apache 2.0 with Commons Clause](LICENSE)
## Development
When developing n8n can be started with `npm run start:dev`.
It will then automatically restart n8n every time a file changes.

View File

@@ -0,0 +1,151 @@
import Vorpal = require('vorpal');
import { Args } from 'vorpal';
import { promises as fs } from 'fs';
import {
CredentialTypes,
Db,
IWorkflowBase,
LoadNodesAndCredentials,
NodeTypes,
GenericHelpers,
WorkflowHelpers,
WorkflowExecuteAdditionalData,
} from "../src";
import {
ActiveExecutions,
UserSettings,
WorkflowExecute,
} from "n8n-core";
import {
INode,
Workflow,
} from "n8n-workflow";
module.exports = (vorpal: Vorpal) => {
return vorpal
.command('run')
// @ts-ignore
.description('Executes a given workflow')
.option('--file <workflow-file>',
'The path to a workflow file to execute')
.option('--id <workflow-id>',
'The id of the workflow to execute')
.option('\n')
// TODO: Add validation
// .validate((args: Args) => {
// })
.action(async (args: Args) => {
// Start directly with the init of the database to improve startup time
const startDbInitPromise = Db.init();
// Load all node and credential types
const loadNodesAndCredentials = LoadNodesAndCredentials();
const loadNodesAndCredentialsPromise = loadNodesAndCredentials.init();
if (!args.options.id && !args.options.file) {
GenericHelpers.logOutput(`Either option "--id" or "--file" have to be set!`);
return Promise.resolve();
}
if (args.options.id && args.options.file) {
GenericHelpers.logOutput(`Either "id" or "file" can be set never both!`);
return Promise.resolve();
}
let workflowId: string | undefined;
let workflowData: IWorkflowBase | undefined = undefined;
if (args.options.file) {
// Path to workflow is given
try {
workflowData = JSON.parse(await fs.readFile(args.options.file, 'utf8'));
} catch (error) {
if (error.code === 'ENOENT') {
GenericHelpers.logOutput(`The file "${args.options.file}" could not be found.`);
return;
}
throw error;
}
// Do a basic check if the data in the file looks right
// TODO: Later check with the help of TypeScript data if it is valid or not
if (workflowData === undefined || workflowData.nodes === undefined || workflowData.connections === undefined) {
GenericHelpers.logOutput(`The file "${args.options.file}" does not contain valid workflow data.`);
return;
}
workflowId = workflowData.id!.toString();
}
// Wait till the database is ready
await startDbInitPromise;
if (args.options.id) {
// Id of workflow is given
workflowId = args.options.id;
workflowData = await Db.collections!.Workflow!.findOne(workflowId);
if (workflowData === undefined) {
GenericHelpers.logOutput(`The workflow with the id "${workflowId}" does not exist.`);
return;
}
}
// Make sure the settings exist
await UserSettings.prepareUserSettings();
// Wait till the n8n-packages have been read
await loadNodesAndCredentialsPromise;
// Add the found types to an instance other parts of the application can use
const nodeTypes = NodeTypes();
await nodeTypes.init(loadNodesAndCredentials.nodeTypes);
const credentialTypes = CredentialTypes();
await credentialTypes.init(loadNodesAndCredentials.credentialTypes);
if (!WorkflowHelpers.isWorkflowIdValid(workflowId)) {
workflowId = undefined;
}
const workflowInstance = new Workflow(workflowId, workflowData!.nodes, workflowData!.connections, true, nodeTypes, workflowData!.staticData);
// Check if the workflow contains the required "Start" node
// "requiredNodeTypes" are also defined in editor-ui/views/NodeView.vue
const requiredNodeTypes = ['n8n-nodes-base.start'];
let startNodeFound = false;
let node: INode;
for (const nodeName of Object.keys(workflowInstance.nodes)) {
node = workflowInstance.nodes[nodeName];
if (requiredNodeTypes.includes(node.type)) {
startNodeFound = true;
}
}
if (startNodeFound === false) {
// If the workflow does not contain a start-node we can not know what
// should be executed and with which data to start.
GenericHelpers.logOutput(`The workflow does not contain a "Start" node. So it can not be executed.`);
return Promise.resolve();
}
const mode = 'cli';
const additionalData = await WorkflowExecuteAdditionalData.get(mode, workflowData!, workflowInstance);
const workflowExecute = new WorkflowExecute(additionalData, mode);
try {
const executionId = await workflowExecute.run(workflowInstance);
const activeExecutions = ActiveExecutions.getInstance();
const data = activeExecutions.getPostExecutePromise(executionId);
console.log('Execution was successfull:');
console.log('====================================');
console.log(JSON.stringify(data, null, 2));
} catch (e) {
console.error('GOT ERROR');
console.log('====================================');
console.error(e);
return;
}
});
};

View File

@@ -0,0 +1,182 @@
import Vorpal = require('vorpal');
import { Args } from 'vorpal';
import { randomBytes } from 'crypto';
import * as config from 'config';
const open = require('open');
import * as localtunnel from 'localtunnel';
import {
ActiveWorkflowRunner,
CredentialTypes,
Db,
GenericHelpers,
LoadNodesAndCredentials,
NodeTypes,
TestWebhooks,
Server,
} from "../src";
import {
UserSettings,
} from "n8n-core";
import { promisify } from "util";
const tunnel = promisify(localtunnel);
let activeWorkflowRunner: ActiveWorkflowRunner.ActiveWorkflowRunner | undefined;
/**
* Opens the UI in browser
*
*/
function openBrowser() {
const editorUrl = GenericHelpers.getBaseUrl();
open(editorUrl, { wait: true })
.catch((error: Error) => {
console.log(`\nWas not able to open URL in browser. Please open manually by visiting:\n${editorUrl}\n`);
});
}
module.exports = (vorpal: Vorpal) => {
return vorpal
.command('start')
// @ts-ignore
.description('Starts n8n. Makes Web-UI available and starts active workflows')
.option('-o --open',
'Opens the UI automatically in browser')
.option('--tunnel',
'Runs the webhooks via a hooks.n8n.cloud tunnel server')
.option('\n')
// TODO: Add validation
// .validate((args: Args) => {
// })
.action((args: Args) => {
if (process.pid === 1) {
console.error(`The n8n node process should not run as process with ID 1 because that will cause
problems with shutting everything down correctly. If started with docker use the
flag "--init" to fix this problem!`);
return;
}
// TODO: Start here the the script in a subprocess which can get restarted when new nodes get added and so new packages have to get installed
// npm install / rm (in other process)
// restart process depending on exit code (lets say 50 means restart)
// Wrap that the process does not close but we can still use async
(async () => {
// Start directly with the init of the database to improve startup time
const startDbInitPromise = Db.init();
// Make sure the settings exist
const userSettings = await UserSettings.prepareUserSettings();
// Load all node and credential types
const loadNodesAndCredentials = LoadNodesAndCredentials();
await loadNodesAndCredentials.init();
// Add the found types to an instance other parts of the application can use
const nodeTypes = NodeTypes();
await nodeTypes.init(loadNodesAndCredentials.nodeTypes);
const credentialTypes = CredentialTypes();
await credentialTypes.init(loadNodesAndCredentials.credentialTypes);
// Wait till the database is ready
await startDbInitPromise;
if (args.options.tunnel !== undefined) {
console.log('\nWaiting for tunnel ...');
if (userSettings.tunnelSubdomain === undefined) {
// When no tunnel subdomain did exist yet create a new random one
const availableCharacters = 'abcdefghijklmnopqrstuvwxyz0123456789';
userSettings.tunnelSubdomain = Array.from({ length: 24 }).map(() => {
return availableCharacters.charAt(Math.floor(Math.random() * availableCharacters.length));
}).join('');
await UserSettings.writeUserSettings(userSettings);
}
const tunnelSettings: localtunnel.TunnelConfig = {
host: 'https://hooks.n8n.cloud',
subdomain: userSettings.tunnelSubdomain,
};
const port = config.get('urls.port') as number;
// @ts-ignore
const webhookTunnel = await tunnel(port, tunnelSettings);
process.env.WEBHOOK_TUNNEL_URL = webhookTunnel.url + '/';
console.log(`Tunnel URL: ${process.env.WEBHOOK_TUNNEL_URL}\n`);
}
Server.start();
// Start to get active workflows and run their triggers
activeWorkflowRunner = ActiveWorkflowRunner.getInstance();
await activeWorkflowRunner.init();
const editorUrl = GenericHelpers.getBaseUrl();
console.log(`\nEditor is now accessible via:\n${editorUrl}`);
// Allow to open n8n editor by pressing "o"
if (Boolean(process.stdout.isTTY) && process.stdin.setRawMode) {
process.stdin.setRawMode(true);
process.stdin.resume();
process.stdin.setEncoding('utf8');
let inputText = '';
if (args.options.browser !== undefined) {
openBrowser();
}
console.log(`\nPress "o" to open in Browser.`);
process.stdin.on("data", (key) => {
if (key === 'o') {
openBrowser();
inputText = '';
} else {
// When anything else got pressed, record it and send it on enter into the child process
if (key.charCodeAt(0) === 13) {
// send to child process and print in terminal
process.stdout.write('\n');
inputText = '';
} else {
// record it and write into terminal
inputText += key;
process.stdout.write(key);
}
}
});
}
})();
vorpal.sigint(async () => {
console.log(`\nStopping n8n...`);
setTimeout(() => {
// In case that something goes wrong with shutdown we
// kill after max. 30 seconds no matter what
process.exit();
}, 30000);
const removePromises = [];
if (activeWorkflowRunner !== undefined) {
removePromises.push(activeWorkflowRunner.removeAll());
}
// Remove all test webhooks
const testWebhooks = TestWebhooks.getInstance();
removePromises.push(testWebhooks.removeAll());
await Promise.all(removePromises);
process.exit();
});
});
};

View File

@@ -0,0 +1,30 @@
module.exports = {
urls: {
endpointRest: 'rest',
endpointWebhook: 'webhook',
endpointWebhookTest: 'webhook-test',
host: 'localhost',
port: 5678,
protocol: 'http',
},
database: {
type: 'sqlite', // Available types: sqlite, mongodb
// MongoDB specific settings
mongodbConfig: {
url: 'mongodb://user:password@localhost:27017/database',
},
},
executions: {
saveManualRuns: false,
},
nodes: {
// Nodes not to load even if found
// exclude: [],
errorTriggerType: 'n8n-nodes-base.errorTrigger',
},
timezone: 'America/New_York',
};

58
packages/cli/index.ts Normal file
View File

@@ -0,0 +1,58 @@
#!/usr/bin/env node
import { join as pathJoin } from 'path';
// Make sure that it also find the config folder when it
// did get started from another folder that the root one.
process.env.NODE_CONFIG_DIR = process.env.NODE_CONFIG_DIR || pathJoin(__dirname, 'config');
import Vorpal = require('vorpal');
import { GenericHelpers } from './src';
// Check if version should be displayed
const versionFlags = [
'-v',
'-V',
'--version'
];
if (versionFlags.includes(process.argv.slice(-1)[0])) {
console.log(require('../package').version);
process.exit(0);
}
if (process.argv.length === 2) {
// When no command is given choose by default start
process.argv.push('start');
}
const command = process.argv[2];
// Check if the command the user did enter is supported else stop
const supportedCommands = [
'help',
'run',
'start',
];
if (!supportedCommands.includes(command)) {
GenericHelpers.logOutput(`The command "${command}" is not known!`);
process.argv.push('help');
}
const vorpal = new Vorpal();
vorpal
.use(require('./commands/run.js'))
.use(require('./commands/start.js'))
.delimiter('')
.show()
.parse(process.argv);
process
.on('unhandledRejection', (reason, p) => {
console.error(reason, 'Unhandled Rejection at Promise', p);
})
.on('uncaughtException', err => {
console.error(err, 'Uncaught Exception thrown');
process.exit(1);
});

93
packages/cli/package.json Normal file
View File

@@ -0,0 +1,93 @@
{
"name": "n8n",
"version": "0.1.2",
"description": "n8n Workflow Automation Tool",
"license": "SEE LICENSE IN LICENSE",
"author": {
"name": "Jan Oberhauser",
"email": "jan@n8n.io"
},
"main": "dist/index",
"types": "dist/src/index.d.ts",
"scripts": {
"build": "tsc",
"start": "node dist/index.js start",
"start:dev": "nodemon",
"test": "jest",
"tslint": "tslint -p tsconfig.json -c tslint.json",
"watch": "tsc --watch"
},
"bin": {
"n8n": "./dist/index.js"
},
"keywords": [
"automate",
"automation",
"IaaS",
"iPaaS",
"n8n",
"workflow"
],
"engines": {
"node": ">=8.0.0"
},
"files": [
"dist"
],
"devDependencies": {
"@types/config": "0.0.34",
"@types/connect-history-api-fallback": "^1.3.1",
"@types/express": "^4.16.1",
"@types/jest": "^23.3.2",
"@types/localtunnel": "^1.9.0",
"@types/node": "^10.10.1",
"@types/open": "^6.1.0",
"@types/parseurl": "^1.3.1",
"@types/request-promise-native": "^1.0.15",
"@types/vorpal": "^1.11.0",
"jest": "^23.6.0",
"nodemon": "^1.19.1",
"sails-disk": "^1.0.1",
"ts-jest": "^23.10.1",
"tslint": "^5.11.0",
"typescript": "~3.3.0"
},
"dependencies": {
"body-parser": "^1.18.3",
"config": "^3.0.1",
"connect-history-api-fallback": "^1.6.0",
"express": "^4.16.4",
"flatted": "^2.0.0",
"glob-promise": "^3.4.0",
"google-timezones-json": "^1.0.2",
"localtunnel": "^1.9.1",
"mongodb": "^3.2.3",
"n8n-core": "^0.1.0",
"n8n-editor-ui": "^0.1.0",
"n8n-nodes-base": "^0.1.0",
"n8n-workflow": "^0.1.0",
"open": "^6.1.0",
"request-promise-native": "^1.0.7",
"sqlite3": "^4.0.6",
"sse-channel": "^3.1.1",
"typeorm": "^0.2.16",
"vorpal": "^1.12.0"
},
"jest": {
"transform": {
"^.+\\.tsx?$": "ts-jest"
},
"testURL": "http://localhost/",
"testRegex": "(/__tests__/.*|(\\.|/)(test|spec))\\.(jsx?|tsx?)$",
"testPathIgnorePatterns": [
"/dist/",
"/node_modules/"
],
"moduleFileExtensions": [
"ts",
"tsx",
"js",
"json"
]
}
}

View File

@@ -0,0 +1,314 @@
import {
IActivationError,
Db,
NodeTypes,
IResponseCallbackData,
IWorkflowDb,
ResponseHelper,
WebhookHelpers,
WorkflowHelpers,
WorkflowExecuteAdditionalData,
} from './';
import {
ActiveWorkflows,
ActiveWebhooks,
} from 'n8n-core';
import {
IWebhookData,
IWorkflowExecuteAdditionalData as IWorkflowExecuteAdditionalDataWorkflow,
WebhookHttpMethod,
Workflow,
WorkflowExecuteMode,
} from 'n8n-workflow';
import * as express from 'express';
export class ActiveWorkflowRunner {
private activeWorkflows: ActiveWorkflows | null = null;
private activeWebhooks: ActiveWebhooks | null = null;
private activationErrors: {
[key: string]: IActivationError;
} = {};
async init() {
// Get the active workflows from database
const workflowsData: IWorkflowDb[] = await Db.collections.Workflow!.find({ active: true }) as IWorkflowDb[];
this.activeWebhooks = new ActiveWebhooks();
// Add them as active workflows
this.activeWorkflows = new ActiveWorkflows();
if (workflowsData.length !== 0) {
console.log('\n ================================');
console.log(' Start Active Workflows:');
console.log(' ================================');
for (const workflowData of workflowsData) {
console.log(` - ${workflowData.name}`);
try {
await this.add(workflowData.id.toString(), workflowData);
console.log(` => Started`);
} catch (error) {
console.log(` => ERROR: Workflow could not be activated:`);
console.log(` ${error.message}`);
}
}
}
}
/**
* Removes all the currently active workflows
*
* @returns {Promise<void>}
* @memberof ActiveWorkflowRunner
*/
async removeAll(): Promise<void> {
if (this.activeWorkflows === null) {
return;
}
const activeWorkflows = this.activeWorkflows.allActiveWorkflows();
const removePromises = [];
for (const workflowId of activeWorkflows) {
removePromises.push(this.remove(workflowId));
}
await Promise.all(removePromises);
return;
}
/**
* Checks if a webhook for the given method and path exists and executes the workflow.
*
* @param {WebhookHttpMethod} httpMethod
* @param {string} path
* @param {express.Request} req
* @param {express.Response} res
* @returns {Promise<object>}
* @memberof ActiveWorkflowRunner
*/
async executeWebhook(httpMethod: WebhookHttpMethod, path: string, req: express.Request, res: express.Response): Promise<IResponseCallbackData> {
if (this.activeWorkflows === null) {
throw new ResponseHelper.ReponseError('The "activeWorkflows" instance did not get initialized yet.', 404, 404);
}
const webhookData: IWebhookData | undefined = this.activeWebhooks!.get(httpMethod, path);
if (webhookData === undefined) {
// The requested webhook is not registred
throw new ResponseHelper.ReponseError('The requested webhook is not registred.', 404, 404);
}
// Get the node which has the webhook defined to know where to start from and to
// get additional data
const workflowStartNode = webhookData.workflow.getNode(webhookData.node);
if (workflowStartNode === null) {
throw new ResponseHelper.ReponseError('Could not find node to process webhook.', 404, 404);
}
const executionMode = 'webhook';
const workflowData = await Db.collections.Workflow!.findOne(webhookData.workflow.id!);
if (workflowData === undefined) {
throw new ResponseHelper.ReponseError(`Could not find workflow with id "${webhookData.workflow.id}"`, 404, 404);
}
return new Promise((resolve, reject) => {
WebhookHelpers.executeWebhook(webhookData, workflowData, workflowStartNode, executionMode, undefined, req, res, (error: Error | null, data: object) => {
if (error !== null) {
return reject(error);
}
resolve(data);
});
});
}
/**
* Returns the ids of the currently active workflows
*
* @returns {string[]}
* @memberof ActiveWorkflowRunner
*/
getActiveWorkflows(): string[] {
if (this.activeWorkflows === null) {
return [];
}
return this.activeWorkflows.allActiveWorkflows();
}
/**
* Returns if the workflow is active
*
* @param {string} id The id of the workflow to check
* @returns {boolean}
* @memberof ActiveWorkflowRunner
*/
isActive(id: string): boolean {
if (this.activeWorkflows !== null) {
return this.activeWorkflows.isActive(id);
}
return false;
}
/**
* Return error if there was a problem activating the workflow
*
* @param {string} id The id of the workflow to return the error of
* @returns {(IActivationError | undefined)}
* @memberof ActiveWorkflowRunner
*/
getActivationError(id: string): IActivationError | undefined {
if (this.activationErrors[id] === undefined) {
return undefined;
}
return this.activationErrors[id];
}
/**
* Adds all the webhooks of the workflow
*
* @param {Workflow} workflow
* @param {IWorkflowExecuteAdditionalDataWorkflow} additionalData
* @param {WorkflowExecuteMode} mode
* @returns {Promise<void>}
* @memberof ActiveWorkflowRunner
*/
async addWorkflowWebhooks(workflow: Workflow, additionalData: IWorkflowExecuteAdditionalDataWorkflow, mode: WorkflowExecuteMode): Promise<void> {
const webhooks = WebhookHelpers.getWorkflowWebhooks(workflow, additionalData);
for (const webhookData of webhooks) {
await this.activeWebhooks!.add(webhookData, mode);
}
}
/**
* Remove all the webhooks of the workflow
*
* @param {string} workflowId
* @returns
* @memberof ActiveWorkflowRunner
*/
removeWorkflowWebhooks(workflowId: string): Promise<boolean> {
return this.activeWebhooks!.removeByWorkflowId(workflowId);
}
/**
* Makes a workflow active
*
* @param {string} workflowId The id of the workflow to activate
* @param {IWorkflowDb} [workflowData] If workflowData is given it saves the DB query
* @returns {Promise<void>}
* @memberof ActiveWorkflowRunner
*/
async add(workflowId: string, workflowData?: IWorkflowDb): Promise<void> {
if (this.activeWorkflows === null) {
throw new Error(`The "activeWorkflows" instance did not get initialized yet.`);
}
let workflowInstance: Workflow;
try {
if (workflowData === undefined) {
workflowData = await Db.collections.Workflow!.findOne(workflowId) as IWorkflowDb;
}
if (!workflowData) {
throw new Error(`Could not find workflow with id "${workflowId}".`);
}
const nodeTypes = NodeTypes();
workflowInstance = new Workflow(workflowId, workflowData.nodes, workflowData.connections, workflowData.active, nodeTypes, workflowData.staticData, workflowData.settings);
const canBeActivated = workflowInstance.checkIfWorkflowCanBeActivated(['n8n-nodes-base.start']);
if (canBeActivated === false) {
throw new Error(`The workflow can not be activated because it does not contain any nodes which could start the workflow. Only workflows which have trigger or webhook nodes can be activated.`);
}
const mode = 'trigger';
const additionalData = await WorkflowExecuteAdditionalData.get(mode, workflowData, workflowInstance);
// Add the workflows which have webhooks defined
await this.addWorkflowWebhooks(workflowInstance, additionalData, mode);
await this.activeWorkflows.add(workflowId, workflowInstance, additionalData);
if (this.activationErrors[workflowId] !== undefined) {
// If there were any activation errors delete them
delete this.activationErrors[workflowId];
}
} catch (error) {
// There was a problem activating the workflow
// Save the error
this.activationErrors[workflowId] = {
time: new Date().getTime(),
error: {
message: error.message,
},
};
throw error;
}
await WorkflowHelpers.saveStaticData(workflowInstance!);
}
/**
* Makes a workflow inactive
*
* @param {string} workflowId The id of the workflow to deactivate
* @returns {Promise<void>}
* @memberof ActiveWorkflowRunner
*/
async remove(workflowId: string): Promise<void> {
if (this.activeWorkflows !== null) {
const workflowData = this.activeWorkflows.get(workflowId);
// Remove all the webhooks of the workflow
await this.removeWorkflowWebhooks(workflowId);
if (workflowData) {
// Save the static workflow data if needed
await WorkflowHelpers.saveStaticData(workflowData.workflow);
}
if (this.activationErrors[workflowId] !== undefined) {
// If there were any activation errors delete them
delete this.activationErrors[workflowId];
}
// Remove the workflow from the "list" of active workflows
return this.activeWorkflows.remove(workflowId);
}
throw new Error(`The "activeWorkflows" instance did not get initialized yet.`);
}
}
let workflowRunnerInstance: ActiveWorkflowRunner | undefined;
export function getInstance(): ActiveWorkflowRunner {
if (workflowRunnerInstance === undefined) {
workflowRunnerInstance = new ActiveWorkflowRunner();
}
return workflowRunnerInstance;
}

View File

@@ -0,0 +1,37 @@
import {
ICredentialType,
ICredentialTypes as ICredentialTypesInterface,
} from 'n8n-workflow';
class CredentialTypesClass implements ICredentialTypesInterface {
credentialTypes: {
[key: string]: ICredentialType
} = {};
async init(credentialTypes: { [key: string]: ICredentialType }): Promise<void> {
this.credentialTypes = credentialTypes;
}
getAll(): ICredentialType[] {
return Object.values(this.credentialTypes);
}
getByName(credentialType: string): ICredentialType {
return this.credentialTypes[credentialType];
}
}
let credentialTypesInstance: CredentialTypesClass | undefined;
export function CredentialTypes(): CredentialTypesClass {
if (credentialTypesInstance === undefined) {
credentialTypesInstance = new CredentialTypesClass();
}
return credentialTypesInstance;
}

69
packages/cli/src/Db.ts Normal file
View File

@@ -0,0 +1,69 @@
import {
IDatabaseCollections,
DatabaseType,
} from './';
import {
UserSettings,
} from "n8n-core";
import {
ConnectionOptions,
createConnection,
getRepository,
} from "typeorm";
import * as config from 'config';
import {
MongoDb,
SQLite,
} from './db';
export let collections: IDatabaseCollections = {
Credentials: null,
Execution: null,
Workflow: null,
};
import * as path from 'path';
export async function init(): Promise<IDatabaseCollections> {
const dbType = config.get('database.type') as DatabaseType;
const n8nFolder = UserSettings.getUserN8nFolderPath();
let entities;
let connectionOptions: ConnectionOptions;
if (dbType === 'mongodb') {
entities = MongoDb;
connectionOptions = {
type: 'mongodb',
url: config.get('database.mongodbConfig.url') as string,
useNewUrlParser: true,
};
} else if (dbType === 'sqlite') {
entities = SQLite;
connectionOptions = {
type: 'sqlite',
database: path.join(n8nFolder, 'database.sqlite'),
};
} else {
throw new Error(`The database "${dbType}" is currently not supported!`);
}
Object.assign(connectionOptions, {
entities: Object.values(entities),
synchronize: true,
logging: false
});
await createConnection(connectionOptions);
collections.Credentials = getRepository(entities.CredentialsEntity);
collections.Execution = getRepository(entities.ExecutionEntity);
collections.Workflow = getRepository(entities.WorkflowEntity);
return collections;
}

View File

@@ -0,0 +1,48 @@
import * as config from 'config';
import * as express from 'express';
/**
* Displays a message to the user
*
* @export
* @param {string} message The message to display
* @param {string} [level='log']
*/
export function logOutput(message: string, level = 'log'): void {
if (level === 'log') {
console.log(message);
} else if (level === 'error') {
console.error(message);
}
}
/**
* Returns the base URL n8n is reachable from
*
* @export
* @returns {string}
*/
export function getBaseUrl(): string {
const protocol = config.get('urls.protocol') as string;
const host = config.get('urls.host') as string;
const port = config.get('urls.port') as number;
if (protocol === 'http' && port === 80 || protocol === 'https' && port === 443) {
return `${protocol}://${host}/`;
}
return `${protocol}://${host}:${port}/`;
}
/**
* Returns the session id if one is set
*
* @export
* @param {express.Request} req
* @returns {(string | undefined)}
*/
export function getSessionId(req: express.Request): string | undefined {
return req.headers.sessionid as string | undefined;
}

View File

@@ -0,0 +1,248 @@
import {
IConnections,
ICredentialsDecrypted,
ICredentialsEncrypted,
IDataObject,
IExecutionError,
INode,
IRun,
IRunExecutionData,
ITaskData,
IWorkflowSettings,
WorkflowExecuteMode,
} from 'n8n-workflow';
import { ObjectID, Repository } from "typeorm";
import { Url } from 'url';
import { Request } from 'express';
export interface IActivationError {
time: number;
error: {
message: string;
};
}
export interface ICustomRequest extends Request {
parsedUrl: Url | undefined;
}
export interface IDatabaseCollections {
Credentials: Repository<ICredentialsDb> | null;
Execution: Repository<IExecutionFlattedDb> | null;
Workflow: Repository<IWorkflowDb> | null;
}
export interface IWorkflowBase {
id?: number | string | ObjectID;
name: string;
active: boolean;
createdAt: number | string;
updatedAt: number | string;
nodes: INode[];
connections: IConnections;
settings?: IWorkflowSettings;
staticData?: IDataObject;
}
// Almost identical to editor-ui.Interfaces.ts
export interface IWorkflowDb extends IWorkflowBase {
id: number | string | ObjectID;
}
export interface IWorkflowResponse extends IWorkflowBase {
id: string;
}
export interface IWorkflowShortResponse {
id: string;
name: string;
active: boolean;
createdAt: number | string;
updatedAt: number | string;
}
export interface ICredentialsBase {
createdAt: number | string;
updatedAt: number | string;
}
export interface ICredentialsDb extends ICredentialsBase, ICredentialsEncrypted{
id: number | string | ObjectID;
}
export interface ICredentialsResponse extends ICredentialsDb {
id: string;
}
export interface ICredentialsDecryptedDb extends ICredentialsBase, ICredentialsDecrypted {
id: number | string | ObjectID;
}
export interface ICredentialsDecryptedResponse extends ICredentialsDecryptedDb {
id: string;
}
export type DatabaseType = 'mongodb' | 'sqlite';
export interface IExecutionBase {
id?: number | string | ObjectID;
mode: WorkflowExecuteMode;
startedAt: number;
stoppedAt: number;
workflowId?: string; // To be able to filter executions easily //
finished: boolean;
retryOf?: number | string | ObjectID; // If it is a retry, the id of the execution it is a retry of.
retrySuccessId?: number | string | ObjectID; // If it failed and a retry did succeed. The id of the successful retry.
}
// Data in regular format with references
export interface IExecutionDb extends IExecutionBase {
data: IRunExecutionData;
workflowData?: IWorkflowBase;
}
export interface IExecutionPushResponse {
executionId?: string;
waitingForWebhook?: boolean;
}
export interface IExecutionResponse extends IExecutionBase {
id: string;
data: IRunExecutionData;
retryOf?: string;
retrySuccessId?: string;
workflowData: IWorkflowBase;
}
// Flatted data to save memory when saving in database or transfering
// via REST API
export interface IExecutionFlatted extends IExecutionBase {
data: string;
workflowData: IWorkflowBase;
}
export interface IExecutionFlattedDb extends IExecutionBase {
id: number | string | ObjectID;
data: string;
workflowData: IWorkflowBase;
}
export interface IExecutionFlattedResponse extends IExecutionFlatted {
id: string;
retryOf?: string;
}
export interface IExecutionsListResponse {
count: number;
// results: IExecutionShortResponse[];
results: IExecutionsSummary[];
}
export interface IExecutionsStopData {
finished?: boolean;
mode: WorkflowExecuteMode;
startedAt: number | string;
stoppedAt: number | string;
}
export interface IExecutionsSummary {
id: string;
mode: WorkflowExecuteMode;
finished?: boolean;
retryOf?: string;
retrySuccessId?: string;
startedAt: number | string;
stoppedAt?: number | string;
workflowId: string;
workflowName?: string;
}
export interface IExecutionDeleteFilter {
deleteBefore?: number;
filters?: IDataObject;
ids?: string[];
}
export interface IN8nConfig {
database: IN8nConfigDatabase;
nodes?: IN8nConfigNodes;
}
export interface IN8nConfigDatabase {
type: DatabaseType;
mongodbConfig?: {
url: string;
};
}
export interface IN8nConfigNodes {
exclude?: string[];
}
export interface IN8nUISettings {
endpointWebhook: string;
endpointWebhookTest: string;
saveManualRuns: boolean;
timezone: string;
urlBaseWebhook: string;
}
export interface IPushData {
data: IPushDataExecutionFinished | IPushDataNodeExecuteAfter | IPushDataNodeExecuteBefore | IPushDataTestWebhook;
type: IPushDataType;
}
export type IPushDataType = 'executionFinished' | 'nodeExecuteAfter' | 'nodeExecuteBefore' | 'testWebhookDeleted' | 'testWebhookReceived';
export interface IPushDataExecutionFinished {
data: IRun;
executionId: string;
}
export interface IPushDataNodeExecuteAfter {
data: ITaskData;
executionId: string;
nodeName: string;
}
export interface IPushDataNodeExecuteBefore {
executionId: string;
nodeName: string;
}
export interface IPushDataTestWebhook {
workflowId: string;
}
export interface IResponseCallbackData {
data?: IDataObject | IDataObject[];
noWebhookResponse?: boolean;
}
export interface IWorkflowErrorData {
[key: string]: IDataObject | string | number | IExecutionError;
execution: {
id?: string;
error: IExecutionError;
lastNodeExecuted: string;
mode: WorkflowExecuteMode;
};
workflow: {
id?: string;
name: string;
};
}

View File

@@ -0,0 +1,261 @@
import {
CUSTOM_EXTENSION_ENV,
UserSettings,
} from 'n8n-core';
import {
ICredentialType,
INodeType,
} from 'n8n-workflow';
import {
IN8nConfigNodes,
} from './';
import { promises as fs } from 'fs';
import * as path from 'path';
import * as glob from 'glob-promise';
import * as config from 'config';
class LoadNodesAndCredentialsClass {
nodeTypes: {
[key: string]: INodeType
} = {};
credentialTypes: {
[key: string]: ICredentialType
} = {};
excludeNodes: string[] | undefined = undefined;
nodeModulesPath = '';
async init(directory?: string) {
// Get the path to the node-modules folder to be later able
// to load the credentials and nodes
const checkPaths = [
// In case "n8n" package is in same node_modules folder.
path.join(__dirname, '..', '..', '..', 'n8n-workflow'),
// In case "n8n" package is the root and the packages are
// in the "node_modules" folder underneath it.
path.join(__dirname, '..', '..', 'node_modules', 'n8n-workflow'),
];
for (const checkPath of checkPaths) {
try {
await fs.access(checkPath);
// Folder exists, so use it.
this.nodeModulesPath = path.dirname(checkPath);
break;
} catch (error) {
// Folder does not exist so get next one
continue;
}
}
if (this.nodeModulesPath === '') {
throw new Error('Could not find "node_modules" folder!');
}
const nodeSettings = config.get('nodes') as IN8nConfigNodes | undefined;
if (nodeSettings !== undefined && nodeSettings.exclude !== undefined) {
this.excludeNodes = nodeSettings.exclude;
}
// Get all the installed packages which contain n8n nodes
const packages = await this.getN8nNodePackages();
for (const packageName of packages) {
await this.loadDataFromPackage(packageName);
}
// Read nodes and credentials from custom directories
const customDirectories = [];
// Add "custom" folder in user-n8n folder
customDirectories.push(UserSettings.getUserN8nFolderCustomExtensionPath());
// Add folders from special environment variable
if (process.env[CUSTOM_EXTENSION_ENV] !== undefined) {
const customExtensionFolders = process.env[CUSTOM_EXTENSION_ENV]!.split(';');
customDirectories.push.apply(customDirectories, customExtensionFolders);
}
for (const directory of customDirectories) {
await this.loadDataFromDirectory('CUSTOM', directory);
}
}
/**
* Returns all the names of the packages which could
* contain n8n nodes
*
* @returns {Promise<string[]>}
* @memberof LoadNodesAndCredentialsClass
*/
async getN8nNodePackages(): Promise<string[]> {
const packages: string[] = [];
for (const file of await fs.readdir(this.nodeModulesPath)) {
if (file.indexOf('n8n-nodes-') !== 0) {
continue;
}
// Check if it is really a folder
if (!(await fs.stat(path.join(this.nodeModulesPath, file))).isDirectory()) {
continue;
}
packages.push(file);
}
return packages;
}
/**
* Loads credentials from a file
*
* @param {string} credentialName The name of the credentials
* @param {string} filePath The file to read credentials from
* @returns {Promise<void>}
* @memberof N8nPackagesInformationClass
*/
async loadCredentialsFromFile(credentialName: string, filePath: string): Promise<void> {
const tempModule = require(filePath);
let tempCredential: ICredentialType;
try {
tempCredential = new tempModule[credentialName]() as ICredentialType;
} catch (e) {
if (e instanceof TypeError) {
throw new Error(`Class with name "${credentialName}" could not be found. Please check if the class is named correctly!`);
} else {
throw e;
}
}
this.credentialTypes[credentialName] = tempCredential;
}
/**
* Loads a node from a file
*
* @param {string} packageName The package name to set for the found nodes
* @param {string} nodeName Tha name of the node
* @param {string} filePath The file to read node from
* @returns {Promise<void>}
* @memberof N8nPackagesInformationClass
*/
async loadNodeFromFile(packageName: string, nodeName: string, filePath: string): Promise<void> {
let tempNode: INodeType;
let fullNodeName: string;
const tempModule = require(filePath);
try {
tempNode = new tempModule[nodeName]() as INodeType;
} catch (error) {
console.error(`Error loading node "${nodeName}" from: "${filePath}"`);
throw error;
}
fullNodeName = packageName + '.' + tempNode.description.name;
tempNode.description.name = fullNodeName;
if (tempNode.description.icon !== undefined &&
tempNode.description.icon.startsWith('file:')) {
// If a file icon gets used add the full path
tempNode.description.icon = 'file:' + path.join(path.dirname(filePath), tempNode.description.icon.substr(5));
}
// Check if the node should be skipped
if (this.excludeNodes !== undefined && this.excludeNodes.includes(fullNodeName)) {
return;
}
this.nodeTypes[fullNodeName] = tempNode;
}
/**
* Loads nodes and credentials from the given directory
*
* @param {string} setPackageName The package name to set for the found nodes
* @param {string} directory The directory to look in
* @returns {Promise<void>}
* @memberof N8nPackagesInformationClass
*/
async loadDataFromDirectory(setPackageName: string, directory: string): Promise<void> {
const files = await glob(path.join(directory, '*\.@(node|credentials)\.js'));
let fileName: string;
let type: string;
const loadPromises = [];
for (const filePath of files) {
[fileName, type] = path.parse(filePath).name.split('.');
if (type === 'node') {
loadPromises.push(this.loadNodeFromFile(setPackageName, fileName, filePath));
} else if (type === 'credentials') {
loadPromises.push(this.loadCredentialsFromFile(fileName, filePath));
}
}
await Promise.all(loadPromises);
}
/**
* Loads nodes and credentials from the package with the given name
*
* @param {string} packageName The name to read data from
* @returns {Promise<void>}
* @memberof N8nPackagesInformationClass
*/
async loadDataFromPackage(packageName: string): Promise<void> {
// Get the absolute path of the package
const packagePath = path.join(this.nodeModulesPath, packageName);
// Read the data from the package.json file to see if any n8n data is defiend
const packageFileString = await fs.readFile(path.join(packagePath, 'package.json'), 'utf8');
const packageFile = JSON.parse(packageFileString);
if (!packageFile.hasOwnProperty('n8n')) {
return;
}
let tempPath: string, filePath: string;
// Read all node types
let fileName: string, type: string;
if (packageFile.n8n.hasOwnProperty('nodes') && Array.isArray(packageFile.n8n.nodes)) {
for (filePath of packageFile.n8n.nodes) {
tempPath = path.join(packagePath, filePath);
[fileName, type] = path.parse(filePath).name.split('.');
await this.loadNodeFromFile(packageName, fileName, tempPath);
}
}
// Read all credential types
if (packageFile.n8n.hasOwnProperty('credentials') && Array.isArray(packageFile.n8n.credentials)) {
for (filePath of packageFile.n8n.credentials) {
tempPath = path.join(packagePath, filePath);
[fileName, type] = path.parse(filePath).name.split('.');
this.loadCredentialsFromFile(fileName, tempPath);
}
}
}
}
let packagesInformationInstance: LoadNodesAndCredentialsClass | undefined;
export function LoadNodesAndCredentials(): LoadNodesAndCredentialsClass {
if (packagesInformationInstance === undefined) {
packagesInformationInstance = new LoadNodesAndCredentialsClass();
}
return packagesInformationInstance;
}

View File

@@ -0,0 +1,37 @@
import {
INodeType,
INodeTypes,
} from 'n8n-workflow';
class NodeTypesClass implements INodeTypes {
nodeTypes: {
[key: string]: INodeType
} = {};
async init(nodeTypes: {[key: string]: INodeType }): Promise<void> {
this.nodeTypes = nodeTypes;
}
getAll(): INodeType[] {
return Object.values(this.nodeTypes);
}
getByName(nodeType: string): INodeType | undefined {
return this.nodeTypes[nodeType];
}
}
let nodeTypesInstance: NodeTypesClass | undefined;
export function NodeTypes(): NodeTypesClass {
if (nodeTypesInstance === undefined) {
nodeTypesInstance = new NodeTypesClass();
}
return nodeTypesInstance;
}

86
packages/cli/src/Push.ts Normal file
View File

@@ -0,0 +1,86 @@
// @ts-ignore
import * as sseChannel from 'sse-channel';
import * as express from 'express';
import {
IPushData,
IPushDataType,
} from '.';
export class Push {
private channel: sseChannel;
private connections: {
[key: string]: express.Response;
} = {};
constructor() {
this.channel = new sseChannel({
cors: {
// Allow access also from frontend when developing
origins: ['http://localhost:8080'],
},
});
this.channel.on('disconnect', (channel: string, res: express.Response) => {
if (res.req !== undefined) {
delete this.connections[res.req.query.sessionId];
}
});
}
/**
* Adds a new push connection
*
* @param {string} sessionId The id of the session
* @param {express.Request} req The request
* @param {express.Response} res The response
* @memberof Push
*/
add(sessionId: string, req: express.Request, res: express.Response) {
if (this.connections[sessionId] !== undefined) {
// Make sure to remove existing connection with the same session
// id if one exists already
this.connections[sessionId].end();
this.channel.removeClient(this.connections[sessionId]);
}
this.connections[sessionId] = res;
this.channel.addClient(req, res);
}
/**
* Sends data to the client which is connected via a specific session
*
* @param {string} sessionId The session id of client to send data to
* @param {string} type Type of data to send
* @param {*} data
* @memberof Push
*/
send(sessionId: string, type: IPushDataType, data: any) { // tslint:disable-line:no-any
if (this.connections[sessionId] === undefined) {
// TODO: Log that properly!
console.error(`The session "${sessionId}" is not registred.`);
return;
}
const sendData: IPushData = {
type,
data,
};
this.channel.send(JSON.stringify(sendData));
}
}
let activePushInstance: Push | undefined;
export function getInstance(): Push {
if (activePushInstance === undefined) {
activePushInstance = new Push();
}
return activePushInstance;
}

View File

@@ -0,0 +1,176 @@
import { Request, Response } from 'express';
import { parse, stringify } from 'flatted';
import {
IExecutionDb,
IExecutionFlatted,
IExecutionFlattedDb,
IExecutionResponse,
IWorkflowDb,
} from './';
/**
* Special Error which allows to return also an error code and http status code
*
* @export
* @class ReponseError
* @extends {Error}
*/
export class ReponseError extends Error {
// The HTTP status code of response
httpStatusCode?: number;
// The error code in the resonse
errorCode?: number;
/**
* Creates an instance of ReponseError.
* @param {string} message The error message
* @param {number} [errorCode] The error code which can be used by frontend to identify the actual error
* @param {number} [httpStatusCode] The HTTP status code the response should have
* @memberof ReponseError
*/
constructor(message: string, errorCode?: number, httpStatusCode?: number) {
super(message);
this.name = 'ReponseError';
if (errorCode) {
this.errorCode = errorCode;
}
if (httpStatusCode) {
this.httpStatusCode = httpStatusCode;
}
}
}
export function sendSuccessResponse(res: Response, data: any, raw?: boolean) { // tslint:disable-line:no-any
res.setHeader('Content-Type', 'application/json');
if (raw === true) {
res.send(JSON.stringify(data));
return;
} else {
res.send(JSON.stringify({
data
}));
}
}
export function sendErrorResponse(res: Response, error: ReponseError) {
let httpStatusCode = 500;
if (error.httpStatusCode) {
httpStatusCode = error.httpStatusCode;
}
if (process.env.NODE_ENV !== 'production') {
console.error('ERROR RESPONSE');
console.error(error);
}
const response = {
code: 0,
message: 'Unknown error',
};
if (error.errorCode) {
response.code = error.errorCode;
}
if (error.message) {
response.message = error.message;
}
if (error.stack && process.env.NODE_ENV !== 'production') {
// @ts-ignore
response.stack = error.stack;
}
res.status(httpStatusCode).send(JSON.stringify(response));
}
/**
* A helper function which does not just allow to return Promises it also makes sure that
* all the responses have the same format
*
*
* @export
* @param {(req: Request, res: Response) => Promise<any>} processFunction The actual function to process the request
* @returns
*/
export function send(processFunction: (req: Request, res: Response) => Promise<any>) { // tslint:disable-line:no-any
return async (req: Request, res: Response) => {
try {
const data = await processFunction(req, res);
// Success response
sendSuccessResponse(res, data);
} catch (error) {
// Error response
sendErrorResponse(res, error);
}
};
}
/**
* Flattens the Execution data.
* As it contains a lot of references which normally would be saved as duplicate data
* with regular JSON.stringify it gets flattened which keeps the references in place.
*
* @export
* @param {IExecutionDb} fullExecutionData The data to flatten
* @returns {IExecutionFlatted}
*/
export function flattenExecutionData(fullExecutionData: IExecutionDb): IExecutionFlatted {
// Flatten the data
const returnData: IExecutionFlatted = Object.assign({}, {
data: stringify(fullExecutionData.data),
mode: fullExecutionData.mode,
startedAt: fullExecutionData.startedAt,
stoppedAt: fullExecutionData.stoppedAt,
finished: fullExecutionData.finished ? fullExecutionData.finished : false,
workflowId: fullExecutionData.workflowId,
workflowData: fullExecutionData.workflowData!,
});
if (fullExecutionData.id !== undefined) {
returnData.id = fullExecutionData.id!.toString();
}
if (fullExecutionData.retryOf !== undefined) {
returnData.retryOf = fullExecutionData.retryOf!.toString();
}
if (fullExecutionData.retrySuccessId !== undefined) {
returnData.retrySuccessId = fullExecutionData.retrySuccessId!.toString();
}
return returnData;
}
/**
* Unflattens the Execution data.
*
* @export
* @param {IExecutionFlattedDb} fullExecutionData The data to unflatten
* @returns {IExecutionResponse}
*/
export function unflattenExecutionData(fullExecutionData: IExecutionFlattedDb): IExecutionResponse {
const returnData: IExecutionResponse = Object.assign({}, {
id: fullExecutionData.id.toString(),
workflowData: fullExecutionData.workflowData as IWorkflowDb,
data: parse(fullExecutionData.data),
mode: fullExecutionData.mode,
startedAt: fullExecutionData.startedAt,
stoppedAt: fullExecutionData.stoppedAt,
finished: fullExecutionData.finished ? fullExecutionData.finished : false
});
return returnData;
}

997
packages/cli/src/Server.ts Normal file
View File

@@ -0,0 +1,997 @@
import * as express from 'express';
import * as bodyParser from 'body-parser';
import * as history from 'connect-history-api-fallback';
import * as requestPromise from 'request-promise-native';
import {
IActivationError,
ActiveWorkflowRunner,
ICustomRequest,
ICredentialsDb,
ICredentialsDecryptedDb,
ICredentialsDecryptedResponse,
ICredentialsResponse,
CredentialTypes,
Db,
IExecutionDeleteFilter,
IExecutionFlatted,
IExecutionFlattedDb,
IExecutionFlattedResponse,
IExecutionPushResponse,
IExecutionsListResponse,
IExecutionsStopData,
IExecutionsSummary,
IN8nUISettings,
IWorkflowBase,
IWorkflowShortResponse,
IWorkflowResponse,
NodeTypes,
Push,
ResponseHelper,
TestWebhooks,
WebhookHelpers,
WorkflowExecuteAdditionalData,
WorkflowHelpers,
GenericHelpers,
} from './';
import {
ActiveExecutions,
Credentials,
LoadNodeParameterOptions,
UserSettings,
WorkflowExecute,
} from 'n8n-core';
import {
ICredentialType,
IDataObject,
INodeCredentials,
INodeTypeDescription,
INodePropertyOptions,
IRunData,
Workflow,
} from 'n8n-workflow';
import {
FindManyOptions,
LessThan,
LessThanOrEqual,
} from 'typeorm';
import * as parseUrl from 'parseurl';
import * as config from 'config';
// @ts-ignore
import * as timezones from 'google-timezones-json';
class App {
app: express.Application;
activeWorkflowRunner: ActiveWorkflowRunner.ActiveWorkflowRunner;
testWebhooks: TestWebhooks.TestWebhooks;
endpointWebhook: string;
endpointWebhookTest: string;
saveManualRuns: boolean;
timezone: string;
activeExecutionsInstance: ActiveExecutions.ActiveExecutions;
push: Push.Push;
constructor() {
this.app = express();
this.endpointWebhook = config.get('urls.endpointWebhook') as string;
this.endpointWebhookTest = config.get('urls.endpointWebhookTest') as string;
this.saveManualRuns = config.get('executions.saveManualRuns') as boolean;
this.timezone = config.get('timezone') as string;
this.config();
this.activeWorkflowRunner = ActiveWorkflowRunner.getInstance();
this.testWebhooks = TestWebhooks.getInstance();
this.push = Push.getInstance();
this.activeExecutionsInstance = ActiveExecutions.getInstance();
}
/**
* Returns the current epoch time
*
* @returns {number}
* @memberof App
*/
getCurrentDate(): number {
return Math.floor(new Date().getTime());
}
private config(): void {
// Get push connections
this.app.use((req: express.Request, res: express.Response, next: express.NextFunction) => {
if (req.url.indexOf('/rest/push') === 0) {
// TODO: Later also has to add some kind of authentication token
if (req.query.sessionId === undefined) {
next(new Error('The query parameter "sessionId" is missing!'));
return;
}
this.push.add(req.query.sessionId, req, res);
return;
}
next();
});
// Make sure that each request has the "parsedUrl" parameter
this.app.use((req: express.Request, res: express.Response, next: express.NextFunction) => {
(req as ICustomRequest).parsedUrl = parseUrl(req);
next();
});
// Support application/json type post data
this.app.use(bodyParser.json({ limit: "16mb" }));
// Make sure that Vue history mode works properly
this.app.use(history({
rewrites: [
{
from: new RegExp(`^\/(rest|${this.endpointWebhook}|${this.endpointWebhookTest})\/.*$`),
to: (context) => {
return context.parsedUrl!.pathname!.toString();
}
}
]
}));
//support application/x-www-form-urlencoded post data
this.app.use(bodyParser.urlencoded({ extended: false }));
this.app.use((req: express.Request, res: express.Response, next: express.NextFunction) => {
// Allow access also from frontend when developing
res.header('Access-Control-Allow-Origin', 'http://localhost:8080');
res.header('Access-Control-Allow-Methods', 'GET, POST, OPTIONS, PUT, PATCH, DELETE');
res.header('Access-Control-Allow-Headers', 'Origin, X-Requested-With, Content-Type, Accept, sessionid');
next();
});
this.app.use((req: express.Request, res: express.Response, next: express.NextFunction) => {
if (Db.collections.Workflow === null) {
const error = new ResponseHelper.ReponseError('Database is not ready!', undefined, 503);
return ResponseHelper.sendErrorResponse(res, error);
}
next();
});
// ----------------------------------------
// Workflow
// ----------------------------------------
// Creates a new workflow
this.app.post('/rest/workflows', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowResponse> => {
const newWorkflowData = req.body;
newWorkflowData.createdAt = this.getCurrentDate();
newWorkflowData.updatedAt = this.getCurrentDate();
newWorkflowData.id = undefined;
// Save the workflow in DB
const result = await Db.collections.Workflow!.save(newWorkflowData);
// Convert to response format in which the id is a string
(result as IWorkflowBase as IWorkflowResponse).id = result.id.toString();
return result as IWorkflowBase as IWorkflowResponse;
}));
// Reads and returns workflow data from an URL
this.app.get('/rest/workflows/from-url', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowResponse> => {
if (req.query.url === undefined) {
throw new ResponseHelper.ReponseError(`The parameter "url" is missing!`, undefined, 400);
}
if (!req.query.url.match(/^http[s]?:\/\/.*\.json$/i)) {
throw new ResponseHelper.ReponseError(`The parameter "url" is not valid! It does not seem to be a URL pointing to a n8n workflow JSON file.`, undefined, 400);
}
const data = await requestPromise.get(req.query.url);
let workflowData: IWorkflowResponse | undefined;
try {
workflowData = JSON.parse(data);
} catch (error) {
throw new ResponseHelper.ReponseError(`The URL does not point to valid JSON file!`, undefined, 400);
}
// Do a very basic check if it is really a n8n-workflow-json
if (workflowData === undefined || workflowData.nodes === undefined || !Array.isArray(workflowData.nodes) ||
workflowData.connections === undefined || typeof workflowData.connections !== 'object' ||
Array.isArray(workflowData.connections)) {
throw new ResponseHelper.ReponseError(`The data in the file does not seem to be a n8n workflow JSON file!`, undefined, 400);
}
return workflowData;
}));
// Returns workflows
this.app.get('/rest/workflows', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowShortResponse[]> => {
const findQuery = {} as FindManyOptions;
if (req.query.filter) {
findQuery.where = JSON.parse(req.query.filter);
}
// Return only the fields we need
findQuery.select = ['id', 'name', 'active', 'createdAt', 'updatedAt'];
const results = await Db.collections.Workflow!.find(findQuery);
for (const entry of results) {
(entry as unknown as IWorkflowShortResponse).id = entry.id.toString();
}
return results as unknown as IWorkflowShortResponse[];
}));
// Returns a specific workflow
this.app.get('/rest/workflows/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowResponse | undefined> => {
const result = await Db.collections.Workflow!.findOne(req.params.id);
if (result === undefined) {
return undefined;
}
// Convert to response format in which the id is a string
(result as IWorkflowBase as IWorkflowResponse).id = result.id.toString();
return result as IWorkflowBase as IWorkflowResponse;
}));
// Updates an existing workflow
this.app.patch('/rest/workflows/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowResponse> => {
const newWorkflowData = req.body;
const id = req.params.id;
if (this.activeWorkflowRunner.isActive(id)) {
// When workflow gets saved always remove it as the triggers could have been
// changed and so the changes would not take effect
await this.activeWorkflowRunner.remove(id);
}
if (newWorkflowData.settings) {
if (newWorkflowData.settings.timezone === 'DEFAULT') {
// Do not save the default timezone
delete newWorkflowData.settings.timezone;
}
if (newWorkflowData.settings.saveManualRuns === 'DEFAULT') {
// Do not save when default got set
delete newWorkflowData.settings.saveManualRuns;
}
}
newWorkflowData.updatedAt = this.getCurrentDate();
await Db.collections.Workflow!.update(id, newWorkflowData);
// We sadly get nothing back from "update". Neither if it updated a record
// nor the new value. So query now the hopefully updated entry.
const reponseData = await Db.collections.Workflow!.findOne(id);
if (reponseData === undefined) {
throw new ResponseHelper.ReponseError(`Workflow with id "${id}" could not be found to be updated.`, undefined, 400);
}
if (reponseData.active === true) {
// When the workflow is supposed to be active add it again
try {
await this.activeWorkflowRunner.add(id);
} catch (error) {
// If workflow could not be activated set it again to inactive
newWorkflowData.active = false;
await Db.collections.Workflow!.update(id, newWorkflowData);
// Also set it in the returned data
reponseData.active = false;
// Now return the original error for UI to display
throw error;
}
}
// Convert to response format in which the id is a string
(reponseData as IWorkflowBase as IWorkflowResponse).id = reponseData.id.toString();
return reponseData as IWorkflowBase as IWorkflowResponse;
}));
// Deletes a specific workflow
this.app.delete('/rest/workflows/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<boolean> => {
const id = req.params.id;
if (this.activeWorkflowRunner.isActive(id)) {
// Before deleting a workflow deactivate it
await this.activeWorkflowRunner.remove(id);
}
await Db.collections.Workflow!.delete(id);
return true;
}));
this.app.post('/rest/workflows/run', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IExecutionPushResponse> => {
const workflowData = req.body.workflowData;
const runData: IRunData | undefined = req.body.runData;
const startNodes: string[] | undefined = req.body.startNodes;
const destinationNode: string | undefined = req.body.destinationNode;
const nodeTypes = NodeTypes();
const executionMode = 'manual';
const sessionId = GenericHelpers.getSessionId(req);
// Do not supply the saved static data! Tests always run with initially empty static data.
// The reason is that it contains information like webhook-ids. If a workflow is currently
// active it would see its id and would so not create an own test-webhook. Additionally would
// it also delete the webhook at the service in the end. So that the active workflow would end
// up without still being active but not receiving and webhook requests anymore as it does
// not exist anymore.
const workflowInstance = new Workflow(workflowData.id, workflowData.nodes, workflowData.connections, false, nodeTypes, undefined, workflowData.settings);
const additionalData = await WorkflowExecuteAdditionalData.get(executionMode, workflowData, workflowInstance, sessionId);
const workflowExecute = new WorkflowExecute(additionalData, executionMode);
let executionId: string;
if (runData === undefined || startNodes === undefined || startNodes.length === 0 || destinationNode === undefined) {
// Execute all nodes
if (WorkflowHelpers.isWorkflowIdValid(workflowData.id) === true) {
// Webhooks can only be tested with saved workflows
const needsWebhook = await this.testWebhooks.needsWebhookData(workflowData, workflowInstance, additionalData, executionMode, sessionId, destinationNode);
if (needsWebhook === true) {
return {
waitingForWebhook: true,
};
}
}
// Can execute without webhook so go on
executionId = await workflowExecute.run(workflowInstance, undefined, destinationNode);
} else {
// Execute only the nodes between start and destination nodes
executionId = await workflowExecute.runPartialWorkflow(workflowInstance, runData, startNodes, destinationNode);
}
return {
executionId,
};
}));
// Returns parameter values which normally get loaded from an external API or
// get generated dynamically
this.app.get('/rest/node-parameter-options', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<INodePropertyOptions[]> => {
const nodeType = req.query.nodeType;
let credentials: INodeCredentials | undefined = undefined;
if (req.query.credentials !== undefined) {
credentials = JSON.parse(req.query.credentials);
}
const methodName = req.query.methodName;
const nodeTypes = NodeTypes();
const executionMode = 'manual';
const sessionId = GenericHelpers.getSessionId(req);
const loadDataInstance = new LoadNodeParameterOptions(nodeType, nodeTypes, credentials);
const workflowData = loadDataInstance.getWorkflowData() as IWorkflowBase;
const additionalData = await WorkflowExecuteAdditionalData.get(executionMode, workflowData, loadDataInstance.workflow, sessionId);
return loadDataInstance.getOptions(methodName, additionalData);
}));
// Returns all the node-types
this.app.get('/rest/node-types', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<INodeTypeDescription[]> => {
const returnData: INodeTypeDescription[] = [];
const nodeTypes = NodeTypes();
const allNodes = nodeTypes.getAll();
allNodes.forEach((nodeData) => {
returnData.push(nodeData.description);
});
return returnData;
}));
// ----------------------------------------
// Node-Types
// ----------------------------------------
// Returns the node icon
this.app.get('/rest/node-icon/:nodeType', async (req: express.Request, res: express.Response): Promise<void> => {
const nodeTypeName = req.params.nodeType;
const nodeTypes = NodeTypes();
const nodeType = nodeTypes.getByName(nodeTypeName);
if (nodeType === undefined) {
res.status(404).send('The nodeType is not known.');
return;
}
if (nodeType.description.icon === undefined) {
res.status(404).send('No icon found for node.');
return;
}
if (!nodeType.description.icon.startsWith('file:')) {
res.status(404).send('Node does not have a file icon.');
return;
}
const filepath = nodeType.description.icon.substr(5);
res.sendFile(filepath);
});
// ----------------------------------------
// Active Workflows
// ----------------------------------------
// Returns the active workflow ids
this.app.get('/rest/active', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<string[]> => {
return this.activeWorkflowRunner.getActiveWorkflows();
}));
// Returns if the workflow with the given id had any activation errors
this.app.get('/rest/active/error/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IActivationError | undefined> => {
const id = req.params.id;
return this.activeWorkflowRunner.getActivationError(id);
}));
// ----------------------------------------
// Credentials
// ----------------------------------------
// Deletes a specific credential
this.app.delete('/rest/credentials/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<boolean> => {
const id = req.params.id;
await Db.collections.Credentials!.delete({ id });
return true;
}));
// Creates new credentials
this.app.post('/rest/credentials', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<ICredentialsResponse> => {
const incomingData = req.body;
// Add the added date for node access permissions
for (const nodeAccess of incomingData.nodesAccess) {
nodeAccess.date = this.getCurrentDate();
}
const encryptionKey = await UserSettings.getEncryptionKey();
if (encryptionKey === undefined) {
throw new Error('No encryption key got found to encrypt the credentials!');
}
// Encrypt the data
const credentials = new Credentials(incomingData.name, incomingData.type, incomingData.nodesAccess);
credentials.setData(incomingData.data, encryptionKey);
const newCredentialsData = credentials.getDataToSave() as ICredentialsDb;
// Add special database related data
newCredentialsData.createdAt = this.getCurrentDate();
newCredentialsData.updatedAt = this.getCurrentDate();
// TODO: also add user automatically depending on who is logged in, if anybody is logged in
// Save the credentials in DB
const result = await Db.collections.Credentials!.save(newCredentialsData);
// Convert to response format in which the id is a string
(result as unknown as ICredentialsResponse).id = result.id.toString();
return result as unknown as ICredentialsResponse;
}));
// Updates existing credentials
this.app.patch('/rest/credentials/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<ICredentialsResponse> => {
const incomingData = req.body;
const id = req.params.id;
// Add the date for newly added node access permissions
for (const nodeAccess of incomingData.nodesAccess) {
if (!nodeAccess.date) {
nodeAccess.date = this.getCurrentDate();
}
}
const encryptionKey = await UserSettings.getEncryptionKey();
if (encryptionKey === undefined) {
throw new Error('No encryption key got found to encrypt the credentials!');
}
// Encrypt the data
const credentials = new Credentials(incomingData.name, incomingData.type, incomingData.nodesAccess);
credentials.setData(incomingData.data, encryptionKey);
const newCredentialsData = credentials.getDataToSave() as unknown as ICredentialsDb;
// Add special database related data
newCredentialsData.updatedAt = this.getCurrentDate();
// Update the credentials in DB
await Db.collections.Credentials!.update(id, newCredentialsData);
// We sadly get nothing back from "update". Neither if it updated a record
// nor the new value. So query now the hopefully updated entry.
const reponseData = await Db.collections.Credentials!.findOne(id);
if (reponseData === undefined) {
throw new ResponseHelper.ReponseError(`Credentials with id "${id}" could not be found to be updated.`, undefined, 400);
}
// Remove the encrypted data as it is not needed in the frontend
reponseData.data = '';
// Convert to response format in which the id is a string
(reponseData as unknown as ICredentialsResponse).id = reponseData.id.toString();
return reponseData as unknown as ICredentialsResponse;
}));
// Returns specific credentials
this.app.get('/rest/credentials/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<ICredentialsDecryptedResponse | ICredentialsResponse | undefined> => {
const findQuery = {} as FindManyOptions;
// Make sure the variable has an expected value
if (req.query.includeData === 'true') {
req.query.includeData = true;
} else {
req.query.includeData = false;
}
if (req.query.includeData !== true) {
// Return only the fields we need
findQuery.select = ['id', 'name', 'type', 'nodesAccess', 'createdAt', 'updatedAt'];
}
const result = await Db.collections.Credentials!.findOne(req.params.id);
if (result === undefined) {
return result;
}
let encryptionKey = undefined;
if (req.query.includeData === true) {
encryptionKey = await UserSettings.getEncryptionKey();
if (encryptionKey === undefined) {
throw new Error('No encryption key got found to decrypt the credentials!');
}
const credentials = new Credentials(result.name, result.type, result.nodesAccess, result.data);
(result as ICredentialsDecryptedDb).data = credentials.getData(encryptionKey!);
}
(result as ICredentialsDecryptedResponse).id = result.id.toString();
return result as ICredentialsDecryptedResponse;
}));
// Returns all the saved credentials
this.app.get('/rest/credentials', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<ICredentialsResponse[]> => {
const findQuery = {} as FindManyOptions;
if (req.query.filter) {
findQuery.where = JSON.parse(req.query.filter);
if ((findQuery.where! as IDataObject).id !== undefined) {
// No idea if multiple where parameters make db search
// slower but to be sure that that is not the case we
// remove all unnecessary fields in case the id is defined.
findQuery.where = { id: (findQuery.where! as IDataObject).id };
}
}
findQuery.select = ['id', 'name', 'type', 'nodesAccess', 'createdAt', 'updatedAt'];
const results = await Db.collections.Credentials!.find(findQuery) as unknown as ICredentialsResponse[];
let encryptionKey = undefined;
if (req.query.includeData === true) {
encryptionKey = await UserSettings.getEncryptionKey();
if (encryptionKey === undefined) {
throw new Error('No encryption key got found to decrypt the credentials!');
}
}
let result;
for (result of results) {
(result as ICredentialsDecryptedResponse).id = result.id.toString();
}
return results;
}));
// ----------------------------------------
// Credential-Types
// ----------------------------------------
// Returns all the credential types which are defined in the loaded n8n-modules
this.app.get('/rest/credential-types', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<ICredentialType[]> => {
const returnData: ICredentialType[] = [];
const credentialTypes = CredentialTypes();
credentialTypes.getAll().forEach((credentialData) => {
returnData.push(credentialData);
});
return returnData;
}));
// ----------------------------------------
// Executions
// ----------------------------------------
// Returns all finished executions
this.app.get('/rest/executions', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IExecutionsListResponse> => {
let filter: any = {}; // tslint:disable-line:no-any
if (req.query.filter) {
filter = JSON.parse(req.query.filter);
}
let limit = 20;
if (req.query.limit) {
limit = parseInt(req.query.limit, 10);
}
const countFilter = JSON.parse(JSON.stringify(filter));
if (req.query.lastStartedAt) {
filter.startedAt = LessThan(req.query.lastStartedAt);
}
const resultsPromise = Db.collections.Execution!.find({
where: filter,
order: {
startedAt: "DESC",
},
take: limit,
});
const countPromise = Db.collections.Execution!.count(countFilter);
const results: IExecutionFlattedDb[] = await resultsPromise;
const count = await countPromise;
const returnResults: IExecutionsSummary[] = [];
for (const result of results) {
returnResults.push({
id: result.id!.toString(),
finished: result.finished,
mode: result.mode,
retryOf: result.retryOf ? result.retryOf.toString() : undefined,
retrySuccessId: result.retrySuccessId ? result.retrySuccessId.toString() : undefined,
startedAt: result.startedAt,
stoppedAt: result.stoppedAt,
workflowId: result.workflowData!.id!.toString(),
workflowName: result.workflowData!.name,
});
}
return {
count,
results: returnResults,
};
}));
// Returns a specific execution
this.app.get('/rest/executions/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IExecutionFlattedResponse | undefined> => {
const result = await Db.collections.Execution!.findOne(req.params.id);
if (result === undefined) {
return undefined;
}
// Convert to response format in which the id is a string
(result as IExecutionFlatted as IExecutionFlattedResponse).id = result.id.toString();
return result as IExecutionFlatted as IExecutionFlattedResponse;
}));
// Retries a failed execution
this.app.post('/rest/executions/:id/retry', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<string> => {
// Get the data to execute
const fullExecutionDataFlatted = await Db.collections.Execution!.findOne(req.params.id);
if (fullExecutionDataFlatted === undefined) {
throw new ResponseHelper.ReponseError(`The execution with the id "${req.params.id}" does not exist.`, 404, 404);
}
const fullExecutionData = ResponseHelper.unflattenExecutionData(fullExecutionDataFlatted);
if (fullExecutionData.finished === true) {
throw new Error('The execution did succeed and can so not be retried.');
}
const executionMode = 'retry';
const nodeTypes = NodeTypes();
const workflowInstance = new Workflow(req.params.id, fullExecutionData.workflowData.nodes, fullExecutionData.workflowData.connections, false, nodeTypes, fullExecutionData.workflowData.staticData, fullExecutionData.workflowData.settings);
const additionalData = await WorkflowExecuteAdditionalData.get(executionMode, fullExecutionData.workflowData, workflowInstance, undefined, req.params.id);
const workflowExecute = new WorkflowExecute(additionalData, executionMode);
return workflowExecute.runExecutionData(workflowInstance, fullExecutionData.data);
}));
// Delete Executions
// INFORMATION: We use POST instead of DELETE to not run into any issues
// with the query data getting to long
this.app.post('/rest/executions/delete', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<void> => {
const deleteData = req.body as IExecutionDeleteFilter;
if (deleteData.deleteBefore !== undefined) {
const filters = {
startedAt: LessThanOrEqual(deleteData.deleteBefore),
};
if (deleteData.filters !== undefined) {
Object.assign(filters, deleteData.filters);
}
await Db.collections.Execution!.delete(filters);
} else if (deleteData.ids !== undefined) {
// Deletes all executions with the given ids
await Db.collections.Execution!.delete(deleteData.ids);
} else {
throw new Error('Required body-data "ids" or "deleteBefore" is missing!');
}
}));
// ----------------------------------------
// Executing Workflows
// ----------------------------------------
// Returns all the currently working executions
// this.app.get('/rest/executions-current', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IExecutionsCurrentSummaryExtended[]> => {
this.app.get('/rest/executions-current', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IExecutionsSummary[]> => {
const executingWorkflows = this.activeExecutionsInstance.getActiveExecutions();
const returnData: IExecutionsSummary[] = [];
let filter: any = {}; // tslint:disable-line:no-any
if (req.query.filter) {
filter = JSON.parse(req.query.filter);
}
for (const data of executingWorkflows) {
if (filter.workflowId !== undefined && filter.workflowId !== data.workflowId) {
continue;
}
returnData.push(
{
id: data.id.toString(),
workflowId: data.workflowId,
mode:data.mode,
startedAt: data.startedAt,
}
);
}
return returnData;
}));
// Forces the execution to stop
this.app.post('/rest/executions-current/:id/stop', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IExecutionsStopData> => {
const executionId = req.params.id;
// Stopt he execution and wait till it is done and we got the data
const result = await this.activeExecutionsInstance.stopExecution(executionId);
if (result === undefined) {
throw new Error(`The execution id "${executionId}" could not be found.`);
}
const returnData: IExecutionsStopData = {
mode: result.mode,
startedAt: result.startedAt,
stoppedAt: result.stoppedAt,
finished: result.finished,
};
return returnData;
}));
// Removes a test webhook
this.app.delete('/rest/test-webhook/:id', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<boolean> => {
const workflowId = req.params.id;
return this.testWebhooks.cancelTestWebhook(workflowId);
}));
// ----------------------------------------
// Options
// ----------------------------------------
// Returns all the available timezones
this.app.get('/rest/options/timezones', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<object> => {
return timezones;
}));
// ----------------------------------------
// Settings
// ----------------------------------------
// Returns the settings which are needed in the UI
this.app.get('/rest/settings', ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IN8nUISettings> => {
return {
endpointWebhook: this.endpointWebhook,
endpointWebhookTest: this.endpointWebhookTest,
saveManualRuns: this.saveManualRuns,
timezone: this.timezone,
urlBaseWebhook: WebhookHelpers.getWebhookBaseUrl(),
};
}));
// ----------------------------------------
// Webhooks
// ----------------------------------------
// GET webhook requests
this.app.get(`/${this.endpointWebhook}/*`, async (req: express.Request, res: express.Response) => {
console.log('\n*** WEBHOOK CALLED (GET) ***');
// Cut away the "/webhook/" to get the registred part of the url
const requestUrl = (req as ICustomRequest).parsedUrl!.pathname!.slice(this.endpointWebhook.length + 2);
let response;
try {
response = await this.activeWorkflowRunner.executeWebhook('GET', requestUrl, req, res);
} catch (error) {
ResponseHelper.sendErrorResponse(res, error);
return ;
}
if (response.noWebhookResponse === true) {
// Nothing else to do as the response got already sent
return;
}
ResponseHelper.sendSuccessResponse(res, response.data, true);
});
// POST webhook requests
this.app.post(`/${this.endpointWebhook}/*`, async (req: express.Request, res: express.Response) => {
console.log('\n*** WEBHOOK CALLED (POST) ***');
// Cut away the "/webhook/" to get the registred part of the url
const requestUrl = (req as ICustomRequest).parsedUrl!.pathname!.slice(this.endpointWebhook.length + 2);
let response;
try {
response = await this.activeWorkflowRunner.executeWebhook('POST', requestUrl, req, res);
} catch (error) {
ResponseHelper.sendErrorResponse(res, error);
return;
}
if (response.noWebhookResponse === true) {
// Nothing else to do as the response got already sent
return;
}
ResponseHelper.sendSuccessResponse(res, response.data, true);
});
// GET webhook requests (test for UI)
this.app.get(`/${this.endpointWebhookTest}/*`, async (req: express.Request, res: express.Response) => {
console.log('\n*** WEBHOOK-TEST CALLED (GET) ***');
// Cut away the "/webhook-test/" to get the registred part of the url
const requestUrl = (req as ICustomRequest).parsedUrl!.pathname!.slice(this.endpointWebhookTest.length + 2);
let response;
try {
response = await this.testWebhooks.callTestWebhook('GET', requestUrl, req, res);
} catch (error) {
ResponseHelper.sendErrorResponse(res, error);
return;
}
if (response.noWebhookResponse === true) {
// Nothing else to do as the response got already sent
return;
}
ResponseHelper.sendSuccessResponse(res, response.data, true);
});
// POST webhook requests (test for UI)
this.app.post(`/${this.endpointWebhookTest}/*`, async (req: express.Request, res: express.Response) => {
console.log('\n*** WEBHOOK-TEST CALLED (POST) ***');
// Cut away the "/webhook-test/" to get the registred part of the url
const requestUrl = (req as ICustomRequest).parsedUrl!.pathname!.slice(this.endpointWebhookTest.length + 2);
let response;
try {
response = await this.testWebhooks.callTestWebhook('POST', requestUrl, req, res);
} catch (error) {
ResponseHelper.sendErrorResponse(res, error);
return;
}
if (response.noWebhookResponse === true) {
// Nothing else to do as the response got already sent
return;
}
ResponseHelper.sendSuccessResponse(res, response.data, true);
});
// Serve the website
this.app.use('/', express.static(__dirname + '/../../node_modules/n8n-editor-ui/dist', { index: 'index.html' }));
}
}
export function start() {
const PORT = config.get('urls.port');
const app = new App().app;
app.listen(PORT, () => {
console.log('n8n ready on port ' + PORT);
});
}

View File

@@ -0,0 +1,208 @@
import * as express from 'express';
import {
IResponseCallbackData,
Push,
ResponseHelper,
WebhookHelpers,
IWorkflowDb,
} from './';
import {
ActiveWebhooks,
} from 'n8n-core';
import {
IWebhookData,
IWorkflowExecuteAdditionalData,
WebhookHttpMethod,
Workflow,
WorkflowExecuteMode,
} from 'n8n-workflow';
const pushInstance = Push.getInstance();
export class TestWebhooks {
private testWebhookData: {
[key: string]: {
sessionId?: string;
timeout: NodeJS.Timeout,
workflowData: IWorkflowDb;
};
} = {};
private activeWebhooks: ActiveWebhooks | null = null;
constructor() {
this.activeWebhooks = new ActiveWebhooks();
this.activeWebhooks.testWebhooks = true;
}
/**
* Executes a test-webhook and returns the data. It also makes sure that the
* data gets additionally send to the UI. After the request got handled it
* automatically remove the test-webhook.
*
* @param {WebhookHttpMethod} httpMethod
* @param {string} path
* @param {express.Request} request
* @param {express.Response} response
* @returns {Promise<object>}
* @memberof TestWebhooks
*/
async callTestWebhook(httpMethod: WebhookHttpMethod, path: string, request: express.Request, response: express.Response): Promise<IResponseCallbackData> {
const webhookData: IWebhookData | undefined = this.activeWebhooks!.get(httpMethod, path);
if (webhookData === undefined) {
// The requested webhook is not registred
throw new ResponseHelper.ReponseError('The requested webhook is not registred.', 404, 404);
}
// Get the node which has the webhook defined to know where to start from and to
// get additional data
const workflowStartNode = webhookData.workflow.getNode(webhookData.node);
if (workflowStartNode === null) {
throw new ResponseHelper.ReponseError('Could not find node to process webhook.', 404, 404);
}
const webhookKey = this.activeWebhooks!.getWebhookKey(webhookData.httpMethod, webhookData.path);
return new Promise(async (resolve, reject) => {
try {
const executionMode = 'manual';
const executionId = await WebhookHelpers.executeWebhook(webhookData, this.testWebhookData[webhookKey].workflowData, workflowStartNode, executionMode, this.testWebhookData[webhookKey].sessionId, request, response, (error: Error | null, data: IResponseCallbackData) => {
if (error !== null) {
return reject(error);
}
resolve(data);
});
if (executionId === undefined) {
// The workflow did not run as the request was probably setup related
// or a ping so do not resolve the promise and wait for the real webhook
// request instead.
return;
}
// Inform editor-ui that webhook got received
if (this.testWebhookData[webhookKey].sessionId !== undefined) {
pushInstance.send(this.testWebhookData[webhookKey].sessionId!, 'testWebhookReceived', { workflowId: webhookData.workflow.id });
}
} catch (error) {
// Delete webhook also if an error is thrown
}
// Remove the webhook
clearTimeout(this.testWebhookData[webhookKey].timeout);
delete this.testWebhookData[webhookKey];
this.activeWebhooks!.removeByWorkflowId(webhookData.workflow.id!.toString());
});
}
/**
* Checks if it has to wait for webhook data to execute the workflow. If yes it waits
* for it and resolves with the result of the workflow if not it simply resolves
* with undefined
*
* @param {IWorkflowDb} workflowData
* @param {Workflow} workflow
* @returns {(Promise<IExecutionDb | undefined>)}
* @memberof TestWebhooks
*/
async needsWebhookData(workflowData: IWorkflowDb, workflow: Workflow, additionalData: IWorkflowExecuteAdditionalData, mode: WorkflowExecuteMode, sessionId?: string, destinationNode?: string): Promise<boolean> {
const webhooks = WebhookHelpers.getWorkflowWebhooks(workflow, additionalData, destinationNode);
if (webhooks.length === 0) {
// No Webhooks found
return false;
}
// Remove test-webhooks automatically if they do not get called (after 120 seconds)
const timeout = setTimeout(() => {
this.cancelTestWebhook(workflowData.id.toString());
}, 120000);
let key: string;
for (const webhookData of webhooks) {
await this.activeWebhooks!.add(webhookData, mode);
key = this.activeWebhooks!.getWebhookKey(webhookData.httpMethod, webhookData.path);
this.testWebhookData[key] = {
sessionId,
timeout,
workflowData,
};
}
return true;
}
/**
* Removes a test webhook of the workflow with the given id
*
* @param {string} workflowId
* @returns {boolean}
* @memberof TestWebhooks
*/
cancelTestWebhook(workflowId: string): boolean {
let foundWebhook = false;
for (const webhookKey of Object.keys(this.testWebhookData)) {
const webhookData = this.testWebhookData[webhookKey];
if (webhookData.workflowData.id.toString() !== workflowId) {
continue;
}
foundWebhook = true;
clearTimeout(this.testWebhookData[webhookKey].timeout);
// Inform editor-ui that webhook got received
if (this.testWebhookData[webhookKey].sessionId !== undefined) {
try {
pushInstance.send(this.testWebhookData[webhookKey].sessionId!, 'testWebhookDeleted', { workflowId });
} catch (error) {
// Could not inform editor, probably is not connected anymore. So sipmly go on.
}
}
// Remove the webhook
delete this.testWebhookData[webhookKey];
this.activeWebhooks!.removeByWorkflowId(workflowId);
}
return foundWebhook;
}
/**
* Removes all the currently active test webhooks
*/
async removeAll(): Promise<void> {
if (this.activeWebhooks === null) {
return;
}
return this.activeWebhooks.removeAll();
}
}
let testWebhooksInstance: TestWebhooks | undefined;
export function getInstance(): TestWebhooks {
if (testWebhooksInstance === undefined) {
testWebhooksInstance = new TestWebhooks();
}
return testWebhooksInstance;
}

View File

@@ -0,0 +1,334 @@
import * as express from 'express';
import {
GenericHelpers,
IExecutionDb,
IResponseCallbackData,
IWorkflowDb,
ResponseHelper,
WorkflowExecuteAdditionalData,
} from './';
import {
BINARY_ENCODING,
ActiveExecutions,
NodeExecuteFunctions,
WorkflowExecute,
} from 'n8n-core';
import {
IBinaryKeyData,
IDataObject,
IExecuteData,
INode,
IRun,
IRunExecutionData,
ITaskData,
IWebhookData,
IWorkflowExecuteAdditionalData,
NodeHelpers,
Workflow,
WorkflowExecuteMode,
} from 'n8n-workflow';
const activeExecutions = ActiveExecutions.getInstance();
/**
* Returns the data of the last executed node
*
* @export
* @param {IRun} inputData
* @returns {(ITaskData | undefined)}
*/
export function getDataLastExecutedNodeData(inputData: IRun): ITaskData | undefined {
const runData = inputData.data.resultData.runData;
const lastNodeExecuted = inputData.data.resultData.lastNodeExecuted;
if (lastNodeExecuted === undefined) {
return undefined;
}
if (runData[lastNodeExecuted] === undefined) {
return undefined;
}
return runData[lastNodeExecuted][runData[lastNodeExecuted].length - 1];
}
/**
* Returns all the webhooks which should be created for the give workflow
*
* @export
* @param {string} workflowId
* @param {Workflow} workflow
* @returns {IWebhookData[]}
*/
export function getWorkflowWebhooks(workflow: Workflow, additionalData: IWorkflowExecuteAdditionalData, destinationNode?: string): IWebhookData[] {
// Check all the nodes in the workflow if they have webhooks
const returnData: IWebhookData[] = [];
let parentNodes: string[] | undefined;
if (destinationNode !== undefined) {
parentNodes = workflow.getParentNodes(destinationNode);
}
for (const node of Object.values(workflow.nodes)) {
if (parentNodes !== undefined && !parentNodes.includes(node.name)) {
// If parentNodes are given check only them if they have webhooks
// and no other ones
continue;
}
returnData.push.apply(returnData, NodeHelpers.getNodeWebhooks(workflow, node, additionalData));
}
return returnData;
}
/**
* Executes a webhook
*
* @export
* @param {IWebhookData} webhookData
* @param {IWorkflowDb} workflowData
* @param {INode} workflowStartNode
* @param {WorkflowExecuteMode} executionMode
* @param {(string | undefined)} sessionId
* @param {express.Request} req
* @param {express.Response} res
* @param {((error: Error | null, data: IResponseCallbackData) => void)} responseCallback
* @returns {(Promise<string | undefined>)}
*/
export async function executeWebhook(webhookData: IWebhookData, workflowData: IWorkflowDb, workflowStartNode: INode, executionMode: WorkflowExecuteMode, sessionId: string | undefined, req: express.Request, res: express.Response, responseCallback: (error: Error | null, data: IResponseCallbackData) => void): Promise<string | undefined> {
// Get the nodeType to know which responseMode is set
const nodeType = webhookData.workflow.nodeTypes.getByName(workflowStartNode.type);
if (nodeType === undefined) {
const errorMessage = `The type of the webhook node "${workflowStartNode.name}" is not known.`;
responseCallback(new Error(errorMessage), {});
throw new ResponseHelper.ReponseError(errorMessage, 500, 500);
}
// Get the responseMode
const reponseMode = webhookData.workflow.getWebhookParameterValue(workflowStartNode, webhookData.webhookDescription, 'reponseMode', 'onReceived');
if (!['onReceived', 'lastNode'].includes(reponseMode as string)) {
// If the mode is not known we error. Is probably best like that instead of using
// the default that people know as early as possible (probably already testing phase)
// that something does not resolve properly.
const errorMessage = `The response mode ${reponseMode} is not valid!.`;
responseCallback(new Error(errorMessage), {});
throw new ResponseHelper.ReponseError(errorMessage, 500, 500);
}
// Prepare everything that is needed to run the workflow
const additionalData = await WorkflowExecuteAdditionalData.get(executionMode, workflowData, webhookData.workflow, sessionId);
const workflowExecute = new WorkflowExecute(additionalData, executionMode);
// Add the Response and Request so that this data can be accessed in the node
additionalData.httpRequest = req;
additionalData.httpResponse = res;
let didSendResponse = false;
try {
// Run the webhook function to see what should be returned and if
// the workflow should be executed or not
const webhookResultData = await webhookData.workflow.runWebhook(workflowStartNode, additionalData, NodeExecuteFunctions, executionMode);
if (webhookResultData.noWebhookResponse === true) {
// The response got already send
responseCallback(null, {
noWebhookResponse: true,
});
didSendResponse = true;
}
if (webhookResultData.workflowData === undefined) {
// Workflow should not run
if (webhookResultData.webhookResponse !== undefined) {
// Data to respond with is given
responseCallback(null, {
data: webhookResultData.webhookResponse
});
} else {
// Send default response
responseCallback(null, {
data: {
message: 'Webhook call got received.',
},
});
}
return;
}
// Now that we know that the workflow should run we can return the default respons
// directly if responseMode it set to "onReceived" and a respone should be sent
if (reponseMode === 'onReceived' && didSendResponse === false) {
// Return response directly and do not wait for the workflow to finish
if (webhookResultData.webhookResponse !== undefined) {
// Data to respond with is given
responseCallback(null, {
data: webhookResultData.webhookResponse,
});
} else {
responseCallback(null, {
data: {
message: 'Workflow got started.',
}
});
}
didSendResponse = true;
}
// Initialize the data of the webhook node
const nodeExecutionStack: IExecuteData[] = [];
nodeExecutionStack.push(
{
node: workflowStartNode,
data: {
main: webhookResultData.workflowData,
},
},
);
const runExecutionData: IRunExecutionData = {
startData: {
},
resultData: {
runData: {},
},
executionData: {
contextData: {},
nodeExecutionStack,
waitingExecution: {},
},
};
// Start now to run the workflow
const executionId = await workflowExecute.runExecutionData(webhookData.workflow, runExecutionData);
// Get a promise which resolves when the workflow did execute and send then response
const executePromise = activeExecutions.getPostExecutePromise(executionId) as Promise<IExecutionDb | undefined>;
executePromise.then((data) => {
if (data === undefined) {
if (didSendResponse === false) {
responseCallback(null, {
data: {
message: 'Workflow did execute sucessfully but no data got returned.',
}
});
didSendResponse = true;
}
return undefined;
}
const returnData = getDataLastExecutedNodeData(data);
if (returnData === undefined) {
if (didSendResponse === false) {
responseCallback(null, {
data: {
message: 'Workflow did execute sucessfully but the last node did not return any data.',
}
});
}
didSendResponse = true;
return data;
} else if (returnData.error !== undefined) {
if (didSendResponse === false) {
responseCallback(null, {
data: {
message: 'Workflow did error.',
}
});
}
didSendResponse = true;
return data;
}
const reponseData = webhookData.workflow.getWebhookParameterValue(workflowStartNode, webhookData.webhookDescription, 'reponseData', 'firstEntryJson');
if (didSendResponse === false) {
let data: IDataObject | IDataObject[];
if (reponseData === 'firstEntryJson') {
// Return the JSON data of the first entry
data = returnData.data!.main[0]![0].json;
} else if (reponseData === 'firstEntryBinary') {
// Return the binary data of the first entry
data = returnData.data!.main[0]![0];
if (data.binary === undefined) {
responseCallback(new Error('No binary data to return got found.'), {});
}
const responseBinaryPropertyName = webhookData.workflow.getWebhookParameterValue(workflowStartNode, webhookData.webhookDescription, 'responseBinaryPropertyName', 'data');
if (responseBinaryPropertyName === undefined) {
responseCallback(new Error('No "responseBinaryPropertyName" is set.'), {});
}
const binaryData = (data.binary as IBinaryKeyData)[responseBinaryPropertyName as string];
if (binaryData === undefined) {
responseCallback(new Error(`The binary property "${responseBinaryPropertyName}" which should be returned does not exist.`), {});
}
// Send the webhook response manually
res.setHeader('Content-Type', binaryData.mimeType);
res.end(Buffer.from(binaryData.data, BINARY_ENCODING));
responseCallback(null, {
noWebhookResponse: true,
});
} else {
// Return the JSON data of all the entries
data = [];
for (const entry of returnData.data!.main[0]!) {
data.push(entry.json);
}
}
responseCallback(null, {
data,
});
}
didSendResponse = true;
return data;
})
.catch((e) => {
if (didSendResponse === false) {
responseCallback(new Error('There was a problem executing the workflow.'), {});
}
throw new ResponseHelper.ReponseError(e.message, 500, 500);
});
return executionId;
} catch (e) {
if (didSendResponse === false) {
responseCallback(new Error('There was a problem executing the workflow.'), {});
}
throw new ResponseHelper.ReponseError(e.message, 500, 500);
}
}
/**
* Returns the base URL of the webhooks
*
* @export
* @returns
*/
export function getWebhookBaseUrl() {
let urlBaseWebhook = GenericHelpers.getBaseUrl();
if (process.env.WEBHOOK_TUNNEL_URL !== undefined) {
urlBaseWebhook = process.env.WEBHOOK_TUNNEL_URL;
}
return urlBaseWebhook;
}

View File

@@ -0,0 +1,38 @@
import {
Db,
} from './';
import {
INode,
IWorkflowCredentials
} from 'n8n-workflow';
export async function WorkflowCredentials(nodes: INode[]): Promise<IWorkflowCredentials> {
// Go through all nodes to find which credentials are needed to execute the workflow
const returnCredentials: IWorkflowCredentials = {};
let node, type, name, foundCredentials;
for (node of nodes) {
if (!node.credentials) {
continue;
}
for (type of Object.keys(node.credentials)) {
if (!returnCredentials.hasOwnProperty(type)) {
returnCredentials[type] = {};
}
name = node.credentials[type];
if (!returnCredentials[type].hasOwnProperty(name)) {
foundCredentials = await Db.collections.Credentials!.find({ name, type });
if (!foundCredentials.length) {
throw new Error(`Could not find credentials for type "${type}" with name "${name}".`);
}
returnCredentials[type][name] = foundCredentials[0];
}
}
}
return returnCredentials;
}

View File

@@ -0,0 +1,210 @@
import {
Db,
IExecutionDb,
IExecutionFlattedDb,
IPushDataExecutionFinished,
IPushDataNodeExecuteAfter,
IPushDataNodeExecuteBefore,
IWorkflowBase,
Push,
ResponseHelper,
WebhookHelpers,
WorkflowCredentials,
WorkflowHelpers,
} from './';
import {
UserSettings,
} from "n8n-core";
import {
IRun,
ITaskData,
IWorkflowExecuteAdditionalData,
WorkflowExecuteMode,
Workflow,
} from 'n8n-workflow';
import * as config from 'config';
const pushInstance = Push.getInstance();
/**
* Checks if there was an error and if errorWorkflow is defined. If so it collects
* all the data and executes it
*
* @param {IWorkflowBase} workflowData The workflow which got executed
* @param {IRun} fullRunData The run which produced the error
* @param {WorkflowExecuteMode} mode The mode in which the workflow which did error got started in
* @param {string} [executionId] The id the execution got saved as
*/
function executeErrorWorkflow(workflowData: IWorkflowBase, fullRunData: IRun, mode: WorkflowExecuteMode, executionId?: string): void {
// Check if there was an error and if so if an errorWorkflow is set
if (fullRunData.data.resultData.error !== undefined && workflowData.settings !== undefined && workflowData.settings.errorWorkflow) {
const workflowErrorData = {
execution: {
id: executionId,
error: fullRunData.data.resultData.error,
lastNodeExecuted: fullRunData.data.resultData.lastNodeExecuted!,
mode,
},
workflow: {
id: workflowData.id !== undefined ? workflowData.id.toString() as string : undefined,
name: workflowData.name,
}
};
// Run the error workflow
WorkflowHelpers.executeErrorWorkflow(workflowData.settings.errorWorkflow as string, workflowErrorData);
}
}
const hooks = (mode: WorkflowExecuteMode, workflowData: IWorkflowBase, workflowInstance: Workflow, sessionId?: string, retryOf?: string) => {
return {
nodeExecuteBefore: [
async (executionId: string, nodeName: string): Promise<void> => {
if (sessionId === undefined) {
return;
}
const sendData: IPushDataNodeExecuteBefore = {
executionId,
nodeName,
};
pushInstance.send(sessionId, 'nodeExecuteBefore', sendData);
},
],
nodeExecuteAfter: [
async (executionId: string, nodeName: string, data: ITaskData): Promise<void> => {
if (sessionId === undefined) {
return;
}
const sendData: IPushDataNodeExecuteAfter = {
executionId,
nodeName,
data,
};
pushInstance.send(sessionId, 'nodeExecuteAfter', sendData);
},
],
workflowExecuteAfter: [
async (fullRunData: IRun, executionId: string): Promise<void> => {
try {
if (sessionId !== undefined) {
// Clone the object except the runData. That one is not supposed
// to be send. Because that data got send piece by piece after
// each node which finished executing
const pushRunData = {
...fullRunData,
data: {
...fullRunData.data,
resultData: {
...fullRunData.data.resultData,
runData: {},
},
},
};
// Push data to editor-ui once workflow finished
const sendData: IPushDataExecutionFinished = {
executionId,
data: pushRunData,
};
pushInstance.send(sessionId, 'executionFinished', sendData);
}
const workflowSavePromise = WorkflowHelpers.saveStaticData(workflowInstance);
let saveManualRuns = config.get('executions.saveManualRuns') as boolean;
if (workflowInstance.settings !== undefined && workflowInstance.settings.saveManualRuns !== undefined) {
// Apply to workflow override
saveManualRuns = workflowInstance.settings.saveManualRuns as boolean;
}
if (mode === 'manual' && saveManualRuns === false) {
if (workflowSavePromise !== undefined) {
// If workflow had to be saved wait till it is done
await workflowSavePromise;
}
// For now do not save manual executions
// TODO: Later that should be configurable. Think about what to do
// with the workflow.id when not saved yet or currently differes from saved version (save diff?!?!)
executeErrorWorkflow(workflowData, fullRunData, mode);
return;
}
// TODO: Should maybe have different log-modes like
// to save all data, only first input, only last node output, ....
// or depending on success to only save all on error to be
// able to start it again where it ended (but would then also have to save active data)
const fullExecutionData: IExecutionDb = {
data: fullRunData.data,
mode: fullRunData.mode,
finished: fullRunData.finished ? fullRunData.finished : false,
startedAt: fullRunData.startedAt,
stoppedAt: fullRunData.stoppedAt,
workflowData,
};
if (retryOf !== undefined) {
fullExecutionData.retryOf = retryOf.toString();
}
if (workflowData.id !== undefined && WorkflowHelpers.isWorkflowIdValid(workflowData.id.toString()) === true) {
fullExecutionData.workflowId = workflowData.id.toString();
}
const executionData = ResponseHelper.flattenExecutionData(fullExecutionData);
// Save the Execution in DB
const executionResult = await Db.collections.Execution!.save(executionData as IExecutionFlattedDb);
if (fullRunData.finished === true && retryOf !== undefined) {
// If the retry was successful save the reference it on the original execution
// await Db.collections.Execution!.save(executionData as IExecutionFlattedDb);
await Db.collections.Execution!.update(retryOf, { retrySuccessId: executionResult.id });
}
if (workflowSavePromise !== undefined) {
// If workflow had to be saved wait till it is done
await workflowSavePromise;
}
executeErrorWorkflow(workflowData, fullRunData, mode, executionResult ? executionResult.id as string : undefined);
} catch (error) {
executeErrorWorkflow(workflowData, fullRunData, mode);
}
},
]
};
};
export async function get(mode: WorkflowExecuteMode, workflowData: IWorkflowBase, workflowInstance: Workflow, sessionId?: string, retryOf?: string): Promise<IWorkflowExecuteAdditionalData> {
const urlBaseWebhook = WebhookHelpers.getWebhookBaseUrl();
const timezone = config.get('timezone') as string;
const webhookBaseUrl = urlBaseWebhook + config.get('urls.endpointWebhook') as string;
const webhookTestBaseUrl = urlBaseWebhook + config.get('urls.endpointWebhookTest') as string;
const encryptionKey = await UserSettings.getEncryptionKey();
if (encryptionKey === undefined) {
throw new Error('No encryption key got found to decrypt the credentials!');
}
return {
credentials: await WorkflowCredentials(workflowData.nodes),
hooks: hooks(mode, workflowData, workflowInstance, sessionId, retryOf),
encryptionKey,
timezone,
webhookBaseUrl,
webhookTestBaseUrl,
};
}

View File

@@ -0,0 +1,152 @@
import {
Db,
IWorkflowErrorData,
NodeTypes,
WorkflowExecuteAdditionalData,
} from './';
import {
WorkflowExecute,
} from 'n8n-core';
import {
IExecuteData,
INode,
IRunExecutionData,
Workflow,
} from 'n8n-workflow';
import * as config from 'config';
const ERROR_TRIGGER_TYPE = config.get('nodes.errorTriggerType') as string;
/**
* Returns if the given id is a valid workflow id
*
* @param {(string | null | undefined)} id The id to check
* @returns {boolean}
* @memberof App
*/
export function isWorkflowIdValid (id: string | null | undefined | number): boolean {
if (typeof id === 'string') {
id = parseInt(id, 10);
}
if (isNaN(id as number)) {
return false;
}
return true;
}
/**
* Executes the error workflow
*
* @export
* @param {string} workflowId The id of the error workflow
* @param {IWorkflowErrorData} workflowErrorData The error data
* @returns {Promise<void>}
*/
export async function executeErrorWorkflow(workflowId: string, workflowErrorData: IWorkflowErrorData): Promise<void> {
// Wrap everything in try/catch to make sure that no errors bubble up and all get caught here
try {
const workflowData = await Db.collections.Workflow!.findOne({ id: workflowId });
if (workflowData === undefined) {
// The error workflow could not be found
console.error(`ERROR: Calling Error Workflow for "${workflowErrorData.workflow.id}". Could not find error workflow "${workflowId}"`);
return;
}
const executionMode = 'error';
const nodeTypes = NodeTypes();
const workflowInstance = new Workflow(workflowId, workflowData.nodes, workflowData.connections, workflowData.active, nodeTypes, undefined, workflowData.settings);
let node: INode;
let workflowStartNode: INode | undefined;
for (const nodeName of Object.keys(workflowInstance.nodes)) {
node = workflowInstance.nodes[nodeName];
if (node.type === ERROR_TRIGGER_TYPE) {
workflowStartNode = node;
}
}
if (workflowStartNode === undefined) {
console.error(`ERROR: Calling Error Workflow for "${workflowErrorData.workflow.id}". Could not find "${ERROR_TRIGGER_TYPE}" in workflow "${workflowId}"`);
return;
}
const additionalData = await WorkflowExecuteAdditionalData.get(executionMode, workflowData, workflowInstance);
// Can execute without webhook so go on
const workflowExecute = new WorkflowExecute(additionalData, executionMode);
// Initialize the data of the webhook node
const nodeExecutionStack: IExecuteData[] = [];
nodeExecutionStack.push(
{
node: workflowStartNode,
data: {
main: [
[
{
json: workflowErrorData
}
]
],
},
},
);
const runExecutionData: IRunExecutionData = {
startData: {
},
resultData: {
runData: {},
},
executionData: {
contextData: {},
nodeExecutionStack,
waitingExecution: {},
},
};
// Start now to run the workflow
await workflowExecute.runExecutionData(workflowInstance, runExecutionData);
} catch (error) {
console.error(`ERROR: Calling Error Workflow for "${workflowErrorData.workflow.id}": ${error.message}`);
}
}
/**
* Saves the static data if it changed
*
* @export
* @param {Workflow} workflow
* @returns {Promise <void>}
*/
export async function saveStaticData(workflow: Workflow): Promise <void> {
if (workflow.staticData.__dataChanged === true) {
// Static data of workflow changed and so has to be saved
if (isWorkflowIdValid(workflow.id) === true) {
// Workflow is saved so update in database
try {
await Db.collections.Workflow!
.update(workflow.id!, {
staticData: workflow.staticData,
});
workflow.staticData.__dataChanged = false;
} catch (e) {
// TODO: Add proper logging!
console.error(`There was a problem saving the workflow with id "${workflow.id}" to save changed staticData: ${e.message}`);
}
}
}
}

View File

@@ -0,0 +1,7 @@
import * as MongoDb from './mongodb';
import * as SQLite from './sqlite';
export {
MongoDb,
SQLite,
};

View File

@@ -0,0 +1,41 @@
import {
ICredentialNodeAccess,
} from 'n8n-workflow';
import {
ICredentialsDb,
} from '../../';
import {
Column,
Entity,
Index,
ObjectID,
ObjectIdColumn,
} from "typeorm";
@Entity()
export class CredentialsEntity implements ICredentialsDb {
@ObjectIdColumn()
id: ObjectID;
@Column()
name: string;
@Column()
data: string;
@Index()
@Column()
type: string;
@Column('json')
nodesAccess: ICredentialNodeAccess[];
@Column()
createdAt: number;
@Column()
updatedAt: number;
}

View File

@@ -0,0 +1,51 @@
import {
WorkflowExecuteMode,
} from 'n8n-workflow';
import {
IExecutionFlattedDb,
IWorkflowDb,
} from '../../';
import {
Column,
Entity,
Index,
ObjectID,
ObjectIdColumn,
} from "typeorm";
@Entity()
export class ExecutionEntity implements IExecutionFlattedDb {
@ObjectIdColumn()
id: ObjectID;
@Column()
data: string;
@Column()
finished: boolean;
@Column()
mode: WorkflowExecuteMode;
@Column()
retryOf: string;
@Column()
retrySuccessId: string;
@Column()
startedAt: number;
@Column()
stoppedAt: number;
@Column('json')
workflowData: IWorkflowDb;
@Index()
@Column()
workflowId: string;
}

View File

@@ -0,0 +1,48 @@
import {
IConnections,
IDataObject,
INode,
IWorkflowSettings,
} from 'n8n-workflow';
import {
IWorkflowDb,
} from '../../';
import {
Column,
Entity,
ObjectID,
ObjectIdColumn,
} from "typeorm";
@Entity()
export class WorkflowEntity implements IWorkflowDb {
@ObjectIdColumn()
id: ObjectID;
@Column()
name: string;
@Column()
active: boolean;
@Column('json')
nodes: INode[];
@Column('json')
connections: IConnections;
@Column()
createdAt: number;
@Column()
updatedAt: number;
@Column('json')
settings?: IWorkflowSettings;
@Column('json')
staticData?: IDataObject;
}

View File

@@ -0,0 +1,3 @@
export * from './CredentialsEntity';
export * from './ExecutionEntity';
export * from './WorkflowEntity';

View File

@@ -0,0 +1,44 @@
import {
ICredentialNodeAccess,
} from 'n8n-workflow';
import {
ICredentialsDb,
} from '../../';
import {
Column,
Entity,
Index,
PrimaryGeneratedColumn,
} from "typeorm";
@Entity()
export class CredentialsEntity implements ICredentialsDb {
@PrimaryGeneratedColumn()
id: number;
@Column({
length: 128
})
name: string;
@Column('text')
data: string;
@Index()
@Column({
length: 32
})
type: string;
@Column('simple-json')
nodesAccess: ICredentialNodeAccess[];
@Column()
createdAt: number;
@Column()
updatedAt: number;
}

View File

@@ -0,0 +1,53 @@
import {
WorkflowExecuteMode,
} from 'n8n-workflow';
import {
IExecutionFlattedDb,
IWorkflowDb,
} from '../../';
import {
Column,
Entity,
Index,
ManyToOne,
PrimaryGeneratedColumn,
} from "typeorm";
import { WorkflowEntity } from './WorkflowEntity';
@Entity()
export class ExecutionEntity implements IExecutionFlattedDb {
@PrimaryGeneratedColumn()
id: number;
@Column('text')
data: string;
@Column()
finished: boolean;
@Column()
mode: WorkflowExecuteMode;
@Column({ nullable: true })
retryOf: string;
@Column({ nullable: true })
retrySuccessId: string;
@Column()
startedAt: number;
@Column()
stoppedAt: number;
@Column('simple-json')
workflowData: IWorkflowDb;
@Index()
@Column({ nullable: true })
workflowId: string;
}

View File

@@ -0,0 +1,55 @@
import {
IConnections,
IDataObject,
INode,
IWorkflowSettings,
} from 'n8n-workflow';
import {
IWorkflowDb,
} from '../../';
import {
Column,
Entity,
PrimaryGeneratedColumn,
} from "typeorm";
@Entity()
export class WorkflowEntity implements IWorkflowDb {
@PrimaryGeneratedColumn()
id: number;
@Column({
length: 128
})
name: string;
@Column()
active: boolean;
@Column('simple-json')
nodes: INode[];
@Column('simple-json')
connections: IConnections;
@Column()
createdAt: number;
@Column()
updatedAt: number;
@Column({
type: 'simple-json',
nullable: true,
})
settings?: IWorkflowSettings;
@Column({
type: 'simple-json',
nullable: true,
})
staticData?: IDataObject;
}

View File

@@ -0,0 +1,3 @@
export * from './CredentialsEntity';
export * from './ExecutionEntity';
export * from './WorkflowEntity';

29
packages/cli/src/index.ts Normal file
View File

@@ -0,0 +1,29 @@
export * from './CredentialTypes';
export * from './Interfaces';
export * from './LoadNodesAndCredentials';
export * from './NodeTypes';
export * from './WorkflowCredentials';
import * as ActiveWorkflowRunner from './ActiveWorkflowRunner';
import * as Db from './Db';
import * as GenericHelpers from './GenericHelpers';
import * as Push from './Push';
import * as ResponseHelper from './ResponseHelper';
import * as Server from './Server';
import * as TestWebhooks from './TestWebhooks';
import * as WebhookHelpers from './WebhookHelpers';
import * as WorkflowExecuteAdditionalData from './WorkflowExecuteAdditionalData';
import * as WorkflowHelpers from './WorkflowHelpers';
export {
ActiveWorkflowRunner,
Db,
GenericHelpers,
Push,
ResponseHelper,
Server,
TestWebhooks,
WebhookHelpers,
WorkflowExecuteAdditionalData,
WorkflowHelpers,
};

View File

@@ -0,0 +1,39 @@
{
"compilerOptions": {
"lib": [
"es2017"
],
"types": [
"node",
"jest"
],
"module": "commonjs",
"noImplicitAny": true,
"removeComments": true,
"strictNullChecks": true,
"forceConsistentCasingInFileNames": true,
"noImplicitReturns": true,
// Have to deactivate for TypeORM
// "strict": true,
"preserveConstEnums": true,
"declaration": true,
"outDir": "./dist/",
"target": "es2017",
"sourceMap": true,
"emitDecoratorMetadata": true,
"experimentalDecorators": true
},
"include": [
"**/*.d.ts",
"commands/**/*",
"index.ts",
"config/**/*",
"src/**/*",
"test/**/*",
],
"exclude": [
"dist/**/*",
"node_modules/**/*",
"**/*.spec.ts"
]
}

103
packages/cli/tslint.json Normal file
View File

@@ -0,0 +1,103 @@
{
"linterOptions": {
"exclude": [
"node_modules/**/*"
]
},
"defaultSeverity": "error",
"jsRules": {},
"rules": {
"array-type": [
true,
"array-simple"
],
"arrow-return-shorthand": true,
"ban": [
true,
{
"name": "Array",
"message": "tsstyle#array-constructor"
}
],
"ban-types": [
true,
[
"Object",
"Use {} instead."
],
[
"String",
"Use 'string' instead."
],
[
"Number",
"Use 'number' instead."
],
[
"Boolean",
"Use 'boolean' instead."
]
],
"class-name": true,
"curly": [
true,
"ignore-same-line"
],
"forin": true,
"jsdoc-format": true,
"label-position": true,
"member-access": [
true,
"no-public"
],
"new-parens": true,
"no-angle-bracket-type-assertion": true,
"no-any": true,
"no-arg": true,
"no-conditional-assignment": true,
"no-construct": true,
"no-debugger": true,
"no-default-export": true,
"no-duplicate-variable": true,
"no-inferrable-types": true,
"no-namespace": [
true,
"allow-declarations"
],
"no-reference": true,
"no-string-throw": true,
"no-unused-expression": true,
"no-var-keyword": true,
"object-literal-shorthand": true,
"only-arrow-functions": [
true,
"allow-declarations",
"allow-named-functions"
],
"prefer-const": true,
"radix": true,
"semicolon": [
true,
"always",
"ignore-bound-class-methods"
],
"switch-default": true,
"triple-equals": [
true,
"allow-null-check"
],
"use-isnan": true,
"quotes": [
"error",
"single"
],
"variable-name": [
true,
"check-format",
"ban-keywords",
"allow-leading-underscore",
"allow-trailing-underscore"
]
},
"rulesDirectory": []
}