Add tagging of workflows (#1647)

* clean up dropdown

* clean up focusoncreate

*  Ignore mistaken ID in POST /workflows

*  Fix undefined tag ID in PATCH /workflows

*  Shorten response for POST /tags

* remove scss mixins

* clean up imports

*  Implement validation with class-validator

* address ivan's comments

* implement modals

* Fix lint issues

* fix disabling shortcuts

* fix focus issues

* fix focus issues

* fix focus issues with modal

* fix linting issues

* use dispatch

* use constants for modal keys

* fix focus

* fix lint issues

* remove unused prop

* add modal root

* fix lint issues

* remove unused methods

* fix shortcut

* remove max width

*  Fix duplicate entry error for pg and MySQL

* update rename messaging

* update order of buttons

* fix firefox overflow on windows

* fix dropdown height

* 🔨 refactor tag crud controllers

* 🧹 remove unused imports

* use variable for number of items

* fix dropdown spacing

*  Restore type to fix build

*  Fix post-refactor PATCH /workflows/:id

*  Fix PATCH /workflows/:id for zero tags

*  Fix usage count becoming stringified

* address max's comments

* fix filter spacing

* fix blur bug

* address most of ivan's comments

* address tags type concern

* remove defaults

*  return tag id as string

* 🔨 add hooks to tag CUD operations

* 🏎 simplify timestamp pruning

* remove blur event

* fix onblur bug

*  Fix fs import to fix build

* address max's comments

* implement responsive tag container

* fix lint issues

* Set default dates in entities

* 👕 Fix lint in migrations

* update tag limits

* address ivan's comments

* remove rename, refactor header, implement new designs for save, remove responsive tag container

* update styling

* update styling

* implement responsive tag container

* implement header tags edit

* implement header tags edit

* fix lint issues

* implement expandable input

* minor fixes

* minor fixes

* use variable

* rename save as

* duplicate fixes

*  Implement unique workflow names

*  Create /workflows/new endpoint

* minor edit fixes

* lint fixes

* style fixes

* hook up saving name

* hook up tags

* clean up impl

* fix dirty state bug

* update limit

* update notification messages

* on click outside

* fix minor bug with count

* lint fixes

*  Add query string params to /workflows/new

* handle minor edge cases

* handle minor edge cases

* handle minor bugs; fix firefox dropdown issue

* Fix min width

* apply tags only after api success

* remove count fix

* 🚧 Adjust to new qs requirements

* clean up workflow tags impl, fix tags delete bug

* fix minor issue

* fix minor spacing issue

* disable wrap for ops

* fix viewport root; save on click in dropdown

* save button loading when saving name/tags

* implement max width on tags container

* implement cleaner create experience

* disable edit while updating

* codacy hex color

* refactor tags container

* fix clickability

* fix workflow open and count

* clean up structure

* fix up lint issues

*  Create migrations for unique workflow names

* fix button size

* increase workflow name limit for larger screen

* tslint fixes

* disable responsiveness for workflow modal

* rename event

* change min width for tags

* clean up pr

*  Adjust quotes in MySQL migration

*  Adjust quotes in Postgres migration

* address max's comments on styles

* remove success toasts

* add hover mode to name

* minor fixes

* refactor name preview

* fix name input not to jiggle

* finish up name input

* Fix up add tags

* clean up param

* clean up scss

* fix resizing name

* fix resizing name

* fix resize bug

* clean up edit spacing

* ignore on esc

* fix input bug

* focus input on clear

* build

* fix up add tags clickablity

* remove scrollbars

* move into folders

* clean up multiple patch req

* remove padding top from edit

* update tags on enter

* build

* rollout blur on enter behavior

* rollout esc behavior

* fix tags bug when duplicating tags

* move key to reload tags

* update header spacing

* build

* update hex case

* refactor workflow title

* remove unusued prop

* keep focus on error, fix bug on error

* Fix bug with name / tags toggle on error

* impl creating new workflow name

*  Refactor endpoint per new guidelines

* support naming endpoint

*  Refactor to support numeric suffixes

* 👕 Lint migrations for unique workflow names

*  Add migrations set default dates to indexes

* fix connection push bug

*  Lowercase default workflow name

*  Add prefixes to set default dates migration

*  Fix indentation on default dates migrations

*  Add temp ts-ignore for unrelated change

*  Adjust default dates migration for MySQL

Remove change to data column in credentials_entity, already covered by Omar's migration. Also, fix quotes from table prefix addition.

*  Adjust quotes in dates migration for PG

* fix safari color bug

* fix count bug

* fix scroll bugs in dropdown

* expand filter size

* apply box-sizing to main header

* update workflow names in executions to be wrapped by quotes

* fix bug where key is same in dropdown

* fix firefox bug

* move up push connection session

* 🔨 Remove mistakenly added nullable property

* 🔥 Remove unneeded index drop-create (PG)

* 🔥 Remove unneeded table copying

*  Merge dates migration with tags migration

* 🔨 Refactor endpoint and make wf name env

* dropdown colors in firefox

* update colors to use variables

* update thumb color

* change error message

* remove 100 char maximum

* fix bug with saving tags dropdowns multiple times

* update error message when no name

*  Update name missing toast message

*  Update workflow already exists message

* disable saving for executions

* fix bug causing modal to close

* make tags in workflow open clickable

* increase workflow limit to 3

* remove success notifications

* update header spacing

* escape tag names

* update tag and table colors

* remove tags from export

* build

* clean up push connection dependencies

* address ben's comments

* revert tags optional interface

* address comments

* update duplicate message

* build

* fix eol

* add one more eol

*  Update comment

* add hover style for workflow open, fix up font weight

Co-authored-by: Mutasem <mutdmour@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Mutasem Aldmour <4711238+mutdmour@users.noreply.github.com>
This commit is contained in:
Ben Hesseldieck
2021-05-29 20:31:21 +02:00
committed by GitHub
parent 335673d329
commit 05eec87d1d
92 changed files with 4602 additions and 1236 deletions

View File

@@ -10,6 +10,7 @@ import {
import {
getConnectionManager,
In,
Like,
} from 'typeorm';
import * as bodyParser from 'body-parser';
require('body-parser-xml')(bodyParser);
@@ -54,10 +55,9 @@ import {
IExternalHooksClass,
IN8nUISettings,
IPackageVersions,
IWorkflowBase,
ITagWithCountDb,
IWorkflowExecutionDataProcess,
IWorkflowResponse,
IWorkflowShortResponse,
LoadNodesAndCredentials,
NodeTypes,
Push,
@@ -67,6 +67,7 @@ import {
WebhookServer,
WorkflowCredentials,
WorkflowExecuteAdditionalData,
WorkflowHelpers,
WorkflowRunner,
} from './';
@@ -85,6 +86,7 @@ import {
INodePropertyOptions,
INodeTypeDescription,
IRunData,
IWorkflowBase,
IWorkflowCredentials,
Workflow,
WorkflowExecuteMode,
@@ -110,6 +112,11 @@ import * as Queue from '../src/Queue';
import { OptionsWithUrl } from 'request-promise-native';
import { Registry } from 'prom-client';
import * as TagHelpers from './TagHelpers';
import { TagEntity } from './databases/entities/TagEntity';
import { WorkflowEntity } from './databases/entities/WorkflowEntity';
import { WorkflowNameRequest } from './WorkflowHelpers';
class App {
app: express.Application;
@@ -119,6 +126,7 @@ class App {
endpointWebhookTest: string;
endpointPresetCredentials: string;
externalHooks: IExternalHooksClass;
defaultWorkflowName: string;
saveDataErrorExecution: string;
saveDataSuccessExecution: string;
saveManualExecutions: boolean;
@@ -142,6 +150,9 @@ class App {
this.endpointWebhook = config.get('endpoints.webhook') as string;
this.endpointWebhookTest = config.get('endpoints.webhookTest') as string;
this.defaultWorkflowName = config.get('workflows.defaultName') as string;
this.saveDataErrorExecution = config.get('executions.saveDataOnError') as string;
this.saveDataSuccessExecution = config.get('executions.saveDataOnSuccess') as string;
this.saveManualExecutions = config.get('executions.saveDataManualExecutions') as boolean;
@@ -484,25 +495,30 @@ class App {
// Creates a new workflow
this.app.post(`/${this.restEndpoint}/workflows`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowResponse> => {
this.app.post(`/${this.restEndpoint}/workflows`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<WorkflowEntity> => {
delete req.body.id; // ignore if sent by mistake
const incomingData = req.body;
const newWorkflowData = req.body as IWorkflowBase;
const newWorkflow = new WorkflowEntity();
newWorkflowData.name = newWorkflowData.name.trim();
newWorkflowData.createdAt = this.getCurrentDate();
newWorkflowData.updatedAt = this.getCurrentDate();
Object.assign(newWorkflow, incomingData);
newWorkflow.name = incomingData.name.trim();
newWorkflowData.id = undefined;
const incomingTagOrder = incomingData.tags.slice();
await this.externalHooks.run('workflow.create', [newWorkflowData]);
if (incomingData.tags.length) {
newWorkflow.tags = await Db.collections.Tag!.findByIds(incomingData.tags, { select: ['id', 'name'] });
}
// Save the workflow in DB
const result = await Db.collections.Workflow!.save(newWorkflowData);
await this.externalHooks.run('workflow.create', [newWorkflow]);
// Convert to response format in which the id is a string
(result as IWorkflowBase as IWorkflowResponse).id = result.id.toString();
return result as IWorkflowBase as IWorkflowResponse;
await WorkflowHelpers.validateWorkflow(newWorkflow);
const savedWorkflow = await Db.collections.Workflow!.save(newWorkflow).catch(WorkflowHelpers.throwDuplicateEntryError) as WorkflowEntity;
savedWorkflow.tags = TagHelpers.sortByRequestOrder(savedWorkflow.tags, incomingTagOrder);
// @ts-ignore
savedWorkflow.id = savedWorkflow.id.toString();
return savedWorkflow;
}));
@@ -535,47 +551,90 @@ class App {
// Returns workflows
this.app.get(`/${this.restEndpoint}/workflows`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowShortResponse[]> => {
const findQuery = {} as FindManyOptions;
this.app.get(`/${this.restEndpoint}/workflows`, ResponseHelper.send(async (req: express.Request, res: express.Response) => {
const findQuery: FindManyOptions<WorkflowEntity> = {
select: ['id', 'name', 'active', 'createdAt', 'updatedAt'],
relations: ['tags'],
};
if (req.query.filter) {
findQuery.where = JSON.parse(req.query.filter as string);
}
// Return only the fields we need
findQuery.select = ['id', 'name', 'active', 'createdAt', 'updatedAt'];
const workflows = await Db.collections.Workflow!.find(findQuery);
const results = await Db.collections.Workflow!.find(findQuery);
workflows.forEach(workflow => {
// @ts-ignore
workflow.id = workflow.id.toString();
// @ts-ignore
workflow.tags = workflow.tags.map(({ id, name }) => ({ id: id.toString(), name }));
});
return workflows;
}));
for (const entry of results) {
(entry as unknown as IWorkflowShortResponse).id = entry.id.toString();
this.app.get(`/${this.restEndpoint}/workflows/new`, ResponseHelper.send(async (req: WorkflowNameRequest, res: express.Response): Promise<{ name: string }> => {
const nameToReturn = req.query.name && req.query.name !== ''
? req.query.name
: this.defaultWorkflowName;
const workflows = await Db.collections.Workflow!.find({
select: ['name'],
where: { name: Like(`${nameToReturn}%`) },
});
// name is unique
if (workflows.length === 0) {
return { name: nameToReturn };
}
return results as unknown as IWorkflowShortResponse[];
const maxSuffix = workflows.reduce((acc: number, { name }) => {
const parts = name.split(`${nameToReturn} `);
if (parts.length > 2) return acc;
const suffix = Number(parts[1]);
if (!isNaN(suffix) && Math.ceil(suffix) > acc) {
acc = Math.ceil(suffix);
}
return acc;
}, 0);
// name is duplicate but no numeric suffixes exist yet
if (maxSuffix === 0) {
return { name: `${nameToReturn} 2` };
}
return { name: `${nameToReturn} ${maxSuffix + 1}` };
}));
// Returns a specific workflow
this.app.get(`/${this.restEndpoint}/workflows/:id`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowResponse | undefined> => {
const result = await Db.collections.Workflow!.findOne(req.params.id);
this.app.get(`/${this.restEndpoint}/workflows/:id`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<WorkflowEntity | undefined> => {
const workflow = await Db.collections.Workflow!.findOne(req.params.id, { relations: ['tags'] });
if (result === undefined) {
if (workflow === undefined) {
return undefined;
}
// Convert to response format in which the id is a string
(result as IWorkflowBase as IWorkflowResponse).id = result.id.toString();
return result as IWorkflowBase as IWorkflowResponse;
// @ts-ignore
workflow.id = workflow.id.toString();
// @ts-ignore
workflow.tags.forEach(tag => tag.id = tag.id.toString());
return workflow;
}));
// Updates an existing workflow
this.app.patch(`/${this.restEndpoint}/workflows/:id`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IWorkflowResponse> => {
this.app.patch(`/${this.restEndpoint}/workflows/:id`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<WorkflowEntity> => {
const { tags, ...updateData } = req.body;
const newWorkflowData = req.body as IWorkflowBase;
const id = req.params.id;
newWorkflowData.id = id;
updateData.id = id;
await this.externalHooks.run('workflow.update', [newWorkflowData]);
await this.externalHooks.run('workflow.update', [updateData]);
const isActive = await this.activeWorkflowRunner.isActive(id);
@@ -585,64 +644,78 @@ class App {
await this.activeWorkflowRunner.remove(id);
}
if (newWorkflowData.settings) {
if (newWorkflowData.settings.timezone === 'DEFAULT') {
if (updateData.settings) {
if (updateData.settings.timezone === 'DEFAULT') {
// Do not save the default timezone
delete newWorkflowData.settings.timezone;
delete updateData.settings.timezone;
}
if (newWorkflowData.settings.saveDataErrorExecution === 'DEFAULT') {
if (updateData.settings.saveDataErrorExecution === 'DEFAULT') {
// Do not save when default got set
delete newWorkflowData.settings.saveDataErrorExecution;
delete updateData.settings.saveDataErrorExecution;
}
if (newWorkflowData.settings.saveDataSuccessExecution === 'DEFAULT') {
if (updateData.settings.saveDataSuccessExecution === 'DEFAULT') {
// Do not save when default got set
delete newWorkflowData.settings.saveDataSuccessExecution;
delete updateData.settings.saveDataSuccessExecution;
}
if (newWorkflowData.settings.saveManualExecutions === 'DEFAULT') {
if (updateData.settings.saveManualExecutions === 'DEFAULT') {
// Do not save when default got set
delete newWorkflowData.settings.saveManualExecutions;
delete updateData.settings.saveManualExecutions;
}
if (parseInt(newWorkflowData.settings.executionTimeout as string, 10) === this.executionTimeout) {
if (parseInt(updateData.settings.executionTimeout as string, 10) === this.executionTimeout) {
// Do not save when default got set
delete newWorkflowData.settings.executionTimeout;
delete updateData.settings.executionTimeout;
}
}
newWorkflowData.updatedAt = this.getCurrentDate();
// required due to atomic update
updateData.updatedAt = this.getCurrentDate();
await Db.collections.Workflow!.update(id, newWorkflowData);
await this.externalHooks.run('workflow.afterUpdate', [newWorkflowData]);
await WorkflowHelpers.validateWorkflow(updateData);
await Db.collections.Workflow!.update(id, updateData).catch(WorkflowHelpers.throwDuplicateEntryError);
const tablePrefix = config.get('database.tablePrefix');
await TagHelpers.removeRelations(req.params.id, tablePrefix);
if (tags?.length) {
await TagHelpers.createRelations(req.params.id, tags, tablePrefix);
}
// We sadly get nothing back from "update". Neither if it updated a record
// nor the new value. So query now the hopefully updated entry.
const responseData = await Db.collections.Workflow!.findOne(id);
const workflow = await Db.collections.Workflow!.findOne(id, { relations: ['tags'] });
if (responseData === undefined) {
if (workflow === undefined) {
throw new ResponseHelper.ResponseError(`Workflow with id "${id}" could not be found to be updated.`, undefined, 400);
}
if (responseData.active === true) {
if (tags?.length) {
workflow.tags = TagHelpers.sortByRequestOrder(workflow.tags, tags);
}
await this.externalHooks.run('workflow.afterUpdate', [workflow]);
if (workflow.active === true) {
// When the workflow is supposed to be active add it again
try {
await this.externalHooks.run('workflow.activate', [responseData]);
await this.externalHooks.run('workflow.activate', [workflow]);
await this.activeWorkflowRunner.add(id, isActive ? 'update' : 'activate');
} catch (error) {
// If workflow could not be activated set it again to inactive
newWorkflowData.active = false;
await Db.collections.Workflow!.update(id, newWorkflowData);
updateData.active = false;
// @ts-ignore
await Db.collections.Workflow!.update(id, updateData);
// Also set it in the returned data
responseData.active = false;
workflow.active = false;
// Now return the original error for UI to display
throw error;
}
}
// Convert to response format in which the id is a string
(responseData as IWorkflowBase as IWorkflowResponse).id = responseData.id.toString();
return responseData as IWorkflowBase as IWorkflowResponse;
// @ts-ignore
workflow.id = workflow.id.toString();
return workflow;
}));
@@ -665,7 +738,6 @@ class App {
return true;
}));
this.app.post(`/${this.restEndpoint}/workflows/run`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<IExecutionPushResponse> => {
const workflowData = req.body.workflowData;
const runData: IRunData | undefined = req.body.runData;
@@ -713,6 +785,69 @@ class App {
};
}));
// Retrieves all tags, with or without usage count
this.app.get(`/${this.restEndpoint}/tags`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<TagEntity[] | ITagWithCountDb[]> => {
if (req.query.withUsageCount === 'true') {
const tablePrefix = config.get('database.tablePrefix');
return TagHelpers.getTagsWithCountDb(tablePrefix);
}
const tags = await Db.collections.Tag!.find({ select: ['id', 'name'] });
// @ts-ignore
tags.forEach(tag => tag.id = tag.id.toString());
return tags;
}));
// Creates a tag
this.app.post(`/${this.restEndpoint}/tags`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<TagEntity | void> => {
const newTag = new TagEntity();
newTag.name = req.body.name.trim();
await this.externalHooks.run('tag.beforeCreate', [newTag]);
await TagHelpers.validateTag(newTag);
const tag = await Db.collections.Tag!.save(newTag).catch(TagHelpers.throwDuplicateEntryError);
await this.externalHooks.run('tag.afterCreate', [tag]);
// @ts-ignore
tag.id = tag.id.toString();
return tag;
}));
// Updates a tag
this.app.patch(`/${this.restEndpoint}/tags/:id`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<TagEntity | void> => {
const { name } = req.body;
const { id } = req.params;
const newTag = new TagEntity();
newTag.id = Number(id);
newTag.name = name.trim();
await this.externalHooks.run('tag.beforeUpdate', [newTag]);
await TagHelpers.validateTag(newTag);
const tag = await Db.collections.Tag!.save(newTag).catch(TagHelpers.throwDuplicateEntryError);
await this.externalHooks.run('tag.afterUpdate', [tag]);
// @ts-ignore
tag.id = tag.id.toString();
return tag;
}));
// Deletes a tag
this.app.delete(`/${this.restEndpoint}/tags/:id`, ResponseHelper.send(async (req: express.Request, res: express.Response): Promise<boolean> => {
const id = Number(req.params.id);
await this.externalHooks.run('tag.beforeDelete', [id]);
await Db.collections.Tag!.delete({ id });
await this.externalHooks.run('tag.afterDelete', [id]);
return true;
}));
// Returns parameter values which normally get loaded from an external API or
// get generated dynamically
@@ -728,6 +863,7 @@ class App {
const nodeTypes = NodeTypes();
// @ts-ignore
const loadDataInstance = new LoadNodeParameterOptions(nodeType, nodeTypes, path, JSON.parse('' + req.query.currentNodeParameters), credentials!);
const workflowData = loadDataInstance.getWorkflowData() as IWorkflowBase;
@@ -892,8 +1028,6 @@ class App {
await this.externalHooks.run('credentials.create', [newCredentialsData]);
// Add special database related data
newCredentialsData.createdAt = this.getCurrentDate();
newCredentialsData.updatedAt = this.getCurrentDate();
// TODO: also add user automatically depending on who is logged in, if anybody is logged in