mohiit1502/Automata
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

feat: Migrate integer primary keys to nanoids (#6345)

Browse Source 
* first commit for postgres migration

* (not working)

* sqlite migration

* quicksave

* fix tests

* fix pg test

* fix postgres

* fix variables import

* fix execution saving

* add user settings fix

* change migration to single lines

* patch preferences endpoint

* cleanup

* improve variable import

* cleanup unusued code

* Update packages/cli/src/PublicApi/v1/handlers/workflows/workflows.handler.ts

Co-authored-by: Omar Ajoue <krynble@gmail.com>

* address review notes

* fix var update/import

* refactor: Separate execution data to its own table (#6323)

* wip: Temporary migration process

* refactor: Create boilerplate repository methods for executions

* fix: Lint issues

* refactor: Added search endpoint to repository

* refactor: Make the execution list work again

* wip: Updating how we create and update executions everywhere

* fix: Lint issues and remove most of the direct access to execution model

* refactor: Remove includeWorkflowData flag and fix more tests

* fix: Lint issues

* fix: Fixed ordering of executions for FE, removed transaction when saving execution and removed unnecessary update

* refactor: Add comment about missing feature

* refactor: Refactor counting executions

* refactor: Add migration for other dbms and fix issues found

* refactor: Fix lint issues

* refactor: Remove unnecessary comment and auto inject repo to internal hooks

* refactor: remove type assertion

* fix: Fix broken tests

* fix: Remove unnecessary import

* Remove unnecessary toString() call

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>

* fix: Address comments after review

* refactor: Remove unused import

* fix: Lint issues

* fix: Add correct migration files

---------

Co-authored-by: Iván Ovejero <ivov.src@gmail.com>

* remove null values from credential export

* fix: Fix an issue with queue mode where all running execution would be returned

* fix: Update n8n node to allow for workflow ids with letters

* set upstream on set branch

* remove typo

* add nodeAccess to credentials

* fix unsaved run check for undefined id

* fix(core): Rename version control feature to source control (#6480)

* rename versionControl to sourceControl

* fix source control tooltip wording

---------

Co-authored-by: Romain Minaud <romain.minaud@gmail.com>

* fix(editor): Pay 548 hide the set up version control button (#6485)

* feat(DebugHelper Node): Fix and include in main app (#6406)

* improve node a bit

* fixing continueOnFail() ton contain error in json

* improve pairedItem

* fix random data returning object results

* fix nanoId length typo

* update pnpm-lock file

---------

Co-authored-by: Marcus <marcus@n8n.io>

* fix(editor): Remove setup source control CTA button

* fix(editor): Remove setup source control CTA button

---------

Co-authored-by: Michael Auerswald <michael.auerswald@gmail.com>
Co-authored-by: Marcus <marcus@n8n.io>

* fix(editor): Update source control docs links (#6488)

* feat(DebugHelper Node): Fix and include in main app (#6406)

* improve node a bit

* fixing continueOnFail() ton contain error in json

* improve pairedItem

* fix random data returning object results

* fix nanoId length typo

* update pnpm-lock file

---------

Co-authored-by: Marcus <marcus@n8n.io>

* feat(editor): Replace root events with event bus events (no-changelog) (#6454)

* feat: replace root events with event bus events

* fix: prevent cypress from replacing global with globalThis in import path

* feat: remove emitter mixin

* fix: replace component events with event bus

* fix: fix linting issue

* fix: fix breaking expression switch

* chore: prettify ndv e2e suite code

* fix(editor): Update source control docs links

---------

Co-authored-by: Michael Auerswald <michael.auerswald@gmail.com>
Co-authored-by: Marcus <marcus@n8n.io>
Co-authored-by: Alex Grozav <alex@grozav.com>

* fix tag endpoint regex

---------

Co-authored-by: Omar Ajoue <krynble@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: Romain Minaud <romain.minaud@gmail.com>
Co-authored-by: Csaba Tuncsik <csaba@n8n.io>
Co-authored-by: Marcus <marcus@n8n.io>
Co-authored-by: Alex Grozav <alex@grozav.com>
This commit is contained in:
Michael Auerswald
2023-06-20 19:13:18 +02:00
committed by GitHub
parent da330f0648
commit c3ba0123ad
156 changed files with 3499 additions and 2594 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
packages/cli/test/integration/audit/credentials.risk.test.ts
View File

@@ -5,6 +5,7 @@ import { audit } from '@/audit';
import { CREDENTIALS_REPORT } from '@/audit/constants';
import { getRiskSection } from './utils';
import * as testDb from '../shared/testDb';
import { generateNanoId } from '@/databases/utils/generators';
beforeAll(async () => {
await testDb.init();
@@ -20,6 +21,7 @@ afterAll(async () => {
test('should report credentials not in any use', async () => {
const credentialDetails = {
id: generateNanoId(),
name: 'My Slack Credential',
data: 'U2FsdGVkX18WjITBG4IDqrGB1xE/uzVNjtwDAG3lP7E=',
type: 'slackApi',
@@ -27,6 +29,7 @@ test('should report credentials not in any use', async () => {
};
const workflowDetails = {
id: generateNanoId(),
name: 'My Test Workflow',
active: false,
connections: {},
@@ -57,13 +60,14 @@ test('should report credentials not in any use', async () => {
expect(section.location).toHaveLength(1);
expect(section.location[0]).toMatchObject({
id: '1',
id: credentialDetails.id,
name: 'My Slack Credential',
});
});
test('should report credentials not in active use', async () => {
const credentialDetails = {
id: generateNanoId(),
name: 'My Slack Credential',
data: 'U2FsdGVkX18WjITBG4IDqrGB1xE/uzVNjtwDAG3lP7E=',
type: 'slackApi',
@@ -73,6 +77,7 @@ test('should report credentials not in active use', async () => {
const credential = await Db.collections.Credentials.save(credentialDetails);
const workflowDetails = {
id: generateNanoId(),
name: 'My Test Workflow',
active: false,
connections: {},
@@ -107,6 +112,7 @@ test('should report credentials not in active use', async () => {
test('should report credential in not recently executed workflow', async () => {
const credentialDetails = {
id: generateNanoId(),
name: 'My Slack Credential',
data: 'U2FsdGVkX18WjITBG4IDqrGB1xE/uzVNjtwDAG3lP7E=',
type: 'slackApi',
@@ -116,6 +122,7 @@ test('should report credential in not recently executed workflow', async () => {
const credential = await Db.collections.Credentials.save(credentialDetails);
const workflowDetails = {
id: generateNanoId(),
name: 'My Test Workflow',
active: false,
connections: {},
@@ -142,16 +149,19 @@ test('should report credential in not recently executed workflow', async () => {
const date = new Date();
date.setDate(date.getDate() - config.getEnv('security.audit.daysAbandonedWorkflow') - 1);
await Db.collections.Execution.save({
data: '[]',
const savedExecution = await Db.collections.Execution.save({
finished: true,
mode: 'manual',
startedAt: date,
stoppedAt: date,
workflowData: workflow,
workflowId: workflow.id,
waitTill: null,
});
await Db.collections.ExecutionData.save({
execution: savedExecution,
data: '[]',
workflowData: workflow,
});
const testAudit = await audit(['credentials']);
@@ -170,6 +180,7 @@ test('should report credential in not recently executed workflow', async () => {
test('should not report credentials in recently executed workflow', async () => {
const credentialDetails = {
id: generateNanoId(),
name: 'My Slack Credential',
data: 'U2FsdGVkX18WjITBG4IDqrGB1xE/uzVNjtwDAG3lP7E=',
type: 'slackApi',
@@ -179,6 +190,7 @@ test('should not report credentials in recently executed workflow', async () =>
const credential = await Db.collections.Credentials.save(credentialDetails);
const workflowDetails = {
id: generateNanoId(),
name: 'My Test Workflow',
active: true,
connections: {},
@@ -205,17 +217,21 @@ test('should not report credentials in recently executed workflow', async () =>
const date = new Date();
date.setDate(date.getDate() - config.getEnv('security.audit.daysAbandonedWorkflow') + 1);
await Db.collections.Execution.save({
data: '[]',
const savedExecution = await Db.collections.Execution.save({
finished: true,
mode: 'manual',
startedAt: date,
stoppedAt: date,
workflowData: workflow,
workflowId: workflow.id,
waitTill: null,
});
await Db.collections.ExecutionData.save({
execution: savedExecution,
data: '[]',
workflowData: workflow,
});
const testAudit = await audit(['credentials']);
expect(testAudit).toBeEmptyArray();

4
packages/cli/test/integration/audit/database.risk.test.ts
View File

@@ -8,6 +8,7 @@ import {
} from '@/audit/constants';
import { getRiskSection, saveManualTriggerWorkflow } from './utils';
import * as testDb from '../shared/testDb';
import { generateNanoId } from '@/databases/utils/generators';
beforeAll(async () => {
await testDb.init();
@@ -28,6 +29,7 @@ test('should report expressions in queries', async () => {
const promises = Object.entries(map).map(async ([nodeType, nodeId]) => {
const details = {
id: generateNanoId(),
name: 'My Test Workflow',
active: false,
connections: {},
@@ -80,6 +82,7 @@ test('should report expressions in query params', async () => {
const promises = Object.entries(map).map(async ([nodeType, nodeId]) => {
const details = {
id: generateNanoId(),
name: 'My Test Workflow',
active: false,
connections: {},
@@ -134,6 +137,7 @@ test('should report unused query params', async () => {
const promises = Object.entries(map).map(async ([nodeType, nodeId]) => {
const details = {
id: generateNanoId(),
name: 'My Test Workflow',
active: false,
connections: {},

7
packages/cli/test/integration/audit/filesystem.risk.test.ts
View File

@@ -4,6 +4,7 @@ import { audit } from '@/audit';
import { FILESYSTEM_INTERACTION_NODE_TYPES, FILESYSTEM_REPORT } from '@/audit/constants';
import { getRiskSection, saveManualTriggerWorkflow } from './utils';
import * as testDb from '../shared/testDb';
import { WorkflowEntity } from '@/databases/entities/WorkflowEntity';
beforeAll(async () => {
await testDb.init();
@@ -26,11 +27,10 @@ test('should report filesystem interaction nodes', async () => {
);
const promises = Object.entries(map).map(async ([nodeType, nodeId]) => {
const details = {
const details = new WorkflowEntity({
name: 'My Test Workflow',
active: false,
connections: {},
nodeTypes: {},
nodes: [
{
id: nodeId,
@@ -38,9 +38,10 @@ test('should report filesystem interaction nodes', async () => {
type: nodeType,
typeVersion: 1,
position: [0, 0] as [number, number],
parameters: {},
},
],
};
});
return Db.collections.Workflow.save(details);
});

4
packages/cli/test/integration/audit/instance.risk.test.ts
View File

@@ -12,6 +12,7 @@ import {
import * as testDb from '../shared/testDb';
import { toReportTitle } from '@/audit/utils';
import config from '@/config';
import { generateNanoId } from '@/databases/utils/generators';
beforeAll(async () => {
await testDb.init();
@@ -31,6 +32,7 @@ test('should report webhook lacking authentication', async () => {
const targetNodeId = uuid();
const details = {
id: generateNanoId(),
name: 'My Test Workflow',
active: true,
nodeTypes: {},
@@ -73,6 +75,7 @@ test('should report webhook lacking authentication', async () => {
test('should not report webhooks having basic or header auth', async () => {
const promises = ['basicAuth', 'headerAuth'].map(async (authType) => {
const details = {
id: generateNanoId(),
name: 'My Test Workflow',
active: true,
nodeTypes: {},
@@ -115,6 +118,7 @@ test('should not report webhooks having basic or header auth', async () => {
test('should not report webhooks validated by direct children', async () => {
const promises = [...WEBHOOK_VALIDATOR_NODE_TYPES].map(async (nodeType) => {
const details = {
id: generateNanoId(),
name: 'My Test Workflow',
active: true,
nodeTypes: {},

7
packages/cli/test/integration/audit/nodes.risk.test.ts
View File

@@ -9,6 +9,7 @@ import { toReportTitle } from '@/audit/utils';
import { mockInstance } from '../shared/utils';
import { LoadNodesAndCredentials } from '@/LoadNodesAndCredentials';
import { NodeTypes } from '@/NodeTypes';
import { WorkflowEntity } from '@/databases/entities/WorkflowEntity';
const nodesAndCredentials = mockInstance(LoadNodesAndCredentials);
nodesAndCredentials.getCustomDirectories.mockReturnValue([]);
@@ -32,11 +33,10 @@ test('should report risky official nodes', async () => {
}, {});
const promises = Object.entries(map).map(async ([nodeType, nodeId]) => {
const details = {
const details = new WorkflowEntity({
name: 'My Test Workflow',
active: false,
connections: {},
nodeTypes: {},
nodes: [
{
id: nodeId,
@@ -44,9 +44,10 @@ test('should report risky official nodes', async () => {
type: nodeType,
typeVersion: 1,
position: [0, 0] as [number, number],
parameters: {},
},
],
};
});
return Db.collections.Workflow.save(details);
});