Node.js Version Support Has Been Updated for the Coalesce CLI

With version 7.24, we have updated the supported Node.js versions for the coa CLI to ensure you are working in a stable and secure environment.

What has changed?

The coa CLI now supports the following Node.js versions:

• 20

• 22

• 24

⚠️ Support for Node.js version 18 has been removed.

Why we made this change

We removed support for Node.js version 18 because it is approaching its end-of-life. 5We have added support for version 24, a Long-Term Support (LTS) version, to ensure continued stability and security. 6

Action Required

If you are running Node.js version 18, you must upgrade to a supported version to continue using the coa CLI. After updating to release 7.24, running the CLI with Node.js version 18 will cause the CLI to exit with the following error message:

Invalid node version 18 not supported

Updates

We’ve made a series of improvements focusing on UI consistency, run visibility, and Git.

UI Changes

• Improved Display of Long Storage Location Names - Long storage location names in your build settings are now shortened to fit the screen. You can hover over the shortened name to see the full name in a tooltip.

• Updated the "Re-Sync Source Columns" Window - We've updated the style of the "Re-sync Source Columns" window to match the rest of the application. It now provides a clearer message when there are no changes to sync.

• Improved the "Delete Column" Confirmation - When you delete one or more columns, the confirmation window now clearly lists the columns you are about to delete. This helps you confirm your changes before they are made.

• Updated Git View After Commit – The Git window has been improved so that after a successful commit and push, the view now clears automatically to show "No Files To View" This provides clearer feedback and ensures the view reflects the current state of your repository."

Run Updates

• Added a "Rendering" Status for Runs - When you start a run, you will now see a "Rendering" status indicator. This provides better visibility into the run process, especially for large jobs that take time to render SQL before execution.

• Added Tooltips to Run Details Page - On the Deploy and Refresh Runs page, you can now hover over truncated node names and error messages to see the full text in a tooltip.

• Redesigned the Run Stage Details Window - We've redesigned the window that shows the details for a run stage. The new design makes it easier to read long SQL queries and error messages, with a wider layout and resizable panels.

Git Updates

• Better Error Handling for Existing Git Branches - When creating a new workspace, you will now see a clear error message if the Git branch name you entered already exists, allowing you to choose a unique name.

Bug Fixes

• Prevented Deployments with Empty Column Names - To prevent errors, you can no longer deploy Nodes that have columns without names. The application will now show an error message, "Empty Column Names, during the deploy plan, prompting you to name all columns before deploying.

• Fixed Drag-and-Drop for Jobs and Subgraphs - We fixed an issue where dragging and dropping nodes into a second or third Job or Subgraph would not work. You can now reliably add nodes to any Job or Subgraph using drag-and-drop.

• Fixed Workspace Loading for Nodes with Invalid Configurations - We resolved an issue that prevented workspaces from loading if a node had an invalid configuration value. The application now automatically corrects these values to ensure your workspace always loads correctly.

• Fixed Resizing of Include/Exclude Filters - We fixed a bug where the include and exclude filter boxes in the browser would shrink unexpectedly after being resized. Both filter boxes now resize correctly.

Package Updates

Base Node Types – Advanced Deploy

• Added comprehensive metadata update checks for Advanced Deployment Templates.

• Fixed block comment issue for metadata update.

Streams and Tasks

• Delta Stream Merge updates:

  • Dual merge logic implemented.

  • Redeployment behavior added for column schema changes.

  • External Table and External Iceberg Table supported as source objects.

  • Deployment phasing introduced.

• Create Templates:

  • Added comprehensive metadata update checks.

  • Fixed comment block parsing.

  • Fixed default scheduler in node definition.

Running Nodes created with any version 2.2.0 and below without adding table key and partition key config options can cause rendering errors.

Create or Alter

• Added comprehensive metadata update checks for Create Templates.

• Fixed comment block parsing.

• Fixed view issues caused by join changes.

External Table

• Partitioning logic updated.

• A default column PARTITION_COLUMN is added with the transformation:

  split_part(metadata$filename, '/', 2)

You can rename this column or adjust the transformation. You can also use another column with the right transformation for partitioning.

Update to Supported Node.js Versions for the Coalesce CLI

Starting with the 7.24 release, we are updating the supported Node.js versions for the coa CLI to ensure you are working in a stable and secure environment.

What's Changing?

The coa CLI will now enforce the use of the following Node.js versions:

• 20

• 22

• 24

⚠️ Support for Node.js version 18 will be removed.

Why We're Making This Change

We're removing support for version 18 because it reaches its end-of-life in April 2025. Support for version 24, a Long-Term Support (LTS) version, has been added to ensure continued stability and security.

Action Required

If you are running Node.js version 18, you must upgrade to a supported version to continue using the coa CLI. After updating to release 7.24, running the CLI with Node.js version 18 will cause the CLI to exit with the following error message:

Invalid node version 18 not supported

Updates

• Column lineage - We’ve improved the process by adding in app guidance and improving the visual indicators making it easier to see the propagation before commiting.

• Snowflake Authentication - The Problem Scanner now alerts you when Snowflake authentication is missing.

• Subgraphs and Jobs

  • The process for creating a Job or Subgraph is now easier. Enter a name in the modal, click Create, and you’ll be taken straight to your new Job or Subgraph.

  • We alert you in the Problem Scanner for duplicate Subgraph or Job names.

  • Previously, when a was Node was added or removed from a Subgraph it would be done silently. Now we let you know it was successful or failed along with either the Node name or number of Nodes.

Bug Fixes

• When editing user roles, the role summary shows in the dropdown correctly.

• Testing a Databricks SQL Warehouse connection will no longer fail with a "Missing token" error when a valid token has been entered.

• Service accounts with Environment Admin permissions for Snowflake environments will no longer receive an "Invalid Connection Information" error on API calls.

• Deploying a Storage Mapping change will no longer result in a "No schema found" error if the old schema or database was deleted directly from the data warehouse before deployment.

• Deleting a source from a column now works consistently when using a trackpad or a mouse.

• Sorting and filtering by Node Status or Stage Status on the deployment runs page will no longer revert or disappear on runs with a large number of objects.

• Deleting a Source Node and creating a new table Node with the same name and location in a single deployment will no longer replace the existing table and cause data loss.

Breaking Change in List Run Endpoint – Version 7.19

Effective in version 7.19,  there is a breaking change to the listRun endpoint:

• The orderBy parameter is now mandatory whenever the startingFrom parameter is included in a request. This change ensures consistent and predictable pagination behavior when using the startingFrom cursor.

What you need to do:

• If your integration uses startingFrom, make sure to include a valid orderBy value in the same request. Requests missing the required orderBy parameter will now return an error.

We are notifying all Coalesce customers using the command-line tools that an immediate upgrade to the Coalesce CLI version 7.17.1 or higher is required. Snowflake’s recent 2025_02 release includes changes that may cause service interruptions for users running older versions of the Coalesce CLI.

To prevent any disruption, please check your CLI version by running:

coa --version

If your version is lower than 7.17.1, upgrade your CLI by running:

npm upgrade -g @coalescesoftware/coa

For more information on the Snowflake bundle changes, please visit the Snowflake Release Notes.

Thank you for your immediate attention to this matter.

In our ongoing efforts to enhance performance and maintainability, we're removing some legacy metadata fields that are no longer needed.

This process is happening in two stages:

• Release 7.14 - Already completed ✅

• Release 7.15 - Coming soonCompleted on March 18, 2025 ✅.

What this means for you:

• No workflows will be disrupted.

• No data will be affected.

When doing a commit, you might see changes for Nodes that you haven’t modified. This is expected when deprecated metadata is removed. You will need to:

1. Making a commit with the changes.

2. Continue on with your workflow.

Example of a commit with metadata changes

Deprecated Properties Removed:

Deprecated fields in 7.15 will affect all Source Node Types regardless of age or activity status. These fields may still appear in our API, but are no longer used in the Node metadata.

• dataset

• table

• metadata.join

• metadata.sourceMapping - already absent from most sources

See our previous changelog post about the changes in 7.14.

Coalesce cleaning up deprecated metadata fields in Nodes. The change will improve data integrity across Coalesce. This will happen in two stages:

• Release 7.14

• Release 7.15

When doing a commit, you might see changes for Nodes that you haven’t modified. This is expected while deprecated metadata is removed. We suggest making a commit with the changes then continuing on with your workflow. Do not delete, edit, or modify the deprecated fields in any way.

Changes in Release 7.14

Deprecated fields in 7.14  will primarily affect older workspaces that haven't been accessed recently. For actively maintained workspaces, the impact should be minimal. However, if you attempt to commit against previously archived or long-dormant workspaces, you may encounter these changes.

Deprecated Properties Removed

The following properties will be completely removed if present:

Top-level column properties:

• ghostRecordValue

• descendedFrom

• hashColumns

• columnsForHash

Inside hashDetails object:

• hashType

• hubHashes

Properties corrected for Invalid Data

Properties set to empty string (""):

• dataType - if not a string type

• transform - if empty

Properties completely removed if invalid

• appliedColumnTests - if null

• config - if null

• hashDetails - if it's a string (should be an object)

Column Reference Fixes

Column Reference (columnReference)

• stepCounter - if null/undefined, set to the node's ID

• All IDs within column references are converted to strings (previously numbers were allowed - this change has been around for a long time)

Source Column References (sourceColumnReferences)

For each reference in the array:

• All IDs within column references are converted to strings

• stepCounter - if null/undefined, set to empty string ("")

Hashed Columns Fixes

For hashedColumns array:

• All IDs within column references are converted to strings

Upcoming Changes in Release 7.15

Deprecated fields in 7.15 will affect all Source Node Types regardless of age or activity status.

Deprecated Properties Removed:

• dataset

• table

• metadata.join

• metadata.sourceMapping - already absent from most sources

Command line users on version 7.5.8 or below should upgrade to coa version 7.10.2 immediately.

Users on`coa` version 7.5.8 and below may encounter deployment and refresh errors. The issue occurs with the presence of`platformKind:snowflake` in the data.yml.

To fix any critical deployment or refresh issues:

• Upgrade to coa version 7.10.2 as soon as possible to prevent further issues.

If you are unable to upgrade your coa version and need to deploy immediately:

• Check your commits for platformKind:snowflake in the data.yml file.

• Manually remove the platformKind:snowflake and commit the changes.

• Deploy as normal

We are working on a change to be delivered within the next 24 hours to allow backwards compatibility for older versions of coa as to not disrupt operations.

On September 5, 2024, Coalesce will begin enabling two upgrades for all organizations.

• Git Repo Format Upgradeis available now. Mandatory upgrades will start September 5, 2024 and expect to be complete by September 12, 2024.

• Role-Based Access Control (RBAC)

We encourage you to prepare for the changes by:

• Upgrade the CLI to the latest version.

• Have your organization's administrators enable the Git Repo Format Upgrade before the mandatory deadline.

• Optionally, contact support to enable the RBAC migration early.

• Review existing roles and understand the permissions they will have under the new system.

By preparing for these changes early, you can avoid unintended consequences, such as being unable to make commits until the upgrade is complete.

We encourage you to review our documentation for more details:

• Git Repo Format Upgrade

• RBAC Migration Guide

Contact our support team to start preparing.

Updates

This release includes performance improvements and bug fixes.

🛑Action Required🛑

Git Repo Format Upgrade must be completed by September 5, 2024.

On September 5, 2024, Coalesce will enable the Git Repo Format Upgrade for all existing customers.

As part of the upgrade, you’ll need to upgrade your CLI and enable the Git Repo Format Upgrade. Review our announcement to learn more.

We're excited to announce that Role-Based Access Control (RBAC) will be enabled for all Coalesce users on September 5, 2024.

RBAC enhances security by ensuring that users have only the access they need, reducing risks and simplifying permission management.

Organizations will be automatically upgraded on September 5, 2024, but if you'd like early access, please reach out to our support team. Take a look at our documentation to get started.

Review our RBAC Migration Guide to understand how these changes will affect your organization.