AI-Generated Documentation: Automatic Descriptions for Nodes, Columns, and Git Commit Messages
These features are currently in Private Preview. Please reach out to your Coalesce account manager to sign up.
Coalesceis introducing AI-generateddescriptions for Node, Columns, and Git Commit messages.
By analyzing your data pipelines lineage and transformation metadata, it creates natural language descriptions. AI-generated descriptions streamlines documentation by creating clear explanations of data transformations and Git commits. Now you can focus on development while making sure your data pipelines are well documented.
Organization admins can opt-in by going to Organization Settings > Preferences and toggling AI Features.
Updates
Run results/ Column Headers now support numeric names, ensuring your data displays accurately.
We've temporarily rolled back Workspace Onboarding based on valuable customer feedback. This change was made to ensure the best customer experience possible. We're working on improvements to deliver a refined version in the future. Sign up for our changelog email to keep to date.
Bug Fixes
We fixed an issue where, when switching tabs in multi-source Nodes the incorrect join string was saved.
Setting up Workspaces in Coalesce just got easier. Our new Workspace onboarding guides you through the processes step-by-step reducing configuration time. Workspace onboarding makes it easier to set up version control, authentication, and Storage Locations/Mappings. This means less setup time and more focus on building pipelines.
Updates
Our Hash editor automatically displays all columns with proper scrolling functionality.
Problem Scanner can detect and alert on duplicate Node names and Storage Mappings, making it easier to find potential problems.
We've made several improvements to enhance platform speed and reliability. These changes are part of our ongoing commitment to deliver the best possible experience for our users.
Bug Fixes
Source filters now correctly populates and displays all available results.
Fixed an issue where, “A write batch can no longer be used after commit() has been called.”, was returning when trying to checkout branches.
Node column details in the lower-left corner now consistently display when clicking Nodes.
Node Editor scrolls correctly when the “Node not enabled for deploy” banner is displayed.
You can now add a "Re-Sync Columns" button to custom Nodes. This feature allows users to sync columns that are not part of a source Node. Learn more in Custom Node Types.
We’ve released Multi-Factor Authentication (MFA) on our platform, a powerful security feature designed to protect your accounts from unauthorized access. MFA adds an extra layer of security by requiring users to verify their identity through an authentication app. Learn how to enable MFA.
Performance Improvements
Performance improvement for Plan file generation during Deploys which has shown improvements of up to 5X or more.
Bug Fixes
Fixed an issue which caused unsupported data type changes to fail on first deployment, but succeed on subsequent deployments.
Fixed a bug which caused concurrent users causing the join text box to overwrite itself.
Improved performance of copy objects tool when copying objects between Workspaces.
Fixed a bug related to copying items from the Column Grid.
We’ve released our built in Scheduler. This enhancement introduces in-app scheduling capabilities for Coalesce jobs, eliminating the requirement for external scheduling tools for basic job orchestration.
We're excited to announce the release of Role-Based Access Control (RBAC). With RBAC, you can assign roles to users based on their responsibilities, ensuring they only have access to the data they need. This minimizes the risk of unauthorized access, streamlining user permissions while enhancing overall security.
We are excited to announce the upcoming launch of Role-based Access Control (RBAC) on our platform, a new feature designed to enhance security and provide more granular access controls within our environment.
Role-based Access Control (RBAC) is a method of restricting system access to authorized users based on their role within an organization.
RBAC is currently in private preview and is available only upon request. To enable RBAC for your organization, please contact the Coalesce support team. We appreciate your feedback and look forward to enhancing this feature based on your insights during this phase.
We’ve updated our Merge Conflicts workflow. Now you can clearly see which lines need to be reviewed, move between each conflict easily, and can decide to keep or reject changes with the click of a button.
Packages are curated collections of node types that make it faster to build data projects in a sustainable and standardized way. Use packages to streamline your development by removing repetitive work and focusing more on your data.
Data teams can choose from a variety of Packages that are designed to:
Speed up foundational development activities with basic, ready-made node types for Work, Dimension, Fact, and View tables.
Maximize the power of Snowflake innovation with support for Dynamic Tables, Streams & Tasks, Materialized Views, Snowflake Cortex ML-based functions, external data loading with Snowpark, and more.
Streamline work on methodology-related initiatives such as building a Data Vault.
Accelerate AI- and ML-focused innovation with custom node types to operationalize anomaly detection, forecasting, sentiment analysis, and more in Snowflake data pipelines.
Read our documentation on Packages to get started.
We have updated the way files are committed into Git. Previously it was split into a data.yml and the specific nodes.yml files. Now you can see the changes for:
Node Types
Environment Mappings
Subgraphs
Jobs
Macros
Packages
How to Opt-In
Organization Administrators can turn it on for the whole organization. Go to Org Settings > Preferences and Enable Git Repository Format Update. Once enabled, all Workspace users will be prompted to switch to the new format. They won't be able to make commits until they switch to the new format.
Users will be notified the upgrade is available, but will not be able to upgrade.
Previously committed metadata can still be deployed after the upgrade.
Git Repo Format Upgrade is opt-in until August 1, 2024. After August 1, 2024, all Git repos will be automatically updated.
Coalesce CLI
Upgrade your Coalesce CLI to version 7.0.18. Versions below 7.0.18 will not be compatible with Coalesce version 7.2.x and above.
If you use the CLI and HAVE NOT upgraded to the new Git Repo Format, you have until August 1, 2024 to upgrade your CLI. After August 1, 2024, the Git Repo Format Upgrade will be turned on for all Coalesce organizations and any CLI versions below 7.0.18 will not be compatible with Coalesce versions 7.2.x and above.
If you have upgraded to the new Git Repo Format and use the Coalesce CLI, please upgrade your CLI version to prevent interruptions in workflow.
Updates
We’ve made our Git integration even better.
Improvements to the Force Checkout screen so it's easier to understand what happens in a force checkout.
Checking out branches has improved. With easier to understand messages and clearer understanding of what happens to commits when changing branches.
Added two columns for Used by Workspace and Deployed In Environment so you know where your commits are being used and deployed.
The Git Modal header now shows your current branch, current commit, and if you have any uncommitted changes.
Fixed an issue where business keys were being duplicated when the column was copied.
Fixed an issue where the Create and Run buttons were removed from User-Defined Nodes.
Fixed an issue where the table description would show in the column documentation.
Fixed an issue where the Node name is cut off when viewing the Column Lineage.
Fixed an issue where during deployment, if the table already exists in Snowflake and a node is deployed with a matching name , but different data, then the table is dropped and re-created during deployment. Now, during deployment, the table is cloned and temporary columns are created.
Fixed an issue where certain Multi-source View nodes that were changed from UNION to UNION ALL were not recognized during the deployment.
Fixed an issue where dragging columns into the Mapping Grid was sometimes unavailable.
Fixed an issue where new Environments configured with Key Pair authentication would not be able to use the Storage Mapping screen.
Documentation
There's a lot of new content to help you use Coalesce.
We've rolled out our new release, Dhaulagiri 7.0.0. This release focuses on making the platform faster and more reliable.
What's New
CLI
CLI option to set Coalesce domain, making it easier for PrivateLink and Non-U.S. based customers to configure coawithout needing to set environment variables.
Config file example
[default] domain=your deployment server. For example, https://app.eu.coalescesoftware.io token=yourtoken snowflakeAuthType=Basic or KeyPair ...
We now support the same SSO user logging into multiple orgs.
Updates
We made some minor UI improvements to:
User and Org settings
Node Type menu
User Credentials form
Node failure in a Workspace, now shows the node name instead of the node ID.
The Node Graph is easier to navigate because of improvements to zooming and panning.
YAML validation was improved by preventing invalid metadata from being checked out.
Storage Mappings alert users if they aren't authorized for the database or if the database doesn't exist.
If an unauthenticated user tries to visit a link that requires authentication, they'll be redirected to the login page. After logging in, they'll be taken back to the page they were trying to visit.
Hidden or enableIf=false config items don't show up in documentation anymore.
Override Create SQL in View Node now adds double quotes to column names.
Bug Fixes
Get Job Status endpoint succeeds for Workspace runs when launched from the API.
Bulk editing only hash columns now succeeds.
Fixed issue with coa CLI not working behind some network proxies.
Documentation
We've made a lot of changes to our documentation.
Organization
We've organized the documentation so you can find the content where you need it. If you're working on Building Your Pipeline or Deploying Your Pipeline, you can find the content you need.
New Content
There's a lot of new content to help you use Coalesce.
