Indicium Targets AI, Data Asset Governance With New Databricks Service

From its Brazil roots, AI and data services provider Indicium continues its U.S. expansion by leveraging its close relationship with big data platform provider Databricks.

Data services provider Indicium is accelerating its expansion in the U.S. with a new unified data and AI asset governance solution for the Databricks platform.

The new Unity Catalog Migration solution is designed to enable unified governance of data and AI assets within the Databricks Data Intelligence Platform, simplifying access, compliance and management across an organization.

The new service is the latest offering from New York-based Indicium, which has been expanding beyond its origins in Brazil into the U.S. data services market over the last year. The company, a Databricks consulting partner, focuses on providing data IT services for financial services customers.

“We are a full-stack data company. We work with data engineering, AI, data infrastructure – all the main service lines for data. And a lot of our work is helping companies modernize, move to the modern data stack, and are using Databricks as their data platform, their AI platform,” said Daniel Avancini, Indicium chief data officer, in an interview with CRN.

The need for data governance for AI models is a key driver right now for solutions like the Unity Catalog Migration offering, he said.

“Most enterprise companies are looking to apply their own proprietary data to AI. And to do that with security, with governance, they need a modern [data] catalog, such as the Databricks Unity Catalog. It's a very interesting use case for AI in general,” Avancini said.

Databricks Unity Catalog, part of the broader Databricks Data Intelligence Platform, is the company’s data governance software for structured and unstructured data. The company announced in June 2024 that the product would be offered as open-source software.

The challenge for businesses and organizations adopting Unity Catalog is migrating from legacy data metastores such as Hive or older Databricks software. That’s the task Indicium is targeting with its Unity Catalog Migration solution.

Avancini said Indicium has been working with Databricks, including its engineering and partnership teams, to develop migration frameworks and processes, “so we can expedite the work and have a better offering for these database clients that want to migrate to Unity Catalog. The idea is really to make these migrations much faster and more streamlined.”

Indicium, in its announcement, said Unity Catalog Migration reduces migration timelines and manual tasks by up to 50 percent. The solution leverages pre-defined frameworks and AI-driven accelerators to deliver such results as a 30-percent improvement in governance efficiency and data accuracy compared to legacy metastores.

Avancini said the bulk of the services involve migrating metadata from legacy systems to the Databricks Unity Catalog and updating and moving data pipelines that serve data processing systems.

“It's almost like a data migration, like a full-scale data migration,” he said. “It's a bit simpler, but there's still a lot of work behind the scenes to make it work.” He described metadata migration as “foundational” for customers looking to leverage the capabilities of the Databricks platform and Indicium’s other tools and services.

The Unity Catalog Migration solution integrates AI and advanced tools such as the Unity Catalog Extension, Databricks CLI, REST APIs and Databricks’ Delta Sharing secure data sharing open-source protocol.

Indicium said the service delivers centralized data access control, standards-compliant security, built-in data auditing and lineage, data discovery and usability, and system table access.