Complex cloud data pipelines are a huge pain point for developers and engineers, which means simplifying data management issues and increasing team agility will be a key factor for success in 2023, according to Sean Knapp, founder and CEO of data automation provider Ascend.io.

Touting plans to double the cloud data automation company's growth following a $31 million Series B funding round last spring, Knapp dove into the difficulties and best practices for enterprise cloud data management with SDxCentral over an email-conducted interview.

SDxCentral: What are the most significant data-related challenges hindering enterprises today?

Knapp: Data is critical to every business, yet data teams, which may be getting leaner in this economic climate, are buckling under the increased pressure to create value and build data applications quickly.

Companies using a multi-tool systems environment are struggling to keep pace because they’re managing a tangle of single-function systems that weren’t designed to work together. Businesses are reaching their breaking point as they tire of assembling and managing upstack tools and not seeing enough returns on those investments.

SDxCentral: How do you recommend enterprises build smarter data pipelines?

Knapp: Switching from bespoke, loosely connected systems to fully integrated platforms can remove the complexities of a multi-tool ecosystem, helping developers manage ongoing changes easier and teams to increase productivity without overstretching resources. Effective data management requires simplifying, consolidating and automating common processes.

SDxCentral: What components make up a successful data stack?

Knapp: A typical data stack addresses the core needs of a data engineering team: ingestion, transformation, orchestration, and observability. A successful data stack goes beyond these by further addressing the issue of scalability – that of the data team by delivering increased productivity as the number of data products produced and maintained continues to climb. With overwhelming demand for data to make business decisions and fuel next-generation products, a successful data stack should greatly reduce the number of standalone tools and services, and involve a high level of automation to lighten the load for data teams and help them produce and iterate on data products with much higher velocity.

SDxCentral: What have you observed as far as the talent demand and market for data professionals?

Knapp: While current job cuts in the tech industry are concerning, hiring skilled data engineers continues to be a top priority. Over the past few years the data industry has begun a shift away from bespoke data platforms to far more productized solutions, and the most impactful shift currently underway is the one towards automation. As a result, the most valuable data engineers in today’s market are quickly becoming those who think automation first, and are able to rapidly deliver data products for the business.