Adapting the Agile Manifesto Principles to Data Warehousing

September 9, 2019 Kent Graziano

Despite the clear benefits of data warehousing programs, business units often question what they perceive to be long delivery timelines and a lack of data accessibility. 

Nearly half (48%) of the respondents in a TDWI survey reported that personnel in their organizations spend at least 61% of their time finding and preparing data. Only 28% said that their business users and analysts can access and analyze new data, including external data, without close IT support.

This is no longer acceptable in the ever-evolving world of data and analytics. To be truly data-driven, your organization must create an agile data environment. The question driving your data warehousing and BI departments should be: How can we deliver results faster with closer integration between data teams and their business stakeholders? 

A key to solving this problem clicked into place for me over a decade ago when a CIO handed me a book on Agile project management, written for software teams. Receiving that book started me on a journey of transforming data warehousing teams to be more nimble and iterative. 

A quick primer on Agile: In 2001, a consortium of software developers published the Agile Manifesto, which outlined 12 Principles of Agile, seeking to codify a more nimble people-focused approach to software development. These Agile Manifesto principles quickly became the industry standard for software teams. As a result, Agile is often described more like a mindset or even a belief system than merely a set of software development tactics. 

Agile development is characterized by the following:

  • Teams divide work into small segments that they refine continuously based on feedback from business stakeholders.
  • Teams work on each of these segments during a predetermined time frame, often known as a sprint.
  • Each sprint focuses on the smallest possible deliverable or set of deliverables for that time frame, sometimes referred to as the minimum viable product (or MVP).

Adapting the Agile Manifesto Principles to Data Warehousing

Simply copying Agile Manifesto principles and associated tactics and approaches from the software world does not fit well with the realities and complications of data warehousing, however. They need to be modified. Two-week sprints, for example, are considered ideal in software development, but they are not always optimal with data warehouse projects. 

As a result, over years of work at multiple organizations, I’ve learned how best to apply and adapt the 12 Agile Manifesto principles for data projects and have seen firsthand just how powerful this hybrid approach can be. 

You’ll find the foundational 12 Principles of Agile, along with my recommendations for adapting them to data warehousing, in my latest ebook, Applying the Agile Manifesto To Your Data Warehousing Program

By borrowing from Agile Manifesto principles and modifying them for our data warehousing needs, we can change how organizations perceive data warehousing programs and teams. We can learn to deliver value more quickly by being adaptable and flexible. I hope you’ll read my ebook and find the approach as transformational as I have.

The post Adapting the Agile Manifesto Principles to Data Warehousing appeared first on Snowflake.

Previous Article
Latest Changes to How Snowflake Handles OCSP
Latest Changes to How Snowflake Handles OCSP

Snowflake client drivers (JDBC, ODBC, Python, and others) connect to a variety of endpoints over HTTPS incl...

Next Article
Data Encryption with Customer-Managed Keys for Azure
Data Encryption with Customer-Managed Keys for Azure

In 2017, Snowflake announced support for customer-managed keys using AWS Key Management Service (KMS). Thes...

×

Subscribe to email updates from the Snowflake Blog

You're subscribed!
Error - something went wrong!