Agile Analytics Demystified
With Special Guest, Lawrence Corr, Author of Agile Data Warehouse Design
Agile analytics and techniques have revolutionized software development and are increasingly being adopted by business intelligence/data warehouse developers. Learn how the early/frequent delivery of working software, customer collaboration and responsiveness to change can transform traditional dimensional modeling and analytics system design.
In this webinar recording, Lawrence Corr, author of the book Agile Data Warehouse Design, describes how agile methods can dramatically improve the design of one of the most critical parts of BI/DW systems: database schemas. Lawrence introduces BEAM (business event analysis and modeling): a set of agile tools and techniques for dimensional modelstorming (modeling + brainstorming) with BI stakeholders and users.
This webinar recording includes Q&A to help apply these valuable concepts to specific needs.
Maximizing business stakeholder involvement during BI solution development coupled with early and repeated solution releases, results in more business benefits early and better BI user adoption.
The classic “waterfall” development method delays solution delivery and places the incorporation of the full volume of useful business data at the end of the lengthy project. This separates the first useable solution delivery from the requirements and design phase so far in time, that often major gaps and surprises that compromise usefulness and adoptability occur.
Shortening the solution release cycle so that useful business data is incorporated earlier and business users continually interact to test and refine the delivered solution in each release, eliminates most surprises and provides incremental business value early and often.
Business Intelligence; Data Modeling; BI Software Development
Business Analysts; Business BI Users; BI Developers; IT Management
Lawrence Corr is a leading data warehouse designer and educator. Early in his career, he was invited by Ralph Kimball to become an associate and teach data warehousing classes for Kimball University in Europe and Africa. He now regularly teaches agile dimensional modeling courses and has taught dimensional BI skills to thousands of students in Europe, North America, the Middle East and South Africa. Using his proven techniques, he has helped optimize DW architectures in healthcare, telecoms, engineering, broadcasting, financial services and retail sectors. He is the author of Agile Data Warehouse Design: Collaborative Dimensional Modeling, from Whiteboard to Star Schema, an Amazon.com #1 bestseller in data warehousing.
Agile Business Intelligence
Agile as applied to information technology (Agile Alliance, 12 years ago, Aug. 2001)
Agile is generally defined as:
- able to move quickly and easily
- active; lively
- mentally quick or acute
Agile Computer SoftwareDevelopment => “Agile Manifesto”
We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
- Individuals and interactions over processes and tools
- Working software over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Twelve principles of agile software development:
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference to the shorter time scale.
- Business people and developers must work together daily throughout the project.
- Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity – the art of maximizing the amount of work not done – is essential.
- The best architectures, requirements and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.
Standard, “waterfall” approach: BDUP => “Big Design Up Front”
Agile [iterative] development: JEDUF = “Just Enough Design Up Front”
( also: JEDI => Just Enough Design Initially )
What’s in the “Analysis, Design and Data Modeling” [red] box above?
Why do the analysis, design and model with BI stakeholders?
- Identify the signi?cant business events worth measuring
Scope and Prioritization
- Discover how business events are described
- Determine how they are measured
Measures, Hierarchies, Comparisons, KPIs
- Unearth budgets, forecasts, targets and other user-controlled data sources.
Extra data, Common summarization levels: Physical Optimization
- Create business ownership and pride in the DW design
Approach to data modeling with business people (business modeling):
Use R.K.’s keywords . . .
I keep six honest serving-men
(They taught me all I knew);
Their names are What and Why and When
And How and Where and Who…
— Rudyard Kipling
The Elephants Child (1902)
Plus: How many or how much? Relating to the Event/Fact (see below). And relating this to the star schema data model:
A new methodology and documentation technique for doing agile business requirements gathering, analysis and data modeling with business stakeholders:
Business Event Analysis &Modeling
Model as a BEAM table – an example data model
If requirement changes are welcomed from release to release (as a result of early BI solution versions and business user feedback), doesn’t that result in a considerable amount of development rework?
If the focus of requirements gathering, analysis and subsequent design is on business processes more so than on a current, point-in-time perception of needed reports, then the basic business dimensions and needed granularity comes out and usually stands the test of time and future releases. Subsequent requirement changes then are mostly about additional and augmented dimension attributes and only rarely do new dimensions or major changes to granularity come about to cause major development rework.
Time and Date Dimension
A rich date and time dimension in a well-thought out data model supports flexible reporting without the need for lengthy, report specific SQL date calculations that need to be developed for each report.