Don't Send Money to Dead People (and Other Lessons for Data Teams)


In June 2020, it was reported that bad data hampered the U.S. government’s ability to roll out its COVID-19 economic recovery programs. In addition to other grievous errors, this data downtime incident sent over $1.4 billion in COVID-19 stimulus checks to dead people.

In part one of a two-part series, I propose a solution to data downtime: data reliability, a concept borrowed from Site Reliability Engineering (SRE) that has been adopted by some of the best data teams in the industry

Image courtesy of Frank Chamaki on Unsplash.

How do we solve for bad data?

I was catching up with a VP of Data I think highly of at a popular public tech company the other day who told me about the impact of data downtime on his company from financial reporting and regulatory reporting to marketing analytics and even customer engagement metrics.

He was jaded by traditional data quality methods as an antidote for solving issues with data.

“Data quality checks only go so far,” he said (and yes, agreed to be quoted anonymously). “I want something that will keep me in the know about data downtime before anyone else — including my boss — knows. Seriously, let me put it this way: I see this as the ‘keep our CFO out of …

Read More on Datafloq
Credit: Source link