The Accidental Data Scientist

In the early days of relational databases, data collection was a large part of the puzzle. Pre-internet, the number of news sources was much smaller, and the metadata involved was much more segmented—names, dates, and places. The format and variety of Big Data have changed just about everything that we believed to be true regarding database contents and creation.
- Posted May 29, 2018
The familiar adage "Everything old is new again" was on my mind as I enjoyed one of my beach reads this summer, How to Lie With Statistics, by Darrell Huff. In a TED talk, Bill Gates said that Huff "shows you how visuals can be used to exaggerate trends and give distorted comparisons." I assumed it was a hot-off-the-press response to the current fixation with fake news and erroneous data. However, the copyright page says Huff first shared these tips and tricks back in 1954.
Given their reliance on statistics, data scientists were stunned when Donald Trump won the U.S. presidential election. Did the faulty numbers and broken algorithms signal the death of Big Data? Or did they underscore the fact that this is a nascent field? The Monday-morning quarterbacking continues, but certain factors may lead to better best practices and a more conscientious climate. At a minimum, the need for higher quality data, sharper code writing, and more contextual, nuanced analysis is apparent.
- July/August 2017 Issue, Posted Aug 08, 2017
Big Data and journalism are becoming inextricably linked. Currently, we seem to be embarking on the slippery slope of presenting statistics and data analysis as solid evidence, sans clarification or context. Even Nate Silver of the FiveThirtyEight website, after putting Donald Trump's chances of becoming the Republican nominee at 12% to 13%, waited to clarify the rationale behind the prediction. To some, a U.S. presidential election feels life-altering. That is literally the case when data are being run through algorithms and used to make decisions, such as mortgage qualification or university admittance.
- March/April 2017 Issue, Posted Apr 11, 2017
MTV's first music video, The Buggles' "Video Killed the Radio Star," became stuck in my head when I read that Facebook EMEA VP Nicola Mendelsohn declared that Facebook will be "all video" in 5 years. Mendelsohn said that text has been declining every year and that Facebook users now view videos 8 billion times per day, up from 1 billion a year ago. Apparently, Facebook thinks video will kill off text, sooner rather than later.
As Sam Roe wrote in the Columbia Journalism Review (CJR) article, for many years, "investigative reporters have approached stories with a similar mindset: Find the bad guys," and "copy down what people said and leave it at that." Big Data, the big disrupter of all industries, might alter that approach. The collection of massive amounts of data has caused practitioners in many fields to reconsider how they do business.
The Paris terrorist attacks in November 2015 caused a seismic shift in many areas, including media discussions of Big Data, which now focus on the Internet of Things (IoT). At first, it may seem difficult to understand how the IoT and terrorism are related, but security involves both a nation's borders and its data. These concerns are prompting new conversations about the IoT's role in the future of the Big Data landscape.
- May 2016 Issue, Posted May 10, 2016