Big Data Applications in the Economics/Financial World Part II: Econometric Modeling in the 21st Century
|"One of the first things taught in introductory statistics books is that correlation is not causation. It is also one of the first things forgotten." – Thomas Sowell
Executive Summary
Data analysis and econometric modeling are major tools that help decision makers design effective polices. This ever evolving world is driven by continuous developments in econometric tools and data sources. One such development in recent years is the use of big data—large and complex datasets that present both opportunities and challenges for analysts and their statistical toolkits. To inform our readers about the potential benefits and limitations of big data applications, we started a two-part series, and the first report looked at the potential benefits from big data for analysts and decision makers.1
This report focuses on issues and solutions related to econometric modeling and forecasting using big data. It is worth mentioning that we recognize that there are other issues, such as privacy and cyber security concerns, that represent huge hurdles for the advancement of utilizing big data on a mass scale. We are not lawyers, information technology or privacy experts, however, and as such, we will solely focus on using big data for modeling and analysis from an economics perspective.
Data as Insight
Typically, analysts utilize a data series to represent or proxy a sector's (or the overall economy's) activities, and by analyzing that series an analyst gains insight about the sector's state. For example, the Bureau of Economic Analysis releases personal spending data every month, and analysts use that series to learn about consumers' behavior and its impact on the overall economy, Figures 1 & 2. This data lags by about one month, however, and is only available in the aggregate. More granular data that include other useful information, such as demographic data, are often unavailable in real-time. Big data, with its hundreds of millions of observations, high frequency and detailed richness, can help to fill this information void. These very traits, however, can create both known and unknown challenges for analysts using traditional statistical techniques and applications. The relative newness of big data analysis likely means that as this phenomenon matures over time, problems with current methods will arise and new statistical tools and techniques will be needed to address these currently unknown problems lurking in the vast sea of big data. For now, however, we cannot know these unknown unknowns, and we must rely on the tools at hand. This report outlines some known potential problems and suggests a few key statistical principles, methods and best practices to bear in mind when conducting big data analysis.
Information on these pages contains forward-looking statements that involve risks and uncertainties. Markets and instruments profiled on this page are for informational purposes only and should not in any way come across as a recommendation to buy or sell in these assets. You should do your own thorough research before making any investment decisions. FXStreet does not in any way guarantee that this information is free from mistakes, errors, or material misstatements. It also does not guarantee that this information is of a timely nature. Investing in Open Markets involves a great deal of risk, including the loss of all or a portion of your investment, as well as emotional distress. All risks, losses and costs associated with investing, including total loss of principal, are your responsibility. The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of FXStreet nor its advertisers.