Economic Data Analysis in Julia

Learn how to use the FRED® API in Julia and update your spreadsheets in an instant

As a decision maker or professional in the corporate sector or financial industry you are often faced with making decisions fast and under a high degree of uncertainty. Having the newest economic data at your disposal within an instant can facilitate that process a lot.

Photo by Sean Lim on Unsplash

Think of a spreadsheet with your Top 10 economic indicators from official statistics that you regularly post process — visualize trends or correlations — in order to support your decision with facts. These indicators usually change on a monthly or quarterly, some even on a daily frequency. If you are lucky, your company provides access to professional data services such as Bloomberg or Thomson Reuters and you can use their spreadsheet add-ins. In an ideal world this is the case.

However, I have often experienced that access to these providers is expensive and thus limited to a small number of departments or even employees. In that case, you will have to update your data manually — or give it to the intern — before you are ready to go. If time is critical, manual updates can be a bottleneck. Furthermore, they are prone to error and frustration. Wouldn’t it be nice to do this by simply pushing a button? Here is how you can make your life a lot easier and automize this task in Julia.

A great example for a database that offers you a vast amount of free and official (macroeconomic) time series is FRED/ALFRED, run by the Federal Reserve Bank of St. Louis. As a macroeconomist by training, I have been using FRED for a long time now. It’s great for research projects and all professional activities concerning US data. Selecting, viewing and downloading time series into spreadsheets works without registration.

Moreover, the St. Louis Fed offers the FRED® API, a web service that facilitates access to and building applications to customize and retrieve data from the central bank’s database. In order to use the FRED® API, you have to register though. After registration, you can request an API.

Once this has been done, there are several ways to proceed. You can either send XML or JSON requests directly and post process the resulting files. How this can be done is explained here. Otherwise you can use the FRED® API from within third-party software in a variety of programming languages. Thanks to the fantastic work of some people out there, this is possible in a lot of languages (C#,C++, Java, Julia, Python, R and many more) already.

For Julia you can either use the FredData.jl package provided by Micah Smith or FredApi.jl by Markushhh on GitHub. For this article, I went for Micah’s package. Please check out his website to see more of his inspiring work.

So how do we get settled? Open the Julia REPL from the command line or your favorite editor. Hit ] to open the package manager. Upon entering — depending on your installed version of Julia — you should see something similar to this:

@v(1.4) pkg>

From here you can install, update or remove all your packages in Julia. To install the package type

@v(1.4) pkg> add FredData

or (in case of unregistered packages) install it directly from GitHub by typing

@v(1.4) pkg> add https://github.com/micahjsmith/FredData.jl

What remains is to connect the FredData package and the API key. Only then can we access the database, remotely. There are two ways to do this. You can either provide the API key directly. However, you have to repeat this step every time you start a new session. For a mock API key this looks as follows:

julia> using FredData
f = Fred("11a32abc1111eg32149287311f123f90") [Hit ENTER]
# it will display
FRED API Connection
    url: https://api.stlouisfed.org/fred/
    key: 11a32abc1111eg32149287311f123f90

In order to store the API key permanently, you have to set up an environment variable in the path of your OS. Following the instructions on Micah Smith’s GitHub account this looks as follows: From within Julia call

julia> open(joinpath(homedir(), ".freddatarc"), "w") do f
           write(f, "11a32abc1111eg32149287311f123f90")
       end

Depending on your OS you then either call

# on macOS/Linux
shell> echo "export FRED_API_KEY=11a32abc1111eg32149287311f123f90" >> ~/.bashrc

# on Windows 7+
shell> setx FRED_API_KEY 11a32abc1111eg32149287311f123f90

from your command line/terminal. Once this is done, we can start downloading and processing our time series.

Let’s set up the Julia environment. We will only need three packages: FredData to manage and download the series from the FRED database, DataFrames and CSV to store and prepare our series for post processing. In Julia’s REPL or your Editor type the following code:

using Pkg;
Pkg.add("DataFrames")
Pkg.add("CSV") #delete these 3 lines if the packages already exist
using DataFrames, CSV, FredData

For the ease of exposition let’s focus on the three most basic macroeconomic time series you can think of: GDP, Consumption and Government Expenditures. Each of this series has a unique identifier which you can find on the central bank’s webpage. This identifier is a necessary condition for any API query, because only that way it knows which series we want to retrieve. For the respective series we find the following information:

Real Gross Domestic Product (GDPC1)

Real Personal Consumption Expenditures (PCE)

Real Government Consumption Expenditures and Gross Investment (GCEC1)

Next, we have to decide on the frequency and the time period we want to look at. While consumption expenditures are available at monthly frequency, the other two series are only updated on a quarterly basis. In addition, while GDP and Government Expenditures are available from the first quarter in 1947, the consumption series only starts in January 1959. For the ease of exposition, let’s download our data at quarterly frequency, starting in the year 2000.

How does downloading a time series from FRED look like? If we just wanted to download the GDP series and extract its values we could type

GDP_pull = get_data(f,"GDPC1", frequency = "q", observation_start = "2000-01-01")
# f is the API key
# "GDPC1" the series key
# frequency "q" is set to quarterly. Choose "a" for annual
# observation_start defines the year the series starts.
df_GDP = GDP_pull.data #dataframe that stores values and a timeline
GDP_values   = df_GDP.value
GDP_timeline = df_GDP.date

Doing this for all of our variables — imagine the list is much longer — produces a lot of code and takes ages to edit. Let’s think of a smarter and shorter way to accomplish this task. We will start by writing a list of series identifiers and loop over them. In addition, we will predefine the frequency, starting/end point and the aggregation method (for the aggregation of consumption expenditures from monthly to quarterly) our variables shall have.

# Identifiers and other settings
series_id = ["GDPC1"; "PCE"; "GCEC1"]
startdate = "2000-01-01"
enddate   = "2020-01-01"
freq      = "q"
agg       = "avg"
# Predefine a dictionary
entryvalues = fill(Any[],size(series_id))
dict = Dict(zip(series_id, entryvalues))
# Loop over the API requests
for (iter, value) in enumerate(series_id)
Fred_Key = "$value" # get identifier
  pull_data = get_data(f,Fred_Key,frequency = freq, observation_start = startdate, observation_end = enddate, aggregation_method = agg)
if iter == 1
dict["timeline"] = pull_data.data[:,3]
  dict[Fred_Key] = pull_data.data[:,4]
else
  dict[Fred_Key] = pull_data.data[:,4]
  
  end
end

Finally we will transform the dictionary into a data frame and use the CSV package to write our data to a file.

# Turn dictionary into Data Frame
export_data = DataFrame(dict)
# Write DataFrame to file
file_name = "new_data.csv"
CSV.write(file_name,export_data)

Thanks for bearing with me. I hope you had fun and this excercise generated some value for you. Please stay tuned for future articles on economic analysis in Julia. Below you will find a complete version of this tutorial for a run-through.