beginner

Using JSON Web APIs from Julia

A common way to access data is through web APIs that return a JSON response. In this post we'll look at how the XKCD.jl package implements a user-friendly interface for accessing the xkcd webcomic's JSON interface.

Josh Day

04 Nov 2021 • 2 min read

Fun with XKCD.jl and Pluto.jl

Enjoying Julia For Data Science? Please share us with a friend and follow us on Twitter at @JuliaForDataSci.

Getting the Data

The xkcd webcomic's JSON interface accepts request of the form:

https://xkcd.com/info.0.json (metadata for most recent comic)
https://xkcd.com/<i>/info.0.json (metadata for i-th comic)

We'll use the HTTP package's get function to retrieve this info:

julia> using HTTP

julia> res = HTTP.get("https://xkcd.com/552/info.0.json");

Interpret Data as JSON

We can then read our result body into JSON using the JSON3 package:

julia> using JSON3

julia> JSON3.read(res.body)

JSON3.Object{Vector{UInt8}, Vector{UInt64}} with 11 entries:
  :month      => "3"
  :num        => 552
  :link       => ""
  :year       => "2009"
  :news       => ""
  :safe_title => "Correlation"
  :transcript => "[[A man is talking to a woman]]\nMan: I used to think correla…
  :alt        => "Correlation doesn't imply causation, but it does waggle its e…
  :img        => "https://imgs.xkcd.com/comics/correlation.png"
  :title      => "Correlation"
  :day        => "6"

Making it User-Friendly

To make using the xkcd JSON interface easier to use, let's create a struct to hold the JSON data and let its constructor do the heavy lifting:

struct Comic 
    json::JSON3.Object
end

Comic() = Comic(JSON3.read(HTTP.get("https://xkcd.com/info.0.json").body))
    
Comic(i::Int) = Comic(JSON3.read(HTTP.get("https://xkcd.com/$i/info.0.json").body))

🎉 Now getting comic metadata is as simple as Comic(552)!

What about Authorization?

The xkcd API does not require authorization. However, many APIs require that you purchase an API key in order to access them. In this case, it's best practice to try to ensure API keys won't end up being accidentally saved in git (or other version control).

To do this, we can rely on environmental variables (ENV in Julia). Suppose the xkcd API requires an apikey parameter. We would want to change our Comic constructor to something like:

function Comic(i::Int)
    apikey = ENV["MY_API_KEY"]
    url = "https://xkcd.com/$i/info.0.json?apikey=$apikey"
    data = JSON3.read(HTTP.get(url).body)
    Comic(data)
end

Imaginary apikey parameter in xkcd API

Other Neat Stuff

Since we are working with images (comics) in the XKCD.jl package, we can add a pretty show method so that the comic image (not just the metadata) will appear in coding environments like Jupyter and Pluto.

function Base.show(io::IO, ::MIME"text/html", c::Comic)
    show(io, MIME"text/html"(), HTML("""
	   <div>
        <h2><code>XKCD.Comic</code> $(c.json.num): $(c.json.title)</h2>
        <img src="$(c.json.img)" alt="$(c.json.alt)" title="$(c.json.alt)">
		<div>
          <a href="https://xkcd.com/$(c.json.num)">
            Link to Original
          </a>
	    </div>
      </div>
    """))
end

🚀 That's It!