Artifacts!

In this article we'll cover what artifacts are and how to use them, both as a package user and a package developer.

What is an Artifact?

In Julia, an artifact is simply a fixed/immutable something that a package needs (that isn't Julia code).  This could be an executable binary, a text file, or any other kind of immutable data.  

The "rules" for artifacts are the following:

• Artifacts must be immutable.
• Artifacts must be a (optionally g-zipped) tarfile (ends with .tar or .tar.gz).
• Artifacts must be publicly available on the internet.

What's the Point of Artifacts?

To understand the usefulness of artifacts, let's take a look at some alternatives:

❌ Storing binary data directly in the git repo.

This is bad practice because git is only meant for tracking changes in text files.  A change to a binary file will require git to save an entirely new version of the file.

❌ Using a deps/build.jl script.

Before artifact support was added to Pkg, this was how you included artifacts along with a package.  When you install a package, this script will run (if it exists).  The script can include downloading files the package needs.  

Downsides to this approach are:

• If two packages use the same artifact, they will both download their own copy.
• Packages that use this method are incompatible with PackageCompiler.jl.
• It is tricky to get platform-specific dependencies working properly.
• The artifact needs to be served from somewhere, which you need to either do yourself or actively monitor to ensure the link doesn't change.

In other words, Pkg's artifact support allows you to:

✅ Avoid bloat in git repos.
✅ Make source code immutable (a package directory's state isn't changed by a build.jl script).
✅ Have packages that share dependencies.
✅ Use PackageCompiler.
✅ Install platform-specific dependencies in a more robust way.
✅ Serve artifacts from the Pkg server infrastructure.

Artifacts as a Package User

As a user, you don't need to make yourself concerned with artifacts.  Whenever a package is installed on your machine, Julia's package manager will automatically download the artifacts it needs!

Artifacts as a Package Developer

As a developer, we highly recommend using the ArtifactUtils package.  See the docs for more info, but here we'll walk through creating a package from scratch that needs an artifact.

1) Let's make a package called Hello

using PkgTemplates

t = Template()

generate(t, "Hello")

2) Navigate to the root directory of Hello

path = joinpath(homedir(), ".julia", "dev", "Hello")

cd(path)

3) Create what we need

• We'll write a file called "hello.txt" into a temporary directory with the contents "Hello!  I am an artifact!".

dir = mktempdir()

file = touch(joinpath(dir, "hello.txt"))

open(file, "w") do io 
    write(io, "Hello!  I am an artifact!")
end

4) Get the ID for our artifact

using ArtifactUtils

artifact_id = artifact_from_directory(dir)

5) Upload the artifact somewhere

• Here we'll use ArtifactUtils to upload to https://gist.github.com.  This code snippet requires you have an SSH Key added to your GitHub account.  See here for generating an SSH Key and here for adding the SSH key to your GitHub account.

gist = upload_to_gist(artifact_id)

6) Create an Artifacts.toml in your Hello package directory

add_artifact!("Artifacts.toml", "hello_artifact", gist)

7) Use the artifact in the package

• Every package can have an __init__ function that runs right after the package is loaded.  In this case, we'll use __init__ to print out the contents of our "hello.txt" artifact.

sourcecode = """
module Hello 

using Artifacts 

function __init__()
    path = joinpath(artifact"hello_artifact", "hello.txt")
    
    println(read(path,String))
end

end #module
"""

open(io -> write(io, sourcecode), joinpath("src", "Hello.jl"), "w")

8) Add the Artifacts dependency and you're done!

using Pkg

# activate the Hello project
Pkg.activate(".")  

# Make sure we add the Artifact dependency.
Pkg.add("Artifacts")  

# get all of Hello's dependencies including artifacts
Pkg.instantiate()     

🎉 Hey it works!

julia> using Hello
Hello!  I am an artifact!

🚀 That's It!

You now know the basics of Julia's artifact system!

Still confused about something?  Did we miss anything important?  Let us know on Twitter at @JuliaForDataSci!

Enjoying Julia For Data Science?  Please share us with a friend and follow us on Twitter at @JuliaForDataSci.

Want to write an article for Julia For Data Science?  Get in touch!  juliafordatascience@gmail.com

Julia For Data Science Numbers:

Hey, this section is new.  We thought it would be fun to include some statistics about Julia For Data Science.  We'll include our member/follower numbers in each post from here on out.

• Newsletter members: 166
• Twitter followers: 836