What is an Artifact?

In Julia, an artifact is simply a fixed/immutable something that a package needs (that isn’t Julia code). Examples include executable binaries, text files, or other immutable data.

Rules for Artifacts:

• Must be immutable
• Must be a (optionally gzipped) tarfile (.tar or .tar.gz)
• Must be publicly available on the internet

What’s the Point of Artifacts?

Using Artifacts as a Package User

Users need no involvement — Julia’s package manager automatically downloads required artifacts during installation.

Creating Artifacts as a Developer

The recommended approach uses the ArtifactUtils package.

using PkgTemplates

t = Template()

generate(t, "Hello")

path = joinpath(homedir(), ".julia", "dev", "Hello")

cd(path)

dir = mktempdir()

file = touch(joinpath(dir, "hello.txt"))

open(file, "w") do io
    write(io, "Hello!  I am an artifact!")
end

using ArtifactUtils

artifact_id = artifact_from_directory(dir)

gist = upload_to_gist(artifact_id)

add_artifact!("Artifacts.toml", "hello_artifact", gist)

sourcecode = """
module Hello

using Artifacts

function __init__()
    path = joinpath(artifact"hello_artifact", "hello.txt")

    println(read(path, String))
end

end #module
"""

open(io -> write(io, sourcecode), joinpath("src", "Hello.jl"), "w")

using Pkg

Pkg.activate(".")

Pkg.add("Artifacts")

Pkg.instantiate()

Result

julia> using Hello
Hello!  I am an artifact!

Julia and Comparisons to Other Languages

• Fast Track to Julia — Collaborative resource covering fundamental Julia concepts.

• Julia/Python/R Side-by-side — Comparison guide enabling developers to reference syntax across three popular data science languages.

• Matlab-Python-Julia Cheatsheet — QuantEcon collaborative project helping users transition between these technical computing platforms.

• Noteworthy Differences from other Languages — Official Julia documentation highlighting unique language features and behavioral differences.

Specific Packages

• Basic Commands in Pluto.jl — Guide for the interactive notebook environment.

• Data Wrangling with DataFrames.jl — PDF reference focused on data manipulation techniques.

• Makie.jl Examples — Visual gallery showcasing visualization capabilities.

• Plots.jl — Reference for the Plots.jl package.

Know a cheatsheet that should be on this list? Reach out on Twitter @JuliaForDataSci.

Getting the Data

The xkcd service provides two API endpoints:

• https://xkcd.com/info.0.json (most recent comic)
• https://xkcd.com/{i}/info.0.json (specific comic by number)

res = HTTP.get("https://xkcd.com/552/info.0.json");

Parsing JSON Responses

The JSON.jl package handles JSON interpretation:

JSON.parse(String(res.body))

JSON.Object{String, Any} with 11 entries:
  "month"      => "3"
  "num"        => 552
  "link"       => ""
  "year"       => "2009"
  "news"       => ""
  "safe_title" => "Correlation"
  "transcript" => "[[A man is talking to a woman]]\nMan: I used to think correl…
  "alt"        => "Correlation doesn't imply causation, but it does waggle its …
  "img"        => "https://imgs.xkcd.com/comics/correlation.png"
  "title"      => "Correlation"
  "day"        => "6"

Creating a User-Friendly Interface

A custom struct simplifies access:

struct Comic
    json::Dict
end

Comic() = Comic(JSON.parse(String(HTTP.get("https://xkcd.com/info.0.json").body)))

Comic(i::Int) = Comic(JSON.parse(String(HTTP.get("https://xkcd.com/$i/info.0.json").body)))

Comic

Obtaining comic metadata becomes as simple as Comic(552):

Comic(552)

Comic(Dict{String, Any}("day" => "6", "alt" => "Correlation doesn't imply causation, but it does waggle its eyebrows suggestively and gesture furtively while mouthing 'look over there'.", "year" => "2009", "num" => 552, "link" => "", "safe_title" => "Correlation", "img" => "https://imgs.xkcd.com/comics/correlation.png", "news" => "", "month" => "3", "title" => "Correlation"…))

Handling API Authorization

For APIs requiring keys, use environment variables to avoid accidental version control commits:

function Comic(i::Int)
    apikey = ENV["MY_API_KEY"]
    url = "https://xkcd.com/$i/info.0.json?apikey=$apikey"
    data = JSON.parse(String(HTTP.get(url).body))
    Comic(data)
end

Enhanced Display Method

Create custom HTML rendering for notebooks and Pluto environments:

function Base.show(io::IO, ::MIME"text/html", c::Comic)
    show(io, MIME"text/html"(), HTML("""
       <div>
        <h2><code>XKCD.Comic</code> $(c.json["num"]): $(c.json["title"])</h2>
        <img src="$(c.json["img"])" alt="$(c.json["alt"])" title="$(c.json["alt"])">
        <div>
          <a href="https://xkcd.com/$(c.json["num"])">
            Link to Original
          </a>
        </div>
      </div>
    """))
end

Comic(552)

XKCD.Comic 552: Correlation

Link to Original

This approach enables seamless comic visualization in interactive Julia environments.

Applied Measure Theory for Probabilistic Modeling (advanced)

Speaker: Chad Scherrer | Watch on YouTube

Overview of MeasureTheory.jl advantages relative to existing distributions packages. Loosely falls under data science but great for those with a mathematical statistics focus.

Bias Audit and Mitigation in Julia (beginner–intermediate)

Speaker: Ashrya Agrawal | Watch on YouTube

Introduces Fairness.jl toolkit for auditing and mitigating machine learning bias. Great introduction to fairness/bias issues with accessible examples.

Clearing the Pipeline Jungle with FeatureTransforms.jl (beginner–intermediate)

Speaker: Glenn Moynihan | Watch on YouTube

Addresses technical debt in feature engineering pipelines and demonstrates FeatureTransforms.jl solutions for sustainable practices without sacrificing flexibility.

DataFrames.jl 1.0 Tutorial — Workshop (beginner–intermediate)

Speaker: Bogumił Kamiński | Watch on YouTube

Comprehensive workshop on loading, transforming, and visualizing data. Assumes prior data frame experience from R or Python. Materials available on GitHub.

Easy, Featureful Parallelism with Dagger.jl (advanced)

Speaker: Julian P Samaroo | Watch on YouTube

Advanced parallelization beyond Distributed.jl, featuring GPU support and fault tolerance. Recommended for those struggling with Julia’s distributed computing primitives.

Introduction to Bayesian Data Analysis — Workshop (intermediate–advanced)

Speakers: Kusti Skytén, Chad Scherrer, Tor Fjelde | Watch on YouTube

In-depth tutorial on applied Bayesian workflows using increasingly sophisticated models. Compares Julia’s probabilistic programming advantages over Stan and Python.

Pluto – One Year Later (beginner–intermediate)

Speaker: Fons van der Plas | Watch on YouTube

Update on Pluto.jl, the interactive Julia notebook IDE. Strongly recommended for those frequently working in notebook environments.

Rewriting Pieces of a Python Codebase in Julia (intermediate)

Speaker: Satvik Souza Beri | Watch on YouTube

Practical guidance on migrating Python code to Julia using PyCall and PyJulia, with documented 10x-30x performance improvements. Covers gotchas and optimal use cases.

The State of DataFrames.jl (beginner)

Speaker: Bogumił Kamiński | Watch on YouTube

Discusses DataFrames.jl design philosophy, recent changes, and future plans without heavy code focus.

State of Julia (intermediate)

Speakers: Jeff Bezanson, Stefan Karpinski, Keno Fischer, Viral Shah | Watch on YouTube

Annual retrospective by Julia’s creators on development and community progress.

Statistics with Julia from the Ground Up — Workshop (beginner)

Speaker: Yoni Nazarathy | Watch on YouTube

My top recommendation for this list — a gentle introduction that works through probability, statistics, dataframes, inference, and regression. Great entry point for statisticians new to Julia.

Installation

] add RCall

Running R Code from Julia

using RCall

# type `$` to enter R mode

R> library(ggplot2)

R> data(diamonds)

R> ggplot(diamonds, aes(x=carat, y=price)) + geom_point()

using RCall

R"y = 2"

Precompiling packages...
    979.1 ms  ✓ QuartoNotebookWorkerTablesExt (serial)
  1 dependency successfully precompiled in 1 seconds
Precompiling packages...
    982.1 ms  ✓ QuartoNotebookWorkerLaTeXStringsExt (serial)
  1 dependency successfully precompiled in 1 seconds
Precompiling packages...
   3774.4 ms  ✓ QuartoNotebookWorkerRCallExt (serial)
  1 dependency successfully precompiled in 4 seconds

RObject{RealSxp}
[1] 2

Sending Julia Variables to R

x = 1

@rput x

R"x"

RObject{IntSxp}
[1] 1

R"y = $x"

R"1 + $x"

Retrieving R Variables in Julia

R"z = 5"

@rget z

z

5.0

robj = R"z"

rcopy(robj)

5.0

Plotting with ggplot2

R"""
library(ggplot2)
data(diamonds)
p <- ggplot(diamonds, aes(x=carat, y=price)) + geom_point()
ggsave('diamonds_plot.png', p, width=6, height=4, dpi=150)
"""

Resources

• RCall Documentation
• Type Conversions Guide

Small CSVs: CSV.File

For datasets comfortably fitting in memory, the CSV package offers CSV.File() for high-performance file reading:

• Automatically determines column types
• Creates objects loadable into DataFrames
• Supports numerous formatting options

using CSV, DataFrames, Plots, Downloads

url = "https://raw.githubusercontent.com/nytimes/covid-19-data/master/us.csv"

file = CSV.File(Downloads.download(url))

df = DataFrame(file)

plot(df.date, df.cases, title="Cumulative COVID-19 Cases in the US",
    lab="", ylab="N", xlab="Date")

Large CSVs: CSV.Rows

For massive files, CSV.Rows creates a memory-efficient iterator loading one row at a time. All data appears as strings requiring manual type parsing.

CSV Combined with OnlineStats

The OnlineStats package provides fast single-pass algorithms using constant memory, enabling statistics calculation on infinitely-sized datasets.

using CSV, OnlineStats, Plots

rows = CSV.Rows("/path/to/large_file.csv", reusebuffer=true)

itr = (parse(Int, r.passenger_count) => parse(Float64, r.fare_amount)
    for r in rows)

o = GroupBy(Int, Hist(0:.5:100))

fit!(o, itr)

plot(plot(o[1]), plot(o[2]), plot(o[3]), plot(o[4]),
    layout=(4,1), link=:all, lab=[1 2 3 4]
)

The example above examines NYC taxi data to visualize fare distributions by passenger count — processing arbitrarily large files with constant memory usage.

Introduction

The Julia Language documentation offers comprehensive performance guidance, though it targets computer scientists rather than data professionals. This article translates those concepts for practical application.

The recommended development workflow emphasizes starting with functional code, then systematically addressing performance bottlenecks through targeted optimization.

Execute Computationally Intensive Tasks Inside Functions

Performance-critical code should reside within functions to ensure optimal execution efficiency. Structure your problem into modular functions rather than lengthy procedural scripts.

# Suboptimal approach
x = 1
f(y) = x + y

# Preferred approach
f(x, y) = x + y

const x = 1
f(y) = x + y

function f(x)
    x < 10 ? "string" : 1
end

x = 1
x = x/2  # Changes type from Int to Float64

Avoid Abstract Types in Containers and Structures

Abstract types force Julia to reason about type sets rather than specific types.

Initialize arrays with known types:

x = []         # eltype(x) == Any
x = Float64[]  # eltype(x) == Float64

Float64[]

Create parametric structs:

# Ambiguous
struct A
    thing::Real
end

# Concrete
struct B{T <: Real}
    thing::T
end

Use isconcretetype to verify type specificity. Abstract types represent conceptual categories rather than instantiable objects.

println(isconcretetype(A))  # false — field type is abstract
println(isconcretetype(B{Float64}))  # true

true
true

Monitor Performance with @time

Temporary variables necessitate garbage collection overhead. The @time macro reveals allocation metrics.

function add100_slow!(x)
    for i in 1:100
        x[:] = x .+ 1
    end
    return x
end

data = randn(10^6);
@time add100_slow!(data);

  0.674279 seconds (239.40 k allocations: 774.794 MiB, 72.09% gc time, 10.90% compilation time)

function add100_fast!(x)
    x .+= 100
end

@time add100_fast!(data);

  0.021860 seconds (109.85 k allocations: 5.465 MiB, 98.85% compilation time)

Performance Improvement Strategies

x = randn(100)

# Broadcast fusion — no intermediate vectors
y = sin.(abs.(x)) .+ 1 ./ (1:100)

100-element Vector{Float64}:
 1.0301276064973934
 1.3346525236905658
 0.9545588616983962
 0.9606027341442998
 1.0331895261649688
 0.6053268084432174
 1.0262242454732222
 1.005335439649936
 0.42695999607626384
 0.9762755589611647
 ⋮
 0.14731715258009556
 0.48045924953026564
 0.8883931680589298
 0.7254725967930264
 0.8270667985754504
 0.788538287684886
 0.5317893698342574
 0.9409477171921796
 0.6586322371392586

x = [3, 1, 2]
sort!(x)  # In-place sorting

3-element Vector{Int64}:
 1
 2
 3

x = [3, 1, 2]
sort(x)   # Returns sorted copy (x unchanged)

3-element Vector{Int64}:
 1
 2
 3

x = randn(3, 3)

x[:, 1:2]          # Creates copy

view(x, :, 1:2)    # View — no copy

3×2 view(::Matrix{Float64}, :, 1:2) with eltype Float64:
 -2.99606     0.0854247
 -0.0899695   0.278193
 -0.521317   -0.709343

x = rand(3,3)

for j in 1:3      # column j
    for i in 1:3  # row i
        x_ij = x[i, j]
        perform_calculation_on_element(x_ij)
    end
end

Summary

These foundational optimization techniques unlock significant performance gains in Julia. The language community prioritizes performance extensively; deeper exploration awaits in the official documentation.

Resources

• Julia Language Performance Tips Documentation

Why Animations are Great

Effective communication represents an undervalued competency in data science. Animations tell a story that static images are unable to tell by adding an extra dimension (often time). Beyond narrative value, animated visualizations demonstrably enhance audience engagement compared to static imagery, making them valuable for presentation purposes.

Animations with Plots.jl

Plots.jl offers unified syntax across multiple plotting backends while providing straightforward utilities for generating animated GIFs. The authors recommend using Pluto notebooks for animation development, as visualizations render directly in the notebook environment.

Method 1: The @gif Macro (Simplest)

@gif for i in 1:50
    plot(sin, 0, i * 2pi / 10)
end

[ Info: Saved animation to /tmp/jl_MVibo1oH2B.gif

Add conditional flags to control frame inclusion:

@gif for i in 1:50
    plot(sin, 0, i * 2pi / 10)
end when i > 30

[ Info: Saved animation to /tmp/jl_MFApSzyfct.gif

Method 2: The @animate Macro (FPS Control)

anim = @animate for i in 1:50
    Random.seed!(123)
    scatter(cumsum(randn(i)), ms=i, lab="", alpha = 1 - i/50,
        xlim=(0,50), ylim=(-5, 7))
end

gif(anim, fps=50)

[ Info: Saved animation to /tmp/jl_SjWJiXs9uZ.gif

Method 3: Direct Plots.Animation (Maximum Control)

a = Animation()

for i in 1:10
    plt = bar(1:i, ylim=(0,10), xlim=(0,10), lab="")
    frame(a, plt)
end

gif(a)

[ Info: Saved animation to /tmp/jl_qtOLo9UHiJ.gif

---

Additional Resources

• Plots.jl Animation Documentation

I’m Stuck. Where Can I Find Help?

Working with Arrays

x = [1, 2, 3, 4]

# A "Range" doesn't store the values between 1 and 4.
y = 1:4

collect(y)  # `1:4` -> `[1, 2, 3, 4]`

4-element Vector{Int64}:
 1
 2
 3
 4

# 1 to 100 with a step size of 3: [1, 4, 7, ..., 94, 97, 100]
1:3:100

1:3:100

# Row-vector (1 x 4)
[1 2 3 4]

1×4 Matrix{Int64}:
 1  2  3  4

# Matrix (2 x 3)
[1 2 3 ; 3 4 5]

2×3 Matrix{Int64}:
 1  2  3
 3  4  5

# Matrix (100 x 3) of random Normal(0, 1) samples
randn(100, 3)

100×3 Matrix{Float64}:
  0.490658   -1.22393    -0.538834
 -1.1079     -1.29614    -0.171739
 -1.13116    -0.469699   -1.31322
 -1.58453    -0.892415   -1.59266
  0.605137    0.3716     -0.0675242
 -0.801192    1.03907     0.315111
 -0.496907   -0.898712    0.576551
 -1.7154     -1.59432    -0.943568
  0.0553514   1.52612    -0.367216
  1.12369     0.212616   -0.607677
  ⋮                      
 -0.0124157  -2.15849    -0.385877
 -0.280765    0.0445803   1.13861
 -0.836502   -1.86265     0.180136
 -1.12289     0.264731   -0.93608
 -1.97876    -0.411737   -0.523138
 -1.04823    -0.607487   -0.206827
 -2.06338     0.251422   -2.07863
 -0.407337    0.246409    1.1479
 -0.161176    1.35028     0.942731

x = rand(100, 2)

x[3, 2]  # retrieve 3rd row of column 2

0.6285777504379272

x = rand(100, 2)

x[105] == x[5, 2]

true

Working With Strings

Key differences from other languages:

• " creates strings
• 's' creates character literals
• String concatenation uses *:

"Hello, " * "World!"

"Hello, World!"

• String interpolation uses $:

x = "World!"
"Hello, $x"

"Hello, World!"

• String macros change interpretation:

r"[a-z]"  # I'm a regular expression!

r"[a-z]"

html"<div>I'm html</div>"  # I'm HTML!

How do I Find/Install/Load Packages?

(@v1.7) pkg> add DataFrames, StatsBase

using DataFrames, StatsBase

# Only bring certain names into the namespace
using StatsBase: countmap, zscore

] activate <dir>

What are Types?

Everything in Julia has a type:

typeof(1)

Int64

Types can be parameterized. A vector of 64-bit integers is Array{Int64, 1}:

typeof([1,2,3])

Vector{Int64} (alias for Array{Int64, 1})

Following Int64 up the type hierarchy reaches Any:

for T in [Int64, Signed, Integer, Real, Number]
    println(supertype(T))
end

Signed
Integer
Real
Number
Any

Abstract types define sets of concrete types. You can instantiate Int64 but not Real.

What is Multiple Dispatch?

Multiple dispatch calls different code based on argument types. This is a core strength of Julia.

f(x::Int) = 1
f(x::Float64) = 2

println(f(1))    # 1
println(f(1.0))  # 2

1
2

Type annotations restrict allowed types but aren’t necessary for functionality:

function f(x::Type1, y::Type2, z::Type3)
    # big computation
end

function f(x, y, z)
    # big computation
end

Julia’s JIT compiler creates specialized methods for each type combination, yielding identical performance either way.

What is Broadcasting?

Broadcasting applies functions to multiple inputs simultaneously by adding a dot .

sin([1,2,3])

MethodError: no method matching sin(::Vector{Int64})
The function `sin` exists, but no method is defined for this combination of argument types.

Closest candidates are:
  sin(::Irrational{:π})
   @ Base mathconstants.jl:146
  sin(::Float16)
   @ Base math.jl:1526
  sin(::Missing)
   @ Base math.jl:1548
  ...

Stacktrace:
 [1] top-level scope
   @ ~/work/juliafordatascience.com/juliafordatascience.com/posts/from_ghost/2021-04-13-quickstart/index.qmd:218

sin.([1,2,3])

3-element Vector{Float64}:
 0.8414709848078965
 0.9092974268256817
 0.1411200080598672

Broadcast fusion combines operations without creating temporary arrays:

x = [1,2,3]
y = [4,5,6]
z = [7,8,9]

x .+ (y .* sin.(z))

3-element Vector{Float64}:
 3.6279463948751562
 6.946791233116909
 5.472710911450539

What should I use to Code in Julia?

According to the 2021 Julia User & Developer Survey:

• 62% use VS Code with the Julia plug-in
• 23% use Pluto.jl
• 21% use Vi/Vim/NeoVim
• 17% use JupyterLab
• 13% use Juno
• 10% use Emacs

What are Macros?

Macros (names starting with @) are functions that operate on expressions before execution. They enable metaprogramming.

Example using @time to measure performance:

@time begin
    sleep(1)
    sleep(2)
end

  3.005262 seconds (9 allocations: 336 bytes)

Additional Resources

• The Julia Programming Language YouTube Channel
• Julialang.org learning resources
• Julia Macros for Beginners

Download Julia

To download Julia, use the instructions at https://julialang.org/downloads. For convenience, we’ll repeat the installation method most people will use:

curl -fsSL https://install.julialang.org | sh

winget install julia -s msstore

The above command will install the juliaup installation manager for Julia. This is generally recommended over a manual download since juliaup will let you know when new versions are available.

Install Packages

Julia has an amazing package manager: Pkg (more on Pkg in a future post) that has two methods of operation:

julia> using Pkg

julia> Pkg.add("StatsBase")

julia>

(@v1.12) pkg> # type ']' to enter Pkg mode

(@v1.12) pkg> add StatsBase

julia>  # type backspace to go back to Julia mode

A New Home

In 2025, the site moved from Ghost to Quarto, an open-source scientific publishing system. The motivations were straightforward:

• No vendor lock-in — content lives in plain Markdown files under version control
• Free to host — deployed to GitHub Pages via GitHub Actions
• Better for technical content — native support for executable Julia code, LaTeX math, and syntax highlighting

All posts from the Ghost era have been preserved and are still available here.

Stay Updated

The newsletter is no more, but you can follow along via the RSS feed — subscribe in your feed reader of choice to get notified when new posts go up.

You can also reach out on Twitter: @JuliaForDataSci

What Learning Rate is “Best”?

Below is an animation of the linear regression loss value () for a changing learning rate parameter (from to ) for SGD and variety of variants that supposedly improve on SGD (hyperparameters chosen as recommended by the papers that introduced the algorithm). The y-axis is cut off at the loss evaluated at the OLS estimate, so that as the lines approach the bottom of the plot, the algorithm approaches the best solution.

It is difficult to pick out a clear winner (except for maybe MSPI…) for a few reasons:

1. Methods are extremely dependent on the learning rate.
2. Learning rate effects are heterogenous.

The takeaway is that we can’t make definitive statements about which method is best for even a straightforward model like linear regression, so be wary of bold claims about a given method’s superiority in deep learning. It seems to me that one should be very concerned with an algorithm’s “robustness” to the choice of learning rate, which doesn’t get much attention in SA literature.

Multiple Dispatch + JIT (Dynamic and Compiled)

When I am asked why I use Julia, my immediate response is “multiple dispatch”. Julia is well-known for performance, but that is only a part of what keeps me using it every day. Multiple dispatch is a feature where different code is called by a function depending on the types of the arguments. Combined with the JIT (Just-in-time compiler), Julia will automatically compile specialized code for each set of argument types the function is called with.

f(x) = x + x

f (generic function with 1 method)

julia> @code_llvm f(1)

define i64 @julia_f_62945(i64) #0 !dbg !5 {
top:
  %1 = shl i64 %0, 1
  ret i64 %1
}

julia> @code_llvm f(1.0)

define double @julia_f_62949(double) #0 !dbg !5 {
top:
  %1 = fadd double %0, %0
  ret double %1
}

cdf(dist, x) - q = 0

using Distributions

function myquantile(d, q)
    out = mean(d)
    for i in 1:10
        out -= (cdf(d, out) - q) / pdf(d, out)
    end
    out
end

myquantile(Normal(0,1), .5)

0.0

myquantile(Gamma(5,1), .7)

5.890361313697006

myquantile(Beta(2, 4), .1)

0.11223495854585855

Takeaway

The language you use has a tremendous effect on how you approach problems (see linguistic relativity). I have a background in statistics, so naturally R was one of the first languages I learned. I don’t mean to bash R (language wars are boring) as it is a fantastic tool for data analysis, but I often find myself asking “how do I solve this without a for loop?” since loops are slow in R. In Julia, I have fewer performance obstacles, so my questions are more along the lines of “what are the methods I’m trying to accomplish this task with?”. If I can reduce a task to the operations that need to be performed, it becomes easy to write abstract yet performant code that works with any types I throw at it.

Multiple dispatch has become invaluable to how I code, and with Julia you get it along with stellar performance. If you want to see how I use multiple dispatch to get a lot done with very little code, check out my package OnlineStats.jl for calculating statistics/models on data streams with single-pass algorithms.

I hope you try Julia for yourself and have the same experience I had.

Come for the speed. Stay for the productivity.

OnlineStats

In OnlineStats.jl, different weighting schemes can be plugged into any statistic or model (see docs on Weights). OnlineStats implements AdaGrad in the mean form, which opens up a variety of learning rates to try rather than just the default . The plot below compares three different choices of learning rate. It’s interesting to note that the blue line, which uses the above default, is the slowest at finding the optimum loss value.

To try out OnlineStats.ADAGRAD, see StatLearn

Definition

Majorization-Minimization (MM) is an important concept in optimization. It is more of a principle than a specific algorithm, and many algorithms can be interpreted under the MM framework (coordinate descent, proximal gradient method, EM Algorithm, etc.). The main idea is the concept of a majorizing function.

A Function is said to majorize at if

Simply put, the surface of lies above , apart from at where they are equal. An MM update consists of minimizing the majorization:

Things to note:

• MM updates are guaranteed to decrease the value of the objective function (descent property).
• Majorizations are often used to split parameters, allowing updates to be done element-wise.

Visualization

• Blue: Objective
• Green: Majorization
• Red: Parameter Estimate

Example: Logistic Regression