Sigmas and for loops

Posted on February 03, 2021

Table of Contents

• Short summary
• Sigma notation
  • Single sigma
  • Nested sigma
• Comments, questions, input, concerns?

Short summary

Sums and products of sequences, represented with "∑" and "∏", respectively, can be thought of as for loops, and vice versa. For those of us who program more than we read/write math, mentally translating them as such can make them faster and easier to understand. A table to translate between the two is below:

This means for the sum of a sequence:

is equivalent to for (i = 1; i <=n; i++) { sum += body }

and for the product of a sequence:

is equivalent to for (i = 1; i <=n; i++) { product *= body }

Sigma notation

Math in manuscripts is often daunting and, for some of us, causes our eyes to glaze over. It really though, is just a concise way to communicate information. Summation and products of sequences, represented by ∑ and ∏, often take me longer than I would like to admit to process and understand, particularly if there are multiple nested ones. Once I had the (obvious in retrospect) "shower thought" that ∑ and ∏ were equivalent to "for" loops in programming, they became much easier to read. Below are some brief examples that hopefully are helpful to others to make the same mental translation. If you see a ∏ instead of ∑ in the literature, remember that it is the same translation, you just need to use multiplication (*=) rather than summation (+=) in the for loop.

Single sigma

Take calculating the log-probability score of a model, S(q), which is calculated by summing the log of the model's probability, q, for each observation, i:

You can translate this to a simple for loop as follows (assuming 1-based array indexing):

1. S(q) becomes log_prob_score
2. ∑ becomes for
3. i = 1 under the ∑ becomes the i = 1 in the for line
4. n over the ∑ becomes i <= n in the for line
5. The "body" of the ∑, log(qᵢ) in this case, is the body of the for loop.

And that's it! See an example written in awk below:

y = get_data();
n = length(y);
q = get_model_predicted_prob(y);

log_prob_score = 0;
for (i = 1; i <= n; i++) {
    log_prob_score += log(q[i]);
}
print "The log prob score is", log_prob_score;

While a functional python version, whose structure is remarkably similar to the math notation, would be:

print(f"The log prob score is {sum(map(log, map(q, get_data())))}")

Nested sigma

Nested sigmas can similarly be translated into nested for loops. Take log-pointwise-predictive-density, the Bayesian version of the log-probability score. This needs to sum the average of the each data observation's probability for each sample of the posterior's. Two "each"es in the previous sentence means we will need two sigmas or for loops. It's mathematically represented as:

which can translate to two for loops:

1. lppd(...) becomes lppd
2. First ∑ becomes outer for
3. i = 1 under first ∑ becomes the i = 1 in the outer for line
4. n on top of the first ∑ becomes i <= n in the outer for line
5. The "body" of the first ∑, log(...) in this case, is the body of the outer for loop.
6. Second ∑ becomes inner for (and the sum will be accumulated with sum_sample_prob)
7. s = 1 under the second ∑ becomes the s = 1 in the inner for line
8. S on top of the first ∑ becomes s <= S in the inner for line
9. The "body" of the second ∑, p(...) in this case, is the body of the inner for loop

This translates to the following awk code:

y = get_data();
n = length(y);
theta = get_posterior_samples();
S = length(theta)

lppd = 0;
for (i = 1; i <= n; i++) {
    sum_sample_prob = 0;
    for (s = 1; s <= S; s++) {
        sum_sample_prob += prob(y[i], theta[s]);
    }
    lppd += log(sum_sample_prob / S);
}
print "The lppd is", lppd;

and a "make-your-eyes-bleed" not great "functional" representation in python that is more similar to the math representation:

print(f"The lppd is {sum(map(lambda y: log(sum(map(lambda theta: prob(y, theta), get_posterior_samples()))/len(y)), get_data()))}")

Forget that line of code above ever existed.

Comments, questions, input, concerns?

I hope this article helps! Please contact me with any questions or input on the article using any of the methods on my contact page.