Which Programming Languages Use the Least Electricity? – The New Stack

Can energy usage data tell us anything about the quality of our programming languages?

Last year a team of six researchers in Portugal from three different universities decided to investigate this question, ultimately releasing a paper titled “Energy Efficiency Across Programming Languages.” They ran the solutions to 10 programming problems written in 27 different languages, while carefully monitoring how much electricity each one used — as well as its speed and memory usage.

Specifically, they used 10 problems from the Computer Language Benchmarks Game, a free software project for comparing performance which includes a standard set of simple algorithmic problems, as well as a framework for running tests. (It was formerly known as “The Great Computer Language Shootout.”) “This allowed us to obtain a comparable, representative, and extensive set of programs… along with the compilation/execution options, and compiler versions.”

It was important to run a variety of benchmark tests because ultimately their results varied depending on which test was being performed. For example, overall the C language turned out to be the fastest and also the most energy efficient. But in the benchmark test which involved scanning a DNA database for a particular genetic sequence, Rust was the most energy-efficient — while C came in third.

Yet even within that same test, the “best” language depends on what your criterion is. For that test C also turned out to be only the second fastest language (again, placing behind Rust). But Rust dropped a full nine positions if the results were sorted by memory usage. And while Fortran was the second most energy efficient language for this test, it also dropped a full six positions when the results were instead sorted by execution time.

A faster language is not always the most energy efficient.

The researchers note that they “strictly followed” the CLBG project’s guidelines about compiler versions and the best optimization flags. Power consumption was measured using a tool from Intel — the Running Average Power Limit tool — with each program executed not just once, but 10 times, “to reduce the impact of cold starts and cache effects, and to be able to analyze the measurements’ consistency and avoid outliers.” (For this reason, they report that “the measured results are quite consistent.”) For added consistency, all of the tests were on a desktop running Linux Ubuntu Server 16.10 (kernel version 4.8.0-22-generic), with 16GB of RAM and a 3.20GHz Haswell Intel Core i5-4460 CPU.

In their paper, the researchers call out some interesting results.

“Lisp, on average, consumes 2.27x more energy (131.34J) than C, while taking 2.44x more time to execute (4926.99ms), and 1.92x more memory (126.64Mb) needed when compared to Pascal.”

They also compared the results from compiled languages versus interpreted languages (with a separate category for languages that run on virtual machines). And the paper also includes a separate comparison of the different programming paradigms — including both functional and imperative programming, plus object-oriented programming and scripting.

Is Faster Greener?

The paper took a hard look at the common assumption that a faster program will always use less energy, pointing out that it’s not as simple as the law of physics that says E(nergy) = T(ime) x P(ower). This is partly because power isn’t expended at a consistent rate, the researchers note, suggesting that may be impacting the work of other researchers investigating whether a program’s running time affects its energy consumption. (“Conclusions regarding this issue diverge sometimes…”) In one of their benchmark tests, a Chapel program took 55 percent less time to execute than an equivalent program written in Pascal — and yet that Pascal program used 10 percent less energy.

So while there’s still a common belief that energy consumption goes down when programs run faster, the researchers state unequivocally that “a faster language is not always the most energy efficient.”

It can be a hard question to answer, since power consumption is affected by many factors (including the quality of the compiler and what libraries are used). But ultimately the researchers were even able to break down energy consumption based on whether it was being consumed by the CPU or DRAM — concluding that the majority of power (around 88 percent) was consumed by the CPU, on average, whether the benchmark program was compiled, interpreted, or run on a virtual machine.

Interestingly, interpreted languages showed a slightly higher variation, with the CPU sometimes consuming as much as 92.90 percent of the power or as little as 81.57 percent.

After studying their results, the researchers also concluded that the relationship between peak usage of DRAM and energy consumption “is almost non-existent.”

The research provides some more insights into the perennial question: is faster greener? Yes, it’s true that “the top five most energy-efficient languages keep their rank when they are sorted by execution time and with very small differences in both energy and time values.”

In fact, for nine out of 10 benchmark problems, the top score (for both speed and energy efficiency) came from one of the top three overall fastest and most energy-efficient languages — which didn’t surprise the researchers. “It is common knowledge that these top three languages (C, C++, and Rust) are known to be heavily optimized and efficient for execution performance, as our data also shows.”

But you don’t see the same order when you rank the other 24 languages by their run-time as you do when you rank them for energy efficiency. “Only four languages maintain the same energy and time rank (OCaml, Haskel, Racket, and Python), while the remainder are completely shuffled.”

And even on individual benchmark tests, there are cases where fast-performing languages are not the most energy efficient.

The Pros of Compiled Languages

There were other interesting results. Compiled languages “tend to be” the most energy-efficient and fastest-running — and their paper can even quantify that difference with a number. “On average, compiled languages consumed 120J [joules] to execute the solutions, while for a virtual machine and interpreted languages this value was 576J and 2365J, respectively.”

The researchers also applied the same precision when comparing execution times, concluding that on average, “compiled languages took 5103ms, virtual machine languages took 20623ms, and interpreted languages took 87614ms.”

Of the top five languages in both categories, four of them were compiled. (The exception? Java.)

The five slowest languages were all interpreted: Lua, Python, Perl, Ruby and Typescript. And the five languages which consumed the most energy were also interpreted: Perl, Python, Ruby, JRuby, and Lua.

But at the same time, when manipulating strings with regular expression, three of the five most energy-efficient languages turn out to be interpreted languages (TypeScript, JavaScript, and PHP), “although they tend to be not very energy efficient in other scenarios.”

Compiled languages also took the top five slots for least amount of memory space used.

“On average, the compiled languages needed 125Mb, the virtual machine languages needed 285Mb, and the interpreted needed 426Mb,” the researchers report. Meanwhile interpreted languages claimed four of the five bottom spots, meaning they consumed the most memory space: JRuby, Dart, Lua, and Perl. (While Erlang is not an interpreted language, it would also appear in the bottom five, between Dart and Lua).

“If sorted by their programming paradigm, the imperative languages needed 116Mb, the object-oriented 249Mb, the functional 251Mb, and finally the scripting needed 421Mb.”

In fact, when comparing the different paradigms, imperative programming often came out on top. Its benchmark programs also used far less energy on average — and ran much faster — than the benchmark programs for object-oriented, functional, and scripting paradigms.

But there’s a lot of factors to consider. “It is clear that different programming paradigms and even languages within the same paradigm have a completely different impact on energy consumption, time, and memory,” the researchers write. Yet which one of those is most important will depend on your scenario. (Background tasks, for example, don’t always need the fastest run-time..)

And some applications require the consideration of two factors — for example, energy usage and execution time. In that case, “C is the best solution, since it is dominant in both single objectives,” the researchers write. If you’re trying to save time while using less memory, C, Pascal, and Go “are equivalent” — and the same is true if you’re watching all three variables (time, energy use, and memory use). But if you’re just trying to save energy while using less memory, your best choices are C or Pascal.

At the end of the paper, the researchers add that for further study, they’d like to examine whether total memory use over time correlates better with energy consumed.

They’re sharing their data online, suggesting it makes it easier for future researchers to compare, for example, .NET languages or JVM languages. For developers working with mobile applications, Internet-of-Things systems, or other apps drawing from limited power supplies, power consumption is a major concern.

But in the end, the study may also leave programmers with the thing they hate most: ambiguity. The researchers report that if you’re looking for a single-best programming language, “this question does not have a concrete and ultimate answer.

“Although the most energy efficient language in each benchmark is almost always the fastest one, the fact is that there is no language which is consistently better than the others,” the researchers conclude. “The situation on which a language is going to be used is a core aspect to determine if that language is the most energy-efficient option.”