In September of last year, consultants keeping watch over
Massachusetts' energy efficiency program issued an update on the state's
ability to track savings. The presentation
included data about public awareness, satisfaction levels after
retrofits, and draft results for residential and commercial programs. It
was all mostly positive. But the slides also included a revealing statement.
"Not
only can we only scratch the surface of recent results, but we can only
scratch the surface of the surface," wrote Ralph Prahl, the lead for
evaluation, measurement and verification (EM&V) at the Massachusetts
Energy Efficiency Advisory Council. The issue, explained Prahl,
was that the state relied heavily on draft results to draw conclusions
at the time. In addition, the utilities and consultants on the advisory
council still couldn't agree on how some programs were performing -- a
first for the state.
"Increasing average lag time between first
and final drafts of EM&V reports means [a] large number of reports
[are] currently in the draft stage," wrote Prahl. The admission worried some onlookers. Massachusetts budgets $500 million a year for efficiency programs and $70 million to monitor results. If the top-ranked state in energy efficiency can't access performance data in a timely way, what does that say about the rest of the country?
{bq-1}
The
problem isn't unique to Massachusetts. A growing number of
energy-efficiency professionals are speaking out about the overly
complex and archaic way that energy efficiency is measured. "We
spend all this money, but our ways to verify if they're saving energy
are not working that well," said Tim Guiterman, the director of EM&V
solutions at Energy Savvy, a software company that touts itself as a
modern alternative to efficiency program management.
At a time
when nearly every product, service and behavior can be tracked in real
time, the efficiency industry still relies on complicated models and
outdated data to verify energy savings, said Guiterman. "Often,
you're getting results three years later. It’s as if you had a
speedometer in your car that told you how fast you went an hour ago," he
said.
The problem is not limited to any one sector, efficiency
company or utility. Rather, say experts, the industry is systematically
plagued by an outdated way of measuring performance -- partly because
efficiency is hard to track compared to energy generation, and partly
because of the industry's inability to modernize.
Utilities in the U.S. now spend
more than $7 billion per year on ratepayer-funded energy efficiency
programs. Within a decade, ratepayers may be supporting up to $15 billion per year in utility efficiency spending, according to government estimates. The low cost of delivering energy efficiency cannot be rivaled -- it is by far the cheapest resource. But do utilities actually know how much they're saving? That's debatable.
Source: Lawrence Berkeley National Laboratory
Economists and analysts tussle over the role of efficiency in nationwide carbon emission reductions, the scope of the rebound effect, and how effective building codes
are in changing energy use. With different statistical assumptions,
wildly different conclusions can be reached about efficiency. A similar debate is taking place on the utility level, where different approaches to modeling often bring different results.
"Jurisdictions
calculate and define savings differently, utilize different deemed
savings values and baseline assumptions, tend to not report uncertainty
in results, and apply different levels of independent review," wrote a group of experts at the DOE's State Energy Efficiency Action Network back in 2011.
The
result: "EM&V is sometimes seen as expensive, not credible, not
timely, not transparent, and as a burden, not a benefit." With
billions of dollars at stake, that uncertainty is inexcusable, said
efficiency industry veteran Michael Blasnik. He and others have found that traditional modeling approaches can be off by double digits.
"To
me, the real scandal is how much utilities spend on consulting firms to
determine cost-effectiveness. They don't use energy data; they use
projections, models and widgets," said Blasnik, now a senior building
scientist at Nest Labs. (Blasnik spoke to GTM before joining Nest, when he ran his own consultancy focused on evaluating energy efficiency.)
Blasnik
has long been critical about the use of pooled models and regression
analysis for evaluating residential efficiency programs, rather than
home-level analysis. His conclusion: "If you need fancy statistics to
find the savings, you’re on shaky ground." "Millions of dollars
get spent doing these studies where no one knows the assumptions. You
could end up with statistical analysis that disproves the laws of
thermodynamics," said Blasnik.
In addition, most utilities rely on
self-reporting to determine net savings. After issuing rebates, a
utility may hire a consultancy to call customers and ask if they would
have retrofitted their home or facility without financial assistance.
The practice is a way of separating legitimate projects from "free
riders."
There's plenty of dispute over self-reporting. Last October, the Missouri utility Ameren sparred with regulators
after it claimed 70 gigawatt-hours of savings more than what
independent auditors measured. Its reported savings were based on
self-reported data. "When you give people money and ask if the
program that gave you money worked, how do you think they'll respond?"
said Energy Savvy's Guiterman.
There is no single answer to the
industry's EM&V problem. Because energy efficiency cannot be
measured in the same way that energy generation can, there are inherent
challenges to measuring behavior change. But new approaches to tracking
savings are emerging. California-based Lime Energy thinks it can track savings on a project-by-project basis better than the status quo.
{bq-2}
Lime
targets the most difficult customer set: small and medium-sized
businesses. The company sets up direct installation programs for
utilities, hires local contractors to complete the projects, and then
charges the power companies for every kilowatt-hour of efficiency
delivered with no administrative fees. The savings are tracked in real
time by Lime's software. "EM&V needs to be continuous and
instantaneous. Old methodologies have not gotten us there," said Arjun
Saroya, the VP of engineering at Lime. "Utilities have become tired of
lack of results from larger administrators. We’ve capitalized on that
trend."
Boston-based EnergySavvy is growing its business based a
similar premise. The company has built an end-to-end software platform
that helps utilities target efficiency opportunities, manage projects
and rebates, and track savings through utility bills and meter data. "What
we do is billing analysis on steroids. And that's never been available
before," said Guiterman. "It gets them away from the customer
self-reported stuff."
These two companies are among dozens in the "intelligent efficiency"
sector using better data loggers, sensors, meters and analytics
software to fine-tune reporting of energy efficiency. They're all
attacking a different part of a major problem: the slow adoption of IT
in measuring efficiency program performance.
"M&V practices
have yet to evolve to take advantage of the smart grid
infrastructure that allows for increased data collection," wrote DOE
efficiency experts back in 2011. Four years later, these companies are
just starting to change the way utilities track energy savings. Efficiency
entrepreneur and policy advocate Matt Golden, a frequent contributor at
GTM, believes utilities are ignoring their most important piece of
infrastructure: the smart meter.
"Moving to energy-efficiency
procurement that pays for efficiency at the meter will unshackle
contractors and the broader energy-efficiency industry from the trap of
current incentive programs and the stifling regulation that inevitably
goes with them," he wrote in a recent op-ed. Golden
put it much more bluntly in an interview: "I'm looking to fundamentally
change the way we think about M&V. If you think about efficiency as
a commodity and think about programs in terms of resource acquisition,
it's way simpler."
The problem, he said, is that utilities
approach efficiency through a coupon-based approach -- sending out
rebates and then calculating savings through complicated models and
self-reporting in the hopes that the program will be deemed effective
months or years later. As a senior consultant to the Investor Confidence Project, Golden is applying his passion for standards to metering. He's currently working on the Open EE Meter, an open-source tool designed to standardize the way savings are measured.
A
group of home-performance professionals, led by Nate Adams and Ted
Kidd, have also been advocating for a simpler way to measure savings and
deliver rebates in the residential efficiency space. Called "One Knob,"
their proposed program is structured around reading savings at the
meter -- delivering incentives simply based on negawatts, not
prescriptive retrofits demanded by a program administrator.
"Is
paying for a negawatt too simple? Well, isn't that what utilities and
public utility commissions want? Incentivizing saved energy directly is
the fastest and simplest path there," wrote Adams in a piece at GTM last fall. Almost everyone agrees that M&V needs to improve. But some are skeptical about the singular obsession with the meter.
"Reading
the meter is helpful, but it doesn’t get to all of the factors that you
have to account for. That’s just the starting point," said Glenn
Garland, the CEO of CLEAResult, the country's biggest efficiency program
administrator. Deeper analysis is needed to account for seasonal
changes, building occupancy shifts and de-tuned equipment, he said.
The
industry is somewhat divided on the severity of the M&V problem in
efficiency. Regulators worried about their reputations and the large
consulting firms traditionally responsible for tracking programs are
generally hesitant to admit the limitations of modeling. Thus, the most
passionate advocates for reform still face a resistance to change. But rapid changes in technology that make reporting easier may finally break through the inertia. "The industry is colliding with new technology in a huge way -- and that's going to change it," said Lime Energy's Saroya.
http://theenergycollective.com/stephenlacey/2235626/lies-damned-lies-and-modeling-energy-efficiencys-problem-tracking-savings