In May, 2010, the ECO released its second Annual Report on the progress of activities in Ontario to reduce or make more efficient use of electricity, natural gas, propane, oil and transportation fuels. Click here for more information on this report, including videos and communications materials.
Executive Summary Setting the Context The Numbers to Date Conclusions and Recommendations	Broadening the Policy Discussion Revisiting Ontario’s Greenhouse Gas Reduction Targets The Role of Forest Biofibre Monitoring, Reporting and Verification Priorities Transportation and the Role of Road Pricing Pricing Carbon in the Economy

In our GHG Progress Report 2008/2009 the ECO indicated its strong support for the government’s intention to include third-party verification of actual (ex post) GHG savings as they become available. The goal of a GHG verification process should be to confirm whether a stated GHG claim has been accurately calculated and truthfully reported. In its CCAP Annual Report 2008-09, the government indicated that the plan’s forward-looking emission reduction forecasts had been validated, and that the government planned to implement a third-party verification process beginning with its 2009-10 annual report. The ECO supported this commitment to “include third-party verification of actual measured savings as they become available.” In time, the ECO would expect the Ontario government’s verification of actual CCAP results to complement the NIR as a source of GHG data. The ECO sees merit in using both sources of data going forward but does not expect the government to reconcile the two.

Contents 1 Defining Terms 2 Verification Fundamentals 3 Verification of CCAP Initiatives Going Forward 3.1 Process Controls 3.2 Scope 3.3 Sampling Plan 3.4 Initiative Implementation Rates 3.5 Additional Concepts Unique to a CCAP Verification

Defining Terms

It is appropriate to begin a discussion about verification by defining terms. The term “verification” has multiple potential meanings to different users. In the context of GHG accounting, verification has a very specific meaning. However, it is often misused when applied to GHG accounting, perhaps because of its more frequent use in general parlance to refer simply to a “double check” on work completed.

In the context of GHG accounting, verification can be defined as a systematic, independent and documented process for evaluating greenhouse gas assertions (or claims) against agreed upon verification criteria. 2 In contrast, validation is the process of confirming that the GHG accounting being done meets the users’ needs. GHG verification is therefore more complex than a “double check” of a GHG calculation. At the least granular level, verification can be said to consist of two components:

1. A management systems review (or conformity assessment), and
2. A quantification review (or materiality assessment).

The goal here is to ensure that the verification fundamentals are understood and will provide the intended users (the government, the ECO and the general public) with the requisite assurance.

Verification Fundamentals

Verification fundamentals require the following parameters be clearly understood: 1) objectives; 2) scope; 3) criteria; 4) level of assurance, and 5) materiality. Objectives set the context and are the test for determining the relevance and veracity of a GHG reduction claim. Scope defines the what, where, when and who of the verification process. Ideally it also defines the how – how much latitude the verifier will be given to assess the original validator’s assumptions. Criteria set the parameters and program rules to assist the verifier in determining overall compliance. Assurance levels are the client’s confidence thresholds used to accept/reject a verifier’s conclusions. And, finally, materiality is related to the error or uncertainty inherent in the reported data.

Verification Fundamentals

Objectives The key objective of the CCAP’s verification should be for the verifier to draw a conclusion that confirms or rejects that a claim regarding actual GHG performance (either for a specific CCAP initiative or for the entire Action Plan) is appropriate and true. More specifically, the objective should be to be able to make a statement on the “truth” of a CCAP initiative’s conformance with criteria (GHG program rules) and with the validated plan (e.g., project design document), material changes (if any), controls for process and data quality, and accuracy. Scope There are four main dimensions to the scope of GHG verification. These define what precisely is being verified, where it is located, when it happened, and who will use the results of the verification. Scope is important for another reason in the context of the CCAP. Will the CCAP verifier be expected to review the assumptions and methodologies that were previously validated and found to be reasonable? If not, the verifier is basically being asked to base its opinion of the accuracy of calculations and the truthfulness of the reporting on the validator’s assurance which, even if it is high, is not a complete guarantee on the assumptions used. Since it is possible that the assumptions used could have material impacts on the GHG reductions of the initiatives, it would be useful to have them re-assessed by the verifier despite the assurance provided previously by the validator. An additional consideration in the discussion of scope for the CCAP verification is whether an uncertainty assessment should be undertaken as part of the verification. Uncertainty assessments represent good practice in GHG verifications because uncertainties affect the GHG claims in two important ways: the verification sampling plan, in that high uncertainties would increase the potential risk of errors in the data itself, requiring the verifier to utilize a more thorough sampling plan to reduce the verification’s detection risk to an acceptable level; and, the materiality assessment, in that high uncertainties may mean the GHG emissions calculated are far enough off their “true” values to cause the GHG claim to be materially incorrect. Criteria In the absence of the CCAP having clearly applicable program rules or criteria against which to determine compliance, the verification could proceed using the International Standards Organization’s GHG quantification principles as general criteria, and the validated emissions modelling methodology for each sector as the detailed criteria. Level of Assurance Level of assurance is the degree of confidence the intended user requires in a verification statement. Most GHG programs require a “reasonable” level of assurance (also known as a high level of assurance, or “positive assurance”) for all verification activities. A reasonable level of assurance is distinguishable from lower levels of assurance in that there is more emphasis on detailed testing of GHG data and information supplied to support the GHG assertion. This is an area of considerable importance to the ECO. The reduction claims that are being verified in the government’s annual CCAP reports and the verification statements that the government is expecting of the verifier should be well-defined before the verification process gets underway. For example, the verifier should be able to provide assurance that each of the CCAP initiatives has an appropriate program management process and that the management process is working as intended. This requirement speaks directly to one of the key concerns the ECO raised in last year’s progress report regarding governance: ideally, the assurance of the program management process would include a review of responsibility, authority and accountability roles within the government with respect to the CCAP initiatives. Materiality In typical GHG verification, materiality is generally either defined explicitly by the client or will be proposed by the verifier to its client based on the verifier’s professional judgment and interpretation regarding the needs of the intended user. The latter approach was taken by the CCAP validator, who first proposed a threshold of 20 per cent for GHG reductions between the government’s projections and validation projections for any given initiative, but then adapted this to be 5 per cent for the overall emissions because "a material issue identified for a single initiative, no matter how small in terms of GHG reductions, could prevent the [validator] from issuing an assurance statement covering the CCAP initiatives analyzed, which was the objective of the project.” A similar assessment and proposal will have to be made by the verifier.

Verification of CCAP Initiatives Going Forward

The ECO recognizes that the verification of the correctness of the technical and process-orientated aspects of CCAP initiatives will likely be a unique assignment for a “typical” GHG verifier, with no prior examples or specific standards or protocols for the verifier to draw upon. This suspicion is supported by the government’s validator, which noted in its October 2009 report that the validation of the CCAP initiatives was "the first known validation of estimates within a climate change mitigation process; hence, there are no directly relevant protocols.” Based on the discussion of verification fundamentals in Section 2.3.2, the following observations are offered to point the way to the development of a CCAP verification process.

Process Controls

Process controls may be the single most important aspect of a CCAP verification. For most initiatives the verifier is unlikely to have very much data on which to base their assessment, likely just the small number of data points provided by the lead ministry in the common reporting templates. Additionally, since the government’s validator did not appear to assess the controls on the data used in the projections, making such an assessment at the verification stage is that much more valuable. Requiring the verifier to assess the controls on the data used in the CCAP initiatives will also provide the assurance the ECO requested in our last progress report around controls (specifically, roles, responsibilities and accountabilities) in the data collection and emission reduction calculation processes.

Scope

An important consideration with respect to the scope of the CCAP verification relates to whether the verification should include a review of assumptions and methodologies that were previously validated and found to be reasonable. Since it is possible that the assumptions and methodologies used could have material impacts on the GHG reductions of the CCAP reduction initiatives, the government may wish to consider having them re-assessed by the verifier despite the assurance provided by the validator.

It would also be useful if the verification scope includes an uncertainty assessment (something that doesn’t appear to have been done by the government’s validator) to determine if the assumptions and methodologies introduce a material uncertainty into the calculation of GHG emission reductions from the CCAP initiatives.

Sampling Plan

The CCAP states that verification will be carried out on the actual results arising from a sample of initiatives. It will be important for the government to use a risk assessment approach to identify the initiatives that will be included in the verification. Since it is clear that most emission reductions over the short term are expected to come from the elimination of coal use for power generation, the ECO expects that the coal phase-out would be one of the initiatives verified. The government should require that a risk assessment be performed prior to each verification to take into account the future likelihood of initiatives, such as cap-and-trade, not proceeding.

Initiative Implementation Rates

One extremely important assumption of the CCAP, noted by both the validator and the ECO, is that the plan’s initiatives are assumed to have an implementation rate of 100 per cent (i.e., will be fully implemented). In verifying any initiatives, the implementation rate will therefore be an important factor for the verifier to assess. The results will provide assurance to the public and the ECO as to which initiative’s implementation rates were quantified and presented appropriately, and it will indicate which initiative implementation rates require further work on the part of the government to quantify in a manner that is defensible.

Additional Concepts Unique to a CCAP Verification

One aspect of the CCAP that is different from a typical project or inventory is that it incorporates multiple “levels” of GHG emissions, removals and reductions information, each of which has its own associated GHG claim, and many of which are arrived at using different methodologies. It will be important for the purposes of transparency that the government reports on the development of other metrics wherever possible for the non-quantifiable CCAP initiatives - such-as the Next Generation Jobs Fund and the Ontario Emerging Technologies Fund - in order to track their contribution to the transition to a low-carbon economy. While the goals and criteria would have to be developed by the government, substantial guidance on a framework in which such criteria could be developed already exists.

As indicated in our 2008/2009 Progress Report, the ECO believes that the verification process can assist in adjusting strategies for the design of new CCAP initiatives, referred to as “policy learning” and to ensure more transparent accountability for results, referred to as “performance management”. This reinforces the need for transparency in articulating both responsibilities and accountabilities for CCAP results.

Recommendation : The ECO recommends that a CCAP verification process ensure that the responsibilities of key line ministries are clear and transparent and that accountabilities for results are clearly articulated.

Previous section: The Role of Forest Biofibre

Next section: Transportation and the Role of Road Pricing

---

Citing This Article:
Environmental Commissioner of Ontario. 2010. Annual Greenhouse Gas Progress Report 2009/2010: Broadening Ontario's Climate Change Policy Agenda. Toronto, ON : Environmental Commissioner of Ontario. pp. 14-18