Ontario polls continue to confuse

Two new polls emerged in the last few days, the first from Oracle Research for Global News and the second (and third) from the Innovative Research Group, the results of which were first discussed on TVO's The Agenda. But there's a hiccup. According to these polls, the Tories are either leading by 11 points or trailing by six.

Yet again, the projection model splits the difference to come up with a near tie in the aggregate: 35.6% for the Tories (or between 34% and 39%) and 35.5% for the Liberals (or between 34% and 39%). The New Democrats remain in third, with 22.6% (or between 21% and 24%).

The Liberals have moved ahead in the seat count again, with 46 seats to 40 for the PCs. The ranges overlap a great deal, however, making the Liberal advantage incredibly soft. The Liberals range from between 39 to 50 seats compared to between 39 and 48 for the Tories. Despite the overall lead for the Liberals, it is still really a toss-up.

And the polls give little indication of which way we should lean.

The Oracle poll pegged the Tories to have 42%, against 31% for the Liberals and 25% for the New Democrats. We haven't heard from Oracle at this level any time recently, though their website does have a poll from early April done for an energy organization. Compared to that poll, this represents an eight-point jump for the Tories and a four-point slip for the Liberals.

The two Innovative polls are interesting, in that the company released the results from both a telephone and online poll done at the exact same time. And the results were almost identical.

The telephone poll gave the Liberals 38.7% support, with the Tories at 32.8% and the NDP at 23.7%. Those results showed no real change from a previous poll done by Innovative in March. The online poll gave the Liberals 38.8% against 32.8% for the Tories and 19.9% for the NDP. This showed a bigger change from an April poll done online, with a swing of three points between the Tories and NDP to the benefit of the PCs.

It is interesting that Innovative shows very little difference between methodologies, except for a higher Green number in the online poll at the expense of the NDP.

Also interesting are the other numbers released by Innovative. The company estimates that 15% of voters are "Core Liberal", with 14% being core PC supporters and 10% being core NDP supporters. That is the base (which includes undecideds), but most fascinating is Innovative's estimate of swing voters. The poll suggests that 5% are PC-NDP swing voters, 7% are Liberal-PC swing voters, while 13% are Liberal-NDP swing voters. This gives one indication of why the Liberals have lurched towards the left in their latest budget - there are more voters to pick up in that direction.

But the polls are still confused, as they have been for some time. A PC lead of the size measured by Oracle is certainly unusual, only replicated in Ipsos Reid's most recent estimate of likely voter support. The large Liberal lead is also hard to swallow, though it has been repeated in some other polls. That the race remains close among a disinterested electorate is probably the safest bet. Hopefully the polls will begin to converge as Ontarians tune in and election day approaches.

Polls clear as mud in Ontario

The Ontario Liberals and Tories are neck-and-neck. Or the Liberals have a wide lead. Or the Tories do. Should we split the difference?

The current aggregate of the polls puts the PCs and Liberals in a tie with 34% apiece, with the New Democrats trailing at 25%. That is probably as clear as it gets in Ontario, as no two recent polls agree on what the situation is in the province.

The chart below shows the province wide results of three polls conducted in Ontario in the last few weeks. All were done within a window stretching from April 7 to April 14, surveying roughly similar numbers of Ontarians but using three different methodologies.

The Forum poll, the earliest of the three, gave the PCs a seven-point lead over the Liberals, with 38% to 31% support.

The Nanos poll, the second of the three but the one released most recently, put the Liberals and Tories in a tie with 36% apiece.

The Innovative poll, the most recent of the three, gave the Liberals a nine-point lead, with 39% to 30% for the Tories.

One might be persuaded to see some progression across these three polls, with the Liberals gaining eight points between April 7 and April 14, all at the expense of the Tories, but that would not be the appropriate conclusion. For one, that is too short of a time span for such a dramatic swing in support. Secondly, different polls conducted by different firms using different methodologies should not be used in that manner. Nevertheless, the three surveys should be in the ballpark.

They aren't even in the ballpark in terms of trends. Nanos has shown a swing from the NDP to the PCs compared with their last survey of Feb. 28-Mar. 3. Innovative has recorded a swing away from the PCs and NDP towards the Liberals since their last poll from March. And Forum has registered a swing from the Liberals to the PCs compared to their poll of March 24.

The one thing the three polls seem to agree on, however, is where the New Democrats stand. All three polls put the NDP at either 22% or 23%, which represents a low-point for the party in the last few months. All three polls also show either stable or negative trends for the NDP, suggesting that Andrea Horwath is not heading into a potential election with any sort of momentum.

But even that is not too clear. The last poll by EKOS Research, out of the field on April 3, just a few days before Nanos and Forum, had the New Democrats at 29% and on the upswing, with the Liberals at 32% and the PCs at just 27%. So if we stretch this string of polls back to EKOS, we then have the Liberals somewhere between 31% and 39%, the PCs between 27% and 38%, and the NDP between 22% and 29%. Got it?

The seat implications for these sort of ranges are enormous. The three most recent polls point to anything from a PC minority government to a Liberal majority.

Seat projections

The seat projection using the Forum poll is the best for the Tories, with 51 seats to 35 for the Liberals and 21 for the NDP. Nanos's results would end up with a similar distribution of seats as currently exists in the Legislative Assembly, with 49 Liberals, 40 PCs, and 18 New Democrats. And the Innovative poll would give the Liberals a majority government of some 59 seats, with the Tories reduced to 27 seats and the NDP winning 21.

The EKOS poll would further confuse things, with 44 seats going to the Liberals and 31 to both the PCs and New Democrats.

Each leader can choose a poll to back-up an election call. Kathleen Wynne would certainly like to see the results from Innovaitve, Tim Hudak would take Forum's poll, while Andrea Horwath would gladly accept EKOS's results. On the other hand, Wynne could be booted from the premier's office according to Forum, Hudak could lose seats according to Innovative (and potentially the Official Opposition role according to EKOS), while Horwath could find herself stripped of influence with a Liberal majority. Who feels lucky?

At the personal level, Wynne and Horwath have better numbers. Innovative did not release any approval or leadership question results, but both Nanos and Forum did. They found somewhat similar results. On who would make the best premier, Wynne stood between 26% (Forum) and 32% (Nanos), while Hudak was second with between 24% (Forum) and 25% (Nanos). That squeezes Horwath out, who managed between 15% (Forum) and 18% (Nanos) support.

On the other hand, Horwath had the highest approval ratings in the Forum poll, at 40% to 34% for Wynne and just 27% for Hudak. A majority of respondents to the Nanos poll felt that both Wynne (54%) and Horwath (53%) had the qualities of a good leader, while just 39% said the same for Hudak.

That makes it difficult for Hudak to bank on a good campaign to propel him to office - his personal numbers are still very low. Horwath, however, may have the most to gain as she is seen the most positively of the three leaders. On the other hand, if the electoral choice becomes one of either Hudak or Wynne, Horwath could see herself shoved aside. Whether to pull the plug on this government will be a difficult call to make for the NDP leader. Predicting the results of this vote may be even harder.

Three-way race in Ontario

Yesterday, the Toronto Star reported the results of a new Innovative Research poll taken after the OLP convention showing the Liberals narrowly edging out the Progressive Conservatives and New Democrats in Ontario. With a small sample, the results of the poll are far from definitive - but it does have a few interesting nuggets.

I wrote about the poll for The Huffington Post Canada and I invite you to read the article, as I will just quickly go over the basic numbers here. I also invite you to take a look at the poll report itself, as it contains a lot of information on other interesting questions.

Before jumping to the conclusion that Kathleen Wynne is boosting Liberal fortunes, it has to be noted that Innovative appears to have been polling regularly (but releasing sparsely, those hoarders) and that the numbers for the Liberals and Tories have actually decreased since December, while the numbers for the NDP have increased. But with a margin of error of almost five points, we really can't see much about what this poll is showing, other than an effective three-way tie (generally what other polls have shown as well).

More interesting are the numbers on who is the best person to be premier: Wynne gets 24% to 18% for Tim Hudak and 13% for Andrea Horwath. That is a big drop since December, when Horwath was leading on this question with 21% to 20% for Hudak and only 19% for Dalton McGuinty. So, by this measure, we can say that Wynne looks to have improved matters for the Liberals to the detriment of the NDP.

With these provincial numbers, the Liberals could eke out another minority government with 45 seats. The Tories would be almost shut out of the Toronto area and win 34, while the New Democrats would take 27 seats. Not shown on the graph is one seat for the Greens. With 9%, or three times their score in 2011, the Greens could potentially win at least one seat, but I suspect that number is inflated (as is often the case with the Greens).

And they're off. If these kinds of numbers are backed up by some other polls, the appetite that the Tories and the NDP might have for an election could drop significantly. It is worth noting that since Wynne's leadership victory, the tone from the press gallery at Queen's Park has been more about an election in the fall or even the spring of 2014, rather than the snap election almost everyone assumed would occur if Sandra Pupatello had taken the leadership. The reaction that Hudak and Horwath will have to Wynne's throne speech will be interesting to see.

Three-way race in Ontario?

Another poll for the suddenly fascinating Ontario political scene was released earlier this week, this time by Innovative Research for TVO's The Agenda. The results show a very close three-way race where any party could come out on top.

Innovative Research was last in the field Oct. 27-Nov. 2, just after the October 2011 election. Since then, the Progressive Conservatives dropped two points to 32%, the New Democrats were up eight points to 31%, and the Liberals were down 11 points to 28%. At 9%, the Greens registered a five point gain.

Aside from the PC slip, all of these changes in support are statistically significant - though the actual significance of that is difficult to say considering Innovative's last poll is a year old. What is more interesting is that the margin of error is large enough to say that no party is definitively in front. Though the Tories have the edge, their advantage over the third-place Liberals is not even statistically significant. That means a very close race.

In fact, the race would be so close that no fewer than 11 ridings would feature three-way races, where the projected margin for the winner would be less than 10 points over the third place party. That means a very unpredictable outcome. Even the Greens could manage a seat win with these results.

But the model gives the Progressive Conservatives a minority government of 42 seats with these numbers, the seats coming almost exclusively from rural Ontario. The New Democrats win 33 seats while the Liberals win 32, all but six of them in and around Toronto.

Undoubtedly, things will shift as the Liberal leadership race gets underway, and Innovative asked some interesting questions concerning the race. For example, 18% of Ontarians said that they would definitely (8%) or probably vote Liberal in the next election, with 30% "not sure one way or the other". Put differently, that gives the Liberals a base of about 18% but the potential for as much as 48% of the vote.

That's a wide range, and is further confirmed by the 45% of respondents who said that the Ontario Liberals' policies and programs were working well or okay, and that they needed no or only minor changes. That makes for a large swathe of the population that is open to voting for the party, but it is worth noting that the proportion of respondents who said that the Liberals' policies and programs were working well and needed no changes was minuscule.

Innovative then asked about potential successors to Dalton McGuinty. Unfortunately, the apparent front-runner Sandra Pupatello was not included in the list. But coming out on top on whether a leader would make someone a lot or somewhat more likely to vote Liberal was Dwight Duncan, at a combined juggernaut of 13%. The only problem is that he has already said he won't run. David McGuinty, Deb Matthews, and Kathleen Wynne tied with 10% each.

A major problem for the Liberals, though, is that only David McGuinty (and some of that may have been people mistaking him for the Premier) and Dwight Duncan managed a majority of respondents who were able to recognize their names. Matthews had 51% non-recognition, while all others had over 60%.

In terms of who Ontarians prefer as leader, 42% don't have a clue and 26% said that they prefer no one. Of the rest, Duncan and Wynne led with 5%, followed by Matthews and McGuinty at 3% and Jim Watson (current mayor of Ottawa who has already ruled out a run) at 2%. Considering the small sample size, these numbers are hardly informative.

Among "core" and "potential" Liberals, however, Wynne averaged 14%, putting her ahead of Duncan (12%) and David McGuinty (7%). It will be interesting to see how things shift when the full list of candidates becomes known (and polled).

But it does not seem that the leadership race will make a huge difference for the Liberals. If voters' preferred candidate wins the race, it bumps up the proportion who say it would make them a lot or somewhat more likely to vote for the party to only 25%.

The goal, then, will be for the next leader to get as many as possible of those 45% of respondents who thought that the party's policies were good enough back into the Liberal fold. The question is whether too many of them are simply fatigued with the Ontario Liberals.

Mulcair's first honeymoon poll

Shortly after Thomas Mulcair was elected leader of the New Democrats, Forum Research conducted a poll that suggests that the Conservatives and NDP are now neck-and-neck in voting intentions.

Forum was last in the field on 2 March, and since then the Conservatives have dropped only two points to 35% - a shift within the margin of error.

The same can't be said for the NDP, who are up seven points since the beginning of March. They are now at 35%. This is the Mulcair bump, but it is remarkable to see the size of the bump and where it has actually taken place.

Shunted off as a result are the Liberals, down six points to 19%, back where they were in the 2011 election. This is very dangerous for the Liberals, as Thomas Mulcair appears to have regained all of the support the NDP had lost in the last year to Bob Rae.

The Bloc Québécois is up two points to 7% while the Greens are down one point to 3%.

The honeymoon bump is strongest in the West and Quebec, as the NDP has gained over 12 points in every region. The New Democrats lead in the Prairies with 44% (+19), British Columbia with 43% (+12), and Quebec with 40% (+12). They are up 15 points in Alberta to 31% as well, an incredible high for any party that isn't Conservative. The increase in Quebec is especially important, as it saves the hide of the bulk of the NDP's caucus.

The Conservatives still rule the roost in Alberta with 52%, though that represents a drop of 17 points since the beginning of March. They have gained four points in each of Ontario (42%) and Atlantic Canada (39%), the Ontario number an important one for the Tories.

The Liberals don't manage more than 25% support anywhere, reaching that level in Atlantic Canada (-4) and 24% in Ontario (-5).

But the significant Tory lead in Ontario throws the seat results out of whack, turning a close race into a near Conservative majority. With these numbers, the Conservatives win 150 seats, five short of a majority. The New Democrats make important gains to reach 122 seats, while the Liberals are reduced to only 29. Though that gives the NDP and the Liberals the ability to out-vote the Tories, they need the Bloc's seven seats for a majority.

The Conservatives win 19 seats in British Columbia, 24 in Alberta, 12 in the Prairies, 71 in Ontario, five in Quebec, 18 in Atlantic Canada, and one in the north.

The New Democrats win 16 seats in British Columbia, four in Alberta, 14 in the Prairies, 24 in Ontario, 56 in Quebec, seven in Atlantic Canada, and one in the north.

The Liberals win one seat in British Columbia, two in the Prairies, 11 in Ontario, seven in Quebec, seven in Atlantic Canada, and one in the north.

The situation is further unbalanced by the expansion of the House to 338 seats. Though the strong NDP performance in British Columbia, Alberta, and Quebec means they can win some of the new seats, the big Conservative advantage in Ontario means they win the bulk of the seats there. In the expanded House, the Conservatives would likely win 168 seats, two short of an absolute majority, while the NDP wins 131, the Liberals 31, and the Bloc eight. The Tories' share of seats increases significantly, to 49.7% from 48.7%.

Ontario, Ontario, Ontario. It is key to the Conservatives' shot at maintaining power, and absolutely essential if the New Democrats are to replace the government. At this stage, the Conservatives have a huge leg-up in the province. Mulcair will have to whittle away Conservative support by a significant degree, as gains have fewer returns for the NDP than they might have for the Liberals.

But it could be a lot worse. A poll by Innovative Research taken just before the convention pegged NDP support at only 25% nationwide, one point up on the Liberals. The Conservatives stood 12 points ahead with 37% support.

In Ontario, Innovative put the Tories at 47% to 33% for the Liberals and only 16% for the NDP. If Mulcair has indeed bumped the New Democrats up to second place, that would be a very positive start for the party.

Innovative also put the NDP neck-and-neck with the Bloc in Quebec, a state of affairs also reported by Forum in a pre-convention poll (30% BQ to 27% NDP). But British Columbia has been relatively steady, as both a Forum poll from 19 March and the Innovative survey put the New Democrats ahead of the Tories. Mulcair seems to have improved the situation somewhat, but B.C. has been leaning NDP for some time.

The honeymoon is certainly off on the right foot for Thomas Mulcair. If he can somehow maintain these impressive levels of support in Western Canada while also being favoured in Quebec (which he has a good shot of doing, merely by being a native son), there will be plenty of time to woo Ontario. But a bump is a bump - and that means there should be some bounce back in the coming months. How much of a retreat, and where it takes place, will be something to watch. But the NDP has demonstrated how a leadership race can erase some of the problems in public opinion a party can suffer between leaders. They should keep this in mind in the run-up to the Liberal leadership race.

The new year

With a new political year on the horizon, each of the parties in the House of Commons could stand to make a few New Year’s resolutions. In addition to encouraging their members to visit the Parliamentary gym more often, here are some of the resolutions each of the parties might want to consider for 2012.

You can read the rest of the article on The Huffington Post Canada website here.

To some extent, I think we're all still getting used to the new reality that is a majority government here in Ottawa. This year will be Canada's first full year of a sitting majority government since 2003. That isn't to say it will be a quiet year. This spring the federal budget will be tabled and the NDP leadership race will come to an end. In the last few months of 2012 the Liberal Party's leadership race will start rolling, and all of this while provincial elections in Alberta and perhaps Quebec take place, not to mention the election south of the border.

Speaking of which, the Republican primary vote in Iowa looks like it will be an incredibly close one between Mitt Romney, Ron Paul, and Rick Santorum. At various times in the last few months, that list was supposed to include Rick Perry, then Herman Cain, then Newt Gingrich, in place of either Paul or Santorum, or both. Meanwhile, in our NDP leadership campaign, it's all a big mystery.

But 2012 will be an important year for almost every party in this country. This is Daniel Paillé's first full year as leader of the Bloc Québécois. The Liberals will be preparing for next year's leadership race. The New Democrats will be choosing the leader who will take them into the 2015 election. The Conservative budget will set the agenda for most of their next few years in government. Could it be the most determinant year on the Hill until 2015?

It will be a big year for ThreeHundredEight, as the projection model goes through more changes and improvements, taking into account more regional data and making what I hope will be better estimations of how the votes will break on election night. Alberta and Quebec (if the latter does indeed go to the polls) should provide their fair share of challenges, what with the Wildrose Party likely to be a major factor throughout the province for the first time in Alberta and the Coalition Avenir Québec being a completely new entity there.

While on the topic of Alberta, which will become the focus of this site as we approach the election period, let's briefly take a look at the story the polls have been telling since January 2010.

It has been a bit of a ride for the Progressive Conservatives, who began 2010 more than 10 points behind Wildrose.

That changed in April 2010 when the Tories moved back in the lead, and that is where they have stayed since. Their numbers were relatively flat until the summer of 2011, when the leadership race galvanized the party. Their levels of support have improved under Alison Redford.

Wildrose, meanwhile, has been on the decline since November 2010. They are still comfortably ensconced in second place, however, holding a double-digit lead over their opposition rivals. But they are far removed from the salad days of late 2009-early 2010.

The Alberta Liberals are also on the decline, and their recent leadership race has not helped them at all. They haven't gotten too close to Wildrose in some time, but until May 2011 they were safely in third place. Since the federal election, however, the Liberals have moved into a tie for third place with the New Democrats, who have seen their levels of support increase slightly. But the likelihood that the NDP could finish third in the 2012 election ahead of the Liberals has more to do with Liberal weakness than it does with NDP strength.

The monthly provincial polling averages chart has been updated to include an Ontario poll by Innovative Research, taken at the end of October and beginning of November. I missed it at the time. The voting intentions results are unchanged from the election, but the poll does have some interesting things to say about why Ontarians voted the way they did. It is also a somewhat confusing slideshow - it looks like a few stray slides from other presentations somehow made their way into it. But it might be worth a look. It seems unlikely that Dalton McGuinty's minority government will fall this year, but at least Ontario gives us the kind of uncertainty that our federal politics used to provide.

Ranking the Pollsters (Updated)

Now that the dust has settled, we can take a look at how each of the pollsters did and assess their performances in the past election.
While it is true that polls are only a snapshot in time, and so are limited in their ability to predict into the future, polls are judged by how closely they align with election results. There is no other way to judge them. Certainly, it is possible that these polls were tracking voting intentions on their field dates accurately, and so cannot be judged for missing out on changing intentions over the last days or hours of the election campaign. While that is a valid argument, a means of assessing pollsters is required and comparing their final poll results to the actual results of the election is the only measuring stick we have. And pollsters themselves use that measuring stick, so we can certainly hold them to their own standards.

All polling firms have been assessed by their final poll results. Nanos Research has been assessed according to their final two-day report that was featured by CTV and The Globe and Mail. Crop, Innovative Research, and Environics are being assessed by their final polls, though they were all taken one week or more before the day of the vote. That may seem unfair, but these are the final numbers we have from these firms and they need to be assessed by some measure. Consider it a penalty for not releasing data closer to the end of the campaign.

Pollsters are assessed by their average error per party. In other words, being off by a total of 20 points for the five national parties combined would be the equivalent of having an average error of 4.0 per party. The party who performed best is highlighted in white. The polling firms are placed in the order of the date of their final poll. We'll start at the national level.

Angus-Reid had the best result for the second consecutive election, with an average error of only 1.0 per party. Nanos Research, Ipsos-Reid, Harris-Decima, and Léger Marketing also performed well, all with an average error of 1.4 points or lower. Of those who reported close to election day, EKOS and Compas performed worst.

See the bottom of this post for a discussion of how the margin of error could be taken into account in this assessment.

Note that only Compas, who was nevertheless very far off, was the only polling firm to over-estimate Conservative support. All others under-estimated their support by almost two points or more. The New Democrats were relatively well-tracked, but only Abacus and Ipsos-Reid had the Liberals lower than their actual result.

Disregarding the results from Environics and Innovative, the best polling method turned out to be the online panel, with an average error of 1.4 points per party. Traditional telephone polling scored 1.6 points, while IVR stood at 2.0 points' worth of error per party.

Also note that those pollsters who do not prompt for the Green Party (Ipsos-Reid) or any party (Nanos Research) generally predicted the Green Party's eventual tally better than those who prompted for the Greens.

Now let's move to the regional assessments, going west-to-east and so starting with British Columbia.

Here, Léger Marketing and Compas scored best with an average error of two points per party. Harris-Decima, at an error of 2.5 points, also did well in this province.

Nanos Research and Ipsos-Reid, who both put the Liberals in the mid-20s, did worst here.

As at the national level, only Compas over-estimated Conservative support. All of the others under-estimated their support, mostly to the benefit of the Liberals and New Democrats. But results varied widely, with Liberal support being pegged at between 10% and 26%, while the NDP was scored at between 25% and 40%. Small sample sizes are partly to blame. Green support, on the other hand, was well tracked.

In Alberta, Angus-Reid did best with an average error of only 1.3 points. The next best was Abacus Data, at 2.1 points. Harris-Decima did worst.

The pollsters had an easier time discerning Conservative support in this province, with two of the pollsters being exactly right. Others (Ipsos-Reid) inflated Tory support while yet others under-estimated them (Harris-Decima, EKOS, Forum). The NDP and Liberals were well tracked, though only Abacus had them in single-digits. All-in-all, the pollsters did well in Alberta.

Compas and Nanos, however, grouped Alberta with Saskatchewan and Manitoba. In these three provinces, Nanos bested Compas by an average of 0.7 points. Compas appeared to give some of the Liberal support to the Tories, while Nanos gave some of the Tory support to the NDP.

In the more usual grouping of Saskatchewan and Manitoba, Angus-Reid did best with an average error of 1.5 points, closely followed by Ipsos-Reid. EKOS struggled here.

Generally, the pollsters had an easier time pinpointing Conservative support here, with three of the pollsters within about a point of their final result. The New Democrats were also well represented, with the biggest error being only of 3.1 points among those polling firms active in the final days of the campaign. The Liberals were also well tracked, while only Ipsos-Reid correctly had the Greens at 3% in these two provinces.

Ontario was the most important province to poll correctly. Angus-Reid did the best here, with an average error of only 1.7 points. They were closely followed by Harris-Decima, while Ipsos-Reid did the worst among those who polled in the final days of the campaign.

But the problem here, as elsewhere, was in recording Conservative support. Again, only Compas over-estimated the Tories while all others had them at 41% or lower. That error had great consequences in determining whether they would win a majority or minority government.

The pollsters, except Compas, over-estimated NDP support, but only by a little. Some (EKOS, Harris-Decima, and Angus-Reid) were very close to accurately predicting the NDP's support, but others had them well over their actual result. The Liberals were generally well polled, however, two pollsters being on the money and another three being within a point.

Quebec provided the biggest surprise on election night, but amazingly the pollsters did very well in the province. They did better than they did in Ontario, which usually has larger sample sizes.

Ipsos-Reid takes the crown in Quebec, but was closely followed by Nanos, Forum, and Léger. Compas and Crop did the worst, though they were still relatively close.

Three pollsters were almost exactly right in predicting the NDP's result, while four others had the NDP over 40%. The pollsters had a bit more trouble with the Bloc, with only Nanos and Forum indicating that they would end up in the low 20s. The pollsters did an excellent job in recording Liberal support, but did a little worse with the Conservatives. All in all, though, the pollsters did an excellent job in Quebec.

That was not the case in Atlantic Canada, which was the worst polled region in Canada. Granted, it usually is polled in very small numbers but the same amount of people are usually surveyed in the Prairies, and there the pollsters did much better.

Harris-Decima was closest with a very good average error of 1.3 points, while Léger was close as well. But EKOS, Angus-Reid, Compas, and Ipsos-Reid were all off by six points or more, with seven pollsters putting the NDP in first in the region.

Results varied wildly, with the Conservatives pegged at between 26% and 44%, the NDP between 28% and 46%, and the Liberals between 11% and 30%, among pollsters active at the end of the campaign. Only Harris-Decima had the NDP below 30%, while most under-estimated Liberal support. Atlantic Canada was a wash.

And that brings us to our final ranking. The pollsters have been ranked on two scores: their national average error (recorded in the first chart at the top of this post) and their average regional error. Combining the two rankings anoints Angus-Reid as the best pollster in 2011, but also awards Harris-Decima, Léger Marketing, and Nanos Research with honourable mentions.

Nationally, Angus-Reid comes out on top but was only marginally better than Nanos, Ipsos-Reid, Harris-Decima, and Léger Marketing.

Forum Research and Abacus Data, in their first federal campaigns, did very respectably.

EKOS struggled while Compas was the worst pollster active in the final days of the campaign. Environics and Innovative Research might have done better had they polled closer to election day.

However, national results can be close merely because all of the regional errors cancel each other out. That was the case with Ipsos-Reid, which drops to 9th place in the average regional error ranking.

Instead, Léger Marketing takes the top spot on its regional results with an average error of 2.4 points. Harris-Decima and Angus-Reid were close behind with an average error of 2.7 and 3.0 points, respectively, while Abacus Data placed 4th on the regional ranking.

Regionally, of those active in the final days of the campaign the online pollsters were off by an average of 2.9 points. Traditional telephone surveys were off by 3.7 points, while IVR surveys were off by 3.9 points. Though it could have been blind luck, the much-maligned online surveys performed best in the 2011 federal election.

Margin of Error Update

Some pollsters and commenters have brought up the issue of the margin of error, and whether that should be taken into account to judge the performances of the polling firms.

From a statistical standpoint, polling firms should be able to accurately predict the result of each party within the poll's margin of error. Though it is hardly the focus of any press release or media report, the margin of error is (or should be) always included in any poll report, and thus if a poll gets the result correct within the margin of error it is a technically accurate poll. But this benefits polling firms with smaller samples, who have larger margins of error within which to work.

If we use this standard, at the national level we would have to eliminate all but Ipsos-Reid, Abacus Data (assuming a random sample for this online pollster), and Nanos Research. These are the only three firms whose national poll findings were within the margin of error of the sample.

If we take it to the next level, assessing each polling firm by the margin of error for each individual party (i.e., the margin of error for the Green Party is not the same as the margin of error for the Conservative Party because of their different levels of support), we have to eliminate all but Nanos Research from this assessment. Ipsos-Reid would fail to pin down NDP support taking this margin of error into consideration, while Abacus would be wrong for the Green Party.

But why stop at the national level? We do not elect presidents - regional data is as important, if not more important, than the national horserace numbers. If we bring it down to the regional level, then even Nanos has to be eliminated as their result of 23.6% for the Liberals in British Columbia (rather than their actual 13.4%) is outside of this particular sample's 7.7% margin of error. No pollster would survive this assessment, as no pollster would fit this criteria even taking into account the 95% confidence.

Just as problematic methodology will not be corrected by over-sampling, it can also be masked by the large margin of error of smaller samples. In the end, Nanos should be commended for having its national poll results within the margin of error, with honourable mentions also going to Ipsos-Reid and Abacus Data. But from my perspective, polling firms that conduct surveys with large samples, thereby giving us more reliable regional results, should not be thrown under the bus. Polls are reported for the consumption of the general public, and the general public is interested in how accurately the polls predict actual outcomes, both nationally and regionally. While I agree that some consideration should be taken for the margin of error in assessing the performance of the pollsters, which I have now done, I believe that my original assessment stands.

Quebec results pulling NDP along in ROC

Four national polls are surprisingly consistent on the situation in Quebec while disagreeing with one another greatly on what is happening in Ontario. But what seems to be clear in these polls from Innovative, EKOS, Nanos, and Angus-Reid is that the rest of Canada is starting to jump on the NDP's Quebec bandwagon.

Whether that bandwagon leads us to a Conservative majority or minority government, however, appears to be within the margin of error.

With the Conservatives around 38% and the Liberals and New Democrats spliting the vote relatively evenly between them, as Innovative and Nanos show, we can expect the Tories to pull a majority government out of the mess. But if the Conservatives are instead at 34-35% with the NDP nipping at their heels, as EKOS and Angus-Reid indicate, then we can expect a re-run of the 2006 election with the New Democrats taking the place of Paul Martin's Liberals.

What is fascinating about these two scenarios is that they have extremely diverging consequences but are both drawn from polls that are not inconsistent, considering their margins of error. We're in such unknown territory that the election result is still very much in the air.

But what's happening to the Conservatives? Even in Nanos's polling the Tories have dropped. Ipsos-Reid might come along this week pegging the Conservatives at 40% or more, but every recent poll points to a drop for the Conservatives. Whether this is being caused by centrist Tories moving over to the Liberals to stop the orange tide or populist Conservatives jumping in with the New Democrats is impossible to say. But the Conservatives are starting to see a strong mandate slip away. They may still win a majority, but it could conceivably come with a smaller share of the vote than in 2008.

Across the polls, not all taken on the same dates but all overlapping, we can see some consistencies. In British Columbia, the New Democrats are clearly back in second and above their 2008 performance. They're currently around 28-30%, whereas they have also grabbed the runner-up position in Alberta as well.

In Quebec, all of the polls are remarkably similar on the NDP's score. These four polls put the party between 36% and 38%, an incredibly consistent result considering the margin of error in Quebec can be as high as six points. The Bloc's support level varies more (24% to 29%), as do that of the Liberals (14% to 20%) and the Conservatives (14% to 18%). They do seem to be indicating that the Tories are stumbling in the province.

But Ontario and Atlantic Canada appear to be toss-ups. That shouldn't be too much of a surprise in Atlantic Canada as the sample sizes are small, but even that region is starting to look erratic by its standards.

Ontario is all over the map. The Conservatives might have a huge lead, the Liberals might be hanging tough, or the NDP might be on track to historic levels of support. We may have to wait a few more days for Ontario to straighten up, or this might be a hint of some new volatility in the province.

A few riding polls from yesterday also caught my attention. There was a poll done by Oracle for the Green Party in the riding of Saanich - Gulf Islands. Polls conducted for parties, and especially those made public by parties, should be treated with suspicion. I do still add them to the model, though at a reduced 10% weighting, compared to 25% for media-funded riding polls.

This particular poll puts Elizabeth May ahead of Gary Lunn at 45% to 38%, with the Liberals and New Democrats out of the race at 9% apiece. The poll was taken on April 18-19 and surveyed 389 people. It has a margin of error of 4.9%, 19 times out of 20. It means the two main parties are statistically tied.

Of course, political parties don't release their privately funded polls without reason. For all we know, the Greens have been polling constantly and this is the first poll to put them in front.

The other riding poll that was brought to my attention was apparently conducted by Léger Marketing in the riding of Lac-Saint-Louis for the Conservative Party. The only information we have is via Ian MacDonald, a columnist. It was taken "after the debates" and included 500 people. Presumeably provided to Mr. MacDonald by Larry Smith's campaign, it puts the Liberals ahead 36% to 30% in the riding, a much closer race than an earlier poll showed. It pegs the NDP at 17%, but the level of support that the Bloc and Green candidates have in the riding is unknown. For that reason it wasn't added to the projection model. Polls need more than a vague mention in a column, particularly when they are paid for and provided by a political party.

Nevertheless, if true these two polls indicate that Saanich - Gulf Islands and Lac-Saint-Louis could be close ridings on Monday night. They were already ones to watch, but now we have more reason to watch closely.

Tomorrow, it looks like we'll have polls from Nanos Research, EKOS Research, Forum Research, and Segma Recherche to add to the model. Hopefully more will pop up. From what I'm being told we can expect a lot of polls over the last four days of the campaign.

Smattering of riding polls and a poll of some interest in Quebec

A bit of an odd day for polls. This morning I added an older Innovative Research poll to the model, today's Nanos, the eye-popping CROP, a partial EKOS, and five riding polls. In the end it caused little change.

Before we get into it, a word about the EKOS poll. It was a bit of a partial report, comprising two days of their full poll which was released today. Those results will be added tomorrow, replacing the national and Quebec numbers added to the projection this morning. At first I was quite confused, and understandably so. iPolitics listed the poll as being conducted from April 18-21, which was quite impossible as it was put up on their site at 12:03 AM on April 21. The EKOS graphic accompanying the article listed the dates as April 18-20, but even that was apparently inaccurate. Because of all this (which, after some inquiries, appears to be nothing but a series of typos and misunderstandings), I haven't included the EKOS poll in the graphic below. As it will be replaced tomorrow by the more recent report, I'll discuss it then.

First off, note that Innovative and Nanos conducted their polls on completely different days. So we shouldn't be comparing them. And secondly, Innovative and CROP are both online pollsters.

We'll start with the older Innovative poll, which has remarkably no change since the last IRG poll taken between April 8-11. All of the national numbers are the same, except the Greens lost one point. The same in Ontario, while in Quebec the Tories have picked up one and the Bloc has dropped one. So, Innovative sees no movement whatsoever.

Now Nanos's poll is a bit more interesting. If we compare it to their last three day poll (April 15-17), we see the Conservatives holding steady (-0.8). The Liberals, however, are down 3.1 points nationally while the NDP is up 4.7 points - in three days. That is a statistically significant shift in support for the NDP.

But it isn't happening in Ontario, where variations were all within 0.4 points. It is taking place in British Columbia, the Prairies, and Atlantic Canada. In Quebec, the NDP is up only 0.4 points and the Liberals 2.5, with the Bloc dropping 3.8 points.

And that poses a bit of a conundrum for the CROP poll. Both Innovative and Nanos were in the field on the same days as CROP, yet they averaged 35% for the Bloc and 20% for the NDP, instead of the 36% for the NDP and the 31% for the Bloc that CROP found. So there is some disagreement of where the NDP stands in Quebec.

On CROP's side of the ledger are two new polls from EKOS and Forum, both showing the NDP and Bloc at similar levels of support as in the CROP poll. So, in the end these three polls might be capturing something real that Nanos and Innovative have not.

But this isn't the first time CROP has come out with an eye-catching poll at odds with others. In mid-March, CROP pegged the Liberals at only 11% in the province. No one else put the party that low.

So, we will have to wait and see what other pollsters find, but three polls released on the same day putting the NDP in front is nothing to scoff at. This could be the new reality.

Now to the riding polls, conducted by Segma Recherche for Le Droit and La Tribune and by Telelink for NTV News. Some pieces of information were missing in the media reports for these polls, and in the Telelink polls I portioned out the undecideds.

We'll start in Sherbrooke, where the only remarkable thing is that Serge Cardin is as far ahead as he is. This was not a riding likely to switch over, and it appears it will stay that way.

In Gatineau, however, Segma has reversed a previous finding that put the NDP well behind the pack. Now Françoise Boivin leads the Bloc's Richard Nadeau 33% to 29%, a lead that ballooned over the last few days in response to reports from Le Devoir over why she left the Liberal Party a few years ago.

This shows that the NDP can win some seats, that province-wide support can be translated into individual victories. But the race is still very close, and technically with the margin of error it is still a three-way race.

The projection was very close to what Segma reported in Gatineau (32% NDP, 28% BQ, 21% LPC, 16% CPC), while in Sherbrooke I didn't have the Bloc that high, with quite a bit of reason. The polls have been added to the model, with this newer Segma replacing the one on the eve of the campaign.

Over in Newfoundland & Labrador, Jack Harris is quite safe in St. John's East with 69% support. The Conservatives are well behind. Some people felt that the NDP would have more of a race on their hands here, but this does not seem to be the case. Comparing it to the projection prior to the inclusion of this poll, I was quite close: within five points for both the NDP and Conservatives. I overestimated Liberal support by eight points, however.

St. John's South - Mount Pearl has been touted as a three-way race, and this poll confirms that it certainly is. The NDP are doing quite well here, but the Liberals still hold the advantage - though with the MOE it could go either way. Here again the projection was performing well, with the Liberal and Conservative numbers pegged within four points of the poll. The NDP was underestimated by seven points, however, though I did have it as a Liberal seat.

Finally, Avalon. This riding has caught some attention in my projection, as I was projecting a Conservative landslide. This is because of the way the model works. In Newfoundland, Tory support has shot up and as that growth is applied proportionately, an already strong performance by the Conservatives in 2008 was buoyed greatly, perhaps more than it should've been. But I was within four points for the Conservatives in the other two ridings, so it appears Avalon is an exception.

In any case, Telelink has the race very close: 48% for the Liberals and 44% for the Conservatives. This indicates that the Liberals are still a strong party in Newfoundland, but that the Conservative rebound in the province is real. The last 10 days of the campaign could send these numbers in any direction. And with things as close as they are to a majority, these two Newfoundland seats could be extremely important on election night.

Pick a poll, any poll

You might as well! The polls that were added to the projection this morning were, in some ways, contradictory. But in other ways they were not, and when we look at the results of the four of them with a closer eye as to why they are different and why they are the same, the polls don't seem as unusual as they might appear at first glance.

No matter, though. Many people will believe the poll they want to believe. And more power to them! There's no way to prove that one poll or another is wrong, especially with more than two weeks to go into the campaign. But I will always default to assuming every pollster takes a professional approach to their work, just as I do with mine. In other words, completely unsubstantiated claims that pollsters cook their numbers will never get any airing here.

So now that that's out of the way, let's get to it. The four polls today are from COMPAS conducted for the Toronto Sun, Innovative Research Group for Maclean's Magazine (and a myriad of radio stations), EKOS Research for iPolitics, and Nanos Research for, of course, CTV and The Globe and Mail. In accordance with the guidelines of Elections Canada (those well-meaning keeners), I should point out that all of these polls sampled eligible Canadian voters, and that Innovative Research uses an online panel.

First off, let's get to the 'controversial' COMPAS poll for the Toronto Sun. As I said yesterday, the biggest problem with this poll is that we have no previous poll from COMPAS for comparison. If we did, and there was a huge swing or no swing it all, we would be better able to judge the validity of their results. This is especially important as the poll is at odds with what others have found. But as far as I can tell, there is nothing in the methodology that easily explains this variation.

Or is there? Although it isn't exactly explained clearly (and my emails to the firm yesterday bounced), it appears that COMPAS's reported results are those of "decided" voters only, excluding "leaners". This explains the huge 35% undecided rate. Virtually every pollster reports on the combined decided and leaning results, and I'm not quite sure why COMPAS would do any different. There doesn't seem to be any compelling evidence that shows that casting aside the leaners brings us to a closer result - a decided voter is not, necessarily, a Canadian who is sure to vote.

Polling firms in the United States have done more work on this "likely voters" issue. We don't really have much to go on up here, and our voting systems are very different (i.e., registered Republican, Democratic, and Independent voters).

COMPAS reports that among decided voters, the Conservatives lead the Liberals by a margin of almost 2:1. This would seem to correspond with the 45% to 24% they report as Canadians' voting intentions. They then go on to say that among leaners the lead is only about 4:3. Well, that's pretty important, isn't it?

So let's try something. Assuming that 65% are decided and 20% are leaners (leaving about 15% undecided, which is what Nanos is reporting), and that the 4:3 lead is the sort of 40% to 30% lead Nanos has found, we narrow down the gap from 21 points to 19, or about 44% to 25%. With rounding (and the fact that I am making a few assumptions), we could even say that the gap is 43% to 26%, or 17 points.

That doesn't look so strange anymore. It is generally where Ipsos Reid has placed the two parties, and is on the outside edges of the margins of error of other recent polls. And when you consider that COMPAS's decided numbers are almost identifical to Ipsos Reid's "absolutely certain to vote" numbers, this poll doesn't look so strange at all.

But the method of reporting the results, and the omission of what questions were asked, is certainly unusual. I doubt that this method of reporting results would meet the MRIA's standards, though more than a few pollsters are truant on that score.

Note that, in the graph above, the regional margins of error have been calculated based on my best estimates of this poll's sample size, as regional breakdowns were not included.

COMPAS's results in Ontario are especially stark, with the Conservatives leading by 49% to only 32% for the Liberals. That would give the Tories 69 seats, with only 27 going to the Liberals and 10 to the New Democrats. But in the Nanos poll, which shows the Liberals leading by 4.4 points, the seat breakdown would be 46 for the Conservatives, 44 for the Liberals, and 16 for the NDP. These two results would create radically different parliaments.

Innovative Research's poll doesn't have anything particularly remarkable in it, and in fact it closely matches my own projection. It does put Michael Ignatieff ahead of Jack Layton on who would make the best Prime Minister, which is new. We last heard from Innovative in a report on Ontario's voting intentions only, taken between April 1 and 3. Since then, the Tories are up three points in the province, which is statistically insignificant.

EKOS's poll, however, is more interesting. It was taken over two days, and those two days will be included in EKOS's weekly report tomorrow for iPolitics. When that poll is released, it will replace this two-day poll in the projection model.

This poll shows a much smaller gap than even Nanos shows: five points. Compared to EKOS's last two-day sneak peek taken between April 4 and 5, the Conservatives have dropped 2.2 points and the Liberals have gained one point. That isn't significant, but the three point gain of the NDP certainly is interesting. Also, note that EKOS has the Liberals in front in the Toronto CMA.

Finally, Nanos. They show a narrowing gap between the Conservatives and Liberals that now stands at 7.8 points, less than it did in 2008. Since their last complete three-day poll, conducted between April 8 and 10, the Conservatives have dropped 2.3 points while the Liberals have gained 0.7. Like EKOS, this isn't significant movement but again the NDP is up, with a 3.1 point gain in three days.

Much of that has come in British Columbia (+9.4) and Ontario (+4.8). In the latter province, the Liberals have gained 5.3 points while the Tories have lost 6.1 points since that last three day poll. Movement definitely seems to be taking place in the battleground province.

In terms of trends, in Atlantic Canada the Conservatives have gained for three consecutive days, as have the Liberals in Ontario and the New Democrats in British Columbia. The Conservatives, however, have lost support for three consecutive days in those two important provinces.

The New Democrats are between 16% and 21% in Quebec in all four of these polls, providing further confirmation of their newfound strength in the province. However, they are also below 20% in Atlantic Canada across the board. That isn't great news for the NDP, but their four MPs in the region are relatively ensconced.

The questions asked by each of the pollsters are as follows: "For those parties you would consider voting federally, could you please rank your top two current local preferences?" (Nanos). "If a federal election were held today, which party would you vote for?" (Innovative). "If a federal election were held tomorrow, which party would you vote for?" (EKOS). The question asked by COMPAS is not available.

Close race in Ontario in two new polls

Today's daily Nanos poll for CTV and The Globe and Mail shows very little change from yesterday, but the close race that Nanos has found to be the case in Ontario has been further confirmed by a new release by the Innovative Research Group, an opinion polling firm with offices in Vancouver and Toronto. Apparently, we should expect to hear from them periodically during the campaign with a look at the situation in Ontario.

The latest Nanos poll shows a bit of a reset in Ontario, with the Conservatives picking up 3.2 points and the Liberals losing 3.2 points, resulting in a more familiar gap between the two parties than yesterday's Liberal edge. But if we compare the newest IRG poll to Nanos's poll taken on the same dates (April 1-3), we see a similar gap. Innovative Research has the two parties at 39% and 35%, for a gap of four points, while Nanos had the parties at 43% and 37.5% on the same dates, a gap of 5.5 points. Considering the margin of error (though as Innovative Research uses an online panel the sampling margin of error does not apply) those are consistent results.

A closer race in Ontario is an important new reality in this campaign. At it's start, the Conservatives had a double-digit lead in the all-important battleground province.  

Note that Maclean's records support as being at 37% and 33%, respectively. This is before discarding the undecideds. The various reports for the radio networks mention support being at the 39% and 35% noted above and used in the model.

The Innovative Research poll also included a few other questions, and broke down support in Ontario by how people voted in the Toronto municipal election. The Conservatives have about 65% of the Rob Ford vote but only 3% of those who voted for Smitherman and the other candidates. Certainly, having most of Mayor Ford's vote is good news for the Conservatives, but not for their hopes of breaking into downtown Toronto. It was the outskirts of the city that voted for Ford, not the downtown core. But it could put a few of the Liberal MPs on the fringe of Toronto's boundaries at risk.

The Liberals have 18% of the Ford vote, but 51% of the "other" vote. Using the 2010's mayoral election results, we can then estimate that the Conservatives have the support of roughly 33% of the people who voted in the Toronto election, compared to 35% for the Liberals.

Stephen Harper and Michael Ignatieff have very close favourability ratings in Ontario: 39% for Ignatieff and 37% for Harper. But when it comes to who would make the best Prime Minister, Harper gets the edge: 34% to 26%.

Aside from the reset in the Nanos poll in Ontario and the drop of 6.5 points by the Tories in British Columbia (not statistically significant, or at least not yet), there is an interesting result in Quebec.

I've harped on it a lot already, but I was sure that Jack Layton's appearance on Tout le monde en parle, the incredibly popular TV show on Radio-Canada, would result in a bump. His party was up a point in Quebec yesterday and is up another 0.9 points today, so we really could be seeing the effect of that appearance. The NDP is now at 18.8% in Quebec, ahead of the Liberals. A lot of minds have to be made up in Quebec, though, as it is the region with the highest number of undecideds in this poll (27%).

But does this bump in Quebec mean any more seats for the New Democrats? Plugging these results into the projection model (and only these results) gives the Bloc Québécois 50 seats, the Liberals 12, the Conservatives 11, and the NDP two. Outremont and Gatineau are the two NDP seats, and that is even with taking into account the Segma polls released just prior to the start of the campaign showing the NDP well behind in the Outaouais.

This would also put them within range of a few other seats as well. The NDP would be within four points of the Liberals in Hull-Aylmer, seven of the Bloc in Jeanne-Le Ber, and three of the Liberals in Westmount - Ville-Marie (though the NDP are probably riding on Anne Lagacé Dowson's 2008 coattails in that riding in the projection). 

The questions asked by each of the pollsters are as follows: "For those parties you would consider voting federally, could you please rank your top two current local preferences?" (Nanos). "If a federal election were held today, which party would you vote for?" (Innovative).