Investigation of the Trillium Line Training Program

Introduction

On the 10th of February, 2025, the Auditor General's Office released a report on the training program for DROs (diesel rail operators) on Lines 2 and 4. This document is less damning for what it says about the training program itself, than for what it says about the City's contractual obligations, and the continued feeling that, to borrow Steve Munro's phrase, "no one is minding the store" at OC Transpo. 

If you want to read the report yourself, it can be found in PDF form here. I will provide page numbers to quotes and key findings; these are the page numbers in the bottom-right corner of the PDF and not the actual PDF pages themselves. I will also annotate the quotes with my interpretation, reading [like this]. 

Background and Overview

Before dissecting the report itself, it would be helpful to review the status of trainees on the line. Chart from the Transit Committee meeting in February.  

A word of caution. The report itself only reviewed allegations made about the training program itself. The scope of the investigation was narrow, compared to a full audit, and focused only on the concerns brought forwards by whistleblowers. To quote the report: 

The scope of our investigation was limited to assessing whether the allegations had merit, and if so, to determine the appropriate course of action that may be required for each of the issues ... We did not conduct a technical review of the training program’s content. Nor did we assess the qualifications of the operators and controllers. This investigation was limited to the applicable regulations and the minimum standards established within the City’s own Training Plans and job requirements. Additionally, this investigation did not include observation of the operators and controllers on the Trillium Line. [we only looked at whether the tips were true]  

While this investigation was conducted in a systematic and professional manner, the extent of activities undertaken by the OAG was narrow compared to an audit and focused solely on the concerns raised to our attention. (pg. 5) 

The investigation was initiated after "several reports [were] received through the City’s Fraud and Waste Hotline (FWHL)." (pg. 2) Apparently, the training program was so shady that multiple people took it upon themselves to report it to the hotline. 

The conclusion of the report is that 

Overall, evidence available could not demonstrate that specific Trillium Line operator instructors had the expected experience and training prior to commencing instruction. This could have had an impact on the quality of the instruction provided to the operators responsible for operating the diesel trains. [the instructors may have been underqualified] 

We further found incomplete training records for the diesel rail operators, controllers and instructors. We understand from management that all required training was completed, and training records were updated before revenue service commenced. (pg. 6) 

It is important to note that the priority level of all of these findings is moderate or lower, which means that: 

The finding represents a moderate internal control deficiency, noncompliance or is a risk to business operations that should be addressed timely. If left uncorrected, this could have a partial impact on business operations, resulting in loss or misappropriation of organizational assets, compromise of data, fines and penalties, or increased regulatory scrutiny. Typically, these issues should be resolved after any high priority findings. (pg. 22)  

Some of the recommendations have already been implemented, and most of these pose no immediate risk to safety on the line. 

The report emphasized that training is only part of a safety management program. However, it is important, and the findings of the investigation reveal a lot about the state of affairs at OC Transpo, TNext, and create a worrying picture of the corporate culture.  

Diesel Rail Operators

1.1: Evidence could not be provided to demonstrate that all diesel rail operator instructors were sufficiently qualified and fully trained before commencing on-job instruction. (pg. 7) 

Lines 2/4 are technically mainline rail, and as such, they are regulated federally, by Transport Canada. According to Transport Canada (TC) regulations, on-job training instructors must have two years of full-time consecutive experience as a locomotive engineer, including three months of service in the area of operations. 

Management at OC Transpo used a much looser definition. All of the instructors met this criteria. However, they did not meet the regulator's criteria. Given the way in which the Trillium Line of old was scheduled, and the years since it operated, meeting these requirements may not have been possible. The investigation then looked at the "intent of the Regulations," and found that: 

While we could find evidence that all 10 on job instructors met management’s definition, the historical scheduling and payroll records showed that some lacked significant rail operating hour experience. [some of the instructors did not have records showing adequate experience operating trains] When consulting directly with a sample of instructors, we found inconsistencies between the system information available as compared to the rail experience described by the instructors themselves. 

We understand from management that data integrity of scheduling and payroll information has historically been a challenge. Given the inconsistencies found, we cannot conclude whether these instructors had the necessary experience before commencing instruction. [management does not have the paperwork to prove that all instructors are sufficiently qualified] (pg. 8) 

 

More damning yet, instructors were not yet fully qualified when they began training operator trainees. The report summarizes it far better than I can. 

In addition to the qualification requirements discussed above, the instructors were subject to a minimum number of on-job training hours themselves, as defined in the Training Plans, to gain the necessary familiarization before instructing. Through interviews, we learned that OC Transpo prioritized the on-job training of the operators over the instructors. This led to instructors training the operators before they, themselves, had completed all their training. Our detailed testing confirmed that most instructors had not met the minimum on-job training hours, per the Training Plans, before they commenced instruction. [the instructors had not completed "instructor training" before starting to instruct trainees] 

Additionally, our review of instructors’ training files found that the instructors were supervising each other during the initial on-job training, even though none of them had completed all their required practical training hours. (pg. 8) 

This is simply a massive failing of management. How this could have happened, I cannot imagine. 

Management's response was: 

Given the highly regulated nature of Trillium Line operations, more oversight and quality assurance are required for both Transit Services and its contractor, TNEXT. A strategy is being developed and will be completed by the end of Q1 2025. 

A full inventory of missing items from training files for Trillium Line employees has already been completed and a plan to complete all requirements is being executed by the end of Q1 2025. All training records subject to regulatory requirements were in place in the employee files prior to the launch of revenue service. 

Management has conducted an analysis to ensure that the qualification of the instructors is documented on file, including the documentation of equivalencies and support for any decisions made by hiring managers. Documentation will be updated on all DROI files by the end of Q1 2025.

The best fit for the instructor positions were experienced legacy operators (LROs) from the pre-construction of the Trillium Line. With the creation of a new system, Management maximized the hiring of uniquely experienced individuals to make up the first group of instructors. (pg. 11-12) 

1.2: Limited evidence was available to support the hiring of several diesel rail instructors.

 According to the report: 

there is a lack of evidence that demonstrates the City assessed instructor candidates on a consistent basis before placing them in the role of instructor. (pg. 9) 

Relief instructors "fill on" on a temporary basis, in case a full-timer is not available. There were nine relief instructors for whom the hiring process was not found. Of these nine instructors, six were rejected in previous rounds for (presumably) the full-time instructor role. 

The investigation could not find how they were hired, as they were not hired through a regular competitive process. 

During our review of competition records, we identified six (6) successful competitions that led to the hiring of candidates for temporary or permanent instructor positions. When we compared the candidates’ experience to the City’s job posting requirements, we found that seven (7) out of 10 successful candidates did not fully meet the minimum experience requirements. The candidates hired for the operator instructor position often lacked experience delivering training. 

We also found that a successful controller instructor candidate lacked the required minimum number of years of certified controller experience for the role. [one hired instructor lacked the legal minimum number of years of experience. They are not, however, actually instructing anyone in the field] We understand that this instructor is not providing on-job instruction and that the City has, to date, continued to retain TNext instructors to provide on-job training to new controllers. (pg. 9) 

The instructors that the City hired did not, according to the report, meet the city's posted requirements for experience. 

It is unclear whether this is human resources passing over more qualified candidates, or whether the city really, really, does not have anyone who wants to work for it. Neither option reflects well on OC Transpo. 

Management response: 

hiring managers exercised some discretion (with regard to interview performance) in the past to select the best available candidates, knowing that the instructor cadre would be completing training together and working together as a holistic group. Given the current status of rail operations on the Trillium Line, there are no concerns that a healthy pool of fully qualified candidates exists for future competitions. 

Management has since conducted a review of the Diesel Rail Operator Instructor job competition in consultation with Human Resources. Any inconsistencies with established City HR practices will be addressed for future competitions. All subsequent instructor job competitions will be monitored to ensure compliance with established City of Ottawa HR procedures and best practices. The Chief Safety Officer has reviewed internal processes and clarified expectations in writing with hiring managers within the Training & Development team to further ensure that the issues do not reoccur. (pg. 12) 

 

1.3: There was limited formal evaluation of the diesel rail instructors’ skills and competencies. 

OC Transpo's internal policies require every instructor to undergo regular, documented evaluations to determine if they are still qualified for their job. According to the report, these should be conducted twice a year for full-time instructors, and once a year for relief instructors. 

The auditor's office requested a copy of these evaluations for all instructors in 2023 and 2024. Well ... 

For 2023, management could not demonstrate that any observations were completed. For 2024 (as of the end of our Conduct Phase in November 2024), a total of three (3) observations were completed. As a result, nine (9) on-job and classroom instructors did not have any performance observations on file for the year. [OC Transpo did not conduct any evaluations in 2023, and only conducted 1/4 of them in 2024]  (pg. 10) 

OC Transpo seems to have a problem following regular processes, including those set out by themselves. This is an urgent priority at OC Transpo that encompasses all facets of its operations: reliability, customer service, and internal management. The agency needs to change, and change rapidly, to implementing regular processes for overview, management, and service.

While there may be an improvement this year, there is no mechanism to ensure that the improvement process is followed the next year. In order for steady improvements to take hold, we need an agency capable of writing and following regular processes for all of its facets. Until then, there will be no systematic improvement. 

As for management: 

This recommendation is already an internal organizational practice, which exceeds compliance with the regulatory requirements ... [we will] continually improve the effectiveness of training through instructor performance observations. The position of Rail Training Section Manager is being filled in Q1 2025 with a new incumbent and this role is responsible to provide this oversight of the rail instructors. (pg. 13) 

Evidence of Training and Future Considerations

2.1: We found incomplete training records for several Trillium Line rail employees. 

As discussed earlier, the Regulations require the City to submit their Training Plans to the Regulator. As a result, we expected to find complete training records, in accordance with the specifications of these Training Plans. (pg. 13) 

Each operator should have proof that all the deliverables had been met. However, the investigation found files with unmarked training records, missing tests, or missing evidence of tests that should have been retaken due to a failing grade. 

Of 33 operators checked, all had the critical ORCR exam except for one. However, 23 were missing one test, and 21 were missing test retakes. 

With this evidence in mind, 

OC Transpo cannot demonstrate successful completion of the training program by Trillium Line employees. (pg. 14) 

Yikes.  

The training plan is a mandatory document (actually, two documents - the Training Certification Plan and the Integrated Training Plan) that is submitted to the regulator. All changes to the plan must be filed with TC within 90 days. The city's training plan was submitted in November, 2023. 

OC Transpo's training plans describe the requirements for both operators and instructors, with a final exam portion. 60 hours of on-the-job training are required to operate both the LINTs and the FLIRTs (it is my understanding that Line 2/4 operators are required to drive both), and 30 hours in a simulator. The investigation found that on-the-job training did not meet these requirements. 

In reality, OC Transpo approved operators for service after 36 hours of training in each vehicle, in contrast to the 60 outlined in their plan. When management became aware of this problem in September 2024, they took "immediate action" to fix this issue. 

There was also evidence that "very few" instructors had reached 36 hours (not 60, not 36) before beginning to instruct operator trainees. There was also missing documentation to confirm yard training - vehicle equipment checklists, yard assessments, etc. - had been completed by either operators or instructors. These gaps were fixed prior to the commencement of revenue service in on the 6th of January, 2025. (pg. 14-15) 

3.1: Recognizing that it was not receiving the training deliverables from TNext in line with expectations, the City supplemented with its own curriculum developers. While this mitigated risk, it increased the cost of the training program to the City. 

At least this one isn't safety critical, except to the penny-pinchers on council who will faint when they hear "increased cost." 

After the lawsuit filed in January comes the revelation (/s) that TNext isn't fulfilling contractual obligations. 

The Project Agreement is clear that TNext was responsible for developing the operations training curriculum and materials that had to meet City standards for training material, including providing material suitable for adult education. However, this section of the Project Agreement neither prescribes the City’s standards for training material nor the expected format for adult education. 

[The] City assigned internal resources as early as 2020 to commence adapting the training material received from TNext to align with City standards.

Had the Project Agreement been more specific about the expectations and requirements of the training program, the City may have been able to avoid taking on some of TNext’s responsibilities for the training program. (pg. 15-16) 

Shockingly, the City of Ottawa wrote a project agreement that did not fully assign responsibility for the development of training material. Fortunately, the city did the right thing, and instead of squabbling with TNext, they wrote their own training materials. 

3.2 Improper in-class testing practices were observed.

The investigator obtained and reviewed some video recordings of TNext's testing environment. Remember that these tests are administered for the safety of passengers. 

The auditors found: 

some instructors provided hints and direction (beyond clarification) and, on occasion, answers to specific exam questions. We also observed inappropriate behaviour among the trainees, such as talking among each other, copying answers, and making jokes about cheating during the exams. In one case, we observed a trainee taking unauthorized photos of the exam answers which could potentially be shared with other training cohorts. (pg. 16) 

This should not make anybody feel good. It seems to me that a broken environment has emerged at TNext, one that disregards safety items as non-essential and views training as an extra. 

According to management, OC Transpo has since taken over the administration of the tests, and has not found evidence that "would suggest that these behaviors have continued." (pg. 17) 

What does this say about us?

The report is intriguing less for its actual content and more for the implications behind its findings. 

One noteworthy piece is that TNext and the City are disagreeing over deliverables - the training program that was eventually designed and paid for by the city. Given the rocky road to the opening of the line, one wonders what else went on behind the scenes. 

Such as the delayed opening. I have a theory: remember that the auditor raised concerns over the lack of training hours with OC Transpo management, who were likely unaware until then. This was in September, about two months before the then-rumoured opening date in November. 

This was eventually delayed to January. 

My theory is that at least part of the delay can be attributed to the lack of operator hours, which had to be made up on the line before it entered revenue service. This, of course, was not communicated at the time, nor was any other reason. Unless the lawsuit reveals, publicly, some documents that we are not privy to, then I doubt we will ever know. 

At the Audit Committee (more on that later), Amilcar said that training was moved from TNext to in-house at OC Transpo. I believe that such things should always be in-house, but it is interesting to hear that TNext is not doing their job properly. 

Another interesting note: the culture at both TNext and the lower levels of OC Transpo management are in poor condition. 

At TNext, instructors are apparently sharing test answers with trainees, trainees are sharing test answers with each other, and nobody seems to know what tests are required and what tests are missing. 

At OC Transpo, nobody is home at the paperwork division (which I'm sure is larger than the scheduling division), nobody is applying proper hiring processes to instructors, and nobody is checking their own training design - the Training Plan is filed by OC Transpo itself - to make sure that they are in compliance. 

This is a critical situation at OC Transpo. While it's good that management eventually fixed the issue, it should not take an investigation to fix safety issues and ensure proper training for operators. Someone, somewhere, from the front line operators to Ms. Amilcar herself, failed. Thank goodness for the auditor general's office. 

What does the city have to say?

On the 10th of February, the Audit Committee, consisting of Cathy Curry, Allan Hubley, Marty Carr, David Brown, David Hill, Theresa Kavanagh, and Catherine Kitts, reviewed the report. 

The video link can be found here. The look at this report begins at 38:30. 

Renee Amilcar began the question period with a statement. Remarks begin at 55:30. 

Let me be clear. The system is safe, and all operators have been properly trained. Most of the issues have already been resolved, including completing all planned hours of training and reviewing the training files for diesel operators, controllers, and instructors ... all these issues were addressed before revenue service launched on January 6th. 

I do not want this investigation to weaken anyone's confidence in the safety of our transit system. 

[We have many tools] ... 

While I am glad that management has taken action on the report, a follow up is coming from the AGO, and I will report on that when it is released. 

While management has taken action on safety on this particular issue, the bevy of systematic errors reveals something broken, deep inside our transit institutions. As its head, I believe that Amilcar has the responsibility to start to fix these issues with culture. 

I would like to hear about culture improvements (or deterioration) at Transit Committee, though I do not expect that to happen at all. 

Marty Carr, of all people, asked a question about systemic problems, with respect to an investigation into bus maintenance last year. Amilcar said that it wasn't a systemic problem (thorough a translator), but "improvements on a continual basis," and did not ask for the money to properly maintain the system. (sigh) 

Amilcar did note that with Line 1, there are already internal processes to ensure safety on the new expansions coming online soon. Sabrina Pasian, the Chief Safety Officer, said that lessons from this investigation will be applied to the Confederation Line as well. 

David Hill pressed Amilcar on a few direct questions on the subject, and said that he wanted to move Line 1 to a different ("more simplistic") regulatory regime. Amilcar skirted the question about whether the service was safely operating before the investigation. 

Council needs to press harder on management to make systematic changes to culture at Transit, both with respect to safety but also in regards to other issues as well. Until then, we will not have an effective transit system. 

Thoughts 

While none of the findings are urgent, they reveal a concerning trend towards lax datakeeping, hiring, training, and other safety aspects at TNext, with implications at OC Transpo being less clear. 

What is clear is that we need housekeeping. Someone on City Council needs to step up and consistently and proactively push for culture change at OC Transpo and Transit Services. This should encompass all aspects of the organization, so that we get a system that is safer, more reliable, and more customer oriented. 

This report also reveals a surprising amount about the relationship with TNext, where failure at the consortium forced the city to take things in-house. This isn't shocking, but it is disappointing, and part of transparency would be increased disclosure on such items. 

That's it on this report. When the AGO does a follow-up, I will report on that as well. Otherwise, stay safe around the snowmelt, and stay warm out there. 

Until next time.