Canada Border Services Agency
Symbol of the Government of Canada

ARCHIVED - Audit of Lookouts – Traveller Mode

Warning This page has been archived.

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

June 2013

This document is also available in PDF (543 KB)
[help with PDF files]

Table of Contents

{*} An asterisk appears where sensitive information has been removed in accordance with the Access to Information Act and the Privacy Act.

1.0 Introduction

The Canada Border Services Agency (CBSA or the Agency) provides integrated border services that support national security and public safety priorities while facilitating the movement of legitimate travellers and goods. The Agency mainly enforces the provisions of the Customs Act and the Immigration and Refugee Protection Act, and is responsible for administering more than 90 acts, regulations and international agreements.

All travellers and goods entering Canada are required to report to the Agency through a designated port of entry (POE). The Agency provides services at approximately 1,200 locations across Canada, including 13 international airports, 117 land border crossings, 4 major marine port facilities, and 27 rails sites.[ 1 ] Each year the Agency processes nearly 100 million travellers and goods at the Canadian POEs across Canada.[ 2 ]

The Agency has designed and implemented lookout processes and systems to manage and intercept high-risk travellers and goods connected to terrorism, organized crime, contraband, proceeds of crime, strategic exports, irregular migrants, and biological threats entering Canada.

Lookouts are intelligence products designed to identify a person, corporation, conveyance or shipment that, according to available information and intelligence, may pose a threat to the health, safety, security, economy, or environment of Canada and Canadians. Lookouts may be issued for a number of reasons, including intelligence information and national security risks.

Border services officers (BSOs) scan or manually input data from travel documents at the primary inspection line (PIL) and travellers' information is verified against various databases and systems. Through automated controls at the primary inspection line, BSOs are notified about which travellers need to be intercepted and examined. High-risk travellers are referred for secondary examination in customs or immigration or both. Travellers found to be violating Canadian legislation may be subject to an enforcement action such as a seizure, an arrest, a monetary penalty or denial of entry to Canada.

The audit includes examination of controls around the lookout "lifecycle" process for Category 1 and 2 lookouts.[ 3 ] Category 1 lookouts have information being put through the intelligence process, and are prepared and issued by Intelligence officers. Category 2 lookouts are the product of CBSA officers other than Intelligence officers, including BSOs, officers at the National Targeting Centre (NTC) and the Border Operations Centre (BOC), as well as criminal investigators and Inland Enforcement officers. Intelligence officers may also enter a Category 2 lookout when information has not yet been put through the full intelligence process to warrant a Category 1 lookout.

A lookout "lifecycle" encompasses three key stages of:

  • Issuance;
  • Maintenance; and
  • Reporting/closing.

2.0 Significance of the Audit

Each year the Agency processes nearly 100 million travellers at Canadian ports of entry across Canada.[ 4 ] The Agency priorities include facilitating travel across Canada's border while at the same time protecting Canada's population from border-related risks. The Agency's controls around the lookout process are integral in the interception of high-risk travellers and goods connected to terrorism, organized crime, irregular migrants, and biological threats. If the Agency's controls around the lookouts processes are not functioning as intended, high-risk travellers and goods may enter Canada and threaten its security, safety and economy.

The audit objective was to provide assurance that the Agency lookouts are appropriately managed and processed. More specifically, the objective was to determine whether controls around the lookout process are well-designed and effective in identifying and intercepting high-risk travellers and goods upon entering Canada.

3.0 Statement of Conformance

The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. The audit approach and methodology followed the International Standards for the Professional Practice of Internal Auditing as defined by the Institute of Internal Auditors and the Internal Auditing Standards for the Government of Canada as required by the Treasury Board Internal Audit Policy.

This audit provides a high level of assurance that the opinion provided therein is appropriate and reflects conditions that existed at the time of the audit.

4.0 Audit Opinion

The Agency has processes and systems to manage and intercept high-risk travellers. Some gaps in the design of controls in lookouts' issuance and maintenance were noted. Controls have been implemented at ports of entry to identify and intercept lookouts and report on the examination results. However, these controls are not always working as intended and effective monitoring at the corporate level is not in place to assess the functioning of the lookout program and gauge whether the Agency's priorities are achieved and risks appropriately managed. This results in a moderately high-risk exposure with respect to some of the Agency's enterprise risks.

5.0 Key Findings

The Agency has processes and systems to manage and intercept high-risk travellers and goods. Intelligence officers issue and maintain their lookouts after applying an intelligence cycle; however, there is limited management oversight and ongoing monitoring in the course of normal operations to ensure lookouts are accurate, and valid.

The Agency does not have reasonable assurance that all lookouts are maintained in a timely manner. {*}. Without documented rationale in the intelligence file, it is unknown {*}. Immigration lookouts are not consistently reviewed to determine if the information in the system warrants the continuation of the lookout.

Intelligence information for lookouts is documented in the Intelligence Management System (IMS) for most lookout files reviewed.  However, the IMS files do not always contain sufficient information to demonstrate the application of the intelligence cycle and to facilitate quality review. A routine management review process for lookouts' quality is not in place.

The electronic scanning of travel documents has been mandatory at all airports since at least 2001. This practice, recently expanded to land borders, will improve the robustness of key controls for lookouts. Practices for referring close matches for secondary examination vary across ports. Recent system enhancements and guidance on dealing with close matches issued to the frontline officers should improve the interception of high-risk travellers.

Examination results are not consistently entered in the systems by BSOs and the quality of narrative reports varies. The completion and the quality of examination results are not systematically reviewed by management. Without sufficient information and timely examination results, reporting on the results of the lookout program and lookouts maintenance is challenging.

Ongoing monitoring is a key element of the risk management approach. Formal monitoring at the corporate level is not performed. Statistics on the lookouts issued, intercepted, un-acquitted and missed are not gathered and the Agency does not regularly assess the design, functioning and effectiveness of controls through periodic reviews. National oversight and monitoring are key to the assessment of the functioning of the lookout program and ensuring that the Agency's priorities and risks are appropriately managed.

The CBSA management established a Lookouts Working Group in September 2012 to assess and develop new strategies for the management of lookouts. The objectives of the Group include the improvement of monitoring, documentation and follow-up of lookouts; the review of lookouts Standard Operating Procedures (SOPs) and guidance; the review of training on the issuance and maintenance of lookouts; the examination of the approval process in lookouts' issuance; and the establishment of a quality assurance and review process, among others. The Group is progressing in the achievement of its objectives.

6.0 Summary of Recommendations

This audit makes four recommendations:

  • Strengthen supervisory review in the lookouts' issuance and maintenance process and to ensure that all lookouts are documented in the Intelligence Management System to enable the review of lookouts' quality.
  • Review and improve system controls to enhance data integrity and enable timely maintenance of lookouts.
  • Establish a performance measurement system to systematically track, monitor, and report on missed and un-acquitted lookouts and implement a quality assurance process to strengthen the quality of examination results.
  • Review and improve the monitoring and reporting function for the overall lookout process.

7.0 Management Response

The Agency agrees with the recommendations of this audit, and its findings are consistent with the conclusions of the CBSA's Lookouts Working Group. Formed in September 2012 to assess, develop and implement new strategies for the management of lookouts, the working group has already accomplished and initiated a great deal of work that addresses many of the audit's recommendations.

Specifically, the CBSA undertook revisions to the Lookouts Policy and Standard Operating Procedures to clarify processes and outline enhanced accountabilities and responsibilities. A National Directive was issued in February 2013 which mandates the consistent and timely reporting and reconciliation of lookouts.

Development of an overall performance framework is underway for the ongoing monitoring of lookouts, with regular reporting requirements to senior and executive level management committees. The Agency has also implemented a quality assurance review of its existing 100,000 lookouts and will complete the review by March 2014.

The CBSA will implement, by March 2014, measures to strengthen the issuance and maintenance of lookouts, the recording of related examination results, quality assurance, and performance measurement.

Business requirements have been developed that would improve Information Technology (IT) system controls to support management oversight and ongoing monitoring and to ensure the accuracy and validity of lookouts. These systems changes will be implemented on a priority basis, commencing in July 2013. In the long term, the CBSA recognizes that the full realization of lookout system controls would require significant investment in a modern and integrated lookouts system that leverages and optimizes enterprise IT tools. To this end, the CBSA will develop a strategy to integrate lookouts into the Agency's systems modernization initiatives.

8.0 Audit Findings

8.1 Lookouts Issuance and Maintenance

Audit Criteria:

  • Controls are in place to ensure that lookouts are issued appropriately.
  • Controls are in place to ensure that lookouts are reviewed, maintained, and closed appropriately.

8.1.1 Issuance of Lookouts

Intelligence officers are responsible for preparing and issuing Category 1 and 2 lookouts. Before a lookout can be issued, the reliability of the source and the accuracy of the information are evaluated to determine if there is sufficient basis for the issuance of a lookout.

Intelligence officers place lookouts under the appropriate category and within the appropriate systems:

  • Integrated Customs Enforcement System (ICES) for customs-related lookouts (for travellers), and
  • Field Operational Support System (FOSS) for immigration-related lookouts.

Subjects of customs lookouts are individuals who are considered high cross-border risks in areas such as transnational organized crime, the illegal movement of currency and the illicit movement of drugs, firearms and other contraband. Subjects of immigration lookouts are foreign nationals who are or may be inadmissible to Canada, are a potential security risk due to prior visa refusals, serious criminality or arrests, organized crime, or suspected national security threat, among others.

{*}. Upon further examination, a denial of visa, seizure of goods or possible arrest may occur.

For the audit period between January 1 and September 30, 2012, the Agency issued approximately 6,600[ 5 ] customs-related lookouts in ICES[ 6 ] and 4,000 immigration-related lookouts in FOSS.[ 7 ] The majority[ 8 ] of lookouts issued were Category 1.

A creation of a lookout involves judgment in applying the intelligence cycle (i.e. steps that transform raw data and information into intelligence). Nevertheless, key controls should be in place to ensure that lookouts are issued appropriately. For quality, lookouts need to be reliable, accurate and relevant. {*}.

The Lookouts SOPs identify key controls during the issuance stage of the lookout process. More specifically, prior to the issuance of a lookout, Intelligence officers are required to: apply an intelligence cycle to the incoming information; establish that the CBSA is acting within its jurisdiction; evaluate the accuracy and reliability of the source of information; and perform various database checks for any record of past enforcement, intelligence, or other data that exists on an individual. They also reach out to the originator of the information and any other relevant law enforcement or customs/immigration partners.

To assess the controls during the issuance and maintenance process, a total of 64[ 9 ] files (32 from ICES and 32 from FOSS) were randomly selected. Most files reviewed did not contain sufficient information that would demonstrate how incoming information was evaluated by the Intelligence officers. As a result, we could not assess the extent to which the intelligence cycle was applied prior to the issuance of lookouts.

Some inconsistencies across the Regions visited and among Intelligence officers in the lookouts issuance may impact the effectiveness of lookouts and data integrity. {*}.

{*}.

{*}.

{*}.

{*}.

Duplicate System Entry of Immigration Lookouts: FOSS is a Citizenship and Immigration Canada (CIC) legacy mainframe system used mainly by the CBSA and CIC. CIC plans to decommission the system in December 2014. In the interim, some Intelligence officers issue immigration lookouts in both ICES and FOSS systems. {*}. However, the duplicate system entry of lookouts would require BSOs at the border to input their examination results into both systems (ICES and FOSS), to successfully close the loop on their intercepted lookouts. The practice of double issuance of lookouts creates problems in closing the loop and impacts data integrity of the Agency lookout systems due to double counting of these lookouts. The Lookouts SOPs do not provide clear direction on this practice and there was no directive from Headquarters on whether this practice was acceptable.

Lookout Categories: Identification of appropriate category of lookout is important to determine the degree of intelligence cycle to be applied as well as for planning and performance measurement reporting purposes. The Lookouts SOPs define the different lookouts categories (Category 1, 2 and 3). However, the interpretation of the definitions was not consistent amongst the Intelligence officers interviewed in that some officers categorized lookouts created on behalf of other departments as Category 1 instead of Category 3. This leads to inconsistent application of the categories and contributes to data integrity issues.

The Lookouts Working Group, established during the audit, is assessing and developing new strategies for the management of lookouts, including the development and provision of training on the issuance and maintenance of lookouts to ensure integrity and national consistency.

Field Operational Support System (FOSS)

For this audit, it is important to provide some background information on FOSS. FOSS is a Citizenship and Immigration Canada (CIC) legacy system used by the CBSA and CIC users for functions associated with the immigration continuum (i.e. the CIC facilitation function, the CBSA-CIC shared functions, and the CBSA enforcement function). The most current version of FOSS III, which is now over 30 years old, is a document management system containing all document-related evidence of the immigration client continuum for the administration of the Immigration and Refugee Protection Act (IRPA). Nearly all the CBSA systems related to traveller issues or immigration processing rely on FOSS immigration enforcement, facilitation, lookouts and lost, stolen, fraudulent document data, either directly or via system-to-system interfaces.

The Agency has identified FOSS as being an obsolete system {*} and the expiry of the FOSS support contract in December 2014. It is important to note that FOSS was not built specifically for managing lookouts. {*}. The Agency has determined that FOSS does not meet the evolving business needs and is viewed as an impediment to managing an effective and efficient lookout program.

As part of CIC's modernization project, FOSS is scheduled to be decommissioned in December 2014. CIC has been developing and implementing its transition to the Global Case Management System (GCMS) to support the CIC facilitation function and some of the CBSA-CIC shared functions of the immigration continuum. However, the CBSA enforcement function was removed from the scope of CIC's GCMS project in 2008. Since then, the Agency has been working towards finding a solution to replace the enforcement functionality currently provided by FOSS and its interfaces. The audit did not examine whether the Agency has adequately identified and managed the risks related to FOSS replacement.

8.1.2 Lookout Maintenance

Lookouts are instructional products developed from information or intelligence, and as such, require ongoing maintenance to provide maximum value to users.

Intelligence officers are responsible for maintaining their respective lookouts. Regular maintenance and review help ensure the integrity of the Agency lookouts. The review should be initiated in a timely manner to ensure that lookouts are continuously updated and remain active to intercept high-risk travellers and their goods at Canadian ports of entry. If lookouts are not reviewed in a timely manner {*}. Lookouts that are no longer relevant should be expired or cancelled in the system(s) and any errors should be noted and corrected to ensure lookouts continue to be of high quality. Otherwise, travellers may be unnecessarily subject to closer questioning and searches at the ports of entry, potentially impacting service delivery and border wait times.  In addition, CBSA resources may be used inappropriately by examining travellers based on lookouts that are no longer current or valid.

Automatic Expiry of Lookouts

The ICES system has an automated control that notifies Intelligence officers in advance of the system expiry of their customs-related lookouts. The expiry of lookouts can be set up to 180 days.[ 10 ] Lookouts that are 'pending expiry' should be reviewed by the Intelligence officers for their continued reliability, accuracy and pertinence to CBSA jurisdiction. {*}.

{*}. There is a requirement under the SOPs to document the details of the lookout's expiry or cancellation in the Intelligence Management System. In the sample of 16 lookouts reviewed for maintenance, {*}. In all cases, the reason or rationale for expiry was not documented. {*}.

{*}. The review date of immigration lookouts can be set for up to 10 years in FOSS. Immigration lookouts in FOSS were not consistently reviewed by Intelligence officers to determine if the information warrants the continuation of the lookout. Interviews confirmed that some FOSS lookouts have exceeded their validity date and should be removed from the system.

The CBSA management established a Lookouts Working Group in September 2012 with a goal of assessing and developing new strategies for the management of lookouts. As part of its several objectives, the Group has initiated a review of all active lookouts in applicable systems[ 11 ] to determine their validity, or if the lookouts should be archived or deleted. This review is particularly important to ensure all data in FOSS is accurate, current and valid prior to the migration of this information into an enhanced CBSA system once FOSS is decommissioned in 2014. The CBSA plans to complete this review by March 2014 with substantial completion by October 2013, as more than 100,000 lookouts need to be reviewed, including those issued by the CBSA on behalf of other government departments (Category 3 lookouts).

Intelligence officers judge the validity of the lookouts in light of events that may have happened over the passage of time and the receipt of additional information. Based on the examination results entered by the BSOs at the port of entry, a decision may be taken to extend, modify, expire or archive it. Therefore, a timely feedback to the Intelligence officers is key for facilitating lookouts maintenance.

{*}.

{*}. If the quality of CBSA lookouts as intelligence products is not rigorously maintained, the lookout effectiveness can be reduced.

8.1.3 Review of Lookouts for Quality

Management supervision and reviews are also an important control activity to ensure the integrity of lookouts. The quality of Agency lookouts relies not only on sound issuance and maintenance processes, but also on a regular quality review of lookouts and timely feedback to Intelligence officers. According to the SOPs, Intelligence managers are responsible for monitoring lookouts created and maintained by their staff.

A routine management review process for lookouts quality and adherence to the Lookouts Policy and Procedures is not in place. The current guidance does not require that lookouts issued by Intelligence officers be authorized by their managers. Accordingly, Intelligence officers issued and also authorized their own lookouts. Occasionally, they discussed more difficult or sensitive cases with their managers prior to issuing or extending lookouts but this is an informal practice. {*}. By integrating supervisory review of lookouts quality into the issuance and maintenance process, the Agency would be able to detect and correct deficiencies in a timely manner.

Appropriate and consistent documentation of lookout information in the right systems enables management to conduct monitoring and quality review of lookouts to assure high-quality intelligence products. {*}. This means that not all lookouts need to have a corresponding IMS file. Intelligence information for lookouts was documented in IMS in most lookout files reviewed (59 out of 64 ICES and FOSS files reviewed for issuance and maintenance). However, the IMS files did not always contain sufficient information to demonstrate the application of the intelligence cycle and to facilitate the review of lookouts for quality. The creation of an IMS file (and STS file where required) for all lookouts with supporting documentation is important as it would enable management to review how incoming information was evaluated by the Intelligence officers and whether the lookout is of satisfactory quality.

Recommendation 1:

The Vice-President of the Programs Branch, in collaboration with the Vice-President of Operations Branch, should ensure that appropriate level of supervisory review in the lookouts' issuance and maintenance process is carried out, and that all lookouts are documented in the Intelligence Management System (IMS) to enable the review of lookouts quality.

Management Response:

Agreed. With respect to the recommendation of ensuring that an appropriate level of supervisory review for the lookouts' issuance and maintenance process is carried out, the CBSA will leverage the work completed by the Lookouts Working Group, including the issuance of a National Directive on improved reporting of lookout outcomes, revisions to the Lookouts Policy and Standard Operating Procedures (SOPs), and development of a training module to address the quality of lookouts.

In 2010, a comprehensive Lookouts Policy was developed in response to the 2007 report of the Office of the Auditor General ("Keeping the Border Open and Secure"). This Lookouts Policy was updated again in 2012 to reflect program improvements and organizational changes to the Enforcement and Intelligence program as a whole. In June 2013, further revisions will be made to the Lookouts Policy and SOPs in order to clarify the accountabilities and responsibilities of regional officers and managers, as well as for managers at National Headquarters.

These revisions will also reinforce the requirement for the mandatory documentation of lookout supporting information and examination results in the Intelligence Management System (IMS) or the Secure Tracking System (STS) and related systems to enable the review of lookouts quality.  A review of compliance with these directives will be completed in December 2013.

Completion Date: December 2013

Recommendation 2:

The Vice-President of the Programs Branch, in collaboration with the Operations and Information, Science and Technology (ISTB) branches, should review and improve system controls to strengthen data integrity and enable timely maintenance of lookouts. This should include data entry, {*}.

Management Response:

Agreed. Programs Branch, in collaboration with Operations and ISTB, will review and improve system controls to strengthen data integrity and enable the timely maintenance of lookouts.

Enhanced business requirements for the lookout management systems (IMS and ICES) have been developed and will become priorities by June 2013. These comprehensive business requirements include improvements to data entry, {*} opportunities to reduce duplicate system entry of immigration lookouts, lookout categories, and expiry of lookouts. As well, requirements have been developed for a robust reporting functionality to assist in the monitoring of the lookout program.

In the short term, implementation of these systems changes will begin in July 2013. In fiscal year 2013-14 and 2014-15, in line with scheduled releases, a number of priority change requests will be implemented for both the ICES and IMS. In addition, the CBSA will develop a long-term IT strategy to integrate improved systems controls for lookouts into the Agency's enterprise-wide systems modernization initiatives. This strategy will be completed by March 2014.

Completion Date: March 2014

8.2 Lookouts Interception at the Border

Audit Criteria:

  • Controls at the Canadian border are in place to ensure that lookouts are intercepted as intended.
  • Controls are in place to ensure the results of lookouts examinations are recorded and appropriately communicated.

When travellers arrive at Canadian ports of entry, they follow a two-stage control process referred to as primary and secondary inspection. At primary inspection, BSOs scan travel documents ({*}), ask questions and make a decision whether to refer travellers and their goods to secondary examination for more detailed checks in customs, immigration, or both. This decision is based on several factors, including information provided by ICES and FOSS. {*} This increases the risk of missed lookouts.   

{*}. In these cases, management is required to record all relevant details.

Lookouts can be missed at primary inspection because of incorrect matching to a traveller. Practices for referring travellers to secondary examination varied among ports visited and among individual BSOs. Some BSOs referred only individuals whose travel documents perfectly matched key data in the lookout {*}. The Lookout Policy and SOPs do not provide guidance on dealing with close matches.

Recently, system improvements were implemented to include mandatory referral of close matches and BSOs were reminded of their accountabilities related to close matches.

Generally, missed lookouts (i.e. those intercepted at primary inspection but for whatever reason did not report to secondary examination) are not tracked and reported on. The Southern Ontario Region is one exception as it does track and report on missed lookouts. The feedback resulting from the analysis of events that led to a missed lookout would enable management to identify potential control weaknesses and lessons learned.

8.2.1 Examination Results

According to the SOPs, BSOs conducting secondary examinations of lookouts are required to record results into appropriate systems immediately following the event or as soon as possible thereafter, regardless of whether or not the interview or examination yields results. Supervisors and managers are responsible for ensuring that examination reports are completed in the appropriate systems. This review is important for confirming the 'closing of the loop' for all intercepted lookouts (i.e. lookouts are tracked, monitored, and followed up to ensure that they are appropriately acquitted in the systems).

The examination results for intercepted customs and immigration lookouts were not always documented in ICES and FOSS by the intercepting officers. In the sample of 56 intercepted ICES lookouts,[ 12 ] seven did not contain examination results in ICES. Information related to interception and examination results for immigration lookouts is not tracked by FOSS and due to this system limitation could not be tested.

Examination results should be detailed enough to provide Intelligence officers with sufficient information to determine whether their lookouts need to be modified, extended, or closed. This includes, but is not limited to, the search results, interview notes, identity of any travelling companions, action taken, etc.

While the SOPs provide broad direction on what information should be captured in the examination results, the understanding and practice varied among BSOs of what information should be included in the results to ensure thorough reporting. Intelligence officers informed us that examination results were not consistently documented in the systems by the intercepting officers, and the quality of reports varied.

Superintendents are responsible for overseeing the completion and the quality of the examination results. Management informed us that they do not systematically review examination results.  In terms of customs-related lookouts, ICES' functionality allows tracking of the interception and documentation of examination results. Therefore, superintendents are able to monitor the lookout process in ICES to ensure that all intercepted lookouts are closed by their BSOs. However, this monitoring was not done consistently across the sites visited.

Some regions visited track all un-acquitted[ 13 ] lookouts and communicate them to the ports of entry for action. We noted that the frequency of reporting was not always sufficient to allow timely follow-up by the superintendents and BSOs. {*}.

As mentioned earlier in this report, {*}.

The frontline supervisors at the ports told us that they generally do not review the quality of the examination results. The Intelligence officers occasionally provide feedback on some reports to frontline supervisors, but this is an ad hoc practice. By monitoring and improving the quality of examination results entered by frontline officers, Intelligence officers would possess more complete information to maintain their lookouts.

It should be noted that while the audit examined a sample of 56 intercepted lookouts, the individuals who were subject to these lookouts in some cases were intercepted multiple times during the audit period. Based on the information from ICES, there were 266 examination results recorded across Canada for these individuals. Of the 266 examinations, 19 had a resultant exam.[ 14 ] The audit did not examine the effectiveness of the examination process as this would be a program evaluation issue. This information is included in this report to provide some context as to the information that is available to management.

Recommendation 3:

The Vice-President of the Operations Branch should implement a quality assurance process to strengthen the quality of examination results and establish a performance measurement system to systematically track, monitor, and report on missed and un-acquitted lookouts.

Management Response:

Agreed. With respect to the implementation of a quality assurance process to strengthen the quality of examination results, the CBSA is currently undertaking a comprehensive quality review of more than 100,000 existing lookouts to ensure compliance with the Agency's policies and procedures. For new lookouts, the Agency will implement a quality assurance framework to strengthen the quality of examination results and to ensure national consistency.

In addition, the Operations Branch, in cooperation with Programs Branch, will establish a system to track, monitor and report on missed and un-acquitted lookouts. The Agency will revisit and build upon existing lookout monitoring processes, such as the best practices undertaken in regions such as Southern Ontario Region, and develop training to ensure national consistency.

The monitoring of examination results and missed or un-acquitted lookouts will include monthly reports to the Operations Branch Executive Committee (OBEC) and regular monitoring of overall program performance by the relevant Program Management Committee. These measures, including the review of existing lookouts, will be fully completed by March 2014.

Completion Date: December 2013

8.3 Monitoring, Reporting and Accountability for Lookouts

Audit Criteria:

  • Roles, responsibilities, and accountabilities of key stakeholders at the entity level for lookouts have been formally defined, documented and communicated.
  • The Agency monitors lookouts results to continuously improve the lookout process for intercepting high-risk travellers entering Canada.
  • Results are reported to senior management and appropriate oversight bodies for strategic direction, risk assessment and decision making.

Roles, responsibilities and accountabilities[ 15 ] of all key stakeholders involved with the Agency's lookout process were defined and documented in the Agency Lookout Policy, SOPs, and other documents.

With respect to corporate monitoring of the lookout program, a broad definition of the roles and responsibilities indicates that the Programs Branch is responsible for the development of the performance management framework and the Operations Branch is tasked with performance data collection and reporting.

There was a lack of formal monitoring at the corporate level. Statistics on the lookouts issued, intercepted, un-acquitted and missed were not gathered, and analysis of controls' design and effectiveness was not performed. The Lookouts Working Group, established during the audit, was tasked with assessing and developing new strategies for the management of lookouts, including the improvement of monitoring.

As a key element of the regional monitoring function, the SOPs assigned an "auditing" role to Intelligence Operations and the Regional Intelligence managers, where "auditing" is defined as "implementing a performance measurement regime that will ensure that the lookout system is properly evaluated and that the quality of CBSA lookouts is rigorously maintained." While "auditing" was expected to be in place, a corporate direction on the methodology was not provided and the auditing function at the Regional level was not carried out.

With respect to reporting, the Programs Branch reported in the Agency's Quarterly Performance Report on the percentage value of goods, shipments and conveyances seized as a result of a lookout, which is a measure related to the effectiveness of lookouts.

This performance indicator, while important for assessing the effectiveness of lookouts, does not provide a full picture of the lookouts' management. Reporting on other measures, including missed and un-acquitted lookouts, would provide useful information to determine the functioning of the lookouts program.

With regular monitoring, the performance of the lookout program will be easier to gauge, and management will have information required to improve the lookout processes and controls. National oversight and monitoring are key to the assessment of the lookouts program and ensuring that the Agency's priorities are achieved and risks are appropriately managed.

Recommendation 4:

The Vice-President of the Programs Branch, in collaboration with the Operations Branch, should review and improve the monitoring and reporting function for the overall lookout process.

Management Response:

Agreed. Programs Branch, in collaboration with the Operations Branch, will review and improve the monitoring and reporting function for the overall lookout process.

The Agency has already issued a National Directive to require regional reporting on lookouts and their outcomes, and will establish a framework for national reporting to the Operations Branch Executive Committee (OBEC) and the Enforcement and Intelligence Program Management Table (see action under Recommendation 1).

The Agency will develop improved performance indicators for the overall lookout process as part of the annual review of the CBSA's Performance Measurement Framework, and will integrate these indicators into its Performance and Service Standards Report (PSSR) and quarterly program performance reports. These measures will be completed by October 2013.

In addition, the CBSA will implement improved reporting functionality in lookout-related systems changes and modernization initiatives, in order to strengthen performance measurement and operational monitoring.

Completion Date: October 2013


Appendix A – About the Audit

Audit Objective and Scope

The audit objective was to provide assurance that Agency responsibilities for lookouts are appropriately managed and processed. More specifically, the objective was to determine whether controls for managing lookouts are well-designed and effective to intercept high-risk people entering Canada.

The audit scope included the review of lookout controls in the air traveller and land streams during the period of January 2012 to April 2013. The audit included visits to the three highest-risk airports and the three highest-risk land borders identified in the CBSA's 2010 National Port Risk Assessment.

The audit excluded lookouts issued on behalf of other government departments and law enforcement agencies (i.e. Category 3 lookouts), commercial lookouts, lookouts of low value shipments (i.e. postal and courier lookouts), and targets.

The audit did not examine whether the Agency has adequately identified and managed the risks related to FOSS replacement.

Risk Assessment

The risk assessment conducted during the planning phase identified a key risk: If the Agency's controls around lookout processes are not functioning as intended, there is a risk that high-risk travellers and goods connected to terrorism, organized crime, irregular migrants, and biological threats could enter Canada and threaten its security, safety and economy.

Approach

To assess the effectiveness of controls during the issuance, maintenance and interception stages of the lookouts process, a sample of 120 lookouts was randomly selected from ICES and FOSS from a population of approximately 10,000 active lookouts issued between January 1 and September 30, 2012. {*} lookouts (Category 1 and 2) across Canada. In addition, the Regional Intelligence Offices associated with these ports issued more than one-third of all customs and immigration lookouts.

The results from the file review cannot be extrapolated to the population because the sample was non-statistical. The results from the interviews and site visits cannot be generalized to ports nationwide; however, visits to these locations allowed us to directly observe the lookouts process and provided insight into the design and effectiveness of the lookouts-related border controls.

The following approach was used:

  • Reviewed and analyzed information from various sources, such as, but not limited to the Lookout Policy, the SOPs, relevant bulletins, directives and reports;
  • Interviewed various stakeholders within the Programs and Operations Branches on their roles and responsibilities, oversight function and monitoring in relation to lookout activities;
  • Collected, reviewed and analyzed documents relating to lookout monitoring processes and monitoring reports;
  • Collected and analyzed lookouts-related information reported to senior management nationally and regionally;
  • Analyzed a random sample of active lookouts from ICES and FOSS issued and intercepted in the Regions included in the audit scope to assess controls related to the issuance, maintenance and interception of lookouts;
  • Analyzed and assessed the design and adequacy of management controls in place at the CBSA for the management of lookouts.

Audit Criteria

The following audit lines of enquiry and criteria were selected.

Lines of Enquiry Audit Criteria
1. Governance and Accountability for Lookouts 1.1 Roles, responsibilities, and accountabilities of key stakeholders at the entity level for lookouts have been formally defined, documented and communicated.
2. Lookouts Creation and Maintenance

2.1 Controls are in place to ensure that lookouts are issued appropriately.

2.2 Controls are in place to ensure that lookouts are reviewed, maintained, and closed appropriately.

3. Lookouts Interception at the Border

3.1 Controls at the Canadian border are in place to ensure that lookouts are intercepted as intended.

3.2 Controls are in place to ensure the results of lookouts examinations are recorded and appropriately communicated.

4. Monitoring and Reporting

4.1 The Agency monitors lookouts results to continuously improve the lookout process for intercepting high-risk travellers entering Canada.

4.2 Results are reported to senior management and appropriate oversight bodies for strategic direction, risk assessment and decision making.


Appendix B – Sampling Approach

A sample of lookouts was selected to assess the effectiveness of controls during the issuance, maintenance and interception stages of the lookouts, and identify gaps and areas in need of improvement. The sample was drawn from ICES and FOSS data systems for the period of January 1, 2012 to September 30, 2012. The sample was linked to the ports scoped in the audit (i.e. stratified) to allow review of lookouts issued, maintained and intercepted at the six highest-risk ports included in the audit scope. The results from our file review cannot be extrapolated to the population because the sample was non-statistical.

Table 1: Sample Size

Region GTA SOR Quebec Pacific Sample Size Population
Port of Entry Pearson Int'l Airport Ambassador Bridge – land border Pierre Elliott Trudeau Airport Lacolle 15 – land border Vancouver Int'l Airport Pacific Hwy – land border    
Issuance & Maintenance (ICES) 8 8 8 8 32 6,587
Issuance & Maintenance (FOSS) 8 8 8 8 32 4,030
{*} {*} {*} {*} {*} {*} {*} {*} {*}
{*} {*} {*} {*} {*} {*} {*} {*} {*}
Total Sample Size 32 24 32 32 120  

{*}.

{*}.

Appendix C – Acronyms

BSO
Border Services Officer
CBSA or the Agency
Canada Border Services Agency
CIC
Citizenship and Immigration Canada
FOSS
Field Operational Support System
GCMS
Global Case Management System
GTA
Greater Toronto Area
ICES
Integrated Customs Enforcement System
IMS
Intelligence Management System
PIL
Primary Inspection Line
SOPs
Standard Operating Procedures
SOR
Southern Ontario Region
STS
Secure Tracking System


Notes

  1. Source: Report on Plans and Priorities, 2012-13. In addition to the above-mentioned CBSA service locations, there are 440 small vessel marina reporting sites, 12 ferry terminals, 3 postal processing plants, 3 detention facilities, 48 international locations staffed with CBSA liaison officers, and 4 major marine port facilities. [Return to text]
  2. CBSA, DPR 2011-2012, Section: Organizational Priority, Summary of Progress Against Priority. [Return to text]
  3. Category 3 lookouts are issued by the CBSA on behalf of other government departments (OGDs) and were excluded from the audit scope. [Return to text]
  4. CBSA, DPR 2011-2012, Section: Organizational Priority, Summary of Progress Against Priority. [Return to text]
  5. Some Category 3 lookouts may have been included in this total due to inconsistent application of the categories by the Intelligence officers. [Return to text]
  6. Based on an extract from the ICES database provided by the auditee. This does not include lookouts issued in previous years and extended in the audit period. [Return to text]
  7. Based on an extract from FOSS obtained from Citizenship and Immigration Canada. This does not include lookouts issued in previous years and extended in the audit period. [Return to text]
  8. 82% of lookouts were labelled Category 1 in ICES, and 97% in FOSS[Return to text]
  9. A non-statistical sampling method was used. Additional information can be found in the 'About the Audit' section of the report. [Return to text]
  10. The automatic expiry of Security and Missing Children lookouts can be set to 365 days. [Return to text]
  11. ICES, FOSS and Accelerated Commercial Release Operations Support System (ACROSS). [Return to text]
  12. One of the 56 intercepted customs lookouts at primary inspection was not examined at secondary (i.e. missed lookout). [Return to text]
  13. Un-acquitted lookouts refer to lookouts that hit at the primary inspection line (PIL), are referred for secondary examination, but have no examinations results recorded in the systems. [Return to text]
  14. The information presented in this paragraph is based on data extracted from ICES at the time of the audit; this information was not audited. [Return to text]
  15. Responsibility is defined as the duty to perform, and accountability as the obligation to report on the fulfillment of the responsibility; OCG, 2011, Audit Criteria related to the Management Accountability Framework, Section Accountability, p. 48. [Return to text]