Archived - Audit of National Targeting

December 2015

Archived Content

Information identified as archived is provided for reference, research or recordkeeping purposes. It is not subject to the Government of Canada Web Standards and has not been altered or updated since it was archived. Please contact us to request a format other than those available.

[*] An asterisk appears where sensitive information has been removed in accordance with the Access to Information Act and the Privacy Act.

1.0 Introduction

The Canada Border Services Agency (CBSA) provides integrated border services that support national security priorities and facilitate the flow of people and goods across the border. The CBSA is an integral part of the Public Safety Portfolio, responsible for integrated national security, emergency management, law enforcement, crime prevention and border management operations. The responsibilities of the Agency include administering legislation that governs the admissibility of people and goods in and out of Canada; identifying, detaining, and removing people who are inadmissible to Canada; interdicting illegal goods at Canada’s border; protecting food safety, plant and animal health, and Canada’s resource base; administering trade legislation and a fair and impartial redress mechanism; and collecting duties and taxes on imported goods.

The Targeting Program supports the CBSA’s mandate by performing pre-arrival risk assessments of traveller, cargo, and conveyance information with the goal of facilitating movement of low risk cargo and people and identifying those which may pose a threat to security and safety of Canada. Legislations support the Targeting Program by requiring carriers to provide the CBSA with Advance Passenger Information (API) and all available Passenger Name Record (PNR) data and Advance Commercial Information (ACI). People and goods that are identified as posing a threat to Canada are referred for verification and examination upon their arrival at a port of entry.

The Targeting Program is further supported by the Agency’s internal intelligence community, partnerships with other Government departments including the Royal Canadian Mounted Police, the Canada Security Intelligence Service, Regional Police, and joint initiatives with other governments, such as embedded targeting officers from the United States.

The Targeting Program is delivered by the National Targeting Centre (NTC), which operates as part of the National Border Operations Centre (NBOC), Operations Branch. The Enforcement and Intelligence (E&I) Programs Directorate, Programs Branch is responsible for providing strategic direction, policy guidance and program management.

Targeting was externally audited by the Office of the Auditor General (OAG) in 2007 as part of a broader CBSA audit, entitled Keeping the Borders Open and Secure. This audit identified three key recommendations related to targeting and suggested that the Agency design and implement a risk management framework for its pre-arrival targeting activities; targeters consistently document their reasons for all referral decisions and officers document the results of subsequent examinations for the purpose of continuous improvement; and that mandatory training be instituted for targeting officers.

Since the 2007 OAG audit, several audits and evaluations have addressed aspects of targeting, such as the Pre-Arrival Targeting Evaluation Study (2008); Facilitating the Flow of Imported Commercial Goods (OAG, 2010); Border Controls on Commercial Imports (OAG, 2012); Audit of the National Targeting Implementation Project (2012); Audit of Border Controls for Marine Ports of Entry (2012); Preventing Illegal Entry into Canada (2013) and Audit of Air Cargo (2014).

In 2012, the CBSA implemented major changes to the targeting business model by centralizing air passenger, air cargo and marine cargo targeting activities. These activities had previously been conducted using a tiered approach which included regional targeting and centralized targeting at the former National Risk Assessment Centre (NRAC). To support the newly centralized business model, the Program established a National Targeting Policy, National Training Standards and training materials to ensure targeters were well trained, as well as a suite of supporting Standard Operating Procedures (SOPs) to guide operations. In 2013, the program developed and implemented Scenario-Based Targeting (SBT) in the air passenger mode, a new and automated process that uses predetermined sets of data elements and rules to identify suspected high-risk travellers prior to their arrival in Canada. In 2015, the Program incorporated data analytics and risk indicator analysis expertise under the NTC.

Given the short time since centralization (2012), findings in the audit must be considered in the context of a program that is in the formative stages of its development. The recommendations offered in this report are intended to assist in the achievement of the program’s objectives and help it mature.

2.0 Significance of the Audit

This audit is of interest to the Agency because the Targeting Program is central to the mandate of the CBSA. Furthermore, while parts of the program have been included in other internal and external audits, this is the CBSA’s first internal audit of the Targeting Program’s management control framework.

The audit objective was to assess the adequacy of the management control framework for the Targeting Program. The scope of this audit covered the period from the inception of the NTC (i.e., fiscal year 2012–2013) to July 2015. [*]

3.0 Statement of Conformance

The audit conforms to the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program. The audit approach and methodology followed the International Standards for the Professional Practice of Internal Auditing as defined by the Institute of Internal Auditors and the Internal Auditing Standards for the Government of Canada, as required by the Treasury Board’s Policy on Internal Audit.

4.0 Audit Opinion

The Targeting Program, as it appears today, is relatively new to the Agency, having been centralized in 2012. As such, findings reflect a program that is still maturing.

Since centralization in 2012, the Program has developed and implemented key management controls which support the effectiveness of the Program, as well as advancements in Scenario-Based Targeting (SBT). To assist in moving the Program forward, the audit has identified three areas for improvement in the management control framework: i) the strengthening of governance (i.e., delineation of program management roles and responsibilities, finalizing a vision), ii) the formalization of a risk management approach to ensure impediments to the Program are systematically identified, assessed, mitigated and monitored, and iii) establishing a process to assess targeting inputs regularly to ensure they effectively support the targeting process.

5.0 Key Findings

While a governance structure is in place to manage the Targeting Program through CBSA’s organizational structure, there are opportunities for improvement by clarifying roles and responsibilities for the management of the Targeting Program; strengthening the oversight provided by Enforcement and Intelligence (E&I) Performance Management Table (PMT); and finalizing a common vision and set of corresponding objectives for Agency stakeholders.

The Targeting Program is operating with an informal risk management process. A formalized process would better position the Program to assess and mitigate current and future program and operational risks as it moves forward with expanding targeting [*] in the coming years.

Effective targets are dependent on the professional discretion and judgement of the targeting officers and the inputs (i.e., data quality, examinations results, and training) which inform their judgement. A review of targeting inputs has occurred in an ad hoc manner, making it difficult to identify which inputs (in specific modes) require improvement.

The Targeting Program has established a continuous improvement function which has the potential for making considerable contributions to the Program. One of the main challenges this function will face going forward is the quality of the examination results entered in the CBSA systems. Without consistent and complete examination data across all modes, the value of this function may not be fully realized.

6.0 Summary of Recommendations

This audit makes the following three recommendations relating to:

  • improving the governance of the Targeting Program by strengthening oversight, clarifying roles and responsibilities for key areas of the program’s management and finalizing a vision to align efforts towards common goals;
  • formalizing the risk management process for the Targeting Program to ensure that key program and operational risks, which may impede the program from reaching its objectives, are mitigated and monitored; and
  • establishing a process to assess targeting inputs regularly to ensure they effectively support the targeting process.

7.0 Management Response

The Programs and Operations Branches support the findings of the Audit and look forward to working collaboratively towards addressing the report recommendations.

As noted within the audit, the targeting program is still evolving and maturing – addressing the recommendations from this audit will further strengthen and improve this fundamental program.

8.0 Audit Findings

8.1 Governance

Audit Criteria:

  • An oversight body(ies) has been established and provides strategic direction as well as monitors the Targeting Program.
  • Senior management and oversight bodies request and receive sufficient, relevant and timely information to support decision-making.
  • Roles, responsibilities, and accountabilities for the Targeting Program have been formally defined and communicated.

Similar to how most programs are delivered in the Agency, accountabilities and operational roles and responsibilities are divided between the Operations (NTC) and Programs branches (Enforcement and Intelligence). Within these branches, roles and responsibilities are further distributed among directorates responsible for the monitoring of forwarder/carrier information (ACI, PNR/API), intelligence, and examination results. The program also relies on systems, data integrity, and security provided by the Agency’s information technology services. These relationships are depicted in Diagram 1.

Diagram 1: Agency Stakeholders in the Targeting Program

The diagram depicts the various Agency stakeholders in the Targeting Program. At the top of the diagram is the Enforcement and Intelligence Program Management Table.

Underneath on the left are three areas of the Programs Branch including Enforcement and Intelligence Programs (Intelligence), Travellers (API/PNR), and Commercial (ACI). Arrows extend from these three areas to the middle of the diagram which represents the NTC of the NBOC.

The middle also shows arrows going to the NTC from partners and from systems and data.

On the right-hand side are three areas of the Operations Branch, including Border Operations (Examination Results), Enforcement and Intelligence Operations (Intelligence) and International Region (Intelligence). Arrows also extend from these three areas to the NTC in the centre of the diagram.

The overall success of the Program is reliant upon the extent to which the various stakeholders within the Operations and Programs branches understand their contributions and how they work together to help the program achieve its objectives. One of the tools that could assist in ensuring the work of the various stakeholders is aligned and program objectives are met is a “strategic vision”.

A draft Targeting Program vision “placemat” has been developed by the Enforcement and Intelligence Programs and indicates future end state of the Targeting Program in the various modes. In addition, a draft NTC vision provides more detailed plans on how the future state of the Targeting Program could be attained. It includes an assessment of the current and future state of eleven key Targeting Program areas, including expansion of the Centralized Targeting Model, partnerships, information technology and data analytics. At the time of the audit, neither the “placemat” nor the NTC vision document have been approved and finalized.

Starting in 2016–2017, directorates within both the Programs and Operations branches will be working together to plan and execute the Program’s expansion. This expansion will see, in the short term, the centralization or delivery of targeting activities in additional modes. [*] This expansion is expected to increase the size and complexity of the Program considerably. Given the number of stakeholders involved in the targeting process, an approved and final vision for the Program would help align all efforts towards common goals.

Oversight for the Achievement of Program Objectives

In order for a program to achieve its objectives and realize its vision, it is essential that an oversight mechanism (such as a committee) be in place to ensure a direction is set and the work of the numerous stakeholders is unified.

The Enforcement and Intelligence Program Management Table (E&I PMT) is the oversight committee for the Targeting Program and meets on a monthly basis to oversee the Program, as well as the other programs within the E&I Directorate. This committee is a Director General level committee that is accountable to the Policy and Program Committee (PPC), which advises Executive Committee (EC) and ensures the ongoing development of CBSA policy and program delivery.

The Terms of Reference (TOR) for the E&I PMT contain many elements for good oversight (i.e., monitoring of risk management, conduct strategic planning). In practice, however, the oversight provided by this committee was limited. Our review of records of decision and meeting minutes suggests the Program is not a standing agenda item for E&I PMT, discussions on the program were infrequent or part of broader discussions on E&I program integrity priorities, and program-level risks were brought to the committee’s attention on an ad hoc basis. Finally, apart from performance reporting through the Agency Performance Reports, the committee has yet to clarify reporting expectations (i.e., type of information required, frequency, format, content, etc.) from the Programs Branch and the NTC to support its oversight role and decision making.

While the extent to which program oversight provided by this committee was limited, it should be noted that E&I PMT has been responsible for overseeing the program since June 2014, and the final TOR clarifying its purpose were only approved in May 2015. This suggests the committee is still in the process of establishing its oversight role.

Roles and Responsibilities

The success of a program is facilitated by the establishment of clear roles and responsibilities at the program management level. Delineated roles and responsibilities ensure that critical components of a program are managed and duplication of effort is minimized. Aspects key to the successful management of a program include: human and financial resource planning; policy development and implementation; performance monitoring; and risk management, among others.

The delineation of roles and responsibilities for the management of the Targeting Program requires greater clarity. The National Targeting Policy (2012) contains information on the operational roles and responsibilities of the Programs and Operations branches, as well as specific directorates/divisions under each Branch. However, information on the interdependencies of these stakeholders showing how they work together to manage the Program was not included in this key document. Furthermore, some of the divisions identified in the Policy no longer exist or have been reorganized (i.e., Traveller and Commercial Targeting and Advance Information Programs; Commercial Targeting Policy Unit), suggesting roles and responsibilities need to be updated. Without clearer roles and responsibilities, key aspects to program management are not occurring in a formal manner (i.e., human resource planning and implementation of formal risk management).

In summary, while a governance structure is in place to oversee the Targeting Program, there are opportunities for improvement by finalizing a vision as well as strategic and operational objectives; strengthening the oversight provided by E&I PMT; and clarifying roles and responsibilities for the management of the program.

Recommendation 1:

The Vice-President of the Programs Branch should improve the governance of the Targeting Program by strengthening oversight, clarifying roles and responsibilities for key areas of program management, and finalizing the Program’s vision.

Management Response Completion Date

Agree.

The Programs Branch is committed to improving the governance of the Targeting program by strengthening oversight, clarifying roles and responsibilities for key areas of the program’s management and finalizing the Program’s vision.

The Enforcement and Intelligence Program Management Table (E&I PMT) is the oversight committee for the Targeting Program and where the strategic management function for the program takes place. The final Terms of Reference clarifying its purpose were approved as of May 2015. To further strengthen the oversight of the program, the mandate of the existing Targeting Program Management Committee (TPMC) will be expanded to become the program management forum and act as a working group that reports on a quarterly basis to the E&I PMT. The E&I PMT will be responsible for monitoring and ensuring progress towards the Targeting Program’s vision and engaging senior level internal and external stakeholders on unresolved issues and continued strategic investments.

July 2016

8.2 Risk Management at the Program Level

Audit Criteria:

  • Program risks are identified, assessed, communicated, mitigated and monitored to ensure the achievement of program objectives.
  • The Agency has determined the level of human resourcing (and skills/competencies) required for the NTC to execute the Targeting Program.

Risk management is the formal and systematic identification, assessment, mitigation and monitoring of risks which could negatively impact the achievement of objectives. According to the Treasury Board Secretariat Guide to Integrated Risk Management and the Committee of Sponsoring Organizations of the Treadway Commission (COSO, 2013), effective risk management should occur across an organization and at all levels within the organization.

While the Agency has formalized risk management at the enterprise and directorate levels, risk management at the program level has not been formalized.

The Targeting Program manages program risks informally. For example, operational challenges are identified and documented through ‘issue sheets’, which are typically developed by the NTC staff. The audit noted that a number of issue sheets had been developed in 2014 to communicate deficiencies related [*]. The issue sheets, however, were not supported by an established process for their communication to senior management, actioning and monitoring.

In addition to the risks identified in past audits, this audit identified emerging risks that may not receive sufficient attention without a more formal risk management approach. As previously mentioned, the Program plans to expand targeting activities in additional modes in the coming years, and this expansion will require more planning and a higher level of resourcing. This will have implications for the people side of the business, particularly in relation to staffing, training and performance management. Within this emerging context, the audit noted that:

  • The program does not have a formal and approved human resource plan to support recruitment, retention and succession. Only various draft recruitment and staffing strategies exist;
  • The effectiveness of the mandatory foundations and modal training courses is unknown given that the training suite has not been assessed for effectiveness. In addition, the current training suite may require enhancement, particularly in the provision of advanced training courses for targeting officers to support continuous learning (i.e., courses on analytical/critical thinking);
  • While most targeting officers have received mandatory training, there are still some who require training. [*];
  • Performance management agreements and personal learning plans, as key performance monitoring controls, are not being leveraged to their full capacity. Areas of improvement and opportunities for growth through mentoring and training were infrequently documented.

In summary, the Targeting Program is operating with an informal risk management process. Without a formal risk management process, key program and operational risks are not being systematically assessed and mitigated. With the expansion of the Targeting Program in the coming years, the complexity of the Program will rise considerably, [*]. Therefore, the current approach to risk management would benefit from being formalized.

Recommendation 2:

The Vice-President of the Programs Branch, in collaboration with the Vice-President of the Operations Branch, should formalize the risk management process for the Targeting Program to ensure that key program risks which may impede the program from reaching its objectives are mitigated and monitored.

Management Response Completion Date

Agree. 

The Programs Branch is committed to the formalization of the programmatic risk management process for the Targeting Program and will work in collaboration with the Operations Branch, Corporate Security Directorate, Information Science and Technology Branch and the Human Resources Branch, to develop a risk profile for the Targeting Program for the approval of E&I PMT. Upon approval of this risk profile, a mitigation action plan will be developed.

December 2016

8.3 Target Creation

Audit Criteria:

  • The process for creating targets is defined.
  • Sufficient information on targets is provided to border security officers (BSOs) to effectively and efficiently conduct examinations.

Referrals for examination are one of the key methods the CBSA uses to assess compliance with the various legislations governing the entry of people and goods. Specifically, travellers and goods can be referred in three distinct methods: 1) Targeted referrals, 2) BSO selective referrals, and 3) Random referrals.

‘Targets’ are products issued by the targeting officers at the NTC who are trained to use a deductive reasoning in the review of multiple sources of information to alert BSOs at the ports of entry of a suspected threat to public safety. Threats may be related to irregular immigration, contraband, national security or other government department (OGD) requirements (i.e., food/plant/animal).

A simplified targeting process appears below in Diagram 2. As illustrated, the decision to issue a target is a result of information review from several sources. The objective of this review is to assess whether the information or data provided supports a decision to release or refer for examination.

Diagram 2: Simplified Targeting Process Illustrating Inputs

The diagram provides a simplified illustration of the various inputs to the targeting process. The inputs are listed in eight boxes as follows:

  • Pre-arrival Data
  • Automated Risk Scoring / Scenarios (SBT)
  • Open and Closed Source Research
  • Training, Guidance (Policies, Procedures)
  • Partners/Colleagues
  • Results (Analysis of Exam Results, Data Analytics)
  • Intelligence (Regional, HQ, NTC)
  • International Intelligence

These inputs flow into the Targeting Officer decision to issue a target or not.

Prior to issuing a target, targeting officers carefully consider the inputs provided to them. For example, a targeting officer will review various elements of the pre-arrival data forwarded by the carrier, as well as intelligence products and information gleaned from their own searches of open and closed sources. The targeting officer also builds on the knowledge and experience they have acquired through mandatory training and previous targets they have personally issued (and followed up on using the examination results entered by BSOs at the port of entry). Due to the reliance on these inputs in the targeting process, it is important that each input be of high quality. Targeting officer discretion and judgement can only be as good as the inputs provided to them.

The Agency assesses the quality of pre-arrival data through compliance monitoring and reporting. The Agency has established some plans to improve the quality of advance information in the air passenger mode and monitors the extent to which carriers comply with the provision of advance information in both the traveller and commercial streams.

The assessment of the other key information inputs to targeting was occurring in an ad hoc manner. This ad hoc approach has involved some inputs being examined by the NTC in specific modes only, while other inputs have been examined more broadly by the Program’s Branch. For instance:

  • the NTC’s ‘issue sheets’ identified limitations related to [*]. To have a more comprehensive understanding of the potential risks and limitations with these key inputs, such information is also needed for [*].
  • E&I’s gap analysis of intelligence products across the Agency in 2014 showed that targeting intelligence products needed to be better tailored to the needs of targeting officers, be presented on requested subject matters and delivered in format that targeting officers found useable. While helpful in identifying challenges for this key targeting input, this analysis did not differentiate between the intelligence products used in different modes. As such, it is not clear whether the product limitations apply generally across all the targeting modes or just specific ones.

A more comprehensive and systematic examination of inputs across all modes would enable a greater understanding of their strengths and deficiencies. This would facilitate the prioritization and allocation of resources where required.

Target Narratives

The results of the target creation process are the narratives that targeting officers enter into various systems (TITAN for commercial targeting and ICES and or FOSS/GCMS for traveller targeting). Narratives are the justifications for why a target was issued and should contain the risk indicators that led the targeting officer to believe a target was warranted for a person or shipment.

The information in the narratives is important for several reasons. First, narratives inform the BSOs at the port of entry of the nature of the suspected risk which can impact the approach that is taken with the interception and examination. Second, a compelling narrative can increase the credibility of the target. Finally, documented narratives enable targeting officers to validate their reasoning, and thereby learn from their own work (which also has implications for the development of new risk indicators and scenarios in the future). For these reasons, it is important that narratives be consistently completed by targeting officers and done in a thorough manner.

It should be noted that targeting officers have to review high volumes of travellers and shipments under significant time constraints, which can make it difficult to provide thorough narratives, especially for unknown risks. Every day, the CBSA processes over 70,000 air passengers, 5,400 marine containers and 9,000 air cargo shipments arriving in Canada at more than 13 international airports and five major marine ports.

A non-statistical sample of 108 target narrativesFootnote 1 was extracted from targets issued in fiscal year 2014–2015. The sample included generally equal proportions of air passenger, marine and air cargo targets. Files were reviewed to determine the quality of the narratives in relation to the required level of detail as specified in the National Targeting Policy and SOPs. Results demonstrated that approximately 80% of narratives were complete (i.e., risk indicators included, justifications were clear and detailed). While there is room for improvement, particularly for marine cargo target narratives, the findings suggest that sufficient information is often provided to facilitate the interception and examination of targets and targeting officers can use the narratives to validate their own reasoning and improve their targeting skills.

In summary, targeting officers are at the cornerstone of the Targeting Program’s effectiveness. The quality of the targeting officer’s work is reliant on the inputs they are provided and without good inputs, the success of the Targeting Program cannot be maximized. To date, review of these inputs has not been systematic, resulting in an incomplete picture of where the strengths and areas of improvement lie. Without this systematic view, it is difficult to prioritize efforts and allocate resources accordingly.

Recommendation 3:

The Vice-President of the Programs Branch, in collaboration with the Vice-President of the Operations Branch, should establish a process to assess targeting inputs regularly to ensure they effectively support the targeting process.

Management Response Completion Date

Agree.

The Programs Branch is committed to working in partnership with Operations to ensure that a robust mechanism is implemented to monitor and evaluate targeting inputs to ensure they continue supporting the strategic priorities of the Agency.

July 2016

8.4 Continuous Improvement

Audit Criteria:

  • A continuous improvement approach has been defined and established to periodically assess the target creation process (as well as the inputs) and adjust as needed.

Continuous improvement is the gradual and ongoing effort of an organization to increase efficiency and effectiveness of processes, products and business results. For the Targeting Program to become more efficient and effective, it is key that resources be dedicated to such activities and that necessary improvements be prioritized, implemented and closely monitored. Continuous improvement is particularly important to the Targeting Program [*].

The Targeting Program has recently incorporated the expertise of three teams which support a continuous improvement function for the Program. These teams include:

  • Targeting Intelligence Unit: A team of intelligence officers who develop intelligence products such as targeting bulletins and analyses of significant contraband seizures. This unit also leads major analytical projects with direct implications for how targeting officers identify potential threats. [*]
  • Target Risk Indicator and Scenarios (TRIS) Unit: A team dedicated to the monitoring and maintenance of scenarios [*] mode and the analysis and maintenance of risk indicators and scores used in commercial targeting. The work of this unit has implications for enhancing target effectiveness by providing targeting officers with scenarios which reflect new and emerging threats and risk scores which have greater predictive capability. 
  • Data Analytics Unit (DAU): A team dedicated to providing statistical support and expertise to the Intelligence and TRIS units. This group uses analytical software against [*] worth of commercial data pulled from multiple CBSA databases to respond to queries for specific indicator and trend analyses.

The strength in the Program’s newly consolidated continuous improvement function lies in its capacity to identify and address deficiencies related to many of the targeting inputs identified in Diagram 2. The function’s combined expertise in intelligence, indicator development and statistical analysis positions it to make significant contributions to the Program going forward (i.e., indicators and scenarios with greater predictive capability, more needs-specific intelligence products, compilation of target profiles using multiple sources of data). In turn, their work can also be used to adjust training, policies and targeting procedures in the future. Moving forward, establishing a common set of priorities/planned projects between the three groups that make up this function will help realize their collective potential.

Examination Results

Key enablers of continuous improvement for the program are the examination results entered by BSOs at the port of entry once a target has been intercepted and an examination conducted. The examination results are important because they provide the Targeting Program with feedback and intelligence on how the examination was conducted, what was found/not found and identify higher risks for future targeting. When analysed over time, this information can help the program establish trends, “lessons learned” and provide valuable insight for the purposes of risk indicator and scenario development. The entering of examination results into the various systems is known as “closing the loop” and is part of broader effort within the Agency.

The “Closing the Loop” initiative implemented in 2011 directs Agency employees to record all examination results, where possible, and to enter thorough and sufficiently detailed notes and comments about conducted examinations. The Agency is gathering volumetric and compliance-oriented data related to “closing the loop” (i.e., number of examinations conducted, the proportion of targets examined, number of overrides, acquittal rates, etc.). Performance is reported on in the Performance and Service Standard and Agency Performance Summary reports that are regularly reviewed by the Program Management Tables, Operations Branch Executive Committee and Executive Committee.

In 2012 the Commercial Border Programs Directorate (CBPD) began monitoring the quality of commercial examination reports. An analysis by CBPD of 150 sampled targets from 2014–2015 showed that 87% of examination results [*] were deemed acceptable, which is 7% higher than the 80% acceptability target established by Operations Branch Executive Committee. The Agency has not yet done a similar analysis [*]. As such, the quality of these results remains unknown.

This audit conducted testing of target examination results [*] to determine the extent to which they were entered in a manner consistent with the level of detail specified in the CBSA Enforcement Manual. A non-statistical sample of 108 [*] targets was extracted from the 2014–2015 fiscal year. Results demonstrated that the level of detail in the examination results was inconsistent.

[*] Incomplete exam results ranged from a few words to nothing at all. Findings suggest that, in general, there is room to improve the quality of target examination results.

The Agency is aware of the importance of quality examination results and the limitations identified in past reviews and audits. Given this, the audit will not identify further recommendations for improvement. Without quality exam results, the Program’s continuous improvement function will be limited in the value it can bring to the Targeting Program.

In brief, the Targeting Program established a continuous improvement function which has the potential for making considerable contributions to the Program in the coming years. One of the main challenges this function will face going forward is the quality of pre-arrival information and the quality of the examination results.

Appendix A – About the Audit

Audit Objectives and Scope

The audit objective was to assess the adequacy of the management control framework for the National Targeting Program.

The scope of this audit covered the period from the inception of the NTC (i.e., fiscal year 2012-2013) to July 2015. [*] The scope of the audit is divided in two parts: i) broad components of management control and ii) operations.

Notable scope exclusions include performance monitoring, which was covered by the 2015 Evaluation of the Targeting Program, and areas related to pre-arrival data quality and compliance, which have been included in several audits and evaluations in recent years.

Risk Assessment

To assist in audit planning and determine the audit scope, a preliminary risk assessment was conducted on the management control framework elements of the Targeting Program. The assessment was based on consultation with staff at the NTC and Programs Branch; review of the available documentation (i.e., policies, procedures, frameworks) at the time and information gleaned from site visits to the Atlantic region, as well as visits to Montréal, the Greater Toronto Area and the Pacific region by the Evaluation team. The following risks represent the risks that the audit engagement was built around:

  • There is a risk that oversight has not been established to provide strategic direction to the Program and monitor progress;
  • There is a risk that senior management does not receive sufficient, relevant and timely information to support decision-making;
  • There is a risk that roles, responsibilities and accountabilities have not been formally defined and communicated;
  • There is a risk that risk management has not been established for the Program;
  • There is a risk that the Agency has not determined the appropriate level of human resourcing required to execute the Program;
  • There is a risk that the method in which targets are created may not identify high risk persons or shipments;
  • There is a risk that BSOs may not have sufficient information to effectively conduct examinations; and
  • There is a risk that the Program has not defined its process of continuous improvement to ensure the quality of targets improves over time.

Approach And Methodology

The audit was conducted in accordance with the Internal Auditing Standards for the Government of Canada. The examination phase of this audit was performed using the following approach:

  • Collect, review and analyze corporate documentation to examine governance, risk management and human resources planning;
  • Collect, review and analyze operational documents (i.e., policies, procedures, standards, directives) to examine targeting procedures and practices;
  • Select a sample of targets to examine the information provided to BSOs; and
  • Conduct interviews with various stakeholders within the NTC, Programs, Comptrollership, Information Systems and Technology Branches.

Audit Criteria

The Audit criteria are aligned with the management control principles from the 2013 Committee of Sponsoring Organizations of the Treadway Commission (COSO). Given the preliminary findings from the planning phase, the following criteria were chosen:

Final Lines of Enquiry Audit Criteria
1. Governance
  • 1.1 An oversight body (ies) has been established and provides strategic direction as well as monitors the Targeting Program.
  • 1.2 Senior management and oversight bodies request and receive sufficient, relevant and timely information to support decision-making.
  • 1.3 Roles, responsibilities, and accountabilities for the Targeting Program have been formally defined and communicated.
2. Risk Management
  • 2.1 Program risks are identified, assessed, communicated, mitigated and monitored to ensure the achievement of program objectives.
3. Human Resource Planning
  • 3.1 The Agency has determined the level of human resourcing (and skills/competencies) required for the NTC to execute the Targeting Program.
4. Targeting Operations
  • 4.1 The process for creating targets is defined.
  • 4.2 Sufficient information on targets is provided to BSOs to effectively and efficiently conduct examinations.
  • 4.3 A continuous improvement approach has been defined and established to periodically assess the target creation process (as well as the inputs) and adjust as needed.

Appendix B – List of Acronyms

ACROSS
Accelerated Commercial Release Operations Support System (commercial system)
ACI
Advance Commerical Information
API
Advance Passenger Information
APS
Agency Performance Summary
BSO
Border Services Officer
CBSA
Canada Border Services Agency
DAU
Data Analytics Unit
E&I
Enforcement and Intelligence
FOSS
Field Operations Support System
ICES
Integrated Customs Enforcement System
IPIL
Integrated Primary Inspection Line
NBOC
National Border Operations Centre
NTC
National Targeting Centre
OAG
Office of the Auditor General
PMT
Program Management Table
PPC
Program Policy Committee
PNR
Passenger Name Record
POE
Port of entry
SBT
Scenario-Based Targeting
SOP
Standard Operating Procedure
TOR
Terms of Reference
TRIS
Targeting Risk Indicators and Scenarios

Notes

Footnotes

Footnote 1

The original sample was 110, but two files were irretrievable in TITAN.

Return to footnote 1 referrer

Date modified: