Tue, 08 Nov 2022 in Refuge: Canada's Journal on Refugees
Life Beyond Refuge: A System Theory of Change for Supporting Refugee Newcomers
Abstract
Canada is internationally recognized as a leader in welcoming refugee newcomers. However, there is limited evidence about how well refugee newcomers fare after arriving in Canada and the effectiveness of resettlement services and supports. A system theory of change was developed to guide assessments of complexity across the refugee-serving sector that seek to investigate refugees’ lived experiences and evaluate practice across multiple levels. This article describes the process of developing the Life Beyond Refuge system theory of change and the implications for community-level practice, public policy, and, ultimately, resettlement outcomes for refugee newcomers.
Resumé
Le Canada est reconnu internationalement comme un chef de file dans l’accueil des nouveaux arrivants réfugiés. Cependant, il y a peu de données sur la manière dont les réfugiés se portent après leur arrivée au Canada et sur l’efficacité des services et des soutiens à la réinstallation. Une théorie systémique du changement a été développée afin de guider les évaluations de la complexité dans le secteur des services aux réfugiés qui cherchent à examiner les expériences vécues des réfugiés et à évaluer les pratiques à plusieurs niveaux. Cet article décrit le processus d’élaboration de la théorie systémique du changement Life Beyond Refuge et ce qu’elle implique pour la pratique au niveau communautaire, les politiques publiques, et, enfin, le bilan de la réinstallation des nouveaux arrivants réfugiés.
Main Text
INTRODUCTION
Canada is a world leader in welcoming refugees. In 1986, the United Nations High Commissioner for Refugees (UNHCR) awarded “the people of Canada” the Nansen Medal for their sustained contribution to the cause of refugees (Beiser, 1999). More recently, Canada’s positive response to the Syrian refugee crisis won international praise (UNHCR,2017). The reality of Canada’s welcoming of refugees may not in fact be as rosy as its reputation. The country’s history of being “unwelcoming” includes examples in which refugees in need of protection were denied (Scotti, 2017), where refugees were selected for economic gain at the expense of the most vulnerable (Canadian Council for Refugees [CCR]2009), and where refugee claimants seeking asylum at the country’s borders experienced lukewarm reception (CCR,1996). Still, the welcome of refugees on humanitarian grounds has remained an enduring component of Canada’s national immigration strategy (El-Assal, 2015).
To distinguish newcomers admitted to Canada on humanitarian grounds from those arriving on economic grounds or through family reunification, Canada created a designated immigration class for refugees in 1976. Since that time, the majority of refugee newcomers have arrived through a proactive resettlement process where refugees are identified overseas and resettled in Canada via three primary streams: as government-assisted refugees (GARs), as privately sponsored refugees (PSRs), and through the newer and smaller shared sponsorship programs (i.e., Blended Visa Office-Referred [BVOR]; Joint Assistance Sponsorship [JAS]). Others have come through the reactive asylum process as refugee claimants who seek asylum after arrival in Canada (Immigration, Refugees, and Citizenship Canada [IRCC],2017). From 2010 to 2017, between 23,000 and 58,000 refugee newcomers were admitted each year through these streams, constituting 12% of all newcomers arriving within the country’s borders during this period (IRCC,2019).
A system of support has unfolded over time to aid refugee newcomers as they resettle and begin the integration process in Canada. For example, federal policy ensures that resettled refugees are financially supported for their first year (whether by the government for GARs or by private sponsors for PSRs). Local, community-based supports are also available to refugee newcomers in the form of informal groups, ethnic associations, and faith communities, as well as professionalized service provider organizations. Some organizations are devoted to serving refugee newcomers by providing resettlement services, while others serve all newcomers through settlement services and/or all residents in the form of public services. As part of the nonprofit sector, these organizations are funded through different levels of government, foundations, and private donations. The federal government provides the main source of funding to the sector, allocating approximately $785 million each year to supporting 550 newcomer service provider organizations across the country (IRCC,2020). The federal government views its investment in this system of support as helping newcomers to integrate and succeed in their new life in Canada and contributing to nation-building (IRCC,2019).
Despite this intentionality in resettlement, Canada does poorly in assessing refugee outcomes. There is a limited body of evidence about how well refugee newcomers fare after arriving in Canada and the effectiveness of available resettlement services and supports (Wilkinson & Garcea, 2017). Past assessments have been sporadic and piecemeal, typically considering limited dimensions of resettlement, such as labour-market integration (Bevelander & Pendakur, 2014), social integration (Simich, Beiser, Stewart, & Mwakarimba, 2005), language (Benseman, 2014), and health (Dorman et al., 2017), or considering specific categories of refugees, such as privately sponsored (Hyndman, Payne, & Jimenez, 2017), government-assisted (Citizenship and Immigration Canada [CIC],2011), or refugee claimants (Jackson & Bauder, 2014). Other assessments have focused on specific refugee groups or particular geographic locations (e.g., Alberta Association of Immigrant Serving Agencies [AAISA],2017; IRCC,2016; Janzen, Leis, & Ochocka,2021; Plasterer, 2011). Equally concerning is the minimal use of evaluation data that do exist among leaders of community-based programs, a fact that hampers the quality of decision-making for local newcomer support (Diener & Thibedeau, 2019). While one comprehensive evaluation of Canada’s refugee programs was recently completed by IRCC (2016), it is unclear why, given its global leadership role in refugee resettlement, Canada has not played a similarly consistent and robust leadership role in the evaluation of refugee resettlement programs. Canada is not alone in this challenge, even if there are many examples of individual resettlement evaluations globally (Dumann & Tissot, 2020; Dunn, Browne, Haworth, Wurie, & Campos-Matos, 2021; Murray, Davidson, & Schweitzer, 2010). The study of migration more generally lacks a clear vision about how best to evaluate migrant well-being in a comprehensive manner (Hendriks & Bartram, 2018).
The purpose of this article is to address the current ad hoc nature of resettlement assessment by proposing a system theory of change that could serve as a holistic framework when evaluating how refugee newcomers are supported. This theory of change was developed through a comprehensive review of the literature that identified outcomes related to refugee resettlement and how these are currently being evaluated. This article describes what a system theory of change is and how it can frame the evaluation of refugee supports within a given country. Next, the parameters of the review are provided before detailing the key components of a consolidated system theory of change for refugee newcomer support. A discussion of the contribution of the system theory of change for both the creation of evidence (conducting evaluation) and the use of evidence in decision-making (mobilizing knowledge) in policy and practice is provided. While the article is grounded primarily in the resettlement sector in Canada, it has implications for other societies impacted by migration in the twenty-first century.
SYSTEM THEORY OF CHANGE
A system theory of change has the potential to facilitate the comprehensive creation and use of evidence in the resettlement sector. In the field of program evaluation, a theory of change explains how the activities of a given intervention are expected to contribute to a chain of intended outcomes (Funnell & Rogers, 2011). A theory of change provides evaluators with a framework when assessing the effectiveness of a given set of activities (process evaluation) and the extent to which the intended outcomes of these activities are reached (outcome evaluation).
A system theory of change is an explanation of interventions within a social system. A social system can be understood as various components (i.e., structures) functioning together as a whole (i.e., process) with intention (i.e., vision) (Foster-Fishman & Behrens, 2007; Janzen, Chapman, & Watson, 2012; Kelly, Ryan, Altman, & Stelzner, 2000). System components are therefore not seen as self-contained units but as interconnected and interdependent within a dynamic and multi-layered context (Schensul, 2009). The health of the system is dependent on developing and accessing resources that facilitate system functioning (Trickett, 2009).
Applied to refugee newcomers, a system theory of change suggests that refugee resettlement cannot be understood in isolation or as a result of a single factor; rather, refugee newcomers and their families are embedded within layers of system components that can both affect and be affected by their resettlement (Janzen et al., 2021). A system theory of change identifies and links the activities and corresponding outcomes across various ecological levels of the refugee system of support. Ecological levels include the micro (e.g., family members, sponsor groups, neighbourhoods, faith community), the exo (e.g., resettlement organizations and other groups in the community with resources and power), and the macro (e.g., policy-makers, media, and other influencers of dominant society). A system theory of change provides a framework for assessing how these various components and their functioning can be improved upon in order to reach desired outcomes. Such a view is consistent with refugee literature in which resettlement is often framed as a negotiation between local realities and external influences that provide higher-level direction to the flow of migration (Janzen et al., 2021).
METHODOLOGY
Our proposed system theory of change was developed by conducting a review of the literature identifying outcomes related to refugee resettlement and how these are currently being evaluated. The theory of change was created by inductively categorizing outcomes found in the literature according to the ecological level of impact and according to how refugee outcomes change over time. Throughout the process of theory development, the emerging system theory of change was shared with stakeholders across the country for verification and resonance. The detailed methodology used in developing the system theory of change, and the project in which it was embedded, is described below.
The system theory of change was developed as part of an evaluation capacity-building initiative led by the Centre for Community Based Research (CCBR) in partnership with the University of Alberta. Funded by the federal immigration department (IRCC), the purpose of this initiative was to equip resettlement organizations and groups to conduct community-based evaluation to improve supports and outcomes for newcomers who come as refugees.
The national project was guided by an intersectoral advisory committee representing various stakeholder perspectives that included people with lived refugee experience, settlement service providers, sponsorship agreement holders, newcomer umbrella network leaders, academics, and the federal government (see http://www.eval4refugee.ca). Over the three-year project (2018–2021), numerous capacity-building strategies were developed and implemented, highlighting the four phases of community-based evaluation and its emphasis on being stakeholder driven, participatory, and action oriented (Janzen, Ochocka, & Stobbe, 2016). Strategies included introductory videos, evaluation readiness tools, ethics support, in-person workshops, a mentorship pool, recorded webinars and online live events, and individualized coaching (see http://www.eval4refugee.ca). Combined, these activities endeavoured to enhance the resources, knowledge, and skills needed to implement community-based approaches for producing and using evaluation evidence and, in turn, to improve Canada’s refugee newcomer system of support. A detailed description and reflection of this capacity-building initiative is available elsewhere (Gokiert, Daniels, Rezvani, & Janzen, 2022).
One foundational activity of the project was to conduct an ongoing literature review. The purpose of the review was to develop a comprehensive understanding of how intended outcomes for refugee resettlement are being conceptualized and evaluated. Research team members from both partner organizations (nine in total) searched online academic journal databases and sectoral websites to identify studies related to refugees, resettlement, evaluation, and outcomes. The review included Canadian and international academic research with refugee newcomers and grey literature produced by community-based organizations, as well as by government (including relevant publications authored by IRCC). Articles included were compiled in an annotated bibliography listing the identified refugee outcomes, indicators of these outcomes, and methods of data collection (Janzen et al., 2020). The annotated bibliography included 46 Canadian-based articles and 39 articles from international research for a total of 85 articles. Approximately half of these were peer-reviewed journal articles, while the remaining half were produced by government and community-based organizations. The majority of articles described primary research or secondary analysis of data with refugee newcomers or community-based organizations, with theoretical articles, literature reviews, and practice notes making up a smaller number. Of the 85 articles included in the annotated bibliography, 16 were evaluations of programs, practices, or policies designed to support refugee newcomers. The vast majority of articles focused on outcomes at the individual level, with a few addressing community- or macro-level outcomes.
Based on this annotated bibliography, the research team then created an outcome inventory document that conceptually organized the outcomes identified in the literature (Janzen et al., 2020). Grounded theory was used to inductively examine how the various refugee outcomes documented in the literature could be meaningfully organized as a whole. Outcomes were first organized by their dominant attributes into categories, referred to as outcome domains. Team members further arranged outcome domains according to ecological level of impact (individual, community, macro), as well as temporally, according to the process of change that refugee newcomers are expected to experience over time in their new home country (claim process, resettlement, settlement and adaptation, integration, and wellness). The outcome inventory used three tables to display outcome domains, arranged by level of impact and colour coded based on time period. The tables elucidate each outcome domain by listing sample outcomes from the literature, and these sample outcomes were cross-referenced to the annotated bibliography. As a living document updated regularly from recent publications, the outcome inventory evolved over the three years of the project to incorporate new insights on refugee outcomes.
The system theory of change, described in the following section, was grounded in this outcome inventory document. Team members synthesized the various dimensions of the resettlement experience as outlined in the outcome inventory into a coherent theory of change capable of charting how interventions to support refugee newcomers and their given activities can contribute to a chain of intended outcomes for refugee newcomers (Funnell & Rogers, 2011). Various iterations of the summary figure were discussed by the nine members of the research team. Member checking was completed by sharing and discussing the theory of change figure with advisory committee members. In addition, two other mechanisms were used to invite broader stakeholder feedback in refining the theory of change so that it optimally reflected current contexts and realities in policy and practice. First, the theory of change was presented and discussed with participants of the Evaluating Refugee Programs capacity-building workshops. In total, 13 full-day workshops were held in communities across the country. Over 300 participants attended these workshops, representing community organizations, newcomer umbrella networks, government, and evaluators (whether academic or private consultants). Second, the theory of change was also presented and discussed at two online live events that the project organized as it shifted to a virtual platform during the global pandemic. In both cases, stakeholder reaction to the theory of change added to the iterative process of honing its conceptualization, verifying its utility, and confirming its resonance with stakeholder experiences.
RESULTS
The system theory of change represents the dynamic and multi-layered context of refugee newcomers’ system of support. As a framework highlighting supportive interventions and corresponding intended outcomes for newcomers arriving in Canada as refugees, it articulates the ecological levels where interventions occur and the time periods during which the process of change from “refugeeness” to “life beyond refuge” (Kyriakides, Bajjali, Mcluhan, & Anderson, 2018) takes place. Figure_1 visually describes these levels and components while linking activities and intended outcomes related to this process. The figure’s title, “Life Beyond Refuge,” suggests that the experience of being a refugee is neither static nor powerless. Instead, it is a process of change where refugee newcomers can be supported over time to achieve meaningful outcomes in life during the claim process (if applicable), during initial resettlement, in settlement and adaptation, and ultimately in attaining integration and wellness in Canadian society. The “Life Beyond Refuge” figure pairs with the outcome inventory, where a full list of outcomes found in the literature are organized. Below, we present and describe the key components of the system theory of change as summarized in Figure_1.
The figure organizes outcomes for refugees according to the ecological level of impact, organized vertically on the figure, and period of time in refugee newcomers’ ideal process of change, presented from left to right. Horizontal arrows show the time period categories from the initial stages of the refugee experience (claim process, resettlement) to the ideal final outcomes where integration and wellness are achieved. Ecological levels of impact include the individual, community, and macro levels. Bi-directional arrows between levels indicate that a reciprocal relationship exists; system components interact across levels to both affect and be affected by corresponding components. The figure includes activities of a given level and time period and the associated outcome domains. The dotted circle on the left indicates that this period does not apply to all refugee newcomers to Canada but only to refugee claimants, while the bold circle on the right represents outcomes ideally shared by refugee newcomers, other newcomers, and all residents of Canada.
Ecological Levels
Individual-Level Outcomes
Individual-, community-, and macro-level outcome domains are organized vertically in the “Life Beyond Refuge” figure. At the top of the figure, individual-level outcomes capture the intended impacts of refugee-serving programs and supports on the lives of individual newcomers who arrived as refugees and their families. The figure makes the point that intended individual-level outcomes evolve and are shared over time and among different subsections of the Canadian population. At the heart of the system, refugee newcomers can move over time beyond their refugeeness in ways that they share with other refugee newcomers (through the claim and resettlement process), with other newcomers (through the settlement and adaptation process), and with other permanent residents in Canada (towards integration and wellness) (Canadian Index of Wellbeing [CIW], 2016; Jedwab & Soroka2014; Kyriakides et al.,2018). Consequently, activities of the individual outcome domains focus on supporting refugee newcomers to achieve outcomes at each time period (e.g., Alberta Association of Immigrant Serving Agencies [AAISA],2017; IRCC,2016; Janzen, Leis, & Ochocka,2021; Plasterer, 2011). In contrast to the community and macro domain levels, most literature discusses outcomes affecting refugees at the individual level. The activities and outcomes associated with the individual level of impact interact with and influence aspects of the community and macro levels while also being shaped by them in turn.
Community-Level Outcomes
Located below the individual level of impact, community-level outcomes are organized by activities related to “welcoming community attitudes” and “welcoming community practices.” Adapted from the Characteristics of a Welcoming Community project (Esses, Hamilton, Bennett-AbuAyyash, & Burnstein, 2010; Ravanera, Esses, & Fernando, 2013), these activities capture how elements of a local community such as its organizations, institutions, local government, businesses, neighbours, and ethnic, faith, and sponsor groups hold attitudes and practices that influence the degree of welcome and support shown to newcomers in their community. Instead of focusing on refugee newcomers themselves, community-level outcome domains highlight the community context into which refugees are placed, emphasizing the responsibility for welcoming newcomers that is shared by all members of the community and its local institutions. Activities devoted to fostering welcoming community attitudes create the conditions for awareness of the refugee experience, recognizing refugee newcomers as members of the community (Atwell, Gifford, & McDonald-Wilmsen, 2009), and favourable and accurate media coverage. Similarly, welcoming community practices create conditions that foster relevant and meaningful employment and educational opportunities (Fang, Sapeha, & Neil, 2018), enhance intra- and inter-group social capital (Im & Rosenberg, 2016; Im, 2018), and support the presence of service provider organizations that effectively meet the needs of newcomers. Additionally, these conditions promote leadership skills development (Im & Rosenberg, 2016); provide newcomer-friendly municipal resources and services that address their needs; foster meaningful religious, political, and social participation (Cheyne-Hazineh, 2020; Donaldson, 2017); ensure mandated services are available and accessible; promote and maintain safe spaces; provide accessible and suitable health care; offer adequate and affordable housing (Rose, 2019); ensure childcare supports are accessible and affordable (CIW,2016; IRCC,2019d) and provide accessible transportation options (English et al.,2017; IRCC,2019d; Stewart et al.,2012). Compared to the individual-level outcome domains, much less attention is given in the literature to community-level outcomes. In particular, beyond the conceptual framework on welcoming communities produced byEsses et al. (2010), research offers little additional insight on the activities or outcomes related to welcoming community attitudes.
Macro-Level Outcomes
Moving down to the bottom of the figure, macro-level outcomes capture how the broader Canadian society can impact the lives of refugee newcomers. Macro-level system components include those entities that influence broad societal conditions of welcome/unwelcome, such as public policy related to immigration and multiculturalism in particularco, media representations of refugees, public institutions including education and government bodies (Gouin, 2016), and public opinion and discourse (Caidi & Allard, 2005; Hinger, Schäfer, & Pott, 2016). Consequently, outcomes achieved at the macro level have the potential to reinforce and stimulate change at the individual and community levels. Despite its critical role in supporting positive outcomes for refugee newcomers, research related to macro-level system components, and subsequently, refugee outcomes at this level, is limited compared with that of the other two levels.
Time Period in the Process of Change
At the individual level, refugee newcomers can be supported to move beyond their refugeeness following the process of change they are expected to experience over time in their new host country. There are four main time period categories represented in the figure: immediate claim process, immediate resettlement and settlement, intermediate adaptation, and ultimate integration and wellness. These periods are organized horizontally in the “Life Beyond Refuge” figure, progressing from left to right.
Immediate Claim Process
For some refugee newcomers, the claim process marks the initial stage of their engagement with Canada. The dotted circle on the left side of the figure represents intended outcomes for those who arrive to Canada seeking asylum as refugee claimants. The circle is dotted to indicate that this domain does not apply to all refugee newcomers to Canada as the majority come as resettled refugees through proactive refugee streams (i.e., private sponsorship, government assistance, or the smaller shared programs). Supporting activities at this stage are primarily intended to help refugee claimants through the claim process. For example, specific outcomes may include increased access to legal support (Wilkinson, Garcea, Bhattacharyya, Abdul-Karim, & Riziki, 2017; Yu, Ouellet, & Warmington, 2007), decreased wait times for claim decisions (Morantz, Rousseau, Banerji, Martin, & Heymann, 2013), and more just and efficient resolutions of claim applications (IRCC,2016). Experiencing positive outcomes in the claim process enables refugee claimants to begin the transition to resettling in their new host country as permanent residents. Despite the importance of this stage in the lives of many refugees, discussions about claimant outcomes are underrepresented in the literature; only three articles in the annotated bibliography identified outcomes associated with the claim process (IRCC,2016; Morantz et al.,2013; Yu et al.,2007).
Immediate Settlement and Resettlement
Proceeding right on the figure, all refugee newcomers who become permanent residents can be supported to begin resettlement in their new home country. Support is aimed at achieving immediate resettlement outcomes that are unique to the forced migration experience of refugees. These outcomes can be shared by all refugee newcomers to Canada whether they arrived as claimants or as resettled refugees via any of the proactive refugee streams. As refugee newcomers shift from a life of fleeing persecution or harm to one of resettlement, establishing a new sense of safety (Esses et al., 2010; Marks, 2014; Puma, Lichtenstein, & Stein, 2018), identity (Fantino & Colak, 2001; Kyriakides et al., 2018; Silove, 2013; Steimel, 2017), and agency (Atwell et al., 2009; Kyriakides et al., 2018; Steimel, 2017) and developing resilience after trauma (Broughton & Shields, 2020; Silove, 2013) are critical. Safety refers to feeling safe from physical harm as well as a psychological sense of economic, social, and physical security (Marks, 2014; Panter-Brick et al., 2018). In the resettlement phase, identity outcomes for refugee newcomers involve shifts away from being narrowly framed as a victim in need of rescuing to defining oneself according to the whole person, including their pre-conflict histories and post-refuge aspirations (Kyriakides et al., 2018). Agency describes refugee newcomers as having the “ability to act” by making decisions that shape their current and future circumstances (Kyriakides et al., 2018). As refugee newcomers begin to resettle, re-claiming their authority to make personal life decisions “confirm[s] their eligibility to exist beyond refuge” (Kyriakides, Bajjali, Mcluhan, & Anderson 2018, p. 70). Outcomes related to developing resilience after trauma include the ability to settle, adapt, and prosper despite personal experiences of trauma and ongoing post-traumatic responses (Broughton & Shields, 2020; Silove, 2013).
Activities supporting these outcome domains can foster positive immediate resettlement outcomes for refugee newcomers. A possible intended outcome of identity, for example, may be a decreased personal affinity with being a refugee as individuals develop a stronger association with becoming a permanent resident (Silove, 2013). While more literature discusses outcomes related to refugee resettlement than the claim process, data for resettlement outcome domains are also relatively limited.
Intermediate Adaptation
Continuing right on the figure, refugee newcomers can be supported in ways that are common for all newcomers to Canada, whether their migration was forced (as in the case of refugees) or voluntary (i.e., newcomers arriving via Canada’s economic or family classes). Corresponding outcomes include those dealing with immediate settlement as well as longer-term (intermediate) adaptation to a new host society. Here is where the majority of literature was found. Activities at this stage support newcomers to achieve outcomes related to receiving accurate and timely settlement information (Agrawal & Zeitouny, 2017; Dorman et al., 2017; Veronis, Tabler, & Ahmed, 2018), finding meaningful employment in their desired field (Canadian Index for Measuring Integration [CIMI],2017; Rioseco et al.,2017; Shields et al.,2016, 1999) securing adequate and affordable housing (CCR,2011; CIW,2016; Hanley et al.,2018) developing and improving language competency (Atwell et al.,2009, Hyndman,2011; IRCC,2019d), navigating shifting family roles and dynamics (Balaghi, Westdal, & Rispoli, 2017; English et al., 2017), achieving and maintaining standards of physical health (Agrawal & Zeitouny,2017; CCR,2011; Rioseco et al.,2017) and achieving and maintaining a positive state of mental health (Cheyne-Hazineh, 2020; Mitra & Hodes, 2019; Sirin, Plass, Homer, Vatanartiran, & Tsai, 2018). Additional literature supporting activities at this stage included accessing quality educational opportunities (Godin, English, Ochocka, & Janzen, 2017; Hyndman, 2011; Puma et al., 2018), building social connections within and across ethnic groups (Ager & Strang,2008; AAISA,2017; Beaman,2012) accessing public transportation options or acquiring means of transport (Woodgate et al., 2017), and attaining skills necessary for navigating a new cultural and social environment (Puma et al., 2018). Employment and language outcomes for newcomers are especially well documented. For example, numerous studies examine newcomers’ labour force participation rates (Bevelander & Pendakur,2014; Mulvey,2015; CIMI,2017; Wilkinson et al.,2017) or their employment satisfaction (AAISA, 2017; Jackson & Bauder,2014; Marks, 2014).
Integration and Wellness
Finally, the system theory of change suggests that the combination of individual-level, community-level, and macro-level supports are necessary if the ultimate outcomes of integration and wellness, identified on the far right of the figure, are to be attained. Newcomer integration refers to the process that immigrant and refugee newcomers undergo as they develop a sense of belonging and contribution to their new host country (IRCC,2019). In our system theory of change, the notion of integration was broadened to include life outcomes pursued by all Canadian residents, both newcomers and non-newcomers alike. In Canada, one common approach to assessing general life outcomes of individuals is through the construct known as well-being (CIW,2016). All residents, including refugee newcomers, can pursue wellness through 10 domains adapted from the Canadian Index of Wellbeing (2016). These include having a sense of belonging (CIMI,2017; Hanley et al.,2018; IRCC,2019d) engaging in democratic activities (CIMI,2017), becoming healthy people (Esses et al., 2010; Hansen, Maidment, & Ahmad, 2016; Hyman & Guruge, 2002), achieving and maintaining food security (CIC,2011; IRCC,2019d) accessing leisure and cultural activities (Nakeyar, Esses, & Reid, 2018; Shields et al., 2016), attaining an acceptable standard of living (Rioseco et al., 2017), determining personal use and designation of time, contributing to environmental goals, pursuing spirituality (Rioseco et al., 2017; Silove, 2013), and living independently in Canadian society (Im & Rosenberg, 2016; Stewart et al., 2012). Each of these outcome domains are further divided into specific indicators. For example, one indicator of democratic engagement includes volunteer involvement through advocacy or political groups (CIW,2016). Little research has been conducted to date on some of these outcomes such as time use and environment, while outcomes related to a sense of community or national belonging are more comprehensively documented in the literature (CIMI,2017; Hanley et al.,2018; IRCC,2019d).
DISCUSSION
Although Canada is a leader in welcoming refugees, the same level of intentionality does not exist for evaluating the policies, programs, and practices intended to support refugee newcomers in the process of their resettlement. In general, a system theory of change can be useful in addressing this gap. The “Life Beyond Refuge” figure illustrates the system theory of change for those at the heart of the system by charting refugee newcomers’ journey beyond their refugeeness across ecological levels and over time and identifying intended outcomes at each stage. Consequently, this framework holds potential to contextualize the evaluation of resettlement supports within a given host country. To begin, it provides a comprehensive yet flexible framework, capable of engaging interdisciplinary researchers and evaluators and useful for linking public policy with community-level practice. It is broad enough to incorporate and synthesize different dimensions of the resettlement experience, drawing from the various available indicators of immigrant settlement and integration (Esses et al., 2010; Jedwab & Soroka, 2014). Perhaps most importantly, as a system theory of change, it is useful in guiding assessments of complexity that seek to investigate lived experience and evaluate practice across multiple levels (Foster-Fishman & Long, 2009) and across locations (Janzen, Nelson, Hausfather, & Ochocka, 2007). In this section, we discuss implications of the system theory of change for systems actors, including policy implications at the macro level, practice implications at the community level, and interactions between the macro and community levels that can influence evaluation in the refugee-serving sector. We conclude with implications for further research, including gaps in the literature of refugee outcomes, as well as the current methodology applied to evaluating supports for refugee newcomers.
Implications for System Actors
Macro-Level System Actors
The system theory of change presented in this article addresses the current gap in the resettlement sector for a holistic framework for evaluating outcomes of refugee support. At the macro level, it provides a framework for system-level research and evaluation through public policy. The macro level in this context includes funders of all types that resource resettlement activities across civil society, including both government and non-governmental sources of funding. Without such a framework to guide policy in the refugee-serving sector, outcomes to be evaluated and associated indicators are often determined at the organizational level, where processes of evaluation and outcomes may vary widely (Thomson, 2010). Furthermore, even in cases where funders require common outcomes to guide evaluation within organizations, documentation and reporting remains piecemeal, allowing organizations to selectively determine what evaluation data they share and how the data are presented (Arvidson & Lyon, 2014). Without evidence generated and documented according to a consistent framework of conceptualizing and reporting outcomes, it becomes difficult to identify similarities or differences in outcomes across the sector, which, in turn, hampers efforts to develop or shift policy intended to support the resettlement of refugee newcomers.
A holistic framework makes it possible to establish a broadly cohesive yet particularly flexible lens through which to assess the collective impact of system components in supporting refugee resettlement. The system theory of change demonstrates that outcomes of each ecological level are interdependent (Schensul, 2009), which should be recognized by macro players as they create evaluation guidelines and build evaluation capacity to reflect this complexity. For example, national-level funders of resettlement support, such as IRCC in Canada, can apply the Life Beyond Refuge theory of change to guide and resource evaluations that collectively assesses outcomes at various levels (individual, community, macro). Such a common evaluation framework can help civil society to broadly gauge its performance nationally, regionally, and locally. Granted, there are limits to the macro-level utility that this system theory of change offers. We are not suggesting that it form the basis of a monolithic evaluation strategy that is standardized; rather, it can serve as a general framework for organizing and reporting on evaluative evidence. It can also guide system players in further examining and improving their practice in a targeted way, ultimately enhancing outcomes for refugee newcomers (Ahad, C, & Beirens, 2020; Beiser, 1999).
Community-Level System Actors
At the community level, the system theory of change can be applied to the evaluative practices of local service provider organizations and networks. In response to the sporadic and piecemeal approach that tends to characterize current evaluation practice in the refugee-serving sector (Ahad et al., 2020), the Life Beyond Refuge framework can act as a compass for planning and conducting multi-faceted assessments within local communities. Specifically, this system theory of change provides a useful menu of outcome options to guide community-based evaluation of individual programs, as well as inform the development of organization-specific or community-wide theories of change (Janzen et al., 2016). The Life Beyond Refuge framework provides structure by identifying outcomes common across the sector while simultaneously being adaptable to the unique circumstances of each local organization and initiative.
Including time periods and ecological levels as components of the framework also serves to influence evaluative practice. The time periods within the process of change highlight the similarities and differences that exist between refugees, other newcomers, and all residents of Canada, making it possible to align activities with target outcomes for a particular group or groups. By recognizing the three ecological levels that can impact refugee newcomers’ experiences of resettlement and the interconnection of these levels, the system theory of change also invites local players to consider the extent of their activities’ impact beyond their primary level of focus (Schensul, 2009). Since the majority of programs and practices tend to focus on addressing individual-level outcomes for refugee newcomers (IRCC,2016), the framework provides the opportunity to evaluate the broader scope of impact by incorporating and synthesizing community- or macro-level outcomes as well. In this way, the creation of real-time evidence makes it possible for local organizations and networks to view their role and location within the broader community context, responding to uncertainty in adaptive and innovative ways and identifying new opportunities to support refugee newcomers’ journeys beyond refuge (Janzen, Seskar-Hencic, Dildar, & Mcfadden, 2012; Suárez-Herrera, Springett, & Kagan, 2009). Similarly, creating more accurate evidence facilitates utilization of this evidence to adjust or improve the practices and programs of those supporting refugee newcomers.
Indeed, we have already witnessed the utility of this system theory of change in informing community-level evaluative practice. Participants of the aforementioned Evaluating Refugee Programs national capacity-building workshops drew on this system theory of change as a menu of options when developing their own localized theories of change in their community-based programs. Participants were able to consider what methodologies would be best to answer the questions they had about their particular theory of change, even while embedding their evaluation within the broader system theory of change. Similarly, we are using this system theory of change to inform the building of an evaluation framework for the private sponsorship of refugees in Canada, and as a reference within an evaluation toolkit for the Global Refugee Sponsorship Initiative (see CCBR, 2022, 2022). All three projects mentioned above were funded by IRCC and demonstrate how the system theory of change can be helpful in designing community-based evaluations and in linking these particular evaluations to the broader resettlement ecosystem.
Interaction of Macro- and Community-Level System Actors
When evaluating refugee-serving programs, there is often the perception of competing agendas between a community-based approach driven by the organization and a standardized assessment process mandated by funders. Community-based organizations face pressure to demonstrate evidence of success according to top-down, funder-driven measures that may be incongruous with their bottom-up, particular theory of change (Arvidson & Lyon, 2014; Liket, Rey-Garcia, & Maas, 2014). Evaluation evidence generated in these conditions runs the risk of lacking utility (Liket et al., 2014) and limiting the genuine assessment of collective impact across the refugee-serving sector. It is not surprising, then, that there is general agreement among practitioners (Sherrell, 2017) and federal policy-makers alike (Donaldson, 2017) about the pressing need for a system-wide, coordinated approach to evaluation.
The system theory of change offers a common framework that can act as a shared foundation for linking the macro and community approaches to evaluation. While the challenges of imposing policy-level evaluation frameworks across community-based settings have long been documented (Sehl, 2004), the Life Beyond Refuge theory of change provides a common language to describe outcomes for refugee newcomers and a shared starting point for conceptualizing integrated policy and programming responses. Instead of continual attempts to fit the square peg of local activities into the round hole of funders’ requirements, funders and civil society can work together to successfully implement their respective theories of change. Cross-sectoral collaboration in developing, implementing, and using evaluations shares responsibility for evaluation processes and outcomes, relieving funders of primary responsibility and empowering civil society to produce useful, verifiable evidence that can be used to improve practices and programs at the community level (Janzen et al., 2021). Such collaboration will require stakeholders to agree upon common core elements of a system-wide evaluation framework while also allowing flexibility for unique community-based elements (consultJanzen, Nelson, Trainor, & Ochocka, 2006, for an example outside of the resettlement field).
Adopting such a system-wide community-based approach to evaluation requires community-based organizations to possess a shared understanding of their common system theory of change and the capacity to conduct effective evaluations (Janzen et al., 2016). Recognizing the need for evaluation capacity building among community-based organizations in Canada (Lasby, 2018; Lasby, 2019), a multi-pronged strategy and corresponding resources have already been developed to provide expert guidance and enhance collective evaluation knowledge and skills across the refugee-serving sector (for examples, see http://www.eval4refugee.ca and http://www.evaluationcapacitynetwork.com). By building capacity for evaluation and adopting a shared approach to assessing and synthesizing theories of change at both the policy and practice levels through the Life Beyond Refuge theory of change, transformative cross-sectoral change becomes attainable.
The system theory of change can further be applied to research by operationalizing well-known theories of refugee integration. For instance, Ager and Strang’s (2008) Domains of Integration framework identifies and describes the key contributors to successful integration for refugees. According to their framework, rights and citizenship, safety and stability, language and cultural knowledge, and social connections are critical to refugees’ integration experiences, as well as for attaining employment, education, and health standards (referred to as “Markers and Means” in the framework). Given its expansive scope and inclusion of multiple domains that contribute to the integration experience, this framework has been widely applied to examining integration primarily from a policy lens (Phillimore, 2012; Puma et al., 2018) and in relation to programs and practices implemented at the community level (Cheyne-Hazineh, 2020; Yohani et al., 2019). Ager and Strang’s framework, however, offers less insight on how to achieve these domains of integration, leaving the application of their framework open to interpretation. Mapping the outcomes of the Life Beyond Refuge theory of change onto the Domains of Integration framework attaches outcomes to Ager and Strang’s corresponding domains, creating a more comprehensive understanding of integration in theory. The system theory of change can be applied to other theories of refugee integration as well, including the recently developed multidimensional integration model that focuses on the role and impact of receiving societies upon integration outcomes for refugee newcomers (Phillimore, 2021). In this way, the system theory of change is adaptable and can be applied to relevant theoretical frameworks as they emerge or evolve.
Implications for Further Research
The process of developing the Life Beyond Refuge system theory of change identified gaps in the literature’s current knowledge of outcomes for refugee newcomers and in current evaluation methodology. We address both of these gaps in turn below.
Gaps in Literature on Outcomes
The literature review revealed that documented outcomes to date primarily relate to the individual level of support for refugee newcomers (Agrawal & Zeitouny, 2017; CCR,2011; Marks,2014; Puma et al.,2018) with fewer articles describing community- and macro-level outcomes. This disproportionately overemphasizes some outcome domains and minimizes others. With little evidence of welcoming community attitudes and practices, for instance, or of systemic and societal influences, it is difficult to integrate these dimensions into local theories of change or resettlement assessments. Similarly, at the individual level, outcome evidence to date tends to be concentrated within specific time periods in the process of change. The literature review revealed that the bulk of studies at the individual level describe settlement outcomes that apply to all newcomers (AAISA,2017; English et al.,2017; Kwon & Lee,2018; Wilkinson et al.,2017). By contrast, fewer evaluations have examined the unique resettlement experiences of refugee newcomers, and even less pertain to refugee claimants. This gap produces an incomplete picture of outcomes related to the vulnerability in the refugee experience. Perhaps more importantly, it suggests a limitation in understanding how refugee newcomers can best be supported through that vulnerability in their new home country.
Clearly, more evidence is needed to fill the existing gaps in outcomes evidence. Informal groups, organizations, and networks across civil society can conduct more evaluations of underrepresented aspects of the system theory of change (i.e., claims process, resettlement outcomes, community-/macro-level outcomes) and share more widely existing evaluations that can contribute to knowledge of these aspects. Generating and mobilizing evaluation findings where currently few exist is essential to expanding understanding of effective refugee newcomer support for both policy and practice. With more evidence comes a greater depth and breadth of common outcomes and a clearer picture of the current state of supports offered to refugee newcomers (Ahad et al., 2020). In particular, studies that draw on socio-ecological models (Bronfenbrenner, 1977), that consider the multiple ecological layers of resettlement, would be of particular relevance. In sum, the literature review and system theory of change illustrate where more evaluative evidence is needed and how it can be applied to system-level improvements.
Gaps in Evaluation Methodology
From a methodological perspective, there is also a gap in understanding of current approaches to evaluating refugee supports. The literature review indicated that evaluation methods of policy, programs, and practices differ across the sector according to the unique circumstances of each study. To date, no comprehensive inventory exists to identify the various assessment strategies used in the refugee-serving sector, which limits knowledge and implementation of best practices and prevents improvements and coordination of evaluation efforts. Further investigation of both quantitative and qualitative assessment strategies is necessary to understand how evidence is gathered, which then can be shared broadly with relevant stakeholders. Compiling and sharing data-gathering methods used in evaluations of refugee-serving supports will boost collective awareness and understanding of evaluation in the sector and improve future evaluation processes and outcomes as a result. Finally, a limitation of this system theory of change is that it is primarily based on Canadian literature. Future studies could expand on or adapt this theory of change through an examination of evaluations from other countries.
CONCLUSION
This article is timely given that Canada’s recently released annual targets predict refugee arrivals increasing from 43,000 in 2018 to 61,000 by 2023 (IRCC,2017, 2020). These numbers signal a strategic federal policy shift towards sustained growth across all immigration classes, including refugees. As the number of refugee newcomers welcomed to Canada increases, policy, programs, and practices across the refugee-serving sector will have to expand proportional to the increasing need. In these circumstances, it becomes especially important to establish common outcomes and a guiding theory of change at the policy level, which can then be applied to practice among local organizations and networks at the community level. The Life Beyond Refuge system theory of change presented in this article provides both. With Canada as the world leader in refugee resettlement on a per capita basis (Hyndman et al., 2017), other nations are looking to its resettlement model (Levitz, 2016), even as global refugee flows are expected to increase due to climate change (Brown, 2008; Epule, Peng, & Lepage, 2015). In other words, this system theory of change has the potential to both inform Canada’s internal policy and practice and allow Canada to play a global leadership role in assessing refugee outcomes. If Canada becomes a leader not only in accepting refugee newcomers within its borders but also in its coordinated approach to evaluating resettlement initiatives, it can drive a global shift towards enhanced outcomes for refugee newcomers in all nations of their resettlement and, in turn, support their progress beyond refugeeness towards integration and well-being.
ABOUT THE AUTHORS
Rich Janzen is Executive Director at the Centre for Community Based Research, and Adjunct Associate Professor at Renison University College at the University of Waterloo. He can be reached at rich@communitybasedresearch.ca.
Mischa Taylor is a Research Coordinator at the University of Alberta. She can be reached at mischa@ualberta.ca.
Rebecca Gokiert, PhD, RPsych is a Professor in the School of Public Health and the Director of the Evaluation Capacity Network at the University of Alberta. She can be reached at rgokiert@ualberta.ca.
FUNDING
This work was supported by Immigration, Refugees, and Citizenship Canada (IRCC) under its Service Delivery Improvements program.
Abstract
Resumé
Main Text
INTRODUCTION
SYSTEM THEORY OF CHANGE
METHODOLOGY
RESULTS
Ecological Levels
Individual-Level Outcomes
Community-Level Outcomes
Macro-Level Outcomes
Time Period in the Process of Change
Immediate Claim Process
Immediate Settlement and Resettlement
Intermediate Adaptation
Integration and Wellness
DISCUSSION
Implications for System Actors
Macro-Level System Actors
Community-Level System Actors
Interaction of Macro- and Community-Level System Actors
Implications for Further Research
Gaps in Literature on Outcomes
Gaps in Evaluation Methodology
CONCLUSION
ABOUT THE AUTHORS
FUNDING