Identity
City

Brussels Capital Region

Region

Belgium

Country

Belgium

About CALICO

pic

Care and Living in Community (CALICO)’ aims to provide affordable housing in Brussels, with a focus on vulnerable groups in the housing market (such as older adults, women and people with a migratory background) and improving their quality of life, agency and empowerment. The project started in November 2018 and will end in October 2021. At the time of writing this case study, the project is in its third year of implementation.

As a pilot project, apart from providing housing, CALICO also aims to develop a community-based model of care in which informal and self-care practices innovatively interact with professional care services. Analysis of the impact of this model, as well as providing insights and evidence to contribute towards governance and policy adjustments, is also clearly stated as an objective of the project.

The core activity of CALICO is the construction of 34 housing units and the establishment of a Community Land Trust (CLT), which provides an innovative model of community-managed housing. This model ensures the financial accessibility of housing through separating ownership of land from ownership of buildings and providing measures to limit the speculative resale of assets. Historically, CLTs have been used in the United States and the United Kingdom, but they are an innovative practice in continental Europe, including Belgium. Importantly, CALICO focuses not only on the organisational and legal foundations of the CLT, but also on the community dimension of its inhabitants’ lives. This is obtained by including inhabitants in estate management, and providing tools for deliberation and physical spaces for interaction (common and service areas).

CALICO is also an experiment in creating a ‘Community Care Model’, in which people interact at different liminal stages of life – birth, old age and the end of life. This addresses the ‘hyper-specialisation’ and ‘hyper-sectoralisation’ of care in Belgium and is directed towards organising care provided within neighbourhoods by its members, instead of institutions. In practice, this communal dimension will be visible after the housing is constructed (i.e. after the period of UIA funding), however its foundations are being planned for earlier, during the design of shared spaces within the investment and the laying out of expectations and guidelines for community governance. In total, eight partners with various background are involved in project implementation.

This is a case study as part of an UIA report. You can access all of the project's resources on its project collection page.

Evaluation governance

Responsibility for the monitoring and evaluation of CALICO was given to expert academic institutions – research groups for the Belgian Ageing Studies and Cosmopolis from the Vrije Universiteit Brussel (VUB). In terms of project management, monitoring and evaluation were structured as a separate work package. However, as the project was co-created by partners who had cooperated on previous initiatives, monitoring and evaluation were designed in a cooperative manner. Partners wanted to be involved from the beginning and expressed the need for usefulness of insights and experiences stemming from CALICO for their broader actions outside of the project itself.

All the partners […] see this as something new, so they can really learn a lot from this project. And they really wanted to learn and to know what will come out, what will go well and wrong […] Not only to realise the project, but really learn from the project as well.

Source: CALICO project hearing

Group interviews with project partners were organised at an early stage of implementation, providing the opportunity for each to share their motives, perceived challenges, and expectations for the results of the monitoring and evaluation. Consequently, research was structured to address stakeholders’ information needs, and expectations were consensually managed. During the interviews, for example, the need for attention to both ‘factual’, number-based indicators, as well as the gathering of personal stories through adequate, qualitative research techniques was stressed.

The interviews also led to a common understanding among project partners that social impact will take longer to occur than the funding lifetime of the project. Outcomes over the next five years should therefore be taken into account, and some may prove unexpected and difficult to grasp. In cooperation with stakeholders, the initial monitoring and evaluation plan (outlined in the application form) was refined and detailed. Importantly, a thorough list of research questions, covering a broad span of topics and assigned to monitoring and evaluation types, target groups and data collection methods, was created.

Because CALICO uses a participatory social action research methodology, feedback and learning loops are intrinsically tied into monitoring and evaluation and project management. The action researcher does not function as an external observer and assessor, rather as an active participant in various activities (such as committee meetings), providing input and advocating for various actions and groups – especially future inhabitants of the housing.

Participatory approach

Text

Participatory evaluation is an approach that involves the stakeholders of a programme or policy in the evaluation process. This involvement can occur at any stage of the evaluation process, from the evaluation design to the data collection and analysis and the reporting of the study.

Source: Better evaluation

“Participatory research is giving power from the researcher to research participants. The research participants and researchers together control the research agenda, process and actions.”

Source: Dawance, T., Smetcoren, A., Ryckewaert, M.,  Aernouts, N.,  De Donder, L.,  Care and Living in Community, CALICO. Groundwork for evaluation and state-of-play, 2019.

General approach

CALICO employed a participatory model in both project implementation and monitoring and evaluation activities. In terms of monitoring and evaluation, this meant adopting a participatory social action methodology through several guiding principles:

  • The inclusion of participants (stakeholders and target group representatives) alongside researchers in the design, conducting and assessment of research results;
  • A focus on delivering practical solutions relevant to project objectives – in this case, improving the living conditions of inhabitants;
  • Openness to diverse audiences and representatives from groups outside of the ‘usual suspects’ and attention paid to the role of power relationships amongst groups and actors;
  • Governance of the monitoring and evaluation process using deliberation and sociocracy (consent-based decision making);
  • Appreciative inquiry, which means a focus on strengths and opportunities instead of problems and pitfalls.

In CALICO, we chose to do participatory action research because it is not just that you do research, have results, present them and leave it there. Action research allows you, as a researcher, to also push for action.

Source: CALICO project hearing

This approach was useful in the case of CALICO, particularly as it is aimed at addressing vulnerable groups’ needs. The participatory approach strengthens the agency and empowerment of participants, which is line with the overall goals of the project.

By design, research was focused on the results and impact of the project, as well as its processes. Thus, tangible effects were considered and their measurement planned, but means to capture what happened during project implementation –  i.e. how those effects were obtained –  were also planned for. Such a design is especially important in innovative projects.

The CALICO team formulated an extensive list of research questions related to impact evaluation and process evaluation, divided into four categories:

  • Development and realisations;
  • Positive impact;
  • Sustainability and future;
  • Gender equity and inclusion of older people.

You can find the list of all research questions in the report, which was prepared at the initial stage of project implementation.

Appreciative inquiry

Text

Appreciative inquiry is a research perspective that focuses on strengths, both of organisations and individuals. It intends to discover, understand and foster innovations in social-organisational arrangements and processes. As an approach, it is a response to negative deficit approaches, which primarily focus on defining the problems.

Source: Dawance, T., Smetcoren, A., Ryckewaert, M.,  Aernouts, N.,  De Donder, L.,  Care and Living in Community, CALICO. Groundwork for evaluation and state-of-play, 2019, p. 57.  

To learn more about this approach, you can consult e.g.:

CALICO decided to use both quantitative and qualitative research methods for data collection:

  • Standardised questionnaire surveys addressed to all residents of the co-housing schemes, designed as a longitudinal study conducted at two points near the beginning and end of the project. The questionnaire consisted of questions on loneliness, subjective wellbeing, quality of life and housing. Standardised questions and scales widely used by institutions (academic researchers, the World Health Organization) were used to provide data comparability.
  • Data from project monitoring, including the number of participants in various sessions and meetings, and databases created by project partners with information on residents.
  • The analysis of documents created during the project, such as financial documentation, implementation reports and other supplementary material.
  • Focus group interviews with representatives of the project partners, conducted in two series at the beginning of the project and near its end.
  • Individual interviews with representatives of key stakeholder groups (CALICO residents, residents of neighbouring real estate complexes, community professionals and policymakers).
  • Participation of researchers in the various committees and working groups set up in the project.

Overall, a carefully conceptualised monitoring and evaluation framework, based on mixed-methods research, was considered by the project team as the best guarantee for providing quality information and useful insights.

We chose a combination of multi-scale (resident, community, partnership, policies) research design with the use of mixed (quantitative and qualitative) data collection methods, which ensures a broad data collection but also in-depth research on various topics of the project and among various groups.

Source: CALICO project hearing

At the time of preparing this case study, CALICO is in the project implementation stage. As a result, not all the planned research activities have been conducted. Focus group interviews with project partners were held at an early stage with the objective of identifying the motivations, expectations and specific needs of each organisation in relation to the evaluation activities. These discussions provided important input for refinement of the monitoring and evaluation design. The researchers also participated in meetings of project committees from the outset, engaging in the co-creation of the project in line with the action research assumptions – not only collecting data, but also contributing to activities and sharing their expertise.

[The evaluator] is giving as well. We’re presenting things. We are sharing. If there’s a question, we’re not just observing and looking at who is giving answers, then looking at the dynamics, but also participating.

Source: CALICO project hearing

The project also presents an inspiring case wherein research benefits from project partners’ specific capacities. Although not foreseen at the stage of application, representatives of important stakeholders were included in the research (i.e. neighbours living in close proximity to the future CALICO buildings). To this end, students of urban and/or sociology studies at ULB (Université Libre de Bruxelles) were invited to act as interviewers. Importantly, not only were they involved as individuals conducting interviews, but they also engaged more deeply with preparation beforehand. After conducting interviews, a platform for critical reflection was also created; students could provide their insights through an innovative workshop-based method of roleplaying, in which they represented different groups of people. Thus, not only was good quality data gathered, but also valuable input for project implementation and communication.

To some degree, CALICO was negatively impacted by the COVID-19 pandemic, which hindered important project actions and data collection rounds. Although one of the project partners was able to conduct activities aimed at setting up the community, the other two partners experienced interruptions and barriers to organising meetings. Despite these challenges, the selection of future residents was only slightly delayed and was finalised by June 2020. Moreover, the research team were able to collect data despite lockdown measures, and the first rounds of questionnaire surveys and individual interviews were held. As most interactions had to be held online, however, rather than in person, it was more challenging to build trust and relationships with residents.

One significant challenge to data collection which the CALICO team were aware of from the beginning was the limited opportunity to collect data about the impact of the project on its target groups. Since a major action of CALICO was to build the housing – a time-consuming process prone to delays – the risk of not being able to conduct the second wave survey (post-intervention) within the planned timeframe was clearly visible. As the project has not ended at the time of writing this case study, it remains to be seen if residents will have moved to the housing when the second wave survey is conducted.

While this may be a drawback in terms of grasping the initial impact of the project, it has to be stressed that only a study conducted after a considerable amount of time could provide information on long-term effects. CALICO addressed this problem by planning to provide research tools for project partners, which will be used after the period of UIA funding has ended. Impact is also measured and evaluated by other, qualitative research techniques. In fact, even if the second wave survey is conducted before the residents move in, the effects of their participation in community planning and design could be grasped.

We do recognise the limitation of the quantitative part in measuring impact because people won’t have moved, but I wouldn’t go as far as to say that we will not be able to measure any impact […] I think also in the qualitative interviews, we will be measuring impact […] In the community building we can find some changes already.

Source: CALICO project hearing

The research scheme is summarised in the table below:

Quantitative research

Qualitative research

Standardised questionnaire surveys conducted with (future) residents at two points at the beginning and end of the project 2 series of 5 focus group interviews conducted with the project partners (Bruxelles Logement, CLTB, Angela.D, Pass-ages, EVA bxl), carried out at the initial stage and at the end of the project
Routine monitoring of activities and participation 2 series of 9 individual interviews with residents, carried out at the initial stage and at the end of the project
  1 focus group interview with local and regional governance partners, and 2 individual interviews with EU policymakers
  38 interviews with residents of real estate complexes in the neighbourhood of CALICO
  5 interviews with volunteers and professionals involved in the development of the ‘community care model’ (e.g. birth and end-of-life facility)
  Participation of researchers in various project implementation committees
  Analysis of documents created during the project

Mixed-methods approach

Text

Mixed-methods approach can be defined as “research in which the investigator collects and analyzes data, integrates the findings, and draws inferences using both qualitative and quantitative approaches or methods in a single study or a program of inquiry”.

Source: Tashakkori, A., Creswell, J.W., “Editorial: the new era of mixed methods”, J Mixed Methods Res 1: 3–7, 2007.

Horizontal issues

Participation was a prevalent theme in CALICO, as most project actions were based on the ideas of co-creation, co-construction, co-production and co-design by stakeholders and future residents of the housing. The monitoring and evaluation design stemmed from the participatory action research approach, which means involving stakeholders and target groups in the conceptualisation, methodology development, implementation of the research, data collection and assessment. In fact, such an approach goes a little further – affirming the influence of the researchers on the subject of research and research participants (in this case, the CALICO project and future residents of the constructed housing). Reciprocity of the relations between actors (researchers, stakeholders and target groups) is one of the key premises of the participatory social action methodologies. In CALICO, this materialised in holding regular meetings on different levels and organising various project committees in which the researcher is not just a passive observer, but an active participant.

Among the multiple topics included in the research questions were gender equity and the inclusion of older people. Since improving actions related to these issues was one of the goals of CALICO, the questions addressed how equity and inclusion were achieved, and to what extent they influenced other impacts and outcomes. Attention was also paid to identifying success factors and obstacles experienced by project stakeholders while working towards equality and inclusion.

It is also important to note that CALICO was a pilot project aimed at testing solutions for housing governance models, addressing vulnerable groups’ needs and community-based care. One of the monitoring and evaluation objectives was to gather knowledge about the project’s processes, providing insight into CLT management, evidence and recommendations for public policy. This impact was planned for, and political advocacy was initiated early, instead of waiting until the project had ended.

Lessons learnt

CALICO is a strong case for employing participatory social action research in innovative urban projects. It also exemplifies how careful planning of monitoring and evaluation activities benefits implementation, as well as capturing multilevel effects and impacts. The approach used in CALICO is particularly transferrable to projects related to social issues and vulnerable target groups. In this case, monitoring and evaluation activities can strengthen inclusion and empowerment. The highlights from our case study are as follows:

  • Monitoring and evaluation benefit from access to expert institutions and engaged personnel (in this case, academia) with domain-specific and methodological knowledge (CLTs and participatory action research, respectively).
  • As project outcomes – here, for example, the improvement of residents’ quality of life through the provision of housing – are difficult to observe during implementation, the feasibility of gathering meaningful measurements of impact might seem questionable. This challenge can be addressed by preparing tools for project partners to assess impact after the period of funding.
  • It is important to consider not only impact and outcomes, but also the processes involved in monitoring and evaluation. As one of the project team put it, in the case of CALICO “bricks are one thing, but community is what it’s about”. CALICO was not just construction work or physical housing space – it was also building interpersonal relations and creating the foundations for neighbourhood cooperation and co-governance. This dimension is represented by a number of activities in the project, and the whole process is closely monitored.
  • The more engaged the researchers are in active participation in project activities, the more challenging it may be for them to stay objective in their assessment. It may be relatively easier for the evaluation team to act only as observers with an external point of view towards the project.
  • While active participation of the researcher is potentially beneficial for project actions and outcomes, as the person can share expertise, voice stakeholders’ views, and drive change, it may be demanding for them to stay objective.

The direct participation of the researchers in the co-creation process […] They contribute by bringing elements of objective contextualisation obtained through their data collection and analysis and their scientific knowledge.

Source: CALICO project hearing

Text

brochures

Want to replicate this urban practice in your city?
Explore the possibilities for replication offered by EUI

About this resource

Report
Location
Brussels Capital Region, Belgium
About UIA
Urban Innovative Actions
Programme/Initiative
2014-2020
#SCEWC24 treasure hunt:
Reach the next level --> explore this page and find the button "Climate Adaptation", hidden in the "Green" part.

Then, you have to find an "Urban practice" located in Paris. 

 

The Urban Innovative Actions (UIA) is a European Union initiative that provided funding to urban areas across Europe to test new and unproven solutions to urban challenges. The initiative had a total ERDF budget of €372 million for 2014-2020.

Go to profile
More content from UIA
1173 resources
See all

Similar content