Executive Summary
Passport4Work (P4W) is rooted in the Dutch Brainport region, also known as the “silicon valley of the Netherlands”. In this region, there is a growth paradox. Continued economic growth coexists with a growing distance to the labour market and decreased employability for the lower-educated workforce. Stakeholders on both the demand and supply sides of labour have called for a structural change.
P4W addresses this call for a structural change by creating a solution which redesigns the journey to (re)employment for the lower educated workforce. To achieve this, an intersectoral transformation of the labour market is pursued through the development of a skills-based platform: Passport4Work.
In essence, P4W represents a highly personalized and gamified platform, through which individuals are guided in the development of their passport for work (by means of gamified assessment). Where appropriate, this process is assisted by job coaches (for instance, when digital literacy levels prevent autonomous use). This passport can then be used for job matching purposes by sharing it with prospective employers. P4W focuses heavily on soft skills (or: general skills, transversal skills; e.g. presenting or analyzing) to broaden the scope of search to different sectors of industry. These soft skills are measured through gamified assessment by incorporating game-elements in assessments.
For instance, the “active listening” skill is measured by means of an interactive roleplaying game, in which respondents have to evaluate the behaviour displayed by actors in a simulated scenario displayed on video. For the active listening skill, an interaction is displayed between a customer and a sales clerk. The respondent is prompted to evaluate aspects such as maintaining eye contact or interrupting the customer.
In P4W, the city of Eindhoven targets specific groups of beneficiaries, for which there is a high sense of urgency. These include those with a relative distance to the labour market in terms of their educational attainment, and employment history. These beneficiaries are actively involved in the design and use of the P4W instrument. Target beneficiaries are initially addressed in three sectors of industry that have a high regional significance: healthcare, construction and technology.
Ultimately, P4W seeks to create a highly personalized and gamified platform, through which intersectoral mobility can occur on the basis of skills. To achieve this, a lot of time and resources in the project period so far have been allocated to realizing a national skills language, and the creation of the Passport4Work platform, comprising game-based assessments and algorithm-based matching.
Passport4Work (P4W) is rooted in the Dutch Brainport region, also known as the “silicon valley of the Netherlands”. In this region, there is a growth paradox. Continued economic growth coexists with a growing distance to the labour market and decreased employability for the lower-educated workforce. Stakeholders on both the demand and supply sides of labour have called for a structural change.
P4W addresses this call for a structural change by creating a solution which redesigns the journey to (re)employment for the lower educated workforce. To achieve this, an intersectoral transformation of the labour market is pursued through the development of a skills-based platform: Passport4Work.
In essence, P4W represents a highly personalized and gamified platform, through which individuals are guided in the development of their passport for work (by means of gamified assessment). Where appropriate, this process is assisted by job coaches (for instance, when digital literacy levels prevent autonomous use). This passport can then be used for job matching purposes by sharing it with prospective employers. P4W focuses heavily on soft skills (or: general skills, transversal skills; e.g. presenting or analyzing) to broaden the scope of search to different sectors of industry. These soft skills are measured through gamified assessment by incorporating game-elements in assessments.
For instance, the “active listening” skill is measured by means of an interactive roleplaying game, in which respondents have to evaluate the behaviour displayed by actors in a simulated scenario displayed on video. For the active listening skill, an interaction is displayed between a customer and a sales clerk. The respondent is prompted to evaluate aspects such as maintaining eye contact or interrupting the customer.
In P4W, the city of Eindhoven targets specific groups of beneficiaries, for which there is a high sense of urgency. These include those with a relative distance to the labour market in terms of their educational attainment, and employment history. These beneficiaries are actively involved in the design and use of the P4W instrument. Target beneficiaries are initially addressed in three sectors of industry that have a high regional significance: healthcare, construction and technology.
Ultimately, P4W seeks to create a highly personalized and gamified platform, through which intersectoral mobility can occur on the basis of skills. To achieve this, a lot of time and resources in the project period so far have been allocated to realizing a national skills language, and the creation of the Passport4Work platform, comprising game-based assessments and algorithm-based matching.
The P4W project incorporates three lessons learned from previous, similar projects, such as Let’s Connect which ran from 2012 to 2015. Let’s Connect also took place in the Brainport region and shared P4W’s objective of making labour markets more transparent. Several lessons were drawn from this project. These lessons include: first of all, the use of strong incentive systems to encourage individuals to disclose information on their skills (and other relevant attributes); second, the need for a “passport for work” to be selective and structured by nature in terms of its contents, to ensure efficient matching procedures; and third, to maintain acceptable levels of validity and reliability of data on skills (to prevent matching on false grounds). Finally, the passport should be part of a broader platform, linking relevant stakeholders (jobseekers, employers, educational institutes) within one accessible ecosystem.
There are also three barriers for the impact of artificial impact on labour to consider. The first barrier represents the lack of high-quality data about the intricacies of an occupation and its underlying skills. Second, forecasting models are limited due to the uncertainty around technology’s impact on labor. Third, aggregate labour market data lacks regional specificity.
In P4W, the gamified assessment serves to promote the engagement levels of the target users (among which there are large differences in literacy levels). By incorporating principles of gaming through online role-playing games and neurogames, P4W seeks to differentiate itself from more classic, survey-based assessments by catering to motivational and incentive-based aspects of use. Previous research has shown a positive effect of gamification on these aspects. Furthermore, P4W heavily invests in psychometric validation of its tools, to ensure the reliability and validity of data for matching purposes. Finally, P4W is aiming for national as well as regional impact, through its contribution to the Dutch skills language CompetentNL.
Recently, a few significant milestones have been reached.
-
First of all, the national survey among employers and employees to validate the US skill taxonomy O*NET within the Dutch labour market context has been finalized. Highlights from the survey will be shared in an upcoming web article in December 2021. The data from the survey will serve as the cornerstore for the skill-based matching algorithm.
-
Second, while Passport4Work is primarily focused on jobseekers with a relative "distance" to the labour market, the project team has always been adamant about creating a platform which can be used for all jobseekers, regardless of educational attainment levels. As such, several focus group sessions have been held with university students to gauge their interest and collect their feedback. The results of these focus group sessions will also be shared in an upcoming web article (also in December 2021).
-
Third, Passport4Work has invested a lot of time and resources on the psychometric validation of its tool. In the early stages of the project, employers have been very vocal about their need for reliable, high-quality data, would they use Passport4Work for recruitment purposes. To ensure this, psychometric experts were involved in the project to provide guidance to the development of the game-based assessments. By adhering to academic principles, Passport4Work is striving to facilitate robust and evidence-based decision making. More on this in the upcoming Zoom-in webinar.
Passport4Work is setting up a scientific experiment to establish the impact of the Passport4Work platform for job seekers and employers.
Several outcomes and hypotheses have been defined to gauge the impact of Passport for Work for job matching purposes, and to test its asserted value.
The hypotheses include:
-
P4W use increases jobseekers' self-awareness in terms of skills and opportunities (Heijde & Van der Heijden, 2003)
-
P4W contributes to enhanced quality of the recruitment process in the (pre-)selection phase and hiring phase. Match quality will be established through person-job fit: does the matching vacancy suit the job seekers educational level, domain and/or competences
-
P4W increases efficiency of the recruitment process
-
The use of P4W tools increases the probability of a sustainable match with an employer
-
P4W contributes to reducing bias in recruitment procedures
These hypotheses will be tested in a longitudinal randomized experiment with a control group. The experiment group consist of jobseekers pursuing employment while using the P4W platform under the supervision of the Eindhoven municipality, whereas the control group will pursue employment in the traditional manner, without using the platform (also under supervision of the Eindhoven municipality). This will allow us to establish the "net effect" of P4W.
The hypotheses will be tested by means of a questionnaire pre and post-intervention, for both job seekers and employers. For the purpose of data triangulation, this questionnaire is supplemented by municipality’s administrative records on job seeker characteristics (such as employment history, welfare status, labour impeding factors), as well as interviews with job seekers and employers.
Implementation challenges
Leadership in a project of the magnitude of Passport4Work is complex, especially with a combination of 10 private and public organizations. This has required a constant juggling of priorities and expectations among the project partners. A key lesson learned here is to be as transparant as possible, both within the MUA as well as among the project partners. By frequently touching base with the project partners, especially those which are involved more in the peripheral sense, P4W managed to maintain its urgency across its partners (despite the barriers imposed by the pandemic). For instance, it has been difficult to get the upcoming scientific experiment off the ground, in part due to GDPR-issues and capacity constraints due to COVID. By acting open and pro-active on this, and involving all stakeholders in this challenge, the project team has secured the commitment of several project partners, who are all contributing towards this (as opposed to merely one partner managing this, which was the initial strategy). As such, the joint effort made has compensated for the unforeseen externalities (COVID and rigid GDPR legislation). Furthermore, openness is critical to maintain political support levels (which is an important prerequisite for the project's success, more on that in Challenge #3) and also for its recognizability, which is slowly but steadily increasing due to the timeliness and relevance of the jobs & skills topic.
As denoted in the previous journal, this challenge has caused some problems at the start (where the EU guidelines conflicted with increasingly flexible employment contracts, network organizations and the implications of running an innovative project ). To remedy this, Passport4Work has propagated a outcome-based procurement procedure among its partners, rather than one based on extensive specifications, to allow more for more freedom and agility towards the envisioned outcome of procured services. In the past year, this challenge has not manifested itself.
As mentioned in the first challenge, a project run by a municipality hinges on its political support. The MUA has a broad labour market agenda, spanning several projects and initiatives. As such, it is important to maintain the relevance and urgency across all levels of the MUA. On the other hand, it also provides opportunities. For instance, as mentioned there have been some difficulties getting experiments with target users up and running. One of the solutions (apart from involving other project partners than originally enviosned) has been to connect P4W to another regional initiative, with an overlapping target audience. By doing so, the outreach of P4W is increased, the other initiative and its beneficiaries benefit from the P4W platform, and the data collection has become richer. This would not have been possible without short ties and a strong network of involved actors and stakeholders with shared interests (but are otherwise often organized in silo's).
In the first half of the project, this was a big concern. On a day-to-day basis, it had proven to be impossible to maintain continuous points of contact across all project partners, with varying levels of responsibility and allocated resources. As such, a core project team was appointed to manage the project on a day-to-day basis. Other project partners have been involved, but at appropriate times rather than at a fixed frequency. For instance by organized thematic workshops or events (the last one being around the launch of the P4W prototype for employers).This decision has allowed for quicker decision-making, more targeted involvement of partners, and in the past year this has worked well for the project. As such, the urgency of the challenge has been reduced from high to normal.
In the previous journal, this challenge had a "easy" status, but due to unforeseen GDPR and COVID contraints, it has changed to medium/hard. The development of the administrative system, in which information on job seeker skill assessments and job matching is stored, has been impeded by on-going GDPR concerns and barriers. The monitoring and evaluation of the project, which in a large part comprises an elaborate impact analysis, is dependent on the realization of this tracking system. Furthermore, the storing of personal information in general, beyond the scope of monitoring and evaluation, is also subjected to GDPR-related scrutiny and applicable legislative protocols. The remediate this, external expertise has been sought by the project. While the project team is optimistic about the outcome, it has delayed the execution of the impact analysis. Another arising issue here has been the availability of participants (job seekers), which was greatly reduced by COVID due to lockdown measures. However, as mentioned in the first challenge, by combining efforts of several project partners (otherwise not involved), this has been solved.
Communication with the target beneficiaries has increased over the past year. COVID caused some hurdles here, with jobseekers under the supervision of the Eindhoven municipality not being able to visit the premises where coaching took place. As such, it was difficult to organize workshops around using the Passport4Work tool. However, several remote sessions took place, and in the past few months physical meetups started being organized again. This has resulted in several job seekers using Passport4Work, either remote or as part of a job coaching trajectory. This has resulted in detailed feedback on the tool and its user experience, informing future prototype versions. In addition, focus group sessions were held with university students to ascertain the utility of P4W for a broader target group. In both cases, this has resulted in positive anecdotal evidence on the perceived value of Passport4Work. For example, it was reported that the tool has a positive effect on the self-worth and motivation of previously discouraged jobseekers. University students were also keen to use the tool, with the difficulty of finding an appropriate job upon employment being a shared concern among the group. The guidance the tool provides in painting a systematic picture of one's skills and personality was found to be extremely useful in this regard.
Other than targeted end users, communication towards prospective employers and policy makers has also increased. The project team was present at the largest HR conference in The Netherlands, to demonstrate the employer interface of the tool, and its underlying psychometric principles. This was a kick-off of the activation campaign, which is continuing over the remaining period of the project. In the coming weeks, a two-part webinar is being released about skills based matching, and the implications for employers.
To ensure the sustainability of the project and its impact, the project team has decided to adopt a double-sided strategy. On the one hand, completing the technical development and scientific testing of the tool. In parallel, P4W is heavily investing in embedding its outputs in existing, broader initiatives. For instance, the results of the national O*NET survey will be shared with the national skills language initiative "CompetentNL", as well as an upcoming data exchange standard, which exists alongside CompetentNL. The team is in talks with both organizations about the implications thereof. Questions that are being addressed include: who is hosting and owning this data, how do we make sure it remains up-to-date, how can we upscale this skill validation activity to other sectors of industry, beyond the three that are central to P4W? Other matters related to upscaling include the management of GDPR legislation, which is proving to be a major obstacle.
About this resource
The Urban Innovative Actions (UIA) is a European Union initiative that provided funding to urban areas across Europe to test new and unproven solutions to urban challenges. The initiative had a total ERDF budget of €372 million for 2014-2020.
Similar content
