PROJECT | NICKEL
CLIENT | AGE-WELL NCE
ROLE | GRADUATE RESEARCHER
DURATION | 3 YEARS AND ONGOING
Developed by me to help novice researchers in determining the right needs elicitation methods based on participant capabilities and limitations. I am the sole researcher in this project and was given the task of identifying research gaps, user needs gathering, analysis, prototyping and usability testing.
Despite the increased recognition of the importance of designing for a wide audience, older and disabled adults are often left out in the design process beginning from the needs elicitation process . There are various reasons why designers/researchers do not include older and disabled adults in needs elicitation process including
Limited established inclusive User Centered Needs Elicitation Methods (UCNEM) available.
Lack of resources to conduct research.
Designers design for themselves assuming all end users are similar to them.
Designers may lack the knowledge or experience to choose the right needs elicitation methodology for their project and participants .
Older adults may have a varying and diverse set of human capabilities such as cognition, physical and perceptual abilities that are different from younger users. This may make it difficult for designer/researchers to determine a method that can be used this population. As a result, older adults have generally been excluded from the development process, particularly of assistive technologies, or are being consulted at the end of the design process . Despite the importance of human capabilities in the needs gathering process for older adults, the traditional requirements elicitation process only takes factors such as problem statement, project domain and final customers into consideration when UCNEM are selected. The inclusion of human capability demands from the participants to accommodate them in the UCNEM is often overlooked.
To address this issue, a new tool, called NICKEL, has been developed with the goal of providing a more inclusive and user-centered needs elicitation methods decision and selection process for designers based on the prospective participant’s cognitive, motor and perceptual abilities.
While there are numerous gaps found in the outcomes of the initial literature research my thesis will focus on the following:
There appears to be a lack of focus of HF in the needs gathering phase of the UCD/PDP process.
There appears to be a lack of experience among designers to determine which UCNEM fits with specific abilities of older adults and/or people with disabilities.
It appears that participant capabilities are not considered as factors in determining UCNEM for research or design studies.
The goal of this research is to develop and evaluate a decision support tool for matching human capabilities of potential users with a UCNEM. The target users for such a tool are novice and experienced designers or developers of assistive and/or inclusive mainstream technologies and products. So the following questions were answered in the research:
What are the level of human capabilities needed to participate in the different UCNEM in terms of visual and hearing abilities, motor skills and cognitive abilities?
How can the knowledge on the level of human capabilities in terms of visual and hearing abilities, motor skills and cognitive impairment be used to develop a tool that assists in determining the UCNEM?
Is the Needs Elicitation Methods Compatible with Key End User Limitations (NICKEL) tool an effective, viable and usable tool for novice designers, developers, and researchers to determine UCNEM methods compatible with older/disabled participants?
Stage 1 | Initial Literature Research
Understanding the usage of Design methods by designers with older adults
As my knowledge within the realms of design methods was insufficient when I started the research, I created a study plan to help me understand,
the different user centered needs elicitation methods available in design research.
how older adults are involved as stakeholders in different needs elicitation methods.
the influence of human factors in UCNEM.
factors used in existing requirements gathering methods selection tools to determine UCNEM.
I started out by conducted a systematic literature research using online resources, journal research publications, patents to understand the importance of UCNEM, different needs elicitation methods and negligence of older adults in the design community. Then I proceeded to conduct research on the reasons, why older and disabled adults are neglected in the design process. Only limited literature is found where UCNEM includes older and/or disabled adults, which indicates that less effort has been made to involve older and/or disabled adults in the design process. After a systematic literature review of 30 research articles, I funneled it down to the following three major reasons:
To create NICKEL as a tool, a database is required with all UCNEM and the capabilities required by stakeholders to participate in each method. Based on the initial literature review there are is no information on the different methods and the capabilities it demands from older and disabled adults. Therefore, I created a survey to gather data on the expert perspective in the capabilities required by older and/or disabled adults to participate in UCNEM. But there were statistical insignificance in some responses in the survey. The survey result findings were published in the ACE-ODAM international conference in Banff, 2018.
Since there was statistical insignificance in the survey results, a focus group was conducted to gather the missing data. Based on the data collected a database was created which formed the base for the NICKEL tool.
Stage 2 | Define
Synthesizing user findings
From my research findings, I identified two primary personas - an novice researchers working with older and disabled adults and a student taking design courses. I created persona profiles and empathy maps based on collective analysis of participants inputs from user surveys, focus groups and secondary research data.
Based on the personas, I was able to understand the need for a tool in the decision making of the right UCNEM.
Stage 3 | Ideate
I created a workflow based on which the tool will be working.
Stage 4 | Prototyping
Based on the database and task flow, I created algorithms and formulas on how each task flow must work and created an excel based tool to make sure the work flow I designed is as expected and satisfies user requirement. The google sheet with the formulas and the tool workflow can be found at A screeshot of the tool workflow is provided below
I created a working high fidelity prototype of the designed screens using Mockplus. The task flow and screen interaction are as given below
Stage 5 | Tool Evaluation
A user study was carried out to evaluate NICKEL for usability and usefulness. Qualitative and quantitative data were collected using pre and post study questionnaire, and a think aloud protocol logging the interactions with NICKEL by participants. Quantitative and qualitative methods were also used to analyze the data with the level of significance set to p<0.05 for statistical analyses.
The pre-study questionnaire consisted of five questions to capture data on gender, the primary area of interest or discipline, the years of experience in the area of design or research interest or discipline, and in conducting needs gathering studies.
There were a total of 24 questions in the post study questionnaire divided into 8 sections.
The first section captured the participant’s ID (each participant were assigned a unique ID number at the beginning of the study).
The second section consisted of 10 questions that captured the usability characteristics of NICKEL using the standard system usability scale (SUS). Each question in the SUS used a 5-point Likert scale format. The usability characteristics included learnability, efficiency, memorability, errors and satisfaction, which were the five primary components of usability.
The third section of the questionnaire captured the user’s reaction to using NICKEL using a 5-point Likert scale.
Sections 4 and 5 has two questions for participants to rate how helpful NICKEL was in determining UCNEM for older and disabled participants on 5-point Likert scales. Through the literature review, it was identified that designers and engineers did not consider HF in determining which UCNEMs fit best with their participants. The NICKEL user study participants were thus asked to give their opinion on the importance of considering HF in determining a relevant UCNEM after using NICKEL in the user study.
Section 6 has two questions, in which participants were asked whether they would use NICKEL in the future for their own work or research in a single question.
In section 7, four questions were asked where participants were asked whether there was agreement between the classification of the visual, hearing, cognition and physical ability levels provided in the NICKEL user manual and the study participant’s understanding of the levels of human capabilities.
Section 8 has four open ended questions which asks about the likes, dislikes of NICKEL, recommendation of any additional HF criteria to be added, and any recommendation in general.
Think Aloud Protocol
A think aloud protocol approach was used to evaluate the tool, where users where asked to think aloud as they were using the tool. All the study sessions were both audio and video recorded to understand where exactly the users stuck or faced problems when using NICKEL.
The survey data was subjected to descriptive and inferential statistical analysis and the qualitative video data was transcribed and subjected to thematic analysis. Two coders was used to code the video transcription to make sure the themes generated are reliable and also passed the ICC analysis.
Future and More
I submitted a grant to the National Centre of Excellence, Canada and secured funding to commercialize the tool. Work is being done to transform the tool into a web/mobile version and license it in SAAS model. There was also a few bugs identified when using the tool and is being fixed now.