Sampling Techniques in Survey Construction and Data Analysis: Strengthening Validity and Generalizability in Statistical Procedures

Abstract

Librarians around the world are proudly user-focused in making decisions about collections, services, and programs for the patrons that visit our physical libraries and our web-based collections. Assessment activities are a critical aspect of the work we perform, and most of us have some experience with managing surveys and interpreting findings. Conducting surveys is one of the most common ways we collect data on patron needs, through directly mailed letters and through links we provide to surveys on our web sites and in emails to groups. But do we always know whether our approaches are strong enough to make effective decisions based on the results of our surveys? The goals of this presentation are to provide guidance on best practices for different types of sampling--ranging from convenience sampling to random sampling to stratified sampling--in order to strengthen the validity of findings and create a path for generalizability to the populations we serve. Issues that arise from different types of sampling techniques, such as confirmation bias and maturation effects, will be discussed with suggestions on how to limit their influence on respondents. In order to estimate the strength of the sample, statistical techniques such as Confidence Levels (CL) and Confidence Intervals (CI) will also be discussed with hands on examples of how to compute them. As a way to increase the reliability of the surveys we produce, models such as Inter-rater Reliability (IR) and Intraclass Correlation Coefficient (ICC) Analysis will...