Lifelong Learning Assessment

The results below summarize information on the Writing Assessment as reported in the Student Learning Section of the WASC EER Report (PDF) (pages 15-16).
The report also contains all supplemental data, charts and appendixes.

Background

The ULO Project on Lifelong Learning began in Spring 2010, when Kennedy Library conducted a survey of student information skills in consultation with the ULO Lifelong Learning Committee. Information skills are a foundational component of lifelong learning, and they contribute to other ULOs including written and oral communication.

Method

The survey was designed to identify student competencies by measuring performance on the Information Literacy Learning Objectives, which the library established in 2009. The survey presented students with a research scenario and asked them to respond to a series of 20 questions. Two versions were administered during a one-month period: one for lower-division and one for upper-division students. The versions differed by the order in which questions were asked and the wording of some questions. Invitations to participate were emailed to 1,332 lower-division and 2,905 upper-division students. In addition, an open invitation was posted on the library website, and instructors who had previously brought students for library instruction were encouraged to announce the survey to current students. Approximately 98% of the responses came from the email invitations. Without adjusting for the remaining 2%, the lower-division response rate was 28% (367 respondents) and the upper-division response rate was 20% (578 respondents). The high response rate likely resulted from the promise of cash prizes; however, not all respondents answered all questions.

Results

Figure 1.10 (PDF) (page 62) presents the mean scores in terms of percent correct for five questions for which there was a single response. A statistical analysis was conducted to determine whether the correct response to each item was related to Class Level and Instruction; the latter factor distinguished between students who had and had not received library instruction in research methods. In all cases, upper-division students did better than lower-division students. For three of the five items—thesis statement/promising research question, correct identification of citation example, and correct selection of the search term that would yield the fewest results—Class Level had a significant effect, demonstrating value added. There was a marginal effect of Class Level on the correct selection of the search term that would yield the most results. Significant effects of Instruction were found for the thesis statement and correct identification of the citation example. The question on the ethical use of ideas showed no significant effects of either Class Level or Instruction. Across all analyses, no significant interactions between variables were present (see Appendix 1.1 for full statistical analysis. The results demonstrate value added across several items on the survey, indicating higher levels of information literacy at the upper-division level. In addition, promising results for the educational effectiveness of library-related instruction were also found, with some indication that lower-division students attending such instruction consistently scored almost as well as upper-division students who had not attended such sessions. It should be noted that the outcomes measured in this scenario-based questionnaire necessarily focused on the means of finding and identifying information rather than on the more complex evaluative and synthetic skills associated with the critical-thinking aspects of information literacy.

Future Plans

The library plans to re-administer the information literacy survey in Spring 2012 to provide more and better data about student learning as a function of Library Instruction and Class Level. When revising the survey, more attention will be paid to the planned analysis, making sure that the upper- and lower-division questions are directly comparable.

Background, Definition, Assessment Goals and Timeline of Activities from 2008-11

Definition of Lifelong Learning

Lifelong Learning (LLL) is comprised of the foundational attributes that enable graduates to navigate a constantly changing world in which success is highly contingent upon the continuous intake, evaluation, and deployment of information. Lifelong learning is comprised of formal [1], non-formal [2], and informal [3] learning, whether intentional or unanticipated, that occurs at any point after a student has graduated. Many attributes characterize the lifelong learner, and these are best developed during the undergraduate and graduate years.

A Lifelong Learner is One Who:

Takes responsibility for learning and personal and/or professional development
(Attribute 1 - Personal and/or Professional Development Learning Outcomes)
Effectively finds, critically evaluates, and formulates new ways of understanding information
( Attribute 2 - Critical Thinking Learning Outcomes)

Resources for Faculty and Staff:

_________________________________________________________________________________________

Abstract of Plan for Assessment:

Upon the development of the lifelong learning document (2008/09), it was decided to focus on the second attribute – to help students learn the information and critical thinking skills.

Beginning: Integrate the basic information and critical thinking skills (Attribute 2) into the General Education (GE) Area A: Communication A1 – A3 lower-division course series. This was implemented in 2009, and is currently a practice in GE A1, A2, and A3 courses.
Developing: Integrate the intermediate- level information and critical thinking skills into the GE Area C: Arts and Humanities (C1-C3) lower-division course series, the GE Area D: Society and the Individual D1-D4 lower-division course series, and selected major courses. This process has been informally implemented in a few classes, and plans are being developed to formally integrate in all of the GE Area C and D classes, as well as specific major courses.
Mastery-Level: Integrate the advanced level information and critical thinking skills into GE upper-division Arts and Humanities C4 synthesis courses, the GE upper-division Society and the Individual D5 synthesis courses, and Senior Project courses. This integration is presently occurring in the Senior Project courses. This process has been informally implemented in a few classes, and plans are being developed to formally integrate in all of the GE Area C and D upper-division courses, as well as specific major courses.

Assessment Methods:

Administer online surveys to determine the effectiveness of the information and critical thinking skills taught through online and classroom instruction.
Seek feedback from the faculty to assess student's understanding and use of information resources and critical thinking in their class research project(s).
Administer both lower and upper- division scenario-based assessments to determine the level of student information and critical thinking skills.

Time Line/History of Assessment Activities

Year One (2008-09)

Assembled a committee to define lifelong, develop the learning outcomes, and the lifelong learning rubric

Year Two (2009-10)

Integrated the basic information and critical thinking skills (attribute 2) into GE Area A: Communication A1 & A3 lower-division courses
Administered online surveys to determine the effectiveness of the information and critical thinking skills taught through online and classroom instruction
Received feedback from the faculty to assess student's understanding and use of information resources and critical thinking in their class research project(s)
Administered both lower and upper- division scenario-based assessments to determine the level of student information and critical thinking skills.
For analysis of results, see Student Information Skills Assessment: Report Comparing Lower and Upper Division Results (PDF)

Year Three (2010-11)

Integrated the basic information and critical thinking skills (Attribute 2) into the GE Area A: Communication A1, A2, and A3 lower-division course series
Administered online surveys to determine the effectiveness of the information and critical thinking skills taught through online and classroom instruction
Received feedback from the faculty to assess student's understanding and use of information resources and critical thinking in their class research project(s)

Plans Beyond 2011

Summary Lifelong Learning Report August 2011 (PDF)
Integrate the intermediate- level information and critical thinking skills into the GE Area C: Arts and Humanities C1-C3 lower-division courses, GE Area D: Society and the Individual D1-D4 lower-division courses, and selected major courses.
Integrate the advanced level information and critical thinking skills into GE Area C: Arts and Humanities upper-division C4 synthesis courses, the GE Area D: Society and the Individual upper-division D5 synthesis courses, and Senior Project courses.
Administer both lower and upper- division scenario-based assessments to determine the level of student information and critical thinking skills. (2013 - This activity is planned for every three years.)

Brief Statement of Significant Findings:

The scenario-based information skills assessment surveys provided consistent evidence of a positive correlation between higher percentages of appropriate responses and student attendance in library sessions at both the lower and upper-division levels
The in-class survey responses strongly suggested that library instruction was a positive learning experience for all students during their first two years at Cal Poly
The teaching faculty provided positive feedback and satisfaction with student research assignments

Recommendations/Actual Improvements Based on Assessment Results:

Revise Research 101 Tutorial using Panapto
Develop an online basic information and critical thinking skills lecture for GE Area A Communication (A1 – A3) lower-division courses.
Work with GE Area A (A1 – A3) instructors to integrate the online lecture into their syllabus.
Develop and integrate information and critical thinking skills into GE Area C:Arts and Humanities (C1-C4) an GE Area D:Society and the Individual ( D1-D5), and the Senior Project.

Ideas for Outreach, Workshops, Etc:

Schedule regular workshops with the Center for Teaching and Learning (CTL) during each academic year.
Present summaries to the Assessment Council on an annual basis.
Continue to work with associate deans, department chairs and faculty in both major and GE courses.
Continue to update the Lifelong Learning web site with results.

___________________________________________________________________________________

Terms defined from Definition of Lifelong Learning (top of page)

[1] Formal learning is deliberate, structured, and leads to the acquisition of a formal certificate.

[2] Non-formal learning is deliberate, but does not take place in a classroom setting. The emphasis here is on the acquisition of a skill and not a formal certificate.

[3] Informal learning is not deliberate and usually occurs during conversation or observation where one picks up some ideas that are not facts but someone's attitudes, perceptions, or values.

Navigation

Home

For Students

Faculty and Staff