Theories-in-Use and Reflection-in-Action: Core Principles for LIS Education by Phillip M. Edwards, p. 18

Research Articles
Educating Future Academic Librarians: An Analysis of Courses in Academic Librarianship by Edgar C. Bailey, Jr., p. 30

WISE Economics: ROI of Quality and Consortiums by Bruce Kingma and Kathleen Schisa, p. 43

Editors’ Notes
Integrating Social Theories into LIS Research by Laurie J. Bonnici and Manimegalai M Subramaniam, p. 1

Also Included in This Issue by Kathleen Burnett and Michelle M. Kazmer, p. 3

JELIS 2009 Satisfaction Survey by Sung Jae Park and Janet L. Capps, p. 4

Keywords: theories-in-use, reflection-in-action; collection development, search techniques, questionnaires, small group instructional diagnosis

For decades, organizational learning theorists have engaged in critical analyses of professionals’ work practices. Through this work, they have attempted to uncover 1) how individuals develop theories to guide their practices, and (2) how individuals adapt these theories to deal with more complex professional situations. Argyris and Schön (1974/1992), for example, explore how professionals apply theories-in-use during their work, and they characterize conditions (dilemmas) under which these theories-in-use could be challenged. Schön (1983) expands upon these notions to consider reflection-in-action, a process through which professionals engage in inquiry and “reflective conversation” with a complex situation in order to address dilemmas. This article examines the extent to which these two concepts—theories-in-use and reflection-in-action—could align with typical learning outcomes associated with LIS education, using two illustrative case studies: one from an undergraduate-level course on search strategies, and one from a graduate-level course in collection development.

Background and Purpose

Published work on the professions and the nature of professional education is rich and varied. Several touchstone publications (Schein, 1972; Schön, 1987; see also, with a sharper focus on librarianship and the information professions, Abbott, 1988) have contributed to and shaped the contemporary discourse around education for the professions. More recent and overarching shifts in higher education toward learner- or learning-centered teaching have offered an expanded range of approaches for characterizing the roles of teachers and students (e.g., Brookfield, 1995; hooks, 1994; Weimer, 2002). Beyond the teacher-student relationship, the recent conceptualization of “authentic professional learning” by Webster-Wright (2009) is suggestive of promising new directions for the design of continuing education programs for the professions, implicitly reflected in recent proposals by Ball (2008) and Mehra and Robinson (2009). Teaching professional skills and values to diverse sets of students in a manner that respects their prior experiences while reflecting authentic tensions that emerge across a variety of work settings is an on-going challenge within LIS education.

 Application of two concepts from the literature on organizational learning—theories-in-use and reflection-in-action—to the LIS classroom may effectively facilitate the alignment of course content, instructors’ expectations, and students’ needs. In the following discussion of these concepts, preference has been given to two “classic” works. This choice, while sacrificing breadth, permits a deeper exploration of each of the associated models as conceived by the original authors. The current application of these models to LIS practice, as evidenced through literature searches, is relatively sparse; the specific application of these models to LIS education has yet to be fully articulated.

Theories-in-Use

The theories-in-use concept arises from the work of Chris Argyris and Donald A. Schön, most thoroughly described in their 1974 monograph, Theory in Practice: Increasing Professional Effectiveness. Theories, for these authors, serve as “vehicles for explanation, prediction, or control” (p. 5); furthermore, theories and practices are inextricably linked within the professions:

A profession, then, not only has a practice but also a theory of action in which that practice can become a reproducible, valid technique. This means that the job of professional education consists not only in teaching technique but in teaching the methods by which behavioral worlds in which techniques can work can be created. (p. 149)

Argyris and Schön distinguish between espoused theories of action (i.e., what someone says they would do under certain circumstances) and theories-in-use (i.e., what someone actually considered when acting, subject to their assumptions and prior experiences). While there may be some level of incompatibility between an individual’s espoused theories and theories-in-use, the latter serve as useful but inherently limited heuristics for managing the complexity of the professional work environment. What occurs during professional education, therefore, is “the experience-based modification of some elements of theories-in-use—governing variables, action strategies, or assumptions” (Argyris & Schön, 1974/1992, p. 18).

In order to achieve this kind of learning, Argyris and Schön focus on the exposure and resolution of dilemmas arising from students’ theories-in-use, including dilemmas of incongruity, dilemmas of inconsistency, dilemmas of effectiveness, dilemmas of value, and dilemmas of testability:

In an appropriately redesigned professional curriculum, courses would describe these inconsistencies and incongruities and perhaps begin to deal with them… Exploring these issues should have several positive consequences. First, the students would realize from the outset the extent of the gap between their academic training and practice. Second, while at school, they would begin to think about and test with each other their own theories of practice. Third, students could then press for the additional courses they need to make their budding theory of practice more effective. (p. 176)

Given this stance on professional curricula, Argyris and Schön (1974/1992) present two complementary models for student learning, which they label model I and model II. Their tone suggests a moderate preference for the second model:

In general, we believe, those strongly disposed to learn model II possess two important characteristics. First, these individuals are able to listen to feedback about the errors they make as they try to learn model II, so long as that feedback is crafted to produce minimal defensiveness, and to strive to correct their errors once these have been pointed out to them. Secondly, once these individuals have received valid and constructive feedback, they tend to “hang in” in order to learn. For example, even if it happens that every new action they try out seems wrong and they feel stuck, they realize that recognizing their stuck-ness is necessary for learning. (p. xxiii)

Thus, the ideal instructional setting would be “a learning environment that produces valid information about each participant’s espoused theories, theories-in-use, and any inconsistencies within each theory as well as among them” (Argyris & Schön, 1974/1992, p. 97). This approach “makes dilemmas recognizable, which creates tension to resolve them [and] this tension motivates learning” (p. 97). A similar shift appears in Lewin’s (1951/1997) three-step model of change in group performance—a process that requires “unfreezing” of a group’s initial understanding of a situation, “moving” through some form of intervention or dialogue, and “freezing” of the newly negotiated group standards.

Reflection-in-Action

Schön’s later work illustrates how professionals and students resolve these dilemmas through a process he terms reflection-in-action. Schön (1983) suggests:

When the phenomena at hand eludes ordinary categories of knowledge-in-practice, presenting itself as unique or unstable, the practitioner may surface and criticize his initial understanding of the phenomenon, construct a new description of it, and test the new description by an on-the-spot experiment. Sometimes he arrives at a new theory of the phenomenon by articulating a feeling he has about it… When he is confronted with demands that seem incompatible or inconsistent, he may respond by reflecting on the applications which he and others have brought to the situation. Conscious of a dilemma, he may attribute it to the way in which he has set his problem, or even to the way in which he has framed his role. He may then find a way of integrating, or choosing among, the values at stake in the situation. (p. 62-63) 

These notions of reflection are not unique to Schön’s writing, however; perhaps the most notable proponent of reflective thought and its role in education is John Dewey (1910/1997). Schön and Dewey both consider education an essential mechanism through which these habits of reflective practice are learned. Schön (1983) claims that “when a practitioner makes sense of a situation he perceives to be unique, he sees it as something already present in his repertoire [of examples, images, understandings, and actions]” (p. 138). This notion of reflective contemplation as an educative process continues to be carried forward by educational philosophers such as Greene (1988).

Potential Applications of Theories to LIS Education

There is a great deal of conceptual overlap between the notions of theories-in-use and reflection-in-action, which suggests a coherent, constructivist foundation for many graduate- and undergraduate-level courses in LIS. General principles for course design associated with these concepts might include: 1) valuing and incorporating student experiences and interests in class sessions and assignments, particularly those that express espoused theories of developing professional practice (cf. theories-in-use); 2) creating authentic exercises that allow students to test their espoused theories against situations emerging from practice (cf. exposing dilemmas); 3) focusing on the collection of evidence—including lived experience, analysis of internal processes in a particular work context, and gathering information about the various communities being served in a particular setting—in support of their choices as developing professionals (cf. resolving dilemmas); 4) offering students a non-threatening space for dialogue with their peers and with practicing professionals to refine their theories-in-use (cf. resolving dilemmas); and 5) allowing students structured and deliberate opportunities for reflection on course content and their own perceptions, attitudes, and assumptions (cf. reflection-in-action). These principles and the theoretical concepts upon which they are based may serve as elements of evaluation as well, given that these qualities may emerge from students’ reflections on their experiences during the course. In the following cases, the concepts of theories-in-use and reflection-in-action are used in both senses: as aids for course design and as lenses for understanding students’ feedback for a particular course.

Method

The School of Information and Library Science (SILS) at the University of North Carolina at Chapel Hill (UNC-CH) supports four degree programs—the B.S. in Information Science (B.S.I.S), the M.S. in Information Science (M.S.I.S.), the M.S. in Library Science (M.S.L.S.), and the Ph.D. in Information Science—and in the discussion that follows, I consider two courses from these programs as cases: the undergraduate-level INLS 200 (Retrieving and Analyzing Information) and the graduate-level INLS 513 (Resource Selection and Evaluation). Representative syllabi for these courses—including course descriptions and anticipated student learning outcomes—are available from my website as archived course materials (Edwards, n.d.).

As a standard practice in my courses, multiple forms of anonymous feedback are collected from students at various points during each semester. I collect anonymous written feedback from students near the third or fourth week of the term, based around open-ended prompts such as:

• Are there things that we’ve been doing that have been particularly helpful for how you’re learning?
• Are there things that could be modified to better match your needs?

Near the middle of the semester, I invite a consultant from the Center for Faculty Excellence at UNC-CH to visit my classes to conduct Small Group Instructional Diagnosis (SGID) sessions over two class periods. The SGID is a formative approach for instructional improvement that has been used in postsecondary education since the mid-1970s (Clark & Redmond, 1982; Diamond, 2004). During the first class period, the consultant collects written feedback from all enrolled students in response to the prompts:

• List the major strengths in the course. (What is helping you learn in the course?) Please explain briefly or give an example for each strength, and
• List changes that could be made in the course to assist you in learning. Please explain how suggested changes could be made.

In the next class session, the consultant returns to the class with a compiled list of themes that emerged from the written feedback and engages students collectively in a member-checking exercise. The final set of results—the students’ transcribed written responses, categorized into several themes, along with any themes that were confirmed, added, or modified based on member-checking—are returned to me.

The final form of anonymous feedback consists of written responses to a SILS-specific questionnaire, generated by students during the final class session of the course. Completed questionnaires are submitted to the SILS office, routed internally among the SILS administration, and returned to instructors after students’ course grades are recorded. All of the representative qualitative responses that follow were gathered anonymously from students in these courses through the above means of feedback collection during the Fall Semester 2008 (INLS 513, with 31 enrolled students; denoted as [F08] below) and the Spring Semester 2009 (INLS 200, with 42 enrolled students, and INLS 513, with 35 enrolled students; denoted as [S09] below).

Results

Theories-in-use and reflection-in-action in an undergraduate-level course

The B.S.I.S. program is designed to “prepare its graduates for a variety of careers in the information industry, including information architecture; database design and implementation; Web design and implementation; and networking support and information consulting; as well as for graduate study” (SILS, 2004a). The INLS 200 course is a prerequisite that undergraduate students complete prior to their admission to the B.S.I.S. program.

In my sections of INLS 200, students are primarily responsible for researching a self-selected topic for inclusion as an entry in Wikipedia. To assist students in their searching and writing, students complete several graded reviews of print- or digitally-based sources that relate to their topic, several rounds of peer review of their classmates’ in-progress entries, and a summative reflective essay on the role of search in daily life and their experiences while researching and writing during the semester.

One of the challenges in teaching a class such as this relates directly to students’ theories-in-use related to searching, as one student expressed during feedback collected in the fourth week of Spring Semester 2009: “We know what searching is and what the value of information is. We use it every day—we do not need to go over it every other class” [S09]. By the time written feedback was collected during the ninth week of this semester, several students made comments which suggest this reticence was melting, largely a consequence of students’ reflections on how small group work and class discussions had enhanced their learning:

“When we have class discussions, it is very interesting and helpful to hear others’ points of view and the different ways they use and think about the material” [S09];

“Openness to speak up in class; ex. I don’t feel embarrassed to throw out an answer to one of his questions when I do in other courses” [S09];

 “Working in groups to actually apply the things we have learned is very helpful because we can see how they can be used and get help and new ideas from classmates” [S09].

By the ninth week of the course, many students also commented on the helpfulness of relevant, authentic examples and assignments as a key factor in helping them learn:

“After taking this course for half a semester, I have become [a] more savvy Internet and database user” [S09];

“I learned many things that I just passed by—privacy on the Web, use of Blackboard, Sakai, etc.” [S09];

 “The material taught is helpful and applicable to daily life” [S09];

“Articles relate to programs that we as students incorporate into our daily lives [S09]; and

“On-hand [sic: hands-on] relevant work; by creating a Wikipedia page over the course of the semester, it makes topics learned in class more relevant” [S09].

Despite the above feedback, which might be seen as the application of reflection-in-action to dilemmas exposed from the course materials and activities, some students expressed frustration regarding the level at which the course was being taught:

“Assume we know nothing about computers, for example, a lot of us are not majors and need step by step directions” [S09];

“Don’t take students’ knowledge of the subjects for granted; I (plus many others) are unfamiliar with developing a user page on Wikipedia, and wish [the instructor] knew that; he was open to a question today when someone basically said they had no idea” [S09].

Another student voiced a contrary assessment:

“Take this class out of the major/minor requirement for INLS; I haven’t learned anything, most material is intuitive and doesn’t need to be taught” [S09].

By the seventeenth week of the semester, the above sentiments about the intuitiveness of course materials and activities continued to emerge, albeit less frequently, during the summative feedback collected about the course:

“We have no idea how to write Wikipedia articles. I don’t think it is a beneficial/helpful thing to learn, felt like a waste of time” [S09]; in contrast with,

“Class material was either too abstract to be worth learning or too intuitive to be taught” [S09].

Nevertheless, over half of the students (9 out of 17) who completed the summative course evaluations mentioned that the most useful learning experience of the course was the research and writing for their Wikipedia entries. One student mentioned peer review activities as being most helpful, and three mentioned the resource review assignments. The students’ final reflective essays, the analysis of which will appear in a separate report, revealed more insight into the students’ changing perceptions of themselves as not only consumers of information but also as producers of information and designers of information resources.

Theories-in-use and reflection-in-action in a graduate-level course

The two masters’ programs at the School of Information and Library Science, the M.S. in Information Science (M.S.I.S.) and the M.S. in Library Science (M.S.L.S.), are guided by complementary sets of competencies (see SILS, 2004b; SILS, 2004c). Within this framework, graduate students may enroll in classes that satisfy requirements for either program. The INLS 513 course is a requirement for the M.S.L.S. program and an elective for the M.S.I.S. program.

In my sections of INLS 513, students are primarily responsible for three projects: 1) a review of a reference resource or substantive work of fiction or nonfiction; 2) selecting resources for a collection associated with a particular community, supported by insights students gain from a community analysis; and 3) a comparative critique of two collection development policies, self-selected by individual students. As with INLS 200, students complete several rounds of peer review of their classmates’ in-progress drafts, and they write short personal reflections to accompany these assignments throughout the semester.

In contrast with INLS 200, however, the graduate students in INLS 513 appeared to be more initially aware of the importance of others’ ideas for their own learning. For example, a student commented during the fourth week of the semester that: “I really enjoyed class when you brought books in so we could evaluate them and hear from everybody their opinions of what should be considered” [S09]. This attitude—that there was a benefit to students’ learning from small group work and discussion—was reflected in the written feedback collected during the ninth week of the semester as well; students noted that strengths of the course included:

“Facilitating [discussion] so people can bring their experiences, ask questions” [F08];

 “Encouraging students to examine perspectives which may be different from their own; interaction with students and including participation in learning” [S09]; and

“Asking us to think/question” [S09].

These sentiments were not universally shared, however; not all students were as apt to see these course activities as being helpful for their learning. By the ninth week of the semester, several students had remarked:

“Learning primarily as a result of completing assigned readings – prefer more focused class discussion and involvement of teacher” [F08];

“Stop the group work; it’s unrealistic for us as graduate students for scheduling and it doesn’t reflect ‘the real world’. Group work wastes my time since I’ve already been in the real world and end up holding people’s hands to get through group projects” [S09];

“Final assignment: I understand the point of submitting drafts, providing feedback, presenting, etc. but not sure this is necessary at all (for me personally, at least) in the learning process” [S09]; and

“Group work is detrimental to my learning because people get way off topic and screw around until the professor calls us back, I learn nothing” [S09].

As with INLS 200, the authenticity of examples, activities, and assignments was an aspect of the course that, upon reflection, students’ found helpful for their learning at week nine:

“Helpful to hear how concepts from readings might appear in different library settings” [F08];

“Use[s] ‘real’ examples to explain collection development rules – online material, online policy, etc. which made it easy for me to understand” [S09];

“[The instructor’s] ‘welcome to class, do ya have any collection development news?’ is a good way to remind us that all of this is potentially relevant to our future careers” [S09];

 “The assignments feel directly relevant to what we are learning in class as they draw on topics we’ve covered without mere rote memorization” [S09]; and

“There is a lot I like about this class and I don’t have many complaints. This has been one of the most constructive classes in SILS so far. I love how interactive the class is, how it relates theory to practice and I feel the projects are relevant” [S09].

Again, attitudes toward the level of authenticity in course activities were not uniform across the class by the ninth week; several students’ reflections on hindrances to their learning at that point included:

“Don’t feel like I am learning specific things” [F08];

“Utilize more solicitation of student experience for this. Philosophical style is fun and makes class enjoyable, however, it makes the usefulness of readings and class discussion less apparent. Emphasize utility of discussions to future work” [F08];

“The content is a bit obvious; collection development is more common sense than learning skills or such, so the class itself has so far been a waste of my time. Nothing [the instructor] can do about that though since it is a required course” [S09];

“Assignments don’t seem related to real-life work; critiquing and/or deconstructing, evaluating policies is perhaps much more useful than having us write policies that in real life are done by committee” [S09]; and

“More practical interactions with materials; more short exercises of play acting as policy writer, collection developers, etc.” [S09].

By the seventeenth week of the course, students’ reflections on their learning in the course seemed to regard many of the activities as being more helpful for their learning, after the “intuitiveness” of course concepts may have been explicitly challenged to a greater extent. For example, many students noted that the opportunity to learn from others’ experiences and opinions was a benefit:

“I liked the mix of PowerPoint and student input; [the instructor] encouraged people to share their opinions and didn’t make it feel like his ideas were better than yours” [F08];

“I really enjoyed the peer reviews for Assignment 3. Very helpful!” [F08];

“Peer editing of final papers – very useful” [S09];

“I like how he asks for class members to bring up topics at the beginning of class” [S09];

“[The instructor] was respectful and encouraged student input” [S09]; and

“The group conferences were particularly helpful” [S09].

There were limits to this approach, however, several of which were expressed as:

“When someone would ask the teacher a question, instead of answering it he often threw the question out to the class to see what they thought. Sometimes you just want the teacher to answer the question” [S09];

“Group projects with young, inexperienced, highly sensitive students, and insanely busy schedules. We work!” [S09].

By the conclusion of the course, students were also generally more positive about the authenticity of activities, assignments, and examples used in the class:

“The group project [was most useful] because it imitated what we would have to do in real life” [S09];

“I liked the first assignment (evaluating a reference source) the most – it seemed to be the most relevant to what I will do as a librarian” [S09];

“I appreciate [the instructor’s] many practical examples” [S09];

 “The reading which pertained to or were actual institutions’ policies and community analyses were helpful” [S09]; and

 “In general, the practical course readings of actual policy really helped me understand the concepts” [S09].

In some cases, students mentioned being able to better understand the breadth of the topics covered and ways in which course topics related to other aspects of the profession:

“I liked the way [course organization] was done—starting with details and moving out to the bigger picture” [F08];

“Now that I’ve taken this class I’m way more aware of collection development issues and how they fit into other classes I’ve taken” [S09]; and

“The worst part for me was the fact that collections can seem to be a nebulous subject to have a conversation about.” [S09]

Nearly half of the students who completed the summative course evaluations mentioned that the most useful learning experience of the course was the collection building assignment where community analysis techniques were applied as a small group (see Table 1).

Discussion

In the early stages of these courses, at both the undergraduate level and the graduate level, a substantive instructional challenge is posed by many students’ initial perceptions that everything is intuitive—that their existing theories-in-use are sufficient to guide their practices. Argyris and Schön (1974/1992) recognize this challenge:

Especially during the early phases of learning, participants will probably feel frustrated and uncomfortable and doubt whether they can learn model-II behavior; they may even doubt whether model II is valid. (p. 97)

In terms of search techniques, many aspects of the search process are not obvious, even if students have tacitly adopted particular aspects in their existing practices: to identify the “best” sources for addressing a particular information need; to evaluate sources based on their quality, authoritativeness, bias, authorship, credibility, currency, and intended audience; to develop skills for formulating effective search queries (cf. Taylor’s (1968) “compromised need”); and to synthesize the results of searches for a particular audience. In terms of collection development, tensions emerge in professional practice: balancing demands to acquire materials based on patron demand versus professional judgment; managing an effective portfolio of order types (approval plans, standing orders, blanket orders, and firm orders) across multiple vendors; accommodating requests for reconsideration of materials; remaining aware of current trends within the publishing industry (e.g., bundling and licensing); and making selection decisions subject to budgetary constraints.

Introducing authentic materials from practice that confront the “obviousness” of students’ theories-in-use—how they approach the decisions they make as searchers and selectors—appear to provide a space in which students can learn from their peers, reflect on their attitudes and practices, and modify their ways of thinking about their roles in their future careers. In-class discussions and other opportunities for students to work in small groups around authentic examples, as an instructional design strategy, seem to be effective mechanisms for aligning the content of the course with students’ experiences and professional goals.

Within LIS, Bell and Shank (2007) propose the application of an inquiry-/design-based paradigm to librarianship. There is evidence elsewhere in the literature (e.g., Watson-Boone, 2000) that librarianship and the information professions are increasingly incorporating inquiry-, design-, and evidence-based strategies for approaching the complexity of modern practice. One of the more complete characterizations of the role that design plays in the information professions comes from Bell and Shank (2007), who build upon the stages of the ADDIE model—analyze, design, develop, implement, and evaluate—and characterize success of a program or service by the impact it has upon user learning. While not all library services or programs may have learning at their core, it is possible to think of the aim of successful design as producing excellent user experiences, which may include learning as well as, say, positive affective responses. This general model bears some fine-level distinctions with other approaches, such as action research (e.g., Wilson, 2000), evidence-based librarianship (e.g., Booth & Brice, 2004; Eldredge, 2000), information needs assessment (e.g., Davis, 2004; Evans & Saponaro, 2005), community analysis (e.g., Consulting Librarians Group, n.d.), and contextual design (Beyer & Holtzblatt, 1998) as well as emerging approaches in positive design (Faust, 2009) and studio-based design (Snyder, Heckman, & Scialdone, 2009), among others.

Interestingly, the two assignments that students noted as being most helpful for their learning across both of these courses—in INLS 200, the Wikipedia entry, and in INLS 513, building a small core collection for a particular community, were arguably the assignments that were most design-based. While working in Wikipedia, undergraduate students interacted with their classmates as well as Wikipedia administrators and editors, and this process helped shape their understanding of how their final entries should be written, researched, and structured. While working with their community analyses as lenses through which materials could be identified and selected, graduate students grappled with trying to understand the needs, capabilities, and constraints of a particular community when building their collections. Both of these projects involved the collection of some evidence about what “acceptable” products would look like, the reflection on this evidence, and some amount of dialogue about how to design the final product.

Conclusions

The preceding discussion is intended to be persuasive rather than prescriptive, and, at times, it may appear more personal in nature than readers typically observe in JELIS articles. There are many different approaches to teaching within LIS, as evident from the pages of JELIS over the years, and as an author, I am hard-pressed to claim that my application of organizational learning concepts to LIS education represents the single, unassailable way forward. My personal stance on LIS education strives for a balance between behavioral and progressive philosophies of adult education (cf. Elias & Merriam, 2005). Many readers may see glimmers of their own approaches to teaching in these concepts and cases without drawing explicit support from the models discussed above; others may use cognate models—for example, the taxonomy of educational objectives (Anderson, Krathwohl, Airasian, Cruikshank, Mayer, Pintrich, Raths, & Wittrock, 2001), case study approaches (Galvin, 1973), experiential learning (Kolb, 1976), communities of practice (Wenger, 1998), or transformative learning (Mezirow & Associates, 2000)—to guide their teaching.

That said, the five principles derived above from the concepts of theories-in-use and reflection-in-action may be broadly applicable to the design and assessment of the effectiveness of classroom activities, exercises, and assignments. Exposing and resolving dilemmas emerging from professional practice, while 1) not universally perceived by students as being useful educational activities, and 2) seemingly more influential among graduate students than among undergraduate students, appears to benefit many students. Deriving shared yet flexible conceptual frameworks for guiding course design and assessment, driven by the best available evidence on how adult students learn, are clearly on-going and evolving processes. What remains essential for continuing this conversation—the importance of which is highlighted by Wallace (2009)—is a community of educators who are willing to share, openly and transparently, the benefits and challenges of their various approaches.

References

Abbott, A. (1988). The system of professions: An essay on the division of expert labor. Chicago: University of Chicago Press.

Anderson, L. W., Krathwohl, D. R., Airasian, P. W., Cruikshank, K. A., Mayer, R. E., Pintrich, P. R., Raths, J., & Wittrock, M. C. (Eds.). (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives (Complete ed.). New York: Longman.

Argyris, C., & Schön, D. A. (1992). Theory in practice: Increasing professional effectiveness. San Francisco: Jossey-Bass. (Original work published 1974).

Ball, M. A. (2008). Practicums and service learning in LIS education. Journal of Education for Library and Information Science, 49(1), 70-82.

Bell, S. J., & Shank, J. D. (2007). Academic librarianship by design: A blended librarian’s guide to the tools and techniques. Chicago: American Library Association.

Beyer, H., & Holtzblatt, K. (1998). Contextual design: Defining customer-centered systems. San Francisco: Morgan Kaufmann.

Brookfield, S. D. (1995). Becoming a critically reflective teacher. San Francisco: Jossey-Bass.

Booth, A., & Brice, A. (2004). Evidence-based practice for information professionals: A handbook. London: Facet.

Clark, D. J., & Redmond, M. V. (1982). Small group instructional diagnosis: Final report. Washington, D.C.: Fund for the Improvement of Postsecondary Education. (ERIC Document Reproduction Service No. ED 217 954)

Consulting Librarians Group. (n.d.). Community analysis methods and evaluative options: The CAMEO handbook. Retrieved October 6, 2009, from http://skyways.lib.ks.us/pathway/cameo/

Davis, D. M. (2004). Needs assessment. In J. C. Bertot & D. M. Davis (Eds.), Planning and evaluating library networked services and resources (pp. 187-221). Westport, CT: Libraries Unlimited.

Dewey, J. (1997). How we think. Mineola, NY: Dover. (Original work published 1910).

Diamond, M. R. (2004). The usefulness of structured mid-term feedback as a catalyst for change in higher education classes. Active Learning in Higher Education, 5(3), 217-231.

Edwards, P. M. (n.d.). www.unc.edu/home/pmedward/. Retrieved October 6, 2009, from http://www.unc.edu/home/pmedward/

Eldredge, J. D. (2000). Evidence-based librarianship: An overview. Bulletin of the Medical Library Association, 88(4), 289-302.

Elias, J. L., & S. B. Merriam. (2005). Philosophical foundations of adult education (3rd ed.). Malabar, FL: Krieger.

Evans, G. E., & Saponaro, M. Z. (2005). Information needs assessment. In Developing library and information center collections (5th ed.). Westport, CT: Libraries Unlimited.

Faust, J. (2009). Positive design. Journal of the American Society for Information Science & Technology, 60(9), 1887-1894.

Galvin, T. J. (1973). The case method in library education and in-service training. Metuchen, NJ: Scarecrow.

Greene, M. (1988). The dialectic of freedom. New York: Teachers College Press.

hooks, b. (1994). Teaching to transgress: Education as the practice of freedom. New York: Routledge.

Kolb, D. A. (1976). The learning style inventory technical manual. Boston, MA: McBer and Company.

Lewin, K. (1997). Frontiers in group dynamics. In Resolving social conflicts & field theory in social science (pp. 301-336). Washington, D.C.: American Psychological Association. (Original work published 1951).

Mehra, B., & Robinson, W. C. (2009). The Community Engagement Model in library and information science education: A case study of a collection development and management course. Journal of Education for Library and Information Science, 50(1), 15-38.

Mezirow, J., & Associates. (2000). Learning as transformation: Critical perspectives on a theory in progress. San Francisco: Jossey-Bass.

Schein, E. H. (1972). Professional education: Some new directions. Berkeley, CA: Carnegie Commission on Higher Education.

Schön, D. A. (1983). The reflective practitioner: How professionals think in action. New York: Basic Books.

Schön, D. A. (1987). Educating the reflective practitioner: Toward a new design for teaching and learning in the professions. San Francisco: Jossey-Bass.

School of Information and Library Science (SILS), University of North Carolina at Chapel Hill. (2004a). Degrees and programs: BSIS overview. Retrieved October 6, 2009, from http://sils.unc.edu/programs/bsis/

School of Information and Library Science (SILS), University of North Carolina at Chapel Hill. (2004b). Degrees and programs: MSIS overview. Retrieved October 6, 2009, from http://sils.unc.edu/programs/msis/

School of Information and Library Science (SILS), University of North Carolina at Chapel Hill. (2004c). Degrees and programs: MSLS overview. Retrieved October 6, 2009, from http://sils.unc.edu/programs/msls/

Snyder, J., Heckman, R., & Scialdone, M. J. (2009). Information studios: Integrating arts-based learning into education of information professionals. Journal of the American Society for Information Science & Technology, 60(3), 1923-1932.

Taylor, R. S. (1968). Question-negotiation and information seeking in libraries. College & Research Libraries, 29(3), 178-194.

Wallace, D. P. (2009). The iSchools, education for librarianship, and the voice of doom and gloom. Journal of Academic Librarianship, 35(5), 405-409.

Watson-Boone, R. (2000). Academic librarians as practitioner-researchers. Journal of Academic Librarianship, 26(2), 85-93.

Webster-Wright, A. (2009). Reframing professional development through understanding authentic professional learning. Review of Educational Research, 79(2), 702-739.

Weimer, M. (2002). Learner-centered teaching: Five key changes to practice. San Francisco: Jossey-Bass.

Wenger, E. (1988). Communities of practice: Learning, meaning, and identity. Cambridge, UK: Cambridge University Press.

Wilson, T. D. (2000). Recent trends in user studies: Action research and qualitative methods. Information Research, 5(3). Retrieved on October 6, 2009, from http://informationr.net/ir/5-3/paper76.html

JELIS is a scholarly double-blind peer-reviewed publication of ALISE. As such, the goals of ALISE are foundational to the vision statements of the JELIS editors. Capps andHinson noted that, “both co-editors [Burnett and Kazmer] strongly agree with the need to address how the changing needs of libraries affect the education of librarians. Other current goals they hold for JELIS include: (a) what is taught, (b) how to teach the topics, (c) what is known about research, and (d) how the research is used” (2009, p. 64). These goals are in alignment with the ALISE goals; however, if JELIS is to be a venue for research dissemination, it is important to measure the perceptions of how well the journal is serving the needs of the research community.

The last systematic measurement of JELIS subscriber satisfaction was over 25 years ago. In 1984, the Readership Survey was conducted at the urging of the ALISE Board of Directors to the Editorial Board (Patterson, 1985). Patterson, who served as the JELIS editor from 1980 to 1988, stated, “a study had never been done and both boards were concerned about the status of the Journal among other refereed publications in the profession” (1985, p. 302). Patterson highlighted the 31-item questionnaire results in the JELIS 25(4) article titled, “An Assessment of the Status of the Journal” (Patterson, 1985). Based on the opinions gathered through the analysis of 177 usable surveys, Patterson concluded that the journal’s place was “firmly established among the scholarly refereed publications in our profession” (1985, p. 311).

Twenty-two percent of the ALISE membership participated in the 2008 ALISE survey (Cloonan, 2008). Part of this questionnaire was a design to solicit feedback on the value of various member services and benefits. The top three member benefits were identified as networking (64 members), conference (62 members), and JELIS (62 members). With just over half of the respondents placing JELIS as an important component of ALISE membership, the journal serves a vital role in the organization, and as such, JELIS must continue to solicit feedback from its readers.

Research Design

The JELIS Satisfaction survey was designed to assess the satisfaction of readers and to inform the editorial decision processes. Five major categories were devised: (1) JELIS use, (2) Degree of satisfaction, (3) Topics on demand, (4) Loyalty, and (5)Demographic questions. The instrument, hosted through an online survey system, was pre-tested for readability, reliability, and questionnaire completion-time. After a few minor adjustments, the survey was announced on the ALISE/JELIS web site and through other professional venues, such as an ALISE conference announcement, and the jESSE listserv.

The sample was divided into two categories: (1) ALISE members and (2) Non-members. The ALISE member category included institutions and individuals who receive the journal as part of their membership. The non-member category included individuals and institutions (primarily libraries) that obtain the journal through volume-based subscriptions, and other individuals who read or cite articles published in the journal.

Results and Discussion

The survey was conducted from January 21st to February 16th, 2009, and had 108 respondents with 95 usable responses. The questionnaire data was exported to an Excel spreadsheet from the online survey system and analyzed with SPSS. Open-ended question responses were manually coded. As results of this study, respondent profile, JELIS use, and JELIS satisfaction are discussed below. An expanded discussion related to the open-ended responses and the loyalty factor is planned.

Respondent Profile

The demographic questions gathered occupational position, organizational affiliation, and geographic region information. Seventy-four percent of the respondents described their occupational title as faculty and 2% as librarians. The faculty category encompassed deans/department chairs, professors including associate and assistant professors, and lecturers. Most of the respondents (n = 70) stated they were affiliated with doctorate-granting universities as classified by the Carnegie Classification of Institutions of Higher Education (Carnegie Foundation for the Advancement of Teaching, 2009) while 14 people were from master’s colleges and universities.

The majority of the respondents (91%) stated they were from regions within the US, while 6% selected Canada and 3% Asia and Pacific. This appears to be in keeping with the results of the earlier Readership Survey response of 90% of the 177 respondents reporting the US as the region of residents (Patterson, 1985).

JELIS Use

Respondents were asked how they use JELIS in terms of reading, citation, publication, length of subscription, sharing, and retention of the journal. The majority of the respondents (89%) indicated that in the past year they have either read or looked through an issue. Taken independently, this appears respectable; however, 37% (n = 33) state they spent between 16–30 minutes looking through an issue upon receipt. Twenty eight respondents (31%) stated they spent less than 15 minutes, and 72% of the participants reported skimming the titles to be aware of the contents and reading the articles of interest. This strongly indicates that past JELIS articles were not holding the reader’s attention for an extended period.

When the respondents were asked if they had cited JELIS articles in the past year, 70% responded negatively. When asked how often they cite articles in JELIS, 21 percent responded ‘never,’ 42% responded ‘less than once a year,’ and 28% skipped this question entirely. Thirty-two percent indicated they had previously published in JELIS.

The length of subscription has changed since the 1984 survey. Patterson wrote that “fifty-four percent of the respondents subscribed to JELIS for 10 or more years, 35% from 10–15 years, 27% from 4 to 9 years, 19% for more than 15 years, 18% from 1 to 3 years, and one person made no response to this question” (1985, p. 303). The 2009 survey indicates 33% have received JELIS for 10 or more years and 39% for three years and under. Understanding the needs of the latter large group of new subscribers may improve the journal’s citation rate.

When asked if they shared their copy of JELIS with others, three fourths of the respondents reported that they do not share JELIS with anyone. Asked how long they keep a copy of JELIS, 58% responded ‘indefinitely.’ With almost three fourths of the respondents self-selecting a faculty occupational classification, future administrations of this survey may need to investigate the extent to which JELIS articles are shared or used in the classroom as required readings.

                                                                Figure 1. Satisfaction with JELIS

Satisfaction

Respondents were asked to rate their satisfaction with JELIS. The degree of subscriber satisfaction was assessed using a 5-level Likert scale (extremely satisfied, somewhat satisfied, somewhat dissatisfied, extremely dissatisfied, or undecided). As Figure 1 shows, 8% of the respondents were ‘extremely’ satisfied, 52% were ‘somewhat’ satisfied, 23% were ‘somewhat’ dissatisfied, 10% were ‘extremely’ dissatisfied, 7% were undecided, and 18 respondents skipped the question. Overall, 60% of the respondents (n = 54) were satisfied, while 33% (n = 30) were dissatisfied. The overwhelming majority response of Somewhat Satisfied for all of the satisfaction attributes listed in Figure 1 reinforces the value of conducting JELIS Satisfaction Surveys.

Table 1 shows the evaluation of satisfaction elements by the respondents who answered “Extremely Dissatisfied” or “Somewhat Dissatisfied” on overall satisfaction. Even though the statistical significance was not tested due to the small sample size, relevance of content is the major factor of dissatisfaction followed by quality of content, breadth of topics, and timeliness of content. The last two factors, quality of authors and design and layout are weakly related to the dissatisfaction, in that the percentage of satisfaction is higher than the percentage of dissatisfaction responses.

Conclusion

The major goal of this inquiry has been descriptive, with the intent that readers’ perceptions be used to inform future editorial efforts. Results from the 2009 JELIS survey indicate that relevance and quality of content are major factors contributing to dissatisfaction with JELIS. The survey outcomes provide important information to assist the current editors to obtain a closer journal alignment with the needs of its readers and subscribers. See Appendix A for additional demographic and JELIS use survey data.

Acknowledgements

The JELIS co-editors and editorial staff sincerely thank all of the respondents of the JELIS 2009 Satisfaction survey. Your participation and feedback will assist in improving the readability and reputation of JELIS. If you missed the opportunity to share your thoughts in 2009, we look forward to hearing from you in the 2010 JELIS Satisfaction survey. Special thanks to Michèle Cloonan (ALISE President) and Kathleen Combs (Executive Director) for their support in publicizing the survey.

Appendix A– Supplementary Data

References

Capps, J., & Hinson, C. (2009). JELIS connections. Journal of Education for Library and Information Science, 50(2), 59–67.

Carnegie Foundation for the Advancement of Teaching. (2009). The Carnegie classification of institutions of higher education. Retrieved August 9, 2009 from http://www.carnegiefoundation.org/classifications/

Cloonan, M. C. (2008). ALISE 2008 survey results. Retrieved August 9, 2009, from http://data.memberclicks.com/site/ali/ALISE_Survey_Results.pdf.

Patterson, C. D. (1985). An assessment of the status of the journal. Journal of Education for Library and Information Science, 25(4), 301–312.