Tag Archives: web evaluation

Search process as web evaluation

“I usually click on the first thing that I see.” Asked to clarify how she decides to pick the first result, she emphasized, “Well, I know the ones that are […] in here [pointing to the shaded Sponsored Link section on a Google results page] they’re the most relevant to what I’m looking for.”

Article: Young Adults’ Evaluation of Web Content

http://ijoc.org/ojs/index.php/ijoc/article/view/636/423

I chose this article in thinking about because of its relevance to what we all do as instructors in today’s climate – attempting to reach students and speak a language that they understand, in terms of how they look for information and how they find and decide to use it.

This study was somewhat unique in that instead of looking at the steps people used to evaluate sources after they located them, it focused on how they arrive at those sources in the first place as a component of evaluation. I don’t think the information is necessarily revolutionary – any of us could probably pull similar answers out if asked, but this study is nice in that it validates what we know but don’t always have data to back up, and it is research done not by librarians but by communications researchers, all of whom focus on new media.

If you want a good down and dirty overview, here is an entry about it from ReadWriteWeb –

http://www.readwriteweb.com/archives/so-called_digital_natives_not_media_savvy_new_study_shows.php

The research was conducted like a usability test – researchers observed students doing searches for answers to canned questions, audio was recorded, and there was some follow-up.

The reason they focused on how sources were arrived at in the first place was that they felt sources were chosen as much because of where they fell in search results as the results of content evaluation. Some of their related findings that are relevant to us:

  1. Over 25% of respondents chose sites because they were the first result in a search engine, not because of the site’s content
  2. Only 10% of respondents even mentioned an author or credentials when completing tasks (“However, examining the screen captures of the tasks being performed makes it clear that even among those participants, none actually followed through by verifying either the identification or the qualifications of the authors whose sites gave them the information they deemed to be providing the best solution for the tasks at hand.”
  3. Many respondents expressed trust in .org sites, saying that they were more trustworthy than .com sites.
  4. Among our sample of 102 participants, overall 60% stated, at one point or another that they would contact an institution such as a university or governmental agency for information. Broken down by method of contact, 52% of the sample suggested placing a phone call, while 17% said they would send an email to the organization. Professionals, both medical and educational, were second on the list of those whom participants would contact offline with a fifth of our respondents suggesting that they would pursue this method.
  5. One of their conclusions was that instructors “must start by recognizing the level of trust that certain search engines and brand names garner from some users and address this in away that is fruitful to a critical overall evaluation of online materials.”

The questions this article made me think about relate to what we can do speak to our students about web evaluation in a way that is relevant to them, and include more of a focus on the search process itself as a component of evaluation. This is where I would like to focus RIOT discussion . . .

International students and source evaluation

  • “How Helping Chinese ESL Students Write Research Papers can Teach Information Literacy,” Mei-Yun (Annie) Lu, Journal of East Asian Libraries, No. 141, Feb 2007, pp 6-11.
  • “The Adaptation of Asian Masters Students to Westerns Norms of Critical Thinking and Argumentation in the UK,” Kathy Durkin, Intercultural Education, 19(1), Feb 2008, pp15-17.

Last week, Meghan, Cindy and I went to RHE 398T (the pedagogy class for Assistant Instructors new to teaching introductory writing) to introduce support materials for unit 2 and talk about how unit 1 was going.  One Assistant Instructor talked about the difficulty he was having with his international students.  He couldn’t seem to find a way to teach them how to evaluate sources because they just didn’t seem prepared to think critically about them. After a somewhat unsatisfying conversation (nobody really had any decent suggestions for him), I decided to look a bit into this issue in the library and education literature.  The above two articles offered interesting insights into the issue this Assistant Instructor presented, but have repercussions beyond just web evaluation in the introductory writing classroom.

UT Environment:  According to the 2009 preliminary enrollment figures, 9.1% of the UT student population is made up of international students.  Just over 3% of undergraduates are international students and just over 25% of graduate students are.

Because international students are such a varied group, I chose to focus on East Asian students (Chinese especially) to narrow the field a bit.  These two articles do a nice job of describing the cultural norms that East Asian/Chinese (depending on the article) students bring into the Western classroom.  A few items that seem relevant to information literacy instruction and active learning include:

  • emphasis on basic knowledge, memorization and repetition rather than defining and answering your own question
  • teacher as authority figure (not facilitator; not constructivist approach)
  • emphasis on harmony, not disagreeing, not voicing opinions
  • there is a right and a wrong answer; don’t question

This is obviously quite different from the Western classroom.  For example, the Signature Courses curriculum – courses designed to support first year college students in their transition to college academic expectations –  very explicitly emphasizes learning to disagree with one’s classmates and professors, albeit civilly (the opposite of harmony and keeping your opinion to yourself); self-reflection through writing, discussion, presentation;  learning to become a college student who can define and then explore his or her own intellectual questions, etc.  In other words, these facets of critical thinking that we are trying to teach freshmen how to do as they transition from high school to college students are culture shock to East Asian students.

The first article talks about how librarians can help students transition to this Western style, self-directed learning through the research paper assignment.  The author illustrates how the process of research paper writing can help international students begin to learn to question, evaluate and synthesize knowledge.  Its interesting and makes a good point, but doesn’t have much in the way of practical ideas librarians can use.

The second article, which isn’t in the library literature, doesn’t really propose a solution.  Instead it examines a group of East Asian Masters students adjusting from their own culture’s approach to teaching and learning to the Western approach and detrmines that these students, often by choice, never make it all the way there.  They choose a “middle path” that allows some questioning and evaluating but doesn’t give up on harmony.

Some questions to consider:

– Given the fact that a chunk of our students may be having this cultural experience, how can/should/does it impact information literacy and critical thinking instruction?

– If these students don’t like to talk, have opinions, disagree, etc., how do our standard approaches to source/web evaluation work in the classroom?

– if  these students aren’t taught to and don’t value questioning authority, etc., how do we find common ground to talk about source evaluation?

– If these students are used to being lectured to and then memorizing what they’ve been told, where does active learning fit in?