Editor’s note: Guest author Jeremy Brueck is an educator and PhD candidate from the University of Akron, who is researching the use of e-Books and mobile devices in early literacy learning. This is part 5 of a 5 part series for EdReach.us called ResearchEd. You can follow him on Twitter @brueckj23.
So, is Quality Rating Tool 2 (publicly accessible at http://bit.ly/eQRT_v2) any better? We will now hone in on our data to see if we have achieved better reliability with Tool 2. The following 4 charts show the inter-rater reliability on three subscales of version 2 of the e-Book Quality Rating Tool. Ten independent raters evaluated 5 books by assigning each book a score between 1 (strongly disagree) and 5 (strongly agree) on ease of use, multimedia and interaction features. Ratings were considered reliable if they fell within +1 point of the rating assigned by the Expert Rater, the project technology expert.