Jump to content

Research:Which parts of an article do readers read

Add links
From Meta, a Wikimedia project coordination wiki
Presentation slides from a presentation at Wikimania 2018 covering some of the results mentioned on this page

This research topic page provides an overview of results, methods and data sources pertaining to the question "Which parts of a Wikipedia article[notes 1] do users actually view?" .

Clickthrough rate of wikilinks by their position in a page (left = top, right = bottom; for the desktop version of English Wikipedia)
[edit]

A 2015 study of clickstream data (on desktop) from the English Wikipedia found that the rate at which wikilinks (internal links) are clicked decreases from top to bottom of the page, although more gradually than one might expect (see chart).[1]

Other researchers who looked at the same data estimated that wikilinks located in the lead section receive between 26% and 43% of the clicks on wikilinks.[2] A follow-up study found that although the lead and the infobox contain only 17% and 4% of the links of an article, they receive 32% and 18% of clicks, respectively [3]. Links located on the left side of the screen (assuming a browser window that takes up the entire screen on a standard WUXGA display on desktop) are more likely to be clicked[4].

During one month in 2019, "English Wikipedia generated 43M clicks to external websites, in roughly even parts via links in infoboxes, cited references, and article bodies". This corresponded to a much higher click-through rate (CTR) for infobox links (0.9%) than for article body links (0.14%) and reference links (0.03%).[5] (see also review in the Wikimedia Research Newsletter: "'Official' external links on Wikipedia generate $7-13 million worth of monthly traffic"

Presentation slide illustrating the instrumentation of reader interactions with citations

Another study by the same authors focused on clicks on external references on English Wikipedia, "find[ing] that overall engagement with citations is low: about one in 300 page views results in a reference click (0.29% overall; 0.56% on desktop; 0.13% on mobile). [...] clicks occur more frequently on shorter pages and on pages of lower quality, suggesting that references are consulted more commonly when Wikipedia itself does not contain the information sought by the user."[6] (See also: research project page, and a video recording and slides of a presentation in the June 2020 Wikimedia Research Showcase)

Section expansions

[edit]
On most mobile (non-tablet) views, the reader only looks at the article's introduction without opening further sections

On the mobile web version of Wikipedia, sections below the lede are by default collapsed (on devices below a certain screen size, i.e. on smartphones but not on tablets). The readers needs to tap on the section heading to read its content. These actions are counted - for a small sample of readers - in the MobileWebSectionUsage schema.

39.9% of the non-tablet mobile users who viewed a mainspace page on November 30 opened a section there, i.e. the median number of sections opened was 0.[7]

Frequency of section expansion in the article en:Barack Obama
Frequency of section expansion in the article en:World War II
This chart shows how far down into a page users read on the Android Wikipedia app (based on Schema:MobileWikiAppPageScroll; details)
The movement of the view pane (grey) is plotted across views of several Wikipedia articles of differing length (green, red etc.) for a reader researching questions about Diabetes. From Research:Directed diabetes info-seeking behavior in Wikipedia

Scroll actions

[edit]

(This refers to instrumentations that recorded which parts of a page appeared in the viewable area.)


...

On the Android Wikipedia app, around 68% of pageviews involve the reader scrolling down at least once (June 2017, excludes navigation via TOC).[8]

During one week in 2014, 25% of app users (devices) scrolled to the end of a page at least once.[9]

Eyetracking

[edit]

German thesis[10]: E.g. in "lookup" tasks, readers spend >45% of time on scanning TOC and lists ("QL-LI"), in "learn" tasks it's <10%

Related German paper[11]: "To get insights into users' interaction with pictorial and textual contents eye-tracking experiments are conducted. Spread of information within the articles and the relation between text and images are analyzed. ... By now 30 articles have been analyzed according to this scheme. There are 639 contact points leading to images. Results show that 39% of all contact points lead from image to image, in mutual directions (previous or next). All text contact points (T, TC, TB, TE, TN, Cit) sum up to a total of 37%. In 5% of all cases, an introduction triggers a saccade to an image. The remaining types of contact points occur rather rarely."

A 2012 conference paper by four researchers from Scotland, titled "Looking for genre: the use of structural features during search tasks with Wikipedia"[12] described the results of an eye tracking study with 30 participants asked to carry out various research tasks on Wikipedia. A main finding was that readers tended to look first at the table of contents, then at the article's infobox. More generally, they "extensively interacted with layout features, such as tables, titles, bullet lists, contents lists, information boxes, and references", and were also observed to frequently "skim and scroll" long articles.

A 2017 thesis found "that hyperlinked words [in English Wikipedia article text] are not more difficult to process than unlinked words, but readers do focus on hyperlinked words" and advised that "Selecting the most important content of the text as hyperlinks optimally helps the reader to gain the most relevant information of the text faster."[13]

An OpenSym 2021 paper[14] "present[ed] an Attention Feedback (AF) approach for Wikipedia readers. The fundamental idea of the proposed approach comprises the implicit capture of gaze-based feedback of Wikipedia readers using a commodity gaze tracker. [...] For each reading session, along with the gaze density heat map, we also provide a set of sentences where a user-focused while reading along with the time for which each sentence was focused. [...] After processing the sentences, we arrange them in the order they are read along with their gaze quotient. By gaze quotient, we mean the time duration (in seconds) for which a sentence is being gazed at or read. [...] the proposed AF framework also captures some additional information [...]: (1) Wikilink clicks [...] (2) Eye blinks [...] (3) Scroll events [...]". At the time, the "study’s outcomes [were] being discussed in the Wikimedia Foundation for developing specialized tools to capture readers’ implicit feedback."

See also: Demo video of an affordable eyetracking system used on a Wikipedia article (2018)

Page previews

[edit]

The page previews feature was introduced in 2017/18 on desktop Wikipedia. Reader can hover their mouse over a link to see an excerpt of the linked page. An internal dataset contains aggregated numbers on how many previews were viewed for a given link, which (similarly to the clickstream data mentioned above) can be used to generate a heatmap of hovers that indicates which parts of the page were read, but also which topics (links) readers are most interested in looking up briefly from the source page.

Demographic differences

[edit]

A 2019 study found that readers in the Global South spend substantially more time on average reading a page than readers in the Global North.[15]

References

[edit]
  1. ↑ Ashwin Paranjape, Bob West, Jure Leskovec, Leila Zia: Improving Website Hyperlink Structure Using Server Logs. WSDM’16, February 22–25, 2016, San Francisco, CA, USA. PDF
  2. ↑ Lamprecht, Daniel; Helic, Denis; Strohmaier, Markus (2015-04-22). "Quo Vadis? On the Effects of Wikipedia's Policies on Navigation". Ninth International AAAI Conference on Web and Social Media. Ninth International AAAI Conference on Web and Social Media. 
  3. ↑ Lamprecht, Daniel; Lerman, Kristina; Helic, Denis; Strohmaier, Markus (May 2016). "How the structure of Wikipedia articles influences user navigation". New Review of Hypermedia and Multimedia. doi:10.1080/13614568.2016.1179798. Retrieved December 15, 2016. 
  4. ↑ Dimitrov, Dimitar; Singer, Philipp; Lemmerich, Florian; Strohmaier, Markus (2016-04-11). Visual Positions of Links and Clicks on Wikipedia (PDF). 25TH INTERNATIONAL WORLD WIDE WEB CONFERENCE. Montréal, Québec, Canada. p. 2. 
  5. ↑ Piccardi, Tiziano; Redi, Miriam; Colavizza, Giovanni; West, Robert (2021-04-19). "On the Value of Wikipedia as a Gateway to the Web". Proceedings of the Web Conference 2021. New York, NY, USA: Association for Computing Machinery. pp. 249–260. ISBN 9781450383127. arXiv:2102.07385. doi:10.1145/3442381.3450136. Wikidata:Q109589191.  Closed access. Code.
  6. ↑ Piccardi, Tiziano; Redi, Miriam; Colavizza, Giovanni; West, Robert (2020-04-20). "Quantifying Engagement with Citations on Wikipedia". Proceedings of The Web Conference 2020. WWW '20. New York, NY, USA: Association for Computing Machinery. pp. 2365–2376. ISBN 9781450370233. doi:10.1145/3366423.3380300.  Closed access Author's copy
  7. ↑ https://phabricator.wikimedia.org/T118041#1847031
  8. ↑ https://phabricator.wikimedia.org/P5629
  9. ↑ [1] "25% of install base saw at least one read more panel", meaning that these app users read or scrolled to the end of an article at least once (where these panels are located).
  10. ↑ Knäusl, Hanna (2014-12-18). "Situationsabhängige Rezeption von Information bei Verwendung der Wikipedia" (Thesis of the University of Regensburg).  p. 202 (in German, with English abstract), cf. 2012 poster
  11. ↑ Rösch, Barbara (2014). "Investigation of Information Behavior in Wikipedia Articles". Proceedings of the 5th Information Interaction in Context Symposium. IIiX '14. New York, NY, USA: ACM. pp. 351–353. ISBN 978-1-4503-2976-7. doi:10.1145/2637002.2637062.  Closed access
  12. ↑ Clark, Malcolm; Ruthven, Ian; O’Brian Holt, Patrik and Song, Dawei (2012). Looking for genre: the use of structural features during search tasks with Wikipedia. Fourth Information Interaction in Context Conference (IIiX 2012). DOI • PDF
  13. ↑ Martikainen, Hanna (2018-10-31). "Mind the Links! How Hyperlinks Influence Online Reading and Navigation : An Eye Movement Study".  Graduate thesis in psychology, University of Turku, 2017
  14. ↑ Dubey, Neeru; Verma, Amit Arjun; Iyengar, S. R. S.; Setia, Simran (2021-09-15). "Implicit Visual Attention Feedback System for Wikipedia Users". 17th International Symposium on Open Collaboration. New York, NY, USA: Association for Computing Machinery. pp. 1–11. ISBN 9781450385008. 
  15. ↑ TeBlunthuis, Nathan; Bayer, Tilman; Vasileva, Olga (20 August 2019). "Dwelling on Wikipedia: investigating time spent by global encyclopedia readers". Proceedings of the 15th International Symposium on Open Collaboration: 1–14. doi:10.1145/3306446.3340829. 

Notes

[edit]
  1. ↑ (or other pages on Wikimedia projects)


[edit]
Retrieved from "https://meta.wikimedia.org/w/index.php?title=Research:Which_parts_of_an_article_do_readers_read&oldid=28710404"