Lecture: Using evidence synthesis in linguistics and computational linguistics in evidence synthesis

While findings from single studies are limited in external validity, synthesizing evidence from multiple studies allows drawing more robust conclusions about a phenomenon. Evidence synthesis typically involves ‚systematic reviews’ which use transparent and objective methods as well as pre-defined appraisal criteria to synthesize all available research on a research question. This reduces risks of bias that traditional literature reviews are vulnerable to. Statistical methods may be used to aggregate results from included studies (meta-analysis) or not ('narrative synthesis', cf. Gough, Oliver & Thomas 2017).

In linguistics, evidence synthesis is used to advance linguistic theory (see Nicenboim, Roettger & Vasishth 2018 for phonetics; see van den Noort et al. 2019 for bilingualism research), potentially inform language policy (Norris & Ortega 2007; Berthele 2019) and guide clinical practice (e. g. concerning dementia, see Brini et al. 2020).

Current challenges include a lack of high-quality primary studies, redundancy due to duplicate work (for an example from linguistics, cf. Brini et al. 2020), shortcomings of statistical methods such as meta-analyses (cf. Nicenboim, Roettger & Vasishth 2018) and keeping reviews up to date in the light of a faster gowing body of research (cf. Westgate & Lindenmayer 2017).

Technical solutions from computational linguistics such as semantic networks and text mining approaches are already used to tackle some of these challenges by automizing labor-intensive stages of systematic reviews (Westgate & Lindenmayer 2017). Going further, Nakagawa et al. (2020) call for a “new ecosystem of evidence synthesis” in which technological solutions “promote openness and interconnectedness” (p. 3). They propose all empicists be organized in open synthesis communities where “synthesis is recognized as the end goal, as researchers design, undertake and report their work” (p. 1).

We will discuss the benefits of evidence synthesis communities for linguistic research and ask how computational linguistics may advance evidence synthesis methodology.

Berthele, Raphael. 2019. Policy recommendations for language learning: Linguists’ contributions between scholarly debates and pseudoscience. Journal of the European Second Language Association 3(1). 1–11. https://doi.org10.22599/jesla.50.

Brini, Stefano, Hamid R. Sohrabi, Jeffrey J. Hebert, Mitchell R. L. Forrest, Matti Laine, Heikki Hämäläinen, Mira Karrasch, Jeremiah J. Peiffer, Ralph N. Martins & Timothy J. Fairchild. 2020. Bilingualism Is Associated with a Delayed Onset of Dementia but Not with a Lower Risk of Developing it: a Systematic Review with Meta-Analyses. Neuropsychology Review. https://doi.org10.1007/s11065-020-09426-8.

Gough, David, Sandy Oliver & James Thomas. 2017. An Introduction to Systematic Reviews. 2nd edition. Los Angeles: SAGE.

Nakagawa, Shinichi, Adam G. Dunn, Malgorzata Lagisz, Alexandra Bannach-Brown, Eliza M. Grames, Alfredo Sánchez-Tójar, Rose E. O’Dea, et al. 2020. A new ecosystem for evidence synthesis. Nature Ecology & Evolution 4(4). 498–501. https://doi.org10.1038/s41559-020-1153-2.

Nicenboim, Bruno, Timo B. Roettger & Shravan Vasishth. 2018. Using meta-analysis for evidence synthesis: The case of incomplete neutralization in German. Journal of Phonetics 70. 39–55. https://doi.org10.1016/j.wocn.2018.06.001.

Noort, Maurits van den, Esli Struys, Peggy Bosch, Lars Jaswetz, Benoît Perriard, Sujung Yeo, Pia Barisch, Katrien Vermeire, Sook-Hyun Lee & Sabina Lim. 2019. Does the Bilingual Advantage in Cognitive Control Exist and If So, What Are Its Modulating Factors? A Systematic Review. Behavioral Sciences (Basel, Switzerland) 9(3). https://doi.org10.3390/bs9030027.

Norris, John M. & Lourdes Ortega. 2007. The Future of Research Synthesis in Applied Linguistics: Beyond Art or Science. TESOL Quarterly 41(4). 805–815. https://doi.org10.1002/j.1545-7249.2007.tb00105.x.

Westgate, Martin J. & David B. Lindenmayer. 2017. The difficulties of systematic reviews. Conservation Biology 31(5). 1002–1007. https://doi.org10.1111/cobi.12890.

Info

Day: 2020-05-22
Start time: 11:50
Duration: 00:30
Room: Chomsky
Track: Applied Linguistics
Language: en

Links:

Files

Feedback

Click here to let us know how you liked this event.

Concurrent Events