Measuring the readability of exhibit panels in museums


== Resumen ==

Se evalúa la comprensibilidad de textos en español en paneles de museos, considerando que están dirigidos a todos los públicos y escritos en lenguaje estándar. Se recopilan los indicadores propuestos en la bibliografía científica sobre la comprensibilidad y se analiza su aplicación a los paneles de museos. Se ha construido un corpus con los textos de paneles de seis museos y se evalúa la dificultad de los paneles mediante las métricas clásicas y tests sobre la percepción del usuario. Con métodos de aprendizaje automático se analiza la capacidad de estas métricas clásicas para pronosticar la dificultad para el usuario. Se han añadido indicadores lingüísticos y de familiaridad del término para mejorar la precisión del pronóstico. Lo más eficaz para predecir el grado de comprensibilidad es un modelo híbrido de indicadores clásicos, lingüísticos y de familiaridad con los términos.


[https://recyt.fecyt.es/index.php/EPI/article/view/epi.2018.may.10/40071 <span style="color:#0645AD; font-weight: bold">Pulsa aquí para acceder a la version PDF</span>]



== Referencias ==

Aluísio, Sandra; Gasperin, Caroline (2010). “Fostering digital inclusion and accessibility: The PorSimples project for simplification of portuguese texts”. En: Procs of the NAACL HLT 2010 Young investigarors workshop on computational approaches to languages of the Americas, Los Angeles, California, June, pp. 46-53. http://www.aclweb.org/anthology/W10-1607

Aluisio, Sandra; Specia, Lucia; Gasperin, Caroline; Scarton, Carolina (2010). “Readability assessment for text simplification”. En: Fifth workshop on innovative use of NLP for building educational applications, pp. 1-9. https://goo.gl/S8L2WJ

Badarudeen, Sameer; Sabharwal, Sanjeev (2010). “Assessing readability of patient education materials: current role in orthopaedics”. Clinical orthopaedics and related research, v. 468, n. 10, pp. 2572-2580. https://doi.org/10.1007/s11999-010-1380-y

Blanco, Roi; Lioma, Christina (2012). “Graph-based term weighting for information retrieval”. Information retrieval, v. 15, n. 1, pp. 54-92. https://doi.org/10.1007/s10791-011-9172-x

Bott, Stefan; Rello, Luz; Drndarevic, Biljana; Saggion, Horacio (2012). “Can Spanish be simpler? LexSiS: Lexical simplification for Spanish”. En: Proceedings of Coling 2012, pp. 357-374. https://aclweb.org/anthology/C/C12/C12-1023.pdf

Carbon, Claus-Christian (2017). “Art perception in the museum: How we spend time and space in art exhibitions”. i-Perception, v. 8, n. 1. https://doi.org/10.1177/2041669517694184

Dale, Edgar; Chall, Jeanne S. (1948). “A formula for predicting readability”. Educational research bulletin, v. 27, n. 1, pp. 11-28.

Elhadad, Noémie (2006). “Comprehending technical texts: Predicting and defining unfamiliar terms”. En: AMIA Annual symposium proceedings, pp. 239-243. https://www.ncbi.nlm.nih.gov/pubmed/17238339

España (2002). “Ley 34/2002, de 11 de julio, de servicios de la sociedad de la información y de comercio electrónico”. BOE, n. 166, 12 julio. https://www.boe.es/buscar/act.php?id=BOE-A-2002-13758

España (2013) “Ley 19/2013, de 9 de diciembre, de transparencia, acceso a la información pública y buen gobierno”. BOE, n. 295, 10 diciembre. https://www.boe.es/buscar/doc.php?id=BOE-A-2013-12887

Feng, Lijun; Jansche, Martin; Huenerfauth, Matt; Elhadad, Noémie (2010). “A comparison of features for automatic readability assessment”. En: Procs of the 23rd Intl conf on computational linguistics: Posters, pp. 276-284. http://dl.acm.org/citation.cfm?id=1944566.1944598

Ferrando-Belart, Vicky (2004). “La legibilidad: un factor fundamental para comprender un texto”. Atención primaria, v. 34, n. 3, pp. 143-146. https://goo.gl/E3S4Ko

Fleiss, Joseph L. (1971). “Measuring nominal scale agreement among many raters”. Psychological bulletin, v. 76, n. 5, pp. 378-382. https://doi.org/10.1037/h0031619

François, Thomas; Fairon, Cédrick (2012). “An ‘AI readability’ formula for French as a foreign language”. En: Joint conference on empirical methods in natural language processing and computational natural language learning, Jeju Island, Korea, 12-14 July, pp. 466-477. http://www.aclweb.org/anthology/D12-1043

François, Thomas; Miltsakaki, Eleni (2012). “Do NLP and machine learning improve traditional readability formulas?”. En: Proceedings of the First workshop on predicting and improving text readability for target reader populations, pp. 49-57. http://dl.acm.org/citation.cfm?id=2390916.2390925

Graesser, Arthur C.; McNamara, Danielle S.; Kulikowich, Jonna M. (2011). “Coh-Metrix: Providing multilevel analyses of text characteristics”. Educational researcher, v. 40, n. 5, pp. 223-234. https://doi.org/10.2307/2529310

Graesser, Arthur C.; McNamara, Danielle S.; Louwerse, Max M.; Cai, Zhiqiang (2004). “Coh-Metrix: Analysis of text on cohesion and language”. Behavior research methods, instruments, & computers, v. 36, n. 2, pp. 193-202. https://doi.org/10.3758/BF03195564

Instituto de Turismo de España (2014). Museo del Prado. Caracterización de los visitantes. Encuesta a los visitantes del Museo del Prado. https://goo.gl/k5c19g

Kauchak, David; Leroy, Gondy; Hogue, Alan (2017). “Measuring text difficulty using parse-tree frequency”. Journal of the Asociation for Information Science and Technology, v. 68, n. 9, pp. 2088-2100. https://doi.org/10.1002/asi.23855

Keselman, Alla; Tse, Tony; Crowell, Jon; Browne, Allen; Ngo, Long; Zeng, Qing (2007). “Assessing consumer health vocabulary familiarity: An exploratory study”. Journal of medical internet research, v. 9, n. 1, e5. https://doi.org/10.2196/jmir.9.1.e5

Landis, J. Richard; Koch, G. Gary (1977). “The measurement of observer agreement for categorical data”. Biometrics, v. 33, n. 1, pp. 159-174. http://www.ncbi.nlm.nih.gov/pubmed/843571

Larsson, Patrik (2006). Classification into readability levels. Implementation and evaluation. Uppsala Universitet. Department of Linguistics and Philology. https://goo.gl/YuTSvq

Leroy, Gondy; Endicott, James E. (2011). “Term familiarity to indicate perceived and actual difficulty of text in medical digital libraries”. En: Xing, Chunxiao; Crestani, Fabio; Rauber, A. (eds.). Digital libraries for cultural heritage, knowledge dissemination, and future creation: 13th Intl conf on Asia-Pacific digital libraries, Icadl 2011, Beijing, China, October 24-27, v. 7008, pp. 307-310. https://doi.org/10.1007/978-3-642-24826-9_38

Leroy, Gondy; Endicott, James E. (2012). “Combining NLP with evidence-based methods to find text metrics related to perceived and actual text difficulty”. En: Proceedings of the 2nd ACM Sighit Intl health informatics symposium, pp. 749-754. https://doi.org/10.1145/2110363.2110452

Leroy, Gondy; Endicott, James E.; Kauchak, David; Mouradi, Obay; Just, Melissa (2013). “User evaluation of the effects of a text simplification algorithm using term familiarity on perception, understanding, learning, and information retention”. Journal of medical internet research, v. 15, n. 7, e144.

Ministerio de Cultura (2011). Conociendo a nuestros visitantes. Estudio de público en museos del Ministerio de Cultura. Resumen ejecutivo. Ministerio de Educación, Cultura y Deporte. https://goo.gl/Xhirta

Montanero-Fernández, Manuel (2004). “Cómo evaluar la comprensión lectora: alternativas y limitaciones”. Revista de educación, v. 335, pp. 415-424. https://dialnet.unirioja.es/servlet/articulo?codigo=1066564

Morato, Jorge; Llorens, Juan; Génova, Gonzalo; Moreiro-González, José-Antonio (2003). “Experiments in discourse analysis impact on information classification and retrieval algorithms”. Information processing & management, v. 39, n. 6, pp. 825-851. https://doi.org/10.1016/S0306-4573(02)00081-X

Morato, Jorge; Sánchez-Cuadrado, Sonia; Moreno, Valentín; Moreiro-González, José-Antonio (2013). “Evolución de los factores de posicionamiento web y adaptación de las herramientas de optimización”. Revista española de documentación científica, v. 36, n. 3, e018. https://doi.org/10.3989/redc.2013.3.956

Newbold, Neil; McLaughlin, Harry; Gillam, Lee (2010). “Rank by readability: Document weighting for information retrieval”. En: Cunningham, Hamish; Hanbury, Allan; Rüger, Stefan. Advances in multidisciplinary retrieval. IRFC. Lecture notes in computer science, v. 6107, pp. 20-30. https://doi.org/10.1007/978-3-642-13084-7_3

Nomura, Misako; Nielsen, Gyda S.; Tronbacke, Bror (2010). Guidelines for easy-to-read materials. IFLA professional reports. https://www.ifla.org/files/assets/hq/publications/professional-report/120.pdf

OECD (2016). Skills matter. Further results from the survey of adult skills. OECD Publishing. ISBN: 978 92 64 25804 4 https://doi.org/10.1787/9789264258051-en

Padró, Lluís; Collado, Miquel; Reese, Samuel; Lloberes, Marina; Castellón, Irene (2010). “FreeLing 2.1: Five years of open-source language processing tools”. En: Procs of the 7th Intl conf on language resources and evaluation (LREC’10), pp. 931-936. https://goo.gl/bpQb1P

Pitler, Emily; Nenkova, Ani (2008). “Revisiting readability: A unified framework for predicting text quality”. In: Procs of the Conf on empirical methods in natural language processing, pp. 186-195. http://dl.acm.org/citation.cfm?id=1613715.1613742

Sigaud-Sellos, Pedro (2010). Aproximación a los conceptos de legibilidad y lecturabilidad: aplicación a la lectura de textos digitales. Pamplona: Universidad de Navarra. https://goo.gl/2uh4K3

Social Science Consulting (2013). TextQuest. http://www.textquest.de/tq-man42.pdf

Stenner, A. Jackson (1996). “Measuring reading comprehension with the lexile framework”. En: 4th North American conf on adolescent/adult literacy. Washington, D.C. https://goo.gl/tVTSbC

Stenner, A. Jackson; Smith, Malbert; Burdick, Donald S. (1983). “Toward a theory of construct definition”. Journal of educational measurement, v. 20, n. 4, pp. 305-316. https://doi.org/10.1111/j.1745-3984.1983.tb00209.x

Temnikova, Irina; Vieweg, Sarah; Castillo, Carlos (2015). “The case for readability of crisis communications in social media”. In: Procs of the 24th Intl conf on world wide web, pp. 1245-1250. https://doi.org/10.1145/2740908.2741718

Tonelli, Sara; Tran-Manh, Ke; Pianta, Emanuele (2012). “Making readability indices readable”. En: Naacl-HLT 2012 Workshop on predicting and improving text readability for target reader populations, pp. 40-48. http://www.aclweb.org/anthology/W12-2206

Van-Oosten, Philip; Tanghe, Dries; Hoste, Veronique (2010). “Towards an improved methodology for automated readability prediction”. En: LREC 2010: Seventh conference on international language resources and evaluation, pp. 775-782. http://hdl.handle.net/1854/LU-1055826

Venturi, Giulia; Bellandi, Tommaso; Dell’Orletta, Felice; Montemagni, Simonetta (2015). “NLP-based readability assessment of health-related texts: A case study on Italian informed consent forms”. En: Procs of the 6th intl workshop on health text mining and information analysis, pp. 131-141. http://www.aclweb.org/anthology/W15-2618

Witten, Ian H.; Frank, Eibe; Hall, Mark A. (2011). “Data mining: Practical machine learning tools and techniques” (3rd ed.). San Francisco (USA): Morgan Kaufmann Publishers. ISBN: 978 0 123748560

Zeng-Treitler, Qing; Kim, Hyeoneui; Goryachev, Sergey; Keselman, Alla; Slaughter, Laura; Smith, Catherine-Arnott (2007). “Text characteristics of clinical reports and their implications for the readability of personal health records”. Studies in health technology and informatics, v. 129, pp. 1117-1121. http://ebooks.iospress.nl/publication/11151