Validating a CIPP-based Inventory to Evaluate Undergraduate-Level Technical English Translation Courses
Abstract
One of the most important factors influencing a valid evaluation is the existence of an appropriate instrument that can accurately evaluate a given program. The purpose of this study was to design a valid and reliable instrument to evaluate BA-level English translation program in Iran. This study was conducted in 2019 and 2020 at different English language faculties in Iran offering BA programs of English translation for more than 10 years. For the purpose of the study, having consulted the existing sources on Context-Input-Process-Product (CIPP) evaluation model, a researcher-made inventory was developed. The next stage included assessing the validity of the instrument through expert judgment and measuring its content validity index. Finally, the reliability of the questionnaire, was estimated by calculating Cronbach's alpha coefficient and Guttmann's split-half coefficient. The final edition of the questionnaire was then distributed among 147 junior and senior undergraduate students of English translation in six different faculties. According to the results. The observed Cronbach's alpha for the instrument was .89 and the observed Guttman's split-half coefficient was .78.in addition, confirmatory factor analysis was done to indorse the construct validity of the inventory. Accordingly, it was concluded that the inventory enjoys desirable validity and reliability and thus, can be used for evaluating different technical translation courses.
Keywords:
Program Evaluation, CIPP Evaluation Model, Technical Translation Courses, ValidationReferences
Akbari, R., & Yazdanmehr, E. (2011). EFL teachers’ recruitment and dynamic assessment in private language institutes of Iran. Journal of English Language Teaching and Learning, 8, 30–51.
Alikina, E. V., Kushnina, L. V., Naugolnykh, A. Y., & Falko, K. I. (2019). Integrative Methodology of Teaching Translation and Interpreting. In The International Conference Going Global through Social Sciences and Humanities (pp. 245–251). Springer, Cham.
Al-Jardani, K. S. S. (2012). English language curriculum evaluation in Oman. International Journal of English Linguistics, 2(5), 40–44.
Aloreibi, A. and Carey, M. D. (2017). English Language Teaching in Libya After Gaddafi. In: R. Kirkpatrick (Ed.). English Language Education Policy in the Middle East and North Africa (pp. 93–114). Springer International Publishing.
Atai, M. R., Babaii, E., & Mousavi, M. (2016). Exploring standards and developing a measure for evaluating Iranian EFL teachers’ professional competence in the private sector. Iranian Journal of English for Academic Purposes, 5(2),30–5.
Aziz, S., Mahmood, M., & Rehman, Z. (2018). Implementation of CIPP model for quality evaluation at school level: A case study. Journal of Education and Educational Development, 5(1), 189–206.
Barrett, A. M., Duggan, R. C., Lowe, J., Nikel, J., & Ukpo, E. (2006). The concept of quality in education: A review of the international literature on the concept of quality in education. EdQual: RPC.
Birjandi, P., & Nosratinia, M. (2009). The qualitative program evaluation of the postgraduate English Translation major in Iran. The Journal of Modern Thoughts in Education, 4 (4), 37–58.
Colina, S., & Angelelli, C. V. (Eds.). (2017). Translation and interpreting pedagogy in dialogue with other disciplines (Vol. 90). John Benjamins Publishing Company.
Crabb, A. and Leroy, P. (2012). The handbook of environmental policy evaluation. Routledge.
Darussalam, G. (2010). Program Evaluation in Higher Education. International Journal of Research and Review, 5(2), 56–65.
Eryanto, H., Swaramarinda, D. R., & Nurmalasari, D. (2019). Effectiveness of entrepreneurship practice program: Using CIPP program evaluation. Journal of Entrepreneurship Education, 22(1), 1–10.
Gholami, J., Sarkhosh, M., & Abdi, H. (2016). An exploration of teaching practices of private, public, and public-private EFL teachers in Iran. Journal of Teacher Education for Sustainability, 18(1), 16–33. doi:10.1515/jtes-2016–0002
HesabiA. (2014). A Comparative Study of Curriculum in Masters’ Degree of Translation Studies between Western Universities and University of Isfahan. Language and Translation Studies, 46(3). 61–86. https://doi.org/10.22067/lts.v46i3.39394
Jain, C., & Prasad, N. (2018). Quality of secondary education in India. Singapore: Springer Nature.
Kafi, Z., Motallebzadeh, K., Khodabakhshzadeh, H., Zeraatpisheh, M., & Lu, X. (2019). Developing, glocalizing & validating a quality indices rubric in English language teaching: A case of CIPP model. Cogent Education, 6(1), 16–23.
Kaveh, F., & Karimnia, A. (2015). Translation Studies Program Evaluation at Master’s Level in Iran: A Study on Islamic Azad University Curriculum Design. Journal of Applied Linguistics, 1(2), 21–30.
Manan, A., Fadhilah, M. A., Kamarullah, K., & Habiburrahim, H. (2020). Evaluating paper-based TOEFL preparation program using the Context, Input, Process, and Product (CIPP) model. Studies in English Language and Education, 7(2), 457–471.
Mangal, S. K., & Mangal, S. (2013). Research methodology in behavioral sciences. PHI Learning Pvt. Ltd.
McDonald, M. (2002). Systematic assessment of learning outcomes. Jones & Bartlett Learning.
Rezvani, R., & Vakilinejad, M. (2014). An Evaluation of Translation Quality Assessment course: Voices from Instructors. Procedia-Social and Behavioral Sciences, 98, 1563–1571.
Rezvani, R., & Vakilinejad, M. (2015). Examining Curriculum Efficiency from Learners’ Perspective: A Case Study of Translation Criticism Course of Iranian Universities. International Journal of Applied Linguistics and English Literature, 4(1), 151–158.
Rezvani, R., Farahzad, F., & Vakilinejad, M. (2012). Looking at the Overlooked: Students' Needs and Perceptions in MA Translation Quality Assessment Curriculum. Translation Studies 38, 83–98.
Salari, Z., & Khazaee Farid, A. (2016). The Necessity of Updating the English Translation BA Syllabus Based on Translation Market in Iran. Language and Translation Studies, 48(1), 19–51. https://doi.org/10.22067/lts.v48i1.37518
Shishavan, H., & Sadeghi, K. (2009). Characteristics of an effective English language teacher as perceived by Iranian teachers and learners of English. English Language Teaching, 2(4). doi:10.5539/elt.v2n4p130
Sopha, S., & Nanni, A. (2019). The CIPP Model: Applications in Language Program Evaluation. Journal of Asia TEFL, 16(4), 13–60.
Stufflebeam, D. L. (2002). The CIPP model for evaluation. In D. L. Stufflebeam, C. F. Madam, & T. Kellaghan (Eds.), Evaluation models (pp. 279–317). New York: Kluwer Academic Publishers.
Stufflebeam, D. L. (2003). The CIPP model for evaluation. In International handbook of educational evaluation (pp. 31–62). Springer, Dordrecht.
Stufflebeam, D. L. (2005). The CIPP Model. In: S. Mathison (Ed.) Encyclopedia of Evaluation (pp. 60–65). Thousand Oaks, CA: Sage Publications.
Stufflebeam, D. L. and Shinkfield, A. J. (2007). Evaluation theory, models and applications. San Francisco, CA: Jossey Bass.
Stufflebeam, D. L. and Zhang, G. (2017). The CIPP evaluation model: How to evaluate for improvement and accountability. Guilford Publications.
Wang, Y. (2008). Influence of Planning on Students’ Language Performance in Task-Based Language Teaching. English language teaching, 1(1), 83–86.
Zhang, G., Zeller, N., Griffith, N., Metcalf, M., Williams, J., Shea, C., & Misulis, K. (2011). Using the context, input, process, and product evaluation model (CIPP) as a comprehensive framework to guide the planning, implementation, and assessment of service-learning programs. Journal of Higher Education Outreach and Engagement, 15(4), 57–84.
Published
How to Cite
Issue
Section
License
Copyright Licensee: Iranian Journal of Translation Studies. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution–NonCommercial 4.0 International (CC BY-NC 4.0 license).
