Content Validity of an Instrument to Assess Innovation Competencies in Engineering Education

Abstract

Innovation competency has become an essential requirement for engineering graduates as they navigate increasingly complex and technologically advanced environments. This study aims to develop and validate an instrument to assess innovation competencies among engineering students in Malaysian technical and vocational institutions. The construction of the instrument was guided by grounded theory insights obtained from structured interviews with engineering educators, together with a synthesis of current literature and ABET-aligned competency frameworks. This process established three major dimensions knowledge, skills, and personality which formed the basis for an initial pool of 53 items. The content validity of the instrument was evaluated by a panel of five experts using the Content Validity Index (CVI) and the modified Kappa statistic. Experts rated each item for relevance using a four-point scale. The results demonstrated strong overall agreement, with most items achieving I-CVI values between 0.80 and 1.00 and Kappa coefficients indicating good to excellent concordance. Items falling below acceptable thresholds were removed or refined based on expert judgment. As a result, 42 items were retained, representing essential components of innovation competency across the three dimensions. The findings confirm that the instrument possesses strong content validity and is suitable for further psychometric evaluation. This validated item set offers educators, curriculum designers, and policymakers a structured and evidence-based tool for assessing innovation readiness among engineering students. The instrument has potential applications in evaluating program effectiveness, identifying student competency gaps, and supporting targeted instructional improvements. Future research should incorporate pilot testing, reliability analysis, exploratory and confirmatory factor analysis, and multi-institutional validation to strengthen the instrument’s robustness, generalizability, and practical utility.