A user-sensitive resource quality assessment approach for health information portals

Information quality control is a critical issue in online health information provision. As one of effective quality control approaches, metadata-driven health information portals provide direct access to the descriptions of selected online resources that are of perceived high quality and relevancy to targeted portal users. This is achieved via a review process manually undertaken by domain experts, who have expertise in both healthcare and information management areas. Due to the subjective, contextual, and dynamic nature of information quality, the labour-intensity of resource quality (RQ) assessment becomes a bottleneck, especially when taking account of diverse user needs and values. Determining the quality of online information resources is the interplay of domain experts, portal content management systems, and RQ assessment processes. Yet, how to support contextual value judgements on RQ from a user-sensitive viewpoint, have not been sufficiently addressed in the literature and practice. The emergence of socio-technical solutions is imperative to improve the scalability and finally the sustainability of RQ assessment processes. This research endeavours to find new approaches that employ intelligent technologies to support decision-making processes of RQ assessment for health information portals. Using a socio-technical design science research approach, the research investigated RQ assessment issues through a user-sensitive systems development research process. It involved three interconnected research phases of concept building, system building, and system evaluation. As a result, a semi-automated and user-sensitive RQ assessment approach was proposed to standardise and facilitate decision-making processes on RQ. The concept building research phase began with a comprehensive analysis of multi-disciplinary research literature, which investigated user-sensitive RQ assessment issues from theoretical, contextual, and technological perspectives. In addition, an exploratory case study of RQ assessment practices was conducted in the context of two metadata-driven health information portals in order to identify domain expert needs and corresponding design requirements of a RQ assessment approach. The conceptualisation of the approach encompassed a user-sensitive quality assessment framework and an intelligent quality tool. The framework defined the construct of RQ as a composition of Reliability and Relevancy in the healthcare domain. The measure of Reliability dimension was defined using an attribute-based approach. In the system building research phase, the feasibility of the proposed approach was tested through the design and development of the Domain Expert Dashboard (DED) prototype system. Machine learning techniques were applied to implement an intelligent system feature. Several other system features were also developed as part of the quality tool in order to meet the decision support needs of domain experts. In the system evaluation research phase, a functional test and a usefulness and usability study were conducted to assess the effects of the DED prototype system on the RQ decision-making processes and outcomes against a multi-criteria evaluation framework. The results demonstrated that both the processes and the outcomes of RQ assessment were improved through the use of the prototype system. This research makes significant theoretical, methodological, and practical contributions. The proposed user-sensitive RQ assessment framework integrates and extends the context-based information quality assessment theory. The framework measure perceived RQ as a relative and aggregated construct by specifying and mapping resource attributes to the characteristics of user information needs and quality perceptions. A more generic method is also proposed for developing domain-specific and user-sensitive RQ assessment metrics for other domains. Moreover, the study adapts the canonical artefact-centric design science research framework in a socio-technical context. Mixed methodologies have been employed to conceptualise and evaluate a socio-technical solution. Theory-building has played a central role, which informed the concept-building of design artefacts. The practical contributions include the conceptual architecture and the instantiation of a quality tool using intelligent technologies. The DED prototype system provides greater functionality to support domain experts making contextual value judgements on RQ, demonstrating how intelligent learning techniques can be applied to describe the quality attributes of online resources. The system has been integrated as part of an operational health information portal, the Breast Cancer Knowledge Online (BCKOnline) portal.