This study was a retrospective cross-sectional observational psychometric investigation.

The study took place across three university hospitals in metropolitan areas of the United Kingdom between September 2015 and July 2016. Availability sampling was used to identify MTBs from the UK National Health Service (NHS) that represent the most common cancer types. Three MTBs were identified and participated in the study including breast, colorectal, and gynecologic MTBs.

The study participants were 44 tumor board members: 15 breast, 15 colorectal, and 14 gynecologic professionals. The tumor boards had the same composition of surgeons (n = 12), oncologists (n = 6), cancer nurse specialists (CNS) (n = 12), radiologists (n = 6), histopathologists (n = 5), and coordinators (administrative role, n = 3). The groups were at the attending level, with an average 9 years of experience (minimum, 2 years; maximum, 22 years). A detailed team composition breakdown has been published previously.12,22,23 Ethical approvals were given by the North West London Research Ethics Committee and locally by the participating hospitals as part of the original research.23 Oral and written consents were given by the team members.

All case discussions during the study period were video-recorded, including discussions on suspected or confirmed cancer. The study included 146 consecutive case discussions from six tumor boards. The dataset is available on Zenodo.24 The sample size was determined using G*Power 325 for a priori power analysis with a two-tailed test, an effect size of 0.6, a probability of 0.05, and a power of 0.90.

We developed MODe-Lite from the original MODe instrument7 and its previous content validation9 positing a reduction in the number of the original items. A multi-phased approach was undertaken in developing MODe-Lite.

In phase 1 (objective A), two meetings were held with experts (B.W.L., a clinical researcher and consultant surgeon, and T.S., a clinical researcher and psychologist) with more than 5 years of experience in the use of MODe and evidence4,9,12 of proficiency with inter-assessor reliability higher than 0.70.26 The aim was to design a new tool and to assess its content validity against MODe’s previous content validation.9

In phase 2 (objective B), assessor 1 (B.W.L.) scored a sample of 146 cases that had previously been assessed with MODe using the revised shorter version of the tool (MODe-Lite). The aim was to assess convergent validity with the original tool.7 We hypothesized (H1) that the MODe-Lite domains would correlate positively with the relevant items of the original MODe,7 in line with the factor model proposed in previous research.9 In addition, we aimed to validate MODe-Lite externally against case complexity as measured by the previously validated Measure of Case-Discussion Complexity (MeDiC).20,21 We hypothesised (H2) that MODe-Lite would positively correlate with the MeDiC tool, in line with the previous research using the MODe instrument.12

In phase 3 (objective C), both assessors (B.W.L. and T.S.) trained a new assessor (S.M., a consultant surgeon) in the use of the shorter version (MODe-Lite) on video-recorded tumor boards during two 2-h-long sessions. The assessors (B.W.L. and S.M.), blinded to the each other’s scores, scored a subset of 60 cases. Disagreements were subsequently discussed during a single 2-h data-review session to understand how the scoring of the shorter tool could be improved. The aim was to determine feasibility and inter-assessor reliability in the use of MODe-Lite.

The validity of MODe-Lite was assessed using a widely used measure, the item-content validity index (I-CVI).27,28 The criteria for item acceptability depends on the number of experts rating the items.27,28 If the experts are fewer than five, all five must agree for the item to be retained.27,28

We performed convergent validity analysis of MODe-Lite by assessing the correlation between individual domains and the original MODe instrument (the individual items of MODe and the global score). We also used the overall MODe-Lite score for the item-total correlation.

We assessed the reliability between the two assessors (B.W.L. and S.M.) using kappa coefficients for categorical items (i.e., the individual items of MODe-Lite) and interclass correlation coefficients (ICCs) for continuous items (i.e., the global scores). For the ICCs, a generally accepted reliability coefficient of 0.70 or higher was used,26 whereas for the kappa coefficients, the following criteria applied: fair agreement (0.21–0.40), moderate agreement (0.41–0.60), substantial agreement (0.61–0.80), and almost perfect agreement (0.81–1.00).29 Cronbach's alpha was calculated to assess the internal consistency for each MODe-Lite domain (i.e., how closely related these domains are against their corresponding items in the original MODe tool).

Table 2 shows summary statistics for the individual domains of MODe-Lite across the entire dataset (n = 146) comprising three tumor boards, namely, breast, colorectal and gynecologic boards. The holistic input and clinical collaboration domains scored lowest, indicating overall lower quality. Pathology and clinical input scored highest, indicating better quality.Table 2

Summary statistics for the MODe-Lite domains^a

Two focus group meetings were held between two expert assessors (B.W.L. and T.S.) for content validation using all 12 items from the original MODe instrument.7 Guided by the previous content validation of the MODe tool with a large sample (n = 1045) using exploratory factor analysis,9 the factor model containing clinical and holistic (patient history; oncologists’, surgeons’, and nurses’ inputs; psychosocial information; comorbidities; patient view), radiology (radiology information and radiologists’ inputs), and pathology (pathology information and pathologists’ input) components received full agreement (I-CVI = 1) for inclusion into MODe-Lite.

Because of the substantial research evidence supporting holistic information1,4,5,9,11,30,31 and clinical collaboration1,4,5,11,32–34 adequately captured in the assessments for team quality improvements,10,11 it was agreed by the expert assessors (I-CVI = 1) that they should be scored separately, resulting in three domains: clinical input, holistic input, and clinical collaboration. Pathology and radiology were retained as separate domains (I-CVI = 1), in line with the factor analysis.9 Management plan, an outcome variable in the factor analysis and previous research,9 also was retained (I-CVI = 1) as a separate item. Therefore, a total of six domains received full agreement for inclusion in MODe-Lite by the expert assessors (I-CVI = 1) and were subjected to further validity and reliability testing. The tool is represented graphically in Fig. 1.Fig. 1.

Copy of the MODe-LITE tool.

MODe-LITE Copyright 2021 © Soukup Lamb under CC-BY-NC-ND

The six domains of MODe-Lite were next validated against the global scores of MODe-Lite, the original MODe,7 and the MeDiC tool,20,21 (in line with previous research)12, respectively. All the MODe-Lite domains showed significant positive correlation with the MODe-Lite global score (Table 3), indicating good convergent validity across all six domains. In addition, significant positive correlation was evident against the global score for the original MODe tool, indicating good external validity and support for the H1 (i.e. MODe-Lite will correlate positively with the relevant items of the original MODe). The exception was pathology, for which the correlation coefficient, although positive, did not reach statistical significance. Significant positive correlation also was evident against the global score for the MeDiC tool, further indicating good external validity and support for the H2 (i.e. MODe-Lite will correlate positively with the MeDiC tool in line with previous research12).Table 3

Item convergent validity, reliability, and external validity for the MODe-Lite

Further external validation was performed against the corresponding MODe items in line with the content validation in phase 1 and previous research.9 Table 4 shows significant positive correlations throughout between the 6 MODe-Lite domains and their corresponding 12 MODe items. This finding provided further support for H1.Table 4

External validity and internal consistency for MODe-Lite against the original MODe tool^a

The Cronbach alpha, measuring how closely related each set of MODe items is as a domain of MODe-Lite, was good for pathology, radiology, and management plan, and somewhat weaker for the clinical, holistic inputs, and clinical collaboration, warranting further testing on a larger sample.

Inter-assessor agreement on MODe-Lite was examined using kappa coefficients for categorical (i.e., the individual items of MODe-Lite) and ICCs for continuous (i.e., the global scores) variables on a subsample of 60 cases (40 % of the total). Good reliability between the two raters (B.W.L. and S.M.) was evident across all six domains, with the ICCs for global score higher than the generally accepted 0.7026 and the kappa coefficients for individual items ranging from moderate to almost perfect agreement. The tool was reported feasible and straightforward to use by both assessors.

A post-scoring data-review session was undertaken to understand how the scoring of the tool could be improved in the two domains that initially appeared to have lower ICCs: holistic input (0.64) and management plan (0.68). Disagreement with regard to scoring of holistic information was perceived to be a limitation of data quality, specifically, sound quality. Scoring of information on holistic aspects of care in MODe-Lite was not anchored to specific members of the MTB, nor to specific terminology, and therefore was easier for observers to miss when the sound quality of the video recordings was poor.

Another limitation of data quality was disagreement on scoring of the management plan. The observers noted that the MTBs used abbreviations for follow up plans (e.g., abbreviation for a follow-up pathway) that were specific to that team or organization. However, abbreviations or jargon relating to the disease or investigations are more universally understood (e.g., CT scan). Locally specific abbreviations might not be understood by external assessors.

After review and discussion of specific cases of disagreement, the majority were settled in agreement, and revised scores for ICC were recorded (Table 5). A minority of cases remained in disagreement between the observers, which related to perceived differences in the application of the scoring, particularly for management plan. Specifically, the word “treatment” was thought to apply only when treatment was recommended. Where patients were found not to require treatment, it was thought that the word “management” was more appropriate. Otherwise, despite the formation of a good management plan, a top score could not be awarded. The assessors agreed on a slight modification of the anchor behavior for management plan, with a change in wording from “treatment options,” to “management options.”Table 5

Inter-assessor reliability coefficients for the MODe-Lite domains

The change from scoring a Likert scale of 5 points to 3 points was thought by the observers to make scoring easier. The anchor behaviors of MODe-Lite were of a more categorical nature (with scores assigned for the accumulation of different types of behaviors) than those of MDT-MODe,7 which required users to quantify a given amount of a composite behavior (e.g., contribution of a particular speciality). The assessors found that it was easier, and therefore quicker, to apply a score to MODe-Lite than to the original MODe. This increased ease of scoring was thought by the observers to be important when several domains were scored in real time and made the task more feasible.