Table 1 For each type of validity, the question, the comparison, and the rationale for evaluation are shown

Can we correctly identify the diagnosis date?

(Convergent validity)

Diagnosis dates from claims algorithm vs those from tumor registry (n = 11,323)

Tumor registry data are deemed most reliable because registrars abstracted data directly from the chart. However, registry data are not available for all patients, necessitating development of a cohort based on administrative data.

Can we accurately identify bladder cancer care received within VA?

(Criterion validity)

Bladder cancer care received in VA based on administrative data vs chart review (n = 100)

(1) Assure that algorithm does find all patients who did get bladder cancer care (sensitivity).

(2) Assure that patients who were identified as receiving bladder cancer care with the algorithm actually did receive such care (positive predictive value).

If we apply the algorithm to the entire cohort, can we distinguish between groups that are conceptually more or less likely to receive bladder cancer care in VA?

(Concurrent validity)

Bladder cancer care received in VA among patients with vs without full text bladder pathology reports available (n = 26,675)

Patients with full text bladder pathology reports are highly suspected to have received bladder cancer care in VA. Thus, the proportion receiving bladder cancer care in VA should be significantly higher among patients who have full text bladder pathology reports than among those who have not.