When assessing speech, listeners may have different internal standards (Keuning et al., 1999). Thus, to achieve reliable speech results, the listeners must be trained in perceptual speech assessment (e.g., Chapman et al., 2016; Lee et al., 2009; Sell et al., 2009). In research, it is recommended to use several trained and blinded SLPs/SLTs specialized in cleft palate speech (Lohmander & Olsson, 2004; Sell, 2005). The SLPs/SLTs who have treated the children may be biased and should not be used as judges, and SLPs/SLTs from cleft lip and palate centers other than the treating should be preferred (Sell, 2005). When choosing listeners, it is important to be aware of that the listener's linguistic background can affect the perception of speech (Hutters & Henningsson, 2004; Lee et al., 2009). Other factors that may affect intra- and inter-judge agreement are e.g., the presentation and design of the task and attention deficits (Kreiman et al., 1992). When performing perceptual speech assessment, a quiet sound environment and good quality headphones are recommended (Brunnegård & Lohmander, 2007).
* Brunnegård K, Lohmander A. A cross-sectional study of speech in 10-year-old children with cleft palate: results and issues of rater reliability. Cleft Palate-Craniofacial Journal, 2007;44:33–44.
* Chapman KL, Baylis A, Trost-Cardamone J, Cordero KN, Dixon A, Dobbelsteyn C, Thurmes A. The Americleft Speech Project: A Training and Reliability Study. Cleft Palate-Craniofacial Journal, 2016;53:93–108.
* Hutters B, Henningsson G. Speech outcome following treatment in crosslinguistic cleft palate studies: methodological implications. Cleft Palate-Craniofacial Journal, 2004;41:544–549.
* Keuning KH, Wieneke GH, Dejonckere PH. The intrajudge reliability of the perceptual rating of cleft palate speech before and after pharyngeal flap surgery: the effect of judges and speech samples. Cleft Palate-Craniofacial Journal, 1999;36:328–333.
* Kreiman J, Gerratt BR, Precoda K, Berke GS. Individual differences in voice quality perception. Journal of Speech and Hearing Research, 1992;35:512–520.
* Lee A, Whitehill TL, Ciocca V. Effect of listener training on perceptual judgement of hypernasality. Clinical Linguistics & Phonetics, 2009;23:319–334.
* Lohmander A, Olsson M. Methodology for perceptual assessment of speech in patients with cleft palate: a critical review of the literature. Cleft Palate-Craniofacial Journal, 2004;41:64–70.
* Sell D. Issues in perceptual speech analysis in cleft palate and related disorders: a review. International Journal of Language & Communication Disorders, 2005;40:103–121.
* Sell D, John A, Harding-Bell A, Sweeney T, Hegarty F, Freeman J. Cleft audit protocol for speech (CAPS-A): a comprehensive training package for speech analysis. International Journal of Language & Communication Disorders, 2009;44:529–548.