? calls From::from on the error as it leaves. Implement From once, and every ? in the module converts for free:

impl From<ParseIntError> for ConfigError {
    fn from(e: ParseIntError) -> Self {
        ConfigError::BadNumber(e)
    }
}

Now this compiles, even though parse returns ParseIntError and the function returns ConfigError:

fn read_port(raw: &str) -> Result<u16, ConfigError> {
    let port: u16 = raw.parse()?;    // converted on the way out
    Ok(port)
}