Modelling and forecasting functional data (time series, spatial measurements, ...), even with a probabilistic flavour, is a common and well-handled problem nowadays. However, suppose one is collecting data from hundreds of individuals, each measuring several related biological measurements, all evolving continuously over time. Such a context, frequently arising in biological or medical studies, quickly leads to highly correlated datasets in which dependencies arise from different sources (for instance, temporal trends or individual similarities). Explicit modelling of overly large covariance matrices that account for these underlying correlations is generally infeasible due to theoretical and computational limitations. Therefore, practitioners often need to restrict their analysis by working with data subsets or making arguable assumptions (fixing time, studying tasks or individuals independently, ...). To tackle these issues, we proposed a novel paradigm for multi-task Gaussian processes, tailored to handle multiple functional data simultaneously. By sharing information between tasks through a mean process rather than an explicit covariance structure, this method yields a learning and forecasting procedure with linear complexity in the number of tasks. The resulting predictions remain Gaussian distributions and thus offer an elegant probabilistic approach to deal with correlated measurements. Group structures can also be exploited by clustering during the learning procedure to enhance predictive performance. We further formalise the distinction from multi-output Gaussian processes, an alternative strategy long studied in the literature, and discuss the conceptual and practical differences between the two approaches. We finally demonstrate that both are compatible and propose multi-task-multi-output GPs, a general framework that can handle complex and frequently arising modelling problems. Several applied examples are explored, drawn from various fields such as epidemiology, biology, meteorology or sports sciences.